Department of Physics and Astronomy

The Forbes Group

Parallel Computing with IPython

$\newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\uvect}[1]{\hat{#1}} \newcommand{\abs}[1]{\lvert#1\rvert} \newcommand{\norm}[1]{\lVert#1\rVert} \newcommand{\I}{\mathrm{i}} \newcommand{\ket}[1]{\left|#1\right\rangle} \newcommand{\bra}[1]{\left\langle#1\right|} \newcommand{\braket}[1]{\langle#1\rangle} \newcommand{\op}[1]{\mathbf{#1}} \newcommand{\mat}[1]{\mathbf{#1}} \newcommand{\d}{\mathrm{d}} \newcommand{\pdiff}[3][]{\frac{\partial^{#1} #2}{\partial {#3}^{#1}}} \newcommand{\diff}[3][]{\frac{\d^{#1} #2}{\d {#3}^{#1}}} \newcommand{\ddiff}[3][]{\frac{\delta^{#1} #2}{\delta {#3}^{#1}}} \DeclareMathOperator{\erf}{erf} \DeclareMathOperator{\Tr}{Tr} \DeclareMathOperator{\order}{O} \DeclareMathOperator{\diag}{diag} \DeclareMathOperator{\sgn}{sgn} \DeclareMathOperator{\sech}{sech} $

Here we discuss how to use ipyparallel to perform some simple parallel computing tasks.

Installation

  1. I will be using this from a Jupyter notebook. First make sure you install Jupyter and ipyparallel on your system (I recommend using conda).

    conda install jupyter notebook ipyparallel

  2. Next enable the IPython Clusters tab in Jupyter Notebook:

    ipcluster nbextension enable
jupyter nbextension install --py ipyparallel
jupyter nbextension enable --py ipyparallel 
jupyter serverextension enable --py ipyparallel

  3. You will need to restart any Jupyter servers to see the effects of this, but you should now be able to navigate to the IPython Clusters tab and see at least the default profile.

Configuration

The next step is to configure a cluster.

ipython profile create --parallel --profile=local_cluster