Python
We have multiple versions of Python installed in /share/apps. Each version has a set of packages installed, however it will be necessary for you to modify your own Python software stacks.
In order to ‘enable’ a specific version of Python, you should source a source file. These are located at /share/apps/source_files/python. If you wanted to use Python-3.6.4 for example:
source /share/apps/source_files/python/python-3.6.4.source
Pip
We recommend using pip to maintain your Python packages. Anaconda is largely inappropriate for the cluster, principally due to the large number of files (many of which are duplicates) produced when creating new environments. Each environment in Anaconda comes with it’s own individual Python installation, and packages are not shared across envs. As Anaconda is generally installed in your home directory, which has a hard limit on space and number of files, you will rapidly reach your quota. Miniconda is less bloated than Anaconda, however it still causes the same issues.
Instead, we recommend using pip with the --user flag to install Python packages in your home directory. The --user flag will install packages in .local/lib/python<major_version>.<minor_version>/site-packages. Packages installed in this directory take precedence over the system packages (installed within /share/apps).
There are a small number of packages that are only installable via Anaconda, in this case Miniconda is recommended.
Pip installs packages in directories based on the major and minor versions, and does not make a distinction based on patch level. This means that whether you are using python-3.6.4 or python-3.6.9, your local packages will be the same.
Installing packages from a file
Pip can install a list of packages by running the following:
pip install -r requirements.txt --user
This file has the following form:
<package1>==<version>
<package2>==<version>
...
You can get pip to output a list of currently installed packages in the form expected by this command by running pip freeze.
It is not necessary for the file to be called requirements.txt, so it is possible to create different environments by maintaining text files containing package lists.
PyTorch & TensorFlow (GPU)
Using PyTorch and TensorFlow GPU requires you to set up CUDA in your environment. Again, there are a range of source files written for different versions of CUDA. For example, to use CUDA 10.0 run the following:
source /share/apps/source_files/cuda/cuda-10.0.source
It is important to use the correct CUDA version for the versions of PyTorch/TensorFlow GPU that you are using.
TensorFlow maintains a list of corresponding CUDA and Python versions here. If you are having trouble with your software stack, it is worth checking that your CUDA version is appropriate for the TensorFlow version you are using.
The variable CUDA_VISIBLE_DEVICES is set for you. Please do not modify this variable from within your code or submission scripts.
Jupyter Notebook
Using jupyter for heavy compute purposes on the cluster is not allowed. However, running a notebook on a login node in order to do small-scale computational tasks is fine. Examples of appropriate tasks includes creating plots from data stored on the cluster storage infrastructure and manipulating data – NOT testing pipelines or models. Larger-scale computational testing should be performed in interactive sessions, however, you will not be able to connect a browser to these sessions.
Running Jupyter
On a login node, you should specify a port that Jupyter will run on, as well as the no browser option:
jupyter notebook --no-browser --port <port_number>
You can use any port number between 7000 and 9500, you should pick a random number between these to ensure you aren’t picking the same port number as someone else.
Tunnelling
As there isn’t a browser on the cluster, you will need to set up a tunnel from your local machine to the cluster login node that you started the Jupyter notebook on. This is achieved by using the ssh command with a handful of additional options as follows:
ssh -l <your_CS_username> -J <your_CS_username>@tails.cs.ucl.ac.uk -L <local_port>:localhost:<remote_port> <login_node>.cs.ucl.ac.uk
Replacing the fields wrapped in <> symbols with the appropriate values. The local port should be any port number you wish to connect to on your local machine – as long as this number is above 1024 it should be fine. Similarly, the remote port should be the same number that you ran the notebook on the cluster with. The login node is the name of the login node you ran jupyter on.
For example, the user alice running their jupyter notebook on pchuckle from port 8765, who wants to use local port 2222 to create the tunnel would run:
ssh -l alice -J alice@tails.cs.ucl.ac.uk -L 2222:localhost:8765 pchuckle.cs.ucl.ac.uk
Providing this tunnel is created successfully (no errors reported when running the above command), the user would then type the following into the address bar of their browser:
localhost:2222
This will then display the jupyter notebook login page.