Installing Tensorflow and setting up the corresponding JupyterHub kernel

This page explains how to install the Tensorflow package for use with GPUs on the cluster, and how to use it from Jupyter Notebook via JupyterHub.

Installing Tensorflow for GPU node

  1. Request an interactive session on a GPU node. For example:

    $ qsub -I -l select=1:ncpus=16:mem=20gb:ngpus=1:gpu_model=p100,walltime=3:00:00
    
  2. Load the required modules

    $ module load cuda-toolkit/9.0.176
    $ module load cuDNN/9.0v7
    $ module load anaconda3/5.0.1
    
  3. Create a conda environment called tf_env (or any name you like), with Python 3.6:

    $ conda create -n tf_env pip python=3.6
    
  4. Activate the conda environment:

    $ source activate tf_env
    
  5. Following the instructions here for installing Tensorflow:

    $ pip install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.5.0-cp36-cp36m-linux_x86_64.whl
    $ conda install matplotlib
    $ conda install scipy
    $ conda install imageio
    
  6. You can now run Python and test the install:

    $ python
    
    >>> import tensorflow as tf
    

    Each time you login, you will first need to load the required modules and also activate the tf_env conda environment before running Python.

Installing Tensorflow for non-GPU node

  1. Request an interactive session for non-GPU node. For example:

    $ qsub -I -l select=1:ncpus=16:mem=20gb,walltime=3:00:00
    
  2. Load the required modules

    $ module load cuda-toolkit/9.0.176 cuDNN/9.0v7 anaconda3/5.0.1
    
  3. Create a conda environment called tf_env_cpu, with Python 3.6:

    $ conda create -n tf_env_cpu pip python=3.6
    
  4. Activate the conda environment:

    $ source activate tf_env_cpu
    
  5. Following the instructions here for installing Tensorflow:

    $ conda install numba
    $ conda install pandas
    $ conda install keras
    $ conda install cython
    $ conda install matplotlib
    $ conda install scipy
    $ conda install imageio   
    $ pip install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.12.0-cp36-cp36m-linux_x86_64.whl   
    
  6. You can now run Python and test the install:

    $ python
    
    >>> import tensorflow as tf
    

Setting up the kernel for JupyterHub:

If you would like to use Tensorflow from Jupyter Notebook on Palmetto via JupyterHub, you need the following additional steps:

  1. Install Jupyter and set up a Notebook kernel called “Tensorflow”:

    $ conda install jupyter
    $ python -m ipykernel install --user --name tf_env --display-name Tensorflow
    
  2. Create/edit the file ~/.jhubrc:

    $ nano .jhubrc
    

    Add the following two lines to the .jhubrc file, then exit.

    module load cuda-toolkit/9.0.176
    module load cuDNN/9.0v7
    
  3. Log into JupyterHub. Suggested settings:

    Number of chunks: 1 (Never request more than 1 chunk as Tensorflow will only use 1 chunk)
    CPU cores: 16
    Memory: 62gb
    Number of GPUs: 1 or 2 (Note: if GPU node is acquired, at least 2 CPU cores are required)
    Queue: workq
    

Setting up the kernel for non-GPU node is the same as GPU, except using tf_env_cpu