Containers

Running containers

Now days the containerization of applications reached such conservative regarding the performance areas such HPC. We are trying to evaluate possible containers solutions and in particular Singularity, since our preliminary performance tests revealed that Singularity should be preferred option.

The initial singularity configuration and basic usage example:

# Establish interactive session
srun --pty bash
# Load singularity module
module load singularity
# Run singularity command for first time
# this will create ~/.singularity folder where
# local cached images will be kept
singularity help
# Move ~/.singularity directory on ~/work parittion
mv ~/.singularity ~/work/
ln -s ~/work/.singularity ~/.singularity

# Run example Docker container mounting local folder
mkdir -p ~/work/test-folder
singularity shell -B $HOME/work/test-folder:/test-folder docker://centos

There is some already created container images based on Ubuntu 18.04 located in /opt/containers/singularity which re ready for use

  • Gromacs 2020.1 – is based on OpenMPI 3.1.6 and Gromacs 2020.1 build
  • Tensorflow 1.15 – Ready for production machine learning tool set
  • Tensorflow 2.1 – Ready for production machine learning tool set

The Tensorflow containers includes following bundle

** tensorflow 1.15 or 2.1
** opencv/4.2.0
** scipy/1.4.1
** matplotlib/3.2.1
** notebook/6.0.3
** jupyter/1.0
** pydicom/1.4.2
** pillow/7.1.2
** scikit-lean/0.22.2
** pandas/1.0.3

example usage running python within tensorflow environment

module load singularity
export IMAGE_BASE=/opt/containers/singularity
singularity run $IMAGE_BASE/tensorflow/1.15/anaconda-2020.1-tensorflow-1.15.simg -c python my_script.py

in case you want to run image in interactive mode with shell command

module load singularity
export IMAGE_BASE=/opt/containers/singularity
singularity shell $IMAGE_BASE/tensorflow/1.15/anaconda-2020.1-tensorflow-1.15.simg
. /usr/local/etc/profile.d/conda.sh
conda activate tf

and finally when you want to execute with exec

module load singularity
export IMAGE_BASE=/opt/containers/singularity
singularity exec $IMAGE_BASE/tensorflow/1.15/anaconda-2020.1-tensorflow-1.15.simg bash -c ". /usr/local/etc/profile.d/conda.sh && conda activate tf && python test.py"

HPC Laboratory provide public GitHub repository with recipes to build Singularity containers: https://github.com/hpc-lab/singularity

Scroll to Top