Now days the containerization of applications reached such conservative regarding the performance areas such HPC. We are trying to evaluate possible containers solutions and in particular Singularity, since our preliminary performance tests revealed that Singularity should be preferred option.
singularity configuration and basic usage example:
# Establish interactive session srun --pty bash # Load singularity module module load singularity # Run singularity command for first time # this will create ~/.singularity folder where # local cached images will be kept singularity help # Move ~/.singularity directory on ~/work parittion mv ~/.singularity ~/work/ ln -s ~/work/.singularity ~/.singularity # Run example Docker container mounting local folder mkdir -p ~/work/test-folder singularity shell -B $HOME/work/test-folder:/test-folder docker://centos
There is some already created container images based on Ubuntu 18.04 located in
/opt/containers/singularity which re ready for use
- Gromacs 2020.1 – is based on OpenMPI 3.1.6 and Gromacs 2020.1 build
- Tensorflow 1.15 – Ready for production machine learning tool set
- Tensorflow 2.1 – Ready for production machine learning tool set
The Tensorflow containers includes following bundle
** tensorflow 1.15 or 2.1
example usage running python within tensorflow environment
module load singularity export IMAGE_BASE=/opt/containers/singularity singularity run $IMAGE_BASE/tensorflow/1.15/anaconda-2020.1-tensorflow-1.15.simg -c python my_script.py
in case you want to run image in interactive mode with
module load singularity export IMAGE_BASE=/opt/containers/singularity singularity shell $IMAGE_BASE/tensorflow/1.15/anaconda-2020.1-tensorflow-1.15.simg . /usr/local/etc/profile.d/conda.sh conda activate tf
and finally when you want to execute with
module load singularity export IMAGE_BASE=/opt/containers/singularity singularity exec $IMAGE_BASE/tensorflow/1.15/anaconda-2020.1-tensorflow-1.15.simg bash -c ". /usr/local/etc/profile.d/conda.sh && conda activate tf && python test.py"