Quick Start
This guide is intended for running SingularityCE on a computer where you have root (administrative) privileges, and will install SingularityCE from source code. Other installation options, including building an RPM package and installing SingularityCE without root privileges are discussed in the installation section of the admin guide.
If you need to request an installation on your shared resource, see the requesting an installation section for information to send to your system administrator.
For any additional help or support contact the Sylabs team: https://www.sylabs.io/contact/
Quick Installation Steps
You will need a Linux system to run SingularityCE natively. Options for using SingularityCE on Mac and Windows machines, along with alternate Linux installation options, are discussed in the installation section of the admin guide.
Install system dependencies
You must first install development tools and libraries to your host.
On Debian-based systems, including Ubuntu:
# Ensure repositories are up-to-date
sudo apt-get update
# Install debian packages for dependencies
sudo apt-get install -y \
wget \
build-essential \
libseccomp-dev \
libglib2.0-dev \
pkg-config \
squashfs-tools \
cryptsetup \
runc
On CentOS/RHEL:
# Install basic tools for compiling
sudo yum groupinstall -y 'Development Tools'
# Install RPM packages for dependencies
sudo yum install -y \
wget \
libseccomp-devel \
glib2-devel \
squashfs-tools \
cryptsetup \
runc
There are 3 broad steps to installing SingularityCE:
Install Go
SingularityCE is written in Go, and may require a newer version of Go than is available in the repositories of your distribution. We recommend installing the latest version of Go from the official binaries.
SingularityCE aims to maintain support for the two most recent stable versions of Go. This corresponds to the Go Release Maintenance Policy and Security Policy, ensuring critical bug fixes and security patches are available for all supported language versions.
Note
If you have previously installed Go from a download, rather than an operating
system package, you should remove your go directory, e.g.
rm -r /usr/local/go, before installing a newer version. Extracting a new
version of Go over an existing installation can lead to errors when building
Go programs, as it may leave behind old files, which have been removed or
replaced in newer versions.
Visit the Go Downloads page and pick a
package archive suitable to the environment you are in. Once the
Download is complete, extract the archive to /usr/local (or use
other instructions on go installation page). Alternatively, follow the
commands here (make sure to replace the values as needed):
$ export VERSION=1.17.2 OS=linux ARCH=amd64 && \
wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz && \
sudo tar -C /usr/local -xzvf go$VERSION.$OS-$ARCH.tar.gz && \
rm go$VERSION.$OS-$ARCH.tar.gz
Set the Environment variable PATH to point to Go:
$ echo 'export PATH=/usr/local/go/bin:$PATH' >> ~/.bashrc && \
source ~/.bashrc
Download SingularityCE from a release
You can download SingularityCE from one of the releases. To see a full list, visit the GitHub release page. After deciding on a release to install, you can run the following commands to proceed with the installation.
$ export VERSION=3.11.0 && \
wget https://github.com/sylabs/singularity/releases/download/v${VERSION}/singularity-ce-${VERSION}.tar.gz && \
tar -xzf singularity-ce-${VERSION}.tar.gz && \
cd singularity-ce-${VERSION}
Compile the SingularityCE source code
Now you are ready to build SingularityCE. Dependencies will be automatically downloaded. You can build SingularityCE using the following commands:
$ ./mconfig && \
make -C builddir && \
sudo make -C builddir install
Note
SingularityCE must be installed as root to function properly.
Overview of the SingularityCE Interface
SingularityCE’s command line interface allows you to build and interact with containers transparently. You can run programs inside a container as if they were running on your host system. You can easily redirect IO, use pipes, pass arguments, and access files, sockets, and ports on the host system from within a container.
The help command gives an overview of SingularityCE options and
subcommands as follows:
$ singularity help
Linux container platform optimized for High Performance Computing (HPC) and
Enterprise Performance Computing (EPC)
Usage:
singularity [global options...]
Description:
SingularityCE containers provide an application virtualization layer enabling
mobility of compute via both application and environment portability. With
SingularityCE one is capable of building a root file system that runs on any
other Linux system where SingularityCE is installed.
Options:
-d, --debug print debugging information (highest verbosity)
-h, --help help for singularity
--nocolor print without color output (default False)
-q, --quiet suppress normal output
-s, --silent only print errors
-v, --verbose print additional information
Available Commands:
build Build a SingularityCE image
cache Manage the local cache
capability Manage Linux capabilities for users and groups
exec Run a command within a container
help Help about any command
inspect Show metadata for an image
instance Manage containers running as services
key Manage OpenPGP keys
oci Manage OCI containers
plugin Manage singularity plugins
pull Pull an image from a URI
push Upload image to the provided library (default is "cloud.sylabs.io")
remote Manage singularity remote endpoints
run Run the user-defined default command within a container
run-help Show the user-defined help for an image
search Search a Container Library for images
shell Run a shell within a container
sif siftool is a program for Singularity Image Format (SIF) file manipulation
sign Attach a cryptographic signature to an image
test Run the user-defined tests within a container
verify Verify cryptographic signatures attached to an image
version Show the version for SingularityCE
Examples:
$ singularity help <command> [<subcommand>]
$ singularity help build
$ singularity help instance start
For additional help or support, please visit https://www.sylabs.io/docs/
Information about individual subcommands can also be viewed by using the
help command:
$ singularity help verify
Verify cryptographic signatures attached to an image
Usage:
singularity verify [verify options...] <image path>
Description:
The verify command allows a user to verify cryptographic signatures on SIF
container files. There may be multiple signatures for data objects and
multiple data objects signed. By default the command searches for the primary
partition signature. If found, a list of all verification blocks applied on
the primary partition is gathered so that data integrity (hashing) and
signature verification is done for all those blocks.
Options:
-a, --all verify all objects
-g, --group-id uint32 verify objects with the specified group ID
-h, --help help for verify
-j, --json output json
--legacy-insecure enable verification of (insecure) legacy signatures
-l, --local only verify with local keys
-i, --sif-id uint32 verify object with the specified ID
-u, --url string key server URL (default "https://keys.sylabs.io")
Examples:
$ singularity verify container.sif
For additional help or support, please visit https://www.sylabs.io/docs/
SingularityCE uses positional syntax (i.e. the order of commands and options
matters). Global options affecting the behavior of all commands follow
immediately after the main singularity command. Then come subcommands,
followed by their options and arguments.
For example, to pass the --debug option to the main singularity
command and run SingularityCE with debugging messages on:
$ singularity --debug run library://lolcow
To pass the --containall option to the run command and run a
SingularityCE image in an isolated manner:
$ singularity run --containall library://lolcow
SingularityCE 2.4 introduced the concept of command groups. For
instance, to list Linux capabilities for a particular user, you would
use the list command in the capability command group, as
follows:
$ singularity capability list dave
Container authors might also write help docs specific to a container, or
for an internal module called an app. If those help docs exist for a
particular container, you can view them as follows:
$ singularity inspect --helpfile container.sif # See the container's help, if provided
$ singularity inspect --helpfile --app=foo foo.sif # See the help for foo, if provided
Download pre-built images
You can use the search command to locate groups, collections, and
containers of interest on the Container Library .
$ singularity search tensorflow
Found 22 container images for amd64 matching "tensorflow":
library://ajgreen/default/tensorflow2-gpu-py3-r-jupyter:latest
Current software: tensorflow2; py3.7; r; jupyterlab1.2.6
Signed by: 1B8565093D80FA393BC2BD73EA4711C01D881FCB
library://bensonyang/collection/tensorflow-rdma_v4.sif:latest
library://dxtr/default/hpc-tensorflow:0.1
library://emmeff/tensorflow/tensorflow:latest
library://husi253/default/tensorflow:20.01-tf1-py3-mrcnn-2020.10.07
library://husi253/default/tensorflow:20.01-tf1-py3-mrcnn-20201014
library://husi253/default/tensorflow:20.01-tf2-py3-lhx-20201007
library://irinaespejo/default/tensorflow-gan:sha256.0c1b6026ba2d6989242f418835d76cd02fc4cfc8115682986395a71ef015af18
library://jon/default/tensorflow:1.12-gpu
Signed by: D0E30822F7F4B229B1454388597B8AFA69C8EE9F
...
You can use the pull and build commands to download pre-built images from an external resource like the Container Library or Docker Hub.
Using the pull subcommand
When called on a native SingularityCE image like those provided by the
Container Library, pull simply downloads the image file to your
system:
$ singularity pull library://lolcow
You can also use pull with a docker:// URI to reference Docker
images served from a registry. In this case, pull does not just
download an image file. Docker images are stored in layers, so pull
must also combine those layers into a usable SingularityCE file.
$ singularity pull docker://sylabsio/lolcow
Pulling docker images may reduce reproducibility: if you were to pull a Docker image today and then wait six months and pull it again, you are not guaranteed to get the same image from docker on both occasions. If any of the source layers of the docker image has changed, the image will be altered. You can get around this by pulling docker images by digest, as follows:
$ singularity pull docker://alpine@sha256:69665d02cb32192e52e07644d76bc6f25abeb5410edc1c7a81a10ba3f0efb90a
Note
SingularityCE will make a SIF image out of the underlying docker image; and because SIF images contain metadata (including timestamps), resulting SingularityCE images will not be bit-for-bit identical, even if they are created from docker images that were pulled by digest.
If reproducibility is a priority for you, the best option is to always build your images from the Container Library if possible.
Using the build subcommand
You can also use the build command to download pre-built images from
an external resource. When using build you must specify a name for
your container like so:
$ singularity build ubuntu.sif library://ubuntu
$ singularity build lolcow.sif docker://sylabsio/lolcow
Unlike pull, build will convert your image to the latest
SingularityCE image format after downloading it. build is like a
“Swiss Army knife” for container creation. In addition to downloading
images, you can use build to create images from other images or from
scratch using a definition file. You can also
use build to convert an image between the container formats
supported by SingularityCE. To see a comparison of the SingularityCE
definition file with Dockerfile, please see: this section.
Interacting with images
You can interact with images in several ways, each of which can accept image URIs in addition to a local image path.
As an example, the following command will pull a lolcow_latest.sif image
from the Container Library:
$ singularity pull library://lolcow
Shell
The shell command allows you to spawn a new shell within your container and interact with it as though it were a virtual machine.
$ singularity shell lolcow_latest.sif
SingularityCE lolcow_latest.sif:~>
The change in prompt indicates that you have entered the container (though you should not rely on prompt forms to determine whether you are in a container or not).
Once inside of a SingularityCE container, you are the same user as you are on the host system.
SingularityCE lolcow_latest.sif:~> whoami
david
SingularityCE lolcow_latest.sif:~> id
uid=1000(david) gid=1000(david) groups=1000(david),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),116(lpadmin),126(sambashare)
shell also works with the library://, docker://, and
shub:// URIs. This creates an ephemeral container that disappears
when the shell is exited.
$ singularity shell library://lolcow
Executing Commands
The exec
command allows you to execute a custom command within a container by
specifying the image file. For instance, to execute the cowsay
program within the lolcow_latest.sif container:
$ singularity exec lolcow_latest.sif cowsay moo
_____
< moo >
-----
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
exec also works with the library://, docker://, and
shub:// URIs. This creates an ephemeral container that executes a
command and disappears.
$ singularity exec library://lolcow cowsay 'Fresh from the library!'
_________________________
< Fresh from the library! >
-------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
Running a container
SingularityCE containers contain runscripts. These are user-defined scripts that define the actions a container should perform when someone runs it. The runscript can be triggered with the run command, or simply by calling the container as though it were an executable.
$ singularity run lolcow_latest.sif
______________________________
< Mon Aug 16 13:01:55 CDT 2021 >
------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
$ ./lolcow_latest.sif
______________________________
< Mon Aug 16 13:12:50 CDT 2021 >
------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
run also works with the library://, docker://, and
shub:// URIs. This creates an ephemeral container that runs and then
disappears.
$ singularity run library://lolcow
______________________________
< Mon Aug 16 13:12:33 CDT 2021 >
------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
Arguments to run
You can pass arguments to the runscript of a container, if it accepts
them. For example, the default runscript of the library://alpine
container passes any arguments to a shell. We can ask the container
to run echo command in this shell as follows:
$ singularity run library://alpine echo "hello"
hello
Because SingularityCE runscripts are evaluated shell scripts, arguments can behave slightly differently than in Docker/OCI runtimes, if they contain expressions that have special meaning to the shell. Here is an illustrative example:
$ docker run -it --rm alpine echo "\$HOSTNAME"
$HOSTNAME
$ singularity run docker://alpine echo "\$HOSTNAME"
p700
$ singularity run docker://alpine echo "\\\$HOSTNAME"
$HOSTNAME
To replicate Docker/OCI behavior, you may need additional escaping or quoting of arguments.
Unlike the run command, the exec command replicates the Docker/OCI
behavior, as it calls the specified executable directly:
$ singularity exec docker://alpine echo "\$HOSTNAME"
$HOSTNAME
$ singularity exec docker://alpine echo "\\\$HOSTNAME"
\$HOSTNAME
Working with Files
Files on the host are reachable from within the container:
$ echo "Hello from inside the container" > $HOME/hostfile.txt
$ singularity exec lolcow_latest.sif cat $HOME/hostfile.txt
Hello from inside the container
This example works because hostfile.txt exists in the user’s home
directory. By default, SingularityCE bind mounts /home/$USER,
/tmp, and $PWD into your container at runtime.
You can specify additional directories to bind mount into your container
with the --bind option. In this example, the data directory on
the host system is bind mounted to the /mnt directory inside the
container.
$ echo "Drink milk (and never eat hamburgers)." > /data/cow_advice.txt
$ singularity exec --bind /data:/mnt lolcow_latest.sif cat /mnt/cow_advice.txt
Drink milk (and never eat hamburgers).
Pipes and redirects also work with SingularityCE commands, just like they do with normal Linux commands:
$ cat /data/cow_advice.txt | singularity exec lolcow_latest.sif cowsay
________________________________________
< Drink milk (and never eat hamburgers). >
----------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
Building images from scratch
SingularityCE v3.0 and above produces immutable images in the Singularity Image File (SIF) format. This ensures reproducible and verifiable images and allows for many extra benefits such as the ability to sign and verify your containers.
However, during testing and debugging you may want an image format that
is writable. This way you can shell into the image and install
software and dependencies until you are satisfied that your container
will fulfill your needs. For these scenarios, SingularityCE also
supports the sandbox format (which is really just a directory).
Sandbox Directories
To build into a sandbox (container in a directory) use the build
--sandbox command and option:
$ singularity build --sandbox ubuntu/ library://ubuntu
This command creates a sub-directory called ubuntu/ with an entire
Ubuntu operating system and some SingularityCE metadata in your current
working directory.
You can use commands like shell, exec , and run with this
directory just as you would with a SingularityCE image. If you pass the
--writable option when you use your container, you can also write
files within the sandbox directory (provided you have the permissions to
do so).
$ singularity exec --writable ubuntu touch /foo
$ singularity exec ubuntu/ ls /foo
/foo
Converting images from one format to another
The build command allows you to build a new container from an existing
container. This means that you can use it to convert a container from one format
to another. For instance, if you have already created a sandbox (directory) and
want to convert it to the Singularity Image Format you can do so:
$ singularity build new.sif sandbox
Doing so may break reproducibility if you have altered your sandbox outside of the context of a definition file, so you are advised to exercise care.
SingularityCE Definition Files
For a reproducible, verifiable and production-quality container, it is recommended that you build your SIF file using a SingularityCE definition file. This also makes it easy to add files, environment variables, and install custom software, while still starting from your base of choice (e.g., the Container Library).
A definition file has a header and a body. The header determines the base container to begin with, and the body is further divided into sections that perform tasks such as software installation, environment setup, and copying files into the container from host system.
Here is an example of a definition file:
BootStrap: library
From: ubuntu:22.04
%post
apt-get -y update
apt-get -y install cowsay lolcat
%environment
export LC_ALL=C
export PATH=/usr/games:$PATH
%runscript
date | cowsay | lolcat
%labels
Author Sylabs
To build a container from this definition file (assuming it is a file
named lolcow.def), you would call build as follows:
$ sudo singularity build lolcow.sif lolcow.def
In this example, the header tells SingularityCE to use a base Ubuntu 22.04 image from the Container Library. The other sections in this definition file are as follows:
The
%postsection is executed within the container at build time, after the base OS has been installed. The%postsection is therefore the place to perform installations of new libraries and applications.The
%environmentsection defines environment variables that will be available to the container at runtime.The
%runscriptsection defines actions for the container to take when it is executed. (These commands will therefore not be run at build time.)And finally, the
%labelssection allows for custom metadata to be added to the container.
This is a very small example of the things that you can do with a definition file. In addition to building a container from the Container Library, you can start with base images from Docker Hub and use images directly from official repositories such as Ubuntu, Debian, CentOS, Arch, and BusyBox. You can also use an existing container on your host system as a base.
If you want to build SingularityCE images but you don’t have administrative (root) access on your build system, you can build images using the Remote Builder.
This quickstart document just scratches the surface of all of the things you can do with SingularityCE!
If you need additional help or support, contact the Sylabs team: https://www.sylabs.io/contact/