11 Software Containers

A container packages a piece of software together with everything it needs to run — libraries, dependencies, even a specific OS environment — into a single, portable file. Instead of installing a tool and its dependencies by hand and hoping they don’t conflict with anything else on the system, you download one image file and run it. The same image behaves identically on your laptop, a colleague’s machine, and Hazel.

This matters most on shared HPC systems: you don’t have root access to install system libraries, and different tools often need incompatible versions of the same dependency. Containers sidestep both problems by keeping each tool’s environment self-contained.

11.1 Docker vs. Apptainer

Docker is the tool most people encounter first, and the vast majority of pre-built scientific software images are published for it (Docker Hub, quay.io/biocontainers, bioconda, etc.). Docker itself, however, requires a persistent background service running as root — which is exactly what a shared, multi-user cluster cannot allow. Any user with Docker access on a shared machine could use it to gain root on the whole system.

Apptainer (formerly called Singularity) solves this by running containers as your own user, with no background daemon and no elevated privileges. It’s the container runtime installed on Hazel, and it’s fully compatible with images built for Docker — you can pull almost any Docker image and run it with Apptainer unchanged.

	Docker	Apptainer
Requires root daemon	Yes	No
Runs as	root (by default)	your own user
Image format	layered image, cached by the daemon	single `.sif` file
Available on Hazel	No	Yes

Important

Docker is not installed on Hazel and will not be — it is a security risk on shared infrastructure. Use Apptainer instead. Any image published for Docker can be pulled and run with Apptainer.

11.2 Getting a Container Image

The BRC already maintains a curated library of common bioinformatics tools as ready-to-use .sif images in /usr/local/usrapps/brc/brc_modules/images/ — see Loading BRC Modules for how to browse and run those directly. Check there first; if the tool you need already exists, you don’t need to pull anything yourself.

If you need a tool that isn’t in the BRC library, you can pull your own image. apptainer pull downloads a Docker or Apptainer image and converts it into a local .sif file:

$ module load apptainer
$ apptainer pull samtools.sif docker://quay.io/biocontainers/samtools:1.23.1--ha83d96e_0

This creates samtools.sif in your current directory — a single file you can move, share, or reference directly with apptainer exec.

Warning

Compute nodes on Hazel have no outbound internet access, so apptainer pull cannot run inside a regular sbatch job. Run it from the login node for small images, or — for larger pulls, or anything you don’t want tying up the shared login node — use the xfer partition, which has outbound internet access:

$ srun --pty --partition=xfer --qos=xfer --time=1:00:00 bash
$ module load apptainer
$ apptainer pull samtools.sif docker://quay.io/biocontainers/samtools:1.23.1--ha83d96e_0

See Downloading Data onto Hazel for more on where network-dependent tasks are allowed to run.

11.3 Running a Container

Once you have a .sif file — whether pulled yourself or one of the BRC’s — there are three ways to use it:

Command	What it does
`apptainer exec <image> <command>`	Run a single command inside the container, then exit
`apptainer shell <image>`	Drop into an interactive shell inside the container
`apptainer run <image>`	Run the image’s default action (if one is defined)

apptainer exec is what you’ll use most often — it lets you call a tool exactly like you would if it were installed locally, just with the container’s environment substituted in:

$ apptainer exec samtools.sif samtools --help

Program: samtools (Tools for alignments in the SAM format)
Version: 1.23.1 (using htslib 1.23.1)

Usage:  samtools <command> [options]
...

To poke around inside a container interactively — useful for checking what’s installed or debugging a failing command — use shell:

$ apptainer shell samtools.sif
Apptainer> samtools --version
Apptainer> which samtools
Apptainer> exit

Important

Running apptainer exec or apptainer shell still uses whatever compute resources are available where you run it. Never run real analysis workloads on the login node — request an interactive job with srun or submit an sbatch script, and run your apptainer exec command there.

11.4 Bind Paths: Why a Container Can’t See Your Files

A container has its own isolated filesystem. By default, Apptainer only exposes your home directory, your current working directory, and a few system paths (/tmp, /proc, /sys, /dev) inside the container — nothing else on Hazel is visible unless you explicitly “bind” it in.

This means if your data lives on /rs1 or scratch and you aren’t running the command from inside that directory tree, the container won’t be able to see it, even though you can see it fine outside the container:

$ apptainer exec samtools.sif samtools view /rs1/researchers/x/xuser/data/sample.bam
[E::hts_open] fail to open file '/rs1/researchers/x/xuser/data/sample.bam'

The fix is to bind the path explicitly with --bind:

$ apptainer exec --bind /rs1/researchers/x/xuser/data \
    samtools.sif samtools view /rs1/researchers/x/xuser/data/sample.bam

Tip

If you regularly work with data outside your home directory, set the APPTAINER_BINDPATH environment variable once in your .bashrc.d/ (see Customizing Your .bashrc) instead of adding --bind to every command. Loading BRC Modules covers this — along with the module system’s own bind-path quirks — in detail.

11.5 Building Your Own Container

Most users will never need to do this — between the BRC library and apptainer pull, nearly every common bioinformatics tool is already covered. But if you need a fully custom environment (a specific combination of tools, a pinned dependency version, your own analysis code baked in), Apptainer can build one from a definition file (.def) that scripts the container’s setup step by step.

# example.def
Bootstrap: docker
From: ubuntu:22.04

%post
    apt-get update && apt-get install -y samtools bcftools

%runscript
    echo "Custom container ready"

$ apptainer build my_tool.sif example.def

Building pulls a base image and installs software over the network, so — like apptainer pull — it needs to run from the login node or the xfer partition, not from a regular compute job.

Note

If you find yourself building and maintaining a custom container for a widely-used tool, consider requesting that the BRC add it to the shared module library instead — see Loading BRC Modules. That way the whole BRC community benefits, and you’re not responsible for keeping the image up to date yourself.