14  GPU Jobs

Some bioinformatics workloads — deep learning-based tools like AlphaFold, base-calling for long-read sequencing, and other neural-network-driven pipelines — run far faster on a GPU than on any number of CPU cores. Hazel has a dedicated gpu partition for this work, but requesting a GPU takes an explicit resource directive; SLURM never hands one out by default. See CPU vs. GPU for a refresher on when a GPU actually helps.

14.1 Requesting a GPU

To request a GPU you must specify that you want to use a GPU partion, and specifically request what GPU type you want to use with --gres (generic resource). Both of these options can be passed as SBATCH directives to your job script:

#SBATCH --partition=gpu_partners
#SBATCH --gres=gpu:a100:1

--gres=gpu:a100:1 requests one A100 GPU. To request a different model or more than one:

#SBATCH --gres=gpu:h100:2   # two H100 GPUs

You can also specify that your GPU job will be short with:

#SBATCH --qos=short_gpu

This will cap your job time at 2 hours, but will move you up in the queue. This is useful if you know your job will be short, especially if you are in a debugging or testing

Important

--gres requests a GPU device; it does not request CPU cores or RAM for the host-side portion of your job. Most GPU tools still need several CPU cores to feed data to the GPU — set --cpus-per-task and --mem as you would for any other job.

14.2 Example Job Script

#!/bin/bash
# ---------------------------------------
# GPU job script
# ---------------------------------------

# --- Resources ---
#SBATCH --job-name=ML_task
#SBATCH --partition=gpu_partners
#SBATCH --gres=gpu:a100:1
#SBATCH --cpus-per-task=4
#SBATCH --mem=32G
#SBATCH --output=logs/ML_task_%j.out
#SBATCH --error=logs/ML_task_%j.err
#SBATCH --time=2:00:00

# --- Environment ---
module load apptainer

# --- Execute ---
apptainer exec --nv my_model.sif python run_inference.py
Important

Pass --nv to apptainer exec/apptainer run whenever the container needs GPU access. Without it, the NVIDIA driver and CUDA libraries aren’t mounted into the container and GPU-dependent code fails or silently falls back to CPU.

If you aren’t using a container, you may have to load the cuda module to configure your program for use with GPU nodes.

14.3 Checking GPU Availability and Usage

Before submitting, you check what’s free on the gpu partition:

$ sinfo -p gpu                         # node/GPU availability on the gpu partition
$ scontrol show node <NODENAME>        # GRES detail for one node, including GPU type and count

14.4 Chosing the right GPU

Hazel has a number of different GPU types. While they all share core similarities, their memory capibilites and data transfer speeds may differ. See this informational page for additional information about GPUs at NC State.

GPU Type Best For
H200 / H100 Large language models, transformer training, scientific computing, and workloads needing maximum memory bandwidth
A100 Deep learning training, scientific computing, and multi-GPU workloads
L40 / L40S Inference, visualization, and mixed AI/graphics workloads
A30 / A10 Smaller models, inference, and development/testing
P100 / RTX 2080 / GTX 1080 Older generation; development, testing, and smaller workloads

14.5 Practical Tips

  • GPUs on Hazel are a shared, limited resource — request only what your tool can actually use, and release the job as soon as it’s done
  • Test on a short interactive GPU session before submitting a long batch job; a broken CUDA setup fails immediately, an under-resourced one fails after an hour
  • Not every tool benefits from a GPU — check the tool’s documentation for GPU support before requesting one