3  Command Line Basics

If the only computer interface you’ve used is a graphical one — clicking folder icons, dragging files into the Trash, double-clicking programs — the cluster’s command line can feel like staring at a blank wall. There are no folders to click. There is no Trash. The whole system is a text prompt waiting for you to type something.

This chapter is for that situation. It walks through the handful of commands that get you from “I just logged in and have no idea what to do” to “I can move around, edit a file, and run a script.” Every example is something you would actually type on Hazel.

Note

Use this chapter as a reference, a cheat sheet is located at the bottom. After a few days of using the command line, the commands here will be muscle memory.

3.1 The Prompt

When you first SSH into the cluster, you’ll see something like this:

[unityid@login01 ~]$

That whole line is called the prompt. It is the shell telling you what it knows about your situation:

• unityid — your user name
• login01 — the machine you’re connected to
• ~ — the directory you are currently sitting in (the tilde ~ is shorthand for your home directory)
• $ — the cursor where you type

Note

When this chapter shows a command starting with $, that just means “type this at the prompt.” Don’t actually type the $.

3.2 Where Am I? — pwd

pwd stands for print working directory. It tells you exactly where you are in the filesystem.

$ pwd
/home/unityid

If you ever feel lost, type pwd. It is the easiest way to get oriented.

3.3 What’s Here? — ls

ls (short for list) shows the contents of the current directory.

$ ls
data  scripts  results  notes.txt

ls has a handful of options worth knowing:

Command	What it does
ls	List files and directories in the current location
ls -l	Long format — shows size, owner, modification time
ls -a	Show hidden files (anything starting with a .)
ls -h	Human-readable file sizes (use with -l)
ls -t	Sort by modification time, newest first
ls -lrt	Long format, oldest first, newest at the bottom
ls /some/path	List the contents of a different directory

The flags can be combined: ls -lah is “long format, all files, human-readable sizes.”

$ ls -lh
drwxr-xr-x 2 unityid users 4.0K Apr 12 10:14 data
drwxr-xr-x 2 unityid users 4.0K Apr 12 10:14 scripts
-rw-r--r-- 1 unityid users 1.2K Apr 12 10:14 notes.txt

The first letter of each row tells you the type: d is a directory, - is a regular file.

3.4 Moving Around — cd

cd stands for change directory. It is how you walk through the filesystem.

$ cd data            # go into the data folder
$ cd /share/brc      # jump to an absolute path
$ cd ..              # go up one level
$ cd ~               # go home
$ cd                 # also goes home (cd with no argument)
$ cd -               # go back to the last directory you were in

After every cd, run pwd and ls until you trust where you are.

3.5 Paths: ~, ., .., and the difference between absolute and relative

Every file on the cluster has a path — a string that says how to find it. Paths use forward slashes / to separate directory names.

An absolute path starts with / and gives the complete location from the very top of the filesystem:

/rs1/researchers/s/smith/project1/reads.fastq

A relative path is interpreted starting from your current directory. If you are sitting in /rs1/researchers/s/smith and you type project1/reads.fastq, the shell expands that to the absolute path above.

A few special path shortcuts come up constantly:

Symbol	Meaning
~	Your home directory (e.g., /home/$USER)
.	The current directory
..	The parent directory (one level up)
/	The root of the filesystem

So cd ../.. goes up two levels, and ./my_script.sh means “the file my_script.sh in the directory I’m currently in.”

Tip

When in doubt, use absolute paths in job scripts. Relative paths break the moment you submit a job from a different directory.

3.6 Tab Completion, History, and Editing the Command Line

Before you go any further: learn these shortcuts. They will save you hours.

3.6.1 Tab completion

Press Tab while typing a file or directory name and the shell will finish it for you. Press Tab twice and it will show all the possible completions.

$ cd /rs1/res<Tab>
# becomes:
$ cd /rs1/researchers/

This is not just convenience — it’s also how you avoid typos. If Tab doesn’t complete, the file you typed doesn’t exist.

3.6.2 Command history

Key	What it does
Up arrow / Down arrow	Step through previous commands
Ctrl+R	Search backwards through your history (type a few letters)
history	Print your recent commands
!!	Re-run the last command
!123	Re-run command number 123 from history

3.6.3 Line editing

Key	What it does
Ctrl+A	Jump to the start of the line
Ctrl+E	Jump to the end of the line
Ctrl+C	Cancel the current command (or kill a running program)

3.7 Making Directories — mkdir

mkdir creates a new directory.

$ mkdir results
$ mkdir -p project/data/raw    # -p makes parent directories as needed

The -p flag is the one to remember. Without it, mkdir project/data/raw fails if project/ doesn’t exist yet. With it, the whole chain is created in one go and mkdir will not complain if the directory already exists.

3.8 Creating and Viewing Files

3.8.1 Make an empty file — touch

$ touch notes.txt

touch creates an empty file (or, if the file already exists, updates its modification timestamp).

3.8.2 Quickly write text into a file — echo with >

$ echo "hello world" > greeting.txt

The > is a redirection operator — more on that in Section 3.13.

3.8.3 View a whole file — cat

$ cat greeting.txt
hello world

cat dumps the entire contents of a file to the screen. Fine for small files. For anything more than a few dozen lines, use less instead.

3.8.4 Page through a file — less

$ less results.txt

less opens the file in a scrollable viewer. Useful keys inside less:

Key	Action
Space / f	Next page
b	Previous page
g	Jump to the top
G	Jump to the bottom
/word	Search forward for “word”
n / N	Next / previous search match
q	Quit

3.8.5 Just the first or last few lines — head and tail

$ head reads.fastq            # first 10 lines
$ head -n 4 reads.fastq       # first 4 lines
$ tail -n 20 job.log          # last 20 lines
$ tail -f job.log             # follow the file as it grows (Ctrl+C to stop)

tail -f is invaluable for watching a log file while a job is running.

3.9 Copying — cp

cp copies a file to a new location.

$ cp notes.txt notes_backup.txt
$ cp notes.txt /share/brc/$USER/notes.txt
$ cp -r project/ project_v2/                # -r for directories
$ cp -p file1 file2                         # -p preserves timestamps and permissions

The -r (recursive) flag is required when copying a directory — without it, cp refuses.

3.10 Moving and Renaming — mv

mv moves a file from one place to another. There is no separate “rename” command — renaming is just moving a file to a new name in the same directory.

$ mv notes.txt notes_old.txt              # rename
$ mv notes.txt /share/brc/$USER/          # move to another directory
$ mv *.fastq fastq_files/                 # move many files at once (see wildcards below)

Warning

mv will silently overwrite an existing destination file. Use mv -i to be prompted before any overwrite.

3.11 Deleting — rm

rm deletes a file. This is where the cluster differs sharply from a graphical desktop.

Important

There is no Trash and no Undo on the cluster. When you rm a file, it is gone immediately and permanently. Cluster filesystems generally do not have user-accessible snapshots you can restore from.

$ rm old_results.txt
$ rm -r old_project/             # -r for directories (recursive)
$ rm -i confidential.txt         # -i prompts before each delete

You will see rm -rf in tutorials online — it means “recursive, force, don’t prompt for anything.” It is one of the most dangerous things you can type. Read carefully before using it, especially with paths that include * or variables.

To delete an empty directory, you can also use rmdir, which (unlike rm -r) refuses to delete anything that still has files in it — a useful safety net.

$ rmdir empty_folder/

3.12 Wildcards (Globs)

The shell expands certain characters into lists of matching filenames before running your command.

Pattern	Matches
*	Any string of characters (including none)
?	Any single character
[abc]	Any one character from the set a, b, or c
[0-9]	Any single digit
{red,blue}	Either red or blue (brace expansion)

$ ls *.txt                    # every file ending in .txt
$ ls sample_*.txt             # sample_001.txt, sample_abc.txt, ...
$ ls sample_00?.txt           # sample_001 through sample_009 only
$ rm *.tmp                    # delete every .tmp file (be careful!)
$ cp data/*.csv backup/       # copy all csv files into backup/

Warning

Always preview a wildcard with ls before using it with rm or mv. A space in the wrong place — rm * .tmp instead of rm *.tmp — will delete every file in your directory.

3.13 Pipes and Redirection

A program’s output normally goes to your terminal. With redirection and pipes, you can send that output somewhere else.

3.13.1 Redirecting output to a file

Operator	Effect
> file	Send standard output to file, overwriting it
>> file	Append standard output to file
2> file	Send standard error to file
&> file	Send both stdout and stderr to file
2>&1	Merge stderr into stdout
< file	Read input from file instead of the keyboard

$ ls -l > listing.txt                     # save the listing to a file
$ echo "run 1 finished" >> log.txt        # append a line to a log
$ python script.py > out.log 2> err.log
$ python script.py &> all.log             # both streams into one file

3.13.2 /dev/null — the trash chute

/dev/null is a special file that throws away anything written to it. Use it to discard output you don’t care about.

$ noisy_command > /dev/null         # discard stdout
$ noisy_command 2> /dev/null        # discard stderr
$ noisy_command &> /dev/null        # discard everything

3.13.3 Pipes — chaining commands

The pipe | sends one command’s output directly into the next command’s input. This is what makes the shell powerful: small commands compose into bigger workflows.

$ ls -l | less                       # page through a long listing
$ cat data.txt | head -n 8           # first 8 lines of data.txt
$ history | tail -n 20               # last 20 commands you ran
$ ls *.txt | wc -l                   # count how many .txt files exist in the current directory

You will see pipes constantly in HPC workflows — they’re how data flows between tools without writing intermediate files.

3.14 Editing Files in the Terminal

You will need to edit text files directly on the cluster: job scripts, config files, small notes. There are two editors you should know about.

3.14.1 nano — the friendly one

nano is a small, beginner-friendly editor. The list of available commands is always shown at the bottom of the screen.

$ nano my_script.sh

Inside nano, the ^ symbol means Ctrl:

Shortcut	What it does
Ctrl+O then Enter	Save (Write Out)
Ctrl+X	Exit (asks to save if you’ve changed anything)
Ctrl+K	Cut the current line
Ctrl+U	Paste
Ctrl+W	Search
Ctrl+G	Help

If you only learn one editor, learn nano. It is enough to do almost anything.

3.14.2 vim — survival guide

vim is everywhere on Linux systems, including as the default editor for some commands like git. Sooner or later you will open it by accident, and you need to know how to get out.

vim has modes. When it opens, you are in normal mode, where most keys are commands rather than text input. To actually type characters into the file, you have to enter insert mode first.

To do this	Type this
Quit without saving	Esc then :q! then Enter
Save and quit	Esc then :wq then Enter
Save without quitting	Esc then :w then Enter
Enter insert mode (start typing)	i
Leave insert mode	Esc
Undo	Esc then u
Redo	Esc then Ctrl+R

The single most useful sequence is Esc :q! Enter — that will get you out of vim no matter what state it is in, without saving any changes.

Tip

If you prefer not to deal with vim showing up unexpectedly, set your default editor to nano:

$ export EDITOR=nano

Add that line to ~/.bashrc to make it stick across sessions.

3.15 File Permissions and Making Scripts Executable — chmod

Every file on the cluster has permissions that control who can read, write, and execute it. The ls -l output shows them in the first column:

-rwxr-xr-- 1 unityid users 1024 Apr 12 10:14 my_script.sh

The 10 characters break down as:

• Position 1: file type (- for file, d for directory)
• Positions 2–4: owner permissions (rwx = read, write, execute)
• Positions 5–7: group permissions
• Positions 8–10: everyone else’s permissions

For day-to-day use, the only chmod command most people need is the one that makes a script runnable:

$ chmod +x my_script.sh

Without +x, trying to run a script gives Permission denied even if the script is otherwise correct. This is the #1 reason a freshly written script “doesn’t work.”

Other common forms:

$ chmod -x my_script.sh         # remove execute permission
$ chmod 644 notes.txt           # rw-r--r-- (owner read/write, others read-only)
$ chmod 755 my_script.sh        # rwxr-xr-x (typical for scripts)
$ chmod -R 750 my_project/      # apply recursively to a whole directory

The numeric forms are a shorthand: each digit is read(4) + write(2) + execute(1) for owner/group/everyone. 755 means owner gets 4+2+1=7, group gets 4+1=5, everyone gets 4+1=5.

3.16 Writing and Running a Bash Script

A bash script is just a text file containing shell commands, run from top to bottom. Anything you can type at the prompt, you can put in a script.

3.16.1 A minimal script

Create a file called hello.sh with nano:

$ nano hello.sh

Type the following and save:

#!/bin/bash

echo "Hello from $(hostname)"
echo "Today is $(date)"
echo "I am in $(pwd)"

Three things to notice:

1. The shebang line (#!/bin/bash) at the very top tells the system “run this script with bash.” It must be the first line of the file, with no spaces before #!.
2. Comments start with # and are ignored by the shell. Use them liberally to explain what your script is doing.
3. $(command) runs a command and substitutes its output into the line. Here it puts the output of hostname, date, and pwd into the printed text.

3.16.2 Running the script

There are two ways to run a script. They look similar but mean slightly different things.

# Option 1: explicitly pass it to bash
$ bash hello.sh

# Option 2: mark it executable, then run it directly
$ chmod +x hello.sh
$ ./hello.sh

The ./ in front of hello.sh is important. It tells the shell “run the hello.sh that is in this directory.” Without it, the shell would search through $PATH for an installed program called hello.sh and report that none exists.

3.16.3 Variables

Variables hold values. You set them with = (no spaces) and read them with $:

#!/bin/bash

NAME="reads.fastq"
OUT_DIR="/share/brc/$USER/results"

mkdir -p "$OUT_DIR"
echo "Processing $NAME into $OUT_DIR"

A few rules that trip people up:

• No spaces around =. NAME = "x" does not work; NAME="x" does.
• Always quote variables when they hold paths: "$OUT_DIR", not $OUT_DIR. Quoting prevents bugs when paths contain spaces.
• Convention is to use UPPER_CASE for variables you set yourself.

3.16.4 Arguments

A script can accept arguments from the command line. Inside the script, $1 is the first argument, $2 the second, and so on:

#!/bin/bash
# usage: ./greet.sh NAME

PERSON="$1"
echo "Hello, $PERSON"

$ chmod +x greet.sh
$ ./greet.sh Jon
Hello, Jon

Important

Run scripts on a compute node, not the login node, if they do real work. The bash scripts in this chapter are fine for small, quick tasks — but anything that uses real CPU, memory, or time should be wrapped in a SLURM job script and submitted with sbatch. See the Running Jobs chapter.

3.17 A Worked Example

Here is a small workflow that uses most of what’s in this chapter. It sets up a project directory, copies some input data into it, and writes a small script that counts how many lines each input file has.

# 1. Find out where you are.
# If you just logged into hazel, you should be in your home directory
$ pwd
/home/<unityid>

# If not, navigate there
$ cd ~

# 2. Make a new directory for this project
$ mkdir -p line_counts
$ cd line_counts
$ pwd
/home/<unityid>/line_counts

# 3. Copy in some input files.
$ mkdir -p input
$ cp /rs1/shares/brc/trainings/COS_Compute_Handbook/data/*.fastq input/
$ ls input/
sample_001_R1.fastq  sample_001_R2.fastq  sample_002_R1.fastq  ...

# 4. Write the script
$ nano count_lines.sh

In nano, paste:

#!/bin/bash
set -euo pipefail

# Set some variables
IN_DIR="input"
OUT_FILE="line_counts.txt"

# Create an empty output file, or wipe it clean if it already exists.
> "$OUT_FILE"

# Loop over every .fastq file in input/, count its lines, and append the result to OUT_FILE.
for f in "$IN_DIR"/*.fastq; do
    n=$(wc -l < "$f")
    echo "$f $n" >> "$OUT_FILE"
done

# Print a message at the end so you know the script finished.
echo "Done. Results in $OUT_FILE"

Save with Ctrl+O, Enter, then exit with Ctrl+X.

# 5. Make it executable and run it
$ chmod +x count_lines.sh
$ ./count_lines.sh
Done. Results in line_counts.txt

# 6. Look at the results
$ cat line_counts.txt
input/sample_001_R1.fastq 4000
input/sample_001_R2.fastq 4000
input/sample_002_R1.fastq 400
input/sample_002_R2.fastq 400
input/sample_003_R1.fastq 1200
input/sample_003_R2.fastq 1200

That’s a complete shell workflow: directories, files, a script, and a result.

3.18 Cheat Sheet

Task	Command
Where am I?	pwd
What’s here?	ls, ls -lah
Move into a directory	cd directory
Move up one level	cd ..
Go home	cd ~ or just cd
Make a directory	mkdir -p path/to/dir
Make an empty file	touch file.txt
View a file	cat file.txt, less file.txt
First / last lines	head file, tail file, tail -f file
Copy a file	cp src dest
Copy a directory	cp -r src dest
Move / rename	mv src dest
Delete a file	rm file (no undo!)
Delete a directory	rm -r dir (no undo!)
Count lines / words	wc -l file, wc -w file
Send output to a file	command > file
Append output to a file	command >> file
Pipe one command into another	command1 \| command2
Discard output	command > /dev/null 2>&1
Make a script runnable	chmod +x script.sh
Run a script	./script.sh or bash script.sh
Edit a file	nano file.txt
Get out of vim	Esc :q! Enter

3.19 Where to Go Next

Once you’re comfortable moving around, viewing files, and running small scripts, the natural next steps are:

• The File Transfer Basics chapter for transferring files to and from the cluster
• The Running Jobs chapter for turning a bash script into a SLURM job
• man <command> — every command above has a manual page. man ls, man cp, man bash. Press q to exit.
• <command> --help — most commands also accept --help for a quick summary