Globus Setup
Welcome to the Globus Setup workshop! This workshop will cover how to use set up and use Globus to transfer data to and from Hazel.
0.1 Advantages of using Globus
- Resumable transfers — picks up where it left off after a dropped connection;
scprestarts from scratch - Faster on large/bulk transfers — uses parallel, multi-threaded data streams instead of
scp’s single stream - No open session required — runs in the background on Globus’s servers, so you can log off and it keeps going (unlike
scpor an Open OnDemand file browser session) - Third-party transfers — moves data directly between two remote endpoints (e.g., Hazel and another cluster) without routing through your laptop
- Built-in integrity checks — automatically verifies checksums after transfer
- Email notifications — alerts you when a transfer completes or fails
- Better for large files than Open OnDemand — the OnDemand file browser is convenient for quick edits/small files but isn’t built for large or bulk transfers
1 Logging into Globus
You should automatically have a Globus account sponsored through NC State.
- Navigate to the Globus webpage and select
LOG INat the top right. - Enter
North Carolina State Universitywhen prompted. - Log in with your unity ID credentials.
You’re in!
2 Connection Globus to Hazel
As we learned earlier, there are multiple locations on Hazel including your home directory, the scratch directory and reserach storage (rs1). You can connect to all of them, however, Research Storage has a separate endpoint than the rest of Hazel.
- In the Globus web portal, navigate to the
File Managertab. - To connect to anything in
/rs1search for NC State Research Storage in theCollectionsearch bar. Once selected, you should navigate to your Research Storage directories. - To connect to anything else on Hazel, including the scratch directory, search for NC State Hazel HPC Cluster in the
Collectionsearch bar. Once selected, you should see your home (~) directory.
Any path that exists on Hazel that you have read access to can be navigated to in the Path search bar (as long as you are on the proper endpoint).
3 Connecting Globus to your personal computer
To easily transfer large files to and from Hazel to your personal computer, you first need to download the Globus Connect Personal application.
- Navigate to the
collectionstab when logged on to the Globus web portal. - Click
Get Globus Connect Personalin the top right corner and download for your operating system. - Once you have opened the Globus Connect Personal app, click log in. This will take you to the Globus web page.
- Chose a meaningful label, like “your-name laptop”, and click
Allow. Complete a similar authentification in the Globus Personal Connect desktop app.
If you have done these steps properly, you should see a Globus icon appear in you taskbar. Now go back to the Globus web portal.
- In the Globus web portal, navigate to the
File Managertab. - In the
Collectionsearch bar, search for your Personal Collection name. Once selected, you should be able to see your local computer’s file system.
Your Globus Connect Personal desktop app must be active your transfers to work. If you are getting an error when trying to view files, make sure the desktop app is open, and the endpoint is not “paused” in the Globus Connect Personal menu.
4 Transfer Basics
Now that we have identified two endpoints, let’s transfer some files using Globus.
- On the Hazel side of the file manager, navigate to wherever you put your files for Workshops 1 and 2.
- Select the file(s) you want to transfer to your personal computer by clicking the box next to their name.
- On the Personal Collection side of the file manager, select a location where files can be recieved (You can always delete these files after this test).
- (OPTIONAL) Give your transfer a name or configure other transfer settings by clicking on
Transfer & Timer Optionsbetween the twoStartbuttons. - To initiate the transfer, click
Starton the side that contains the file(s) to transfer. - You can monitor on-going transfers from the
Activitytab. - Once the transfer is complete, you should recieve an email notification to your NC State email.
The Globus endpoints are bidirectional, meaning that you can initiate transfers and recieve files from either collection on the File Manager screen.
5 Future Directions
Globus is a powerful tool, and we only scratched the surface today. Any time you need to transfer large files safely and securly, you should ask yourself, “Can Globus be used for this?”
Other file-transfer applications that Globus could be used for:
- Transfer large files between
/sharesratch space and/rs1. - Transfer large files to and from a collaborator at another institution.
- Transfer to and from a cloud server or external storage not connected to Hazel.
- Set up automatic transfers triggered with Globus
Flows.
There is a Globus CLI that may be preferable to some users when developing more complex Globus workflows.