Importing and Exporting Data#
Both importing data into (ingress) and exporting data from (egress) Nightingale must be done following the instructions below to maintain compliance with Northwestern and NIST SP 800-171 standards. Nightingale is set up to protect sensitive and regulated data, but you, as a user, are directly responsible for protecting the data you upload to, or have access to, on Nightingale.
Data Ingress#
The recommended data ingress tool for moving any data onto Nightingale is Globus. Researchers may also use the command scp to securely copy files from their Northwestern-owned and managed devices to Nightingale.
Globus#
Globus is a secure, high-performance data transfer service used by many research institutions which allows you to move data directly between approved storage collections.
To ingress data to Nightingale using Globus, you must first add your NCSA identity to your Globus account.
Visit Globus.org
Select “Log In” from the upper right side
Choose “National Center for Supercomputing Applications” from the dropdown
Log in with your NCSA credentials
Select “Settings” from the left-hand menu
Under the “Account” tab, click “Link Another Identity” on the upper right side
Choose “Northwestern University” from the dropdown
Log in with your Northwestern NetID and password
Once your Northwestern and NCSA identities are linked to your account, you can transfer data from Northwestern Globus collections to Nightingale.
To find the Nightingale collection:
Choose “Collections” from the left hand menu
Search “ncsa#ngale” and choose the resulting “Subscribed Mapped Collection”
Note that depending on when you last logged in to Globus, you may have to reauthenticate to view the collection. See the Northwestern Globus guide or the official Globus documentation for more information on using Globus.
Important!
While Globus allows transfers both to and from Nightingale, transferring any data off of Nightingale or between different project folders on Nightingale will trigger a security alert. Before moving any file off of or between different project folders on Nightingale, users must submit a support request so that Nightingale administrators know to expect the corresponding alert.
scp#
For files that are allowed on your local encrypted device, you may also use the command scp to securely copy files to Nightingale. Data subject to NIST SP 800-171 are not allowed on local devices; HIPAA data may be allowed depending on the management and security of your laptop. Please contact the IT department that manages your device with any questions.
The scp command takes a minimum of two parameters: the source location, and the destination location for your transfer. When the destination is remote, the syntax is similar to that of ssh where you provide your username and the domain name of the remote host.
Because there is a secure bastion host in front of the Nightingale login nodes, you must use the flag -J and provide both the bastion host and the log in node domain names in your command. For example, the command below copies a file called file.txt from the Documents folder on a user’s local computer up to their home directory on the log in node:
scp -J <username>@ngale-bastion-1.ncsa.illinois.edu Documents/file.txt <username>@ng-login01.ngale.internal.ncsa.edu:.
Please replace <username> with your NCSA username, and Documents/file.txt with the correct local path to whatever you are transferring if you copy the above command.
Like when using ssh, you will have to enter your NCSA password and use Duo authentication when using scp.
Data Egress#
Egressing data from Nightingale must follow these steps:
Consider compliance and deidentification needs before moving data off of Nightingale.
Submit a support request outlining details of your data transfer.
Transfer data as outlined in your support request, preferably using Globus.
See each section below for more information on each step.
Compliance and Deidentification#
If you are moving controlled or regulated data off of Nightingale, it must be to another system compliant with the relevant controls or regulations for that data. If you are moving data to a non-compliant system, the data must be deidentified or transformed such that the previous controls or regulations no longer apply.
For HIPAA-regulated data, all HIPAA identifiers, any other PII, or any other information restricted by the data use agreement (DUA) must be removed before any data can be exported from the Nightingale environment.
For data controlled by NIST SP 800-171, deidentification is not sufficient to decontrol these data. Please consult the specific DUA, or data use policy pertaining to your particular dataset to understand when and if any portion of the controlled-access data you are working with can leave the Nightingale environment.
Submitting a Support Request for Data Egress#
All data transfers on Nightingale are monitored and any data movement between projects or off of the system will trigger an alert if NCSA staff are not informed beforehand. Let NCSA staff know of any upcoming data transfers by submitting a support request via email to help@ncsa.illinois.edu with the following information:
A subject line that includes Nightingale
Where your data is currently located
The size of the data you are moving
Where the your data will be moved to
How you are doing the data transfer
What security controls are required for these data
When you will be transferring the data
Support requests should be sent at least one business day before any transfer is started.
As with ingress, the recommended data transfer tool for data egress is Globus.