Managing Software and Code

Managing Software and Code#

The Secure Data Enclave (SDE) supports code development as part of your research workflow. The environment includes a range of approved libraries and package support, along with tools like Git available directly within the Virtual Machines (VM).

To meet security requirements, the SDE blocks access to external sites such as GitHub, online package repositories, and public APIs. As a result, code should be developed in a way that is self-contained, reproducible, and compatible with the resources available within the enclave.

This page explains how to request software and how to bring code into and out of the SDE.

Software and Packages#

Using pre-installed software or packages#

The Virtual Machines in the SDE are provisioned with approved Python and R packages, as well as Integrated Development Environments (IDEs), such as code editors, for coding. To learn more about the existing software in the SDE, see Available Software in the SDE.

Requesting coding packages#

The SDE is a controlled environment with no direct internet connectivity. As a user, software installation is restricted.

If you require additional R or Python libraries:

  1. Submit a request through the resource request process. This will generate a request ticket.

  2. Provide links to each package that you need installed in the request ticket.

  3. This request must be approved by the Information Security Office.

  4. Once approved, Northwestern IT will install the package within the VM.

Requesting Software#

For software installs(E.g. Stata or Matlab), the SDE requires extra review to confirm compatibility.

Requirements#

  1. The SDE VMs run on Linux/Unix (Ubuntu). Your software must be compatible with this operating system and able to run without internet access

  2. Check that your software works with your VM’s CPU, RAM, and storage configuration. You can check your VM’s configuration by following these steps.

  3. Once you’ve verified that your software can run, you can proceed with submitting an installation request.

Request Process#

  1. Submit a request through the resource request process. This will generate a request ticket.

  2. This request must be approved by the Information Security Office.

  3. After approval, download the installer file and give it to your SDE Data Engineer.

  4. Provide the installer file to the Data Engineer in your SDE.

  5. The Data Engineer should ingress the installer file and move it to a bucket in the project with the VM.

  6. If your software requires license keys or tokens, include them in a text file with the installer.

  7. Notify Northwestern IT in the ticket once all required files are available; IT will complete the installation.

Code#

All code in the SDE must be created inside the environment or added through the approved ingress process.

Bringing Code Into the SDE#

Code cannot be pulled into the SDE directly from external repositories (e.g., GitHub).

Bringing code from an existing Ingress Source#

If your code exists in a configured ingress source during the initial build, you can follow the ingress process to bring the code into the SDE.

Bringing code from a new Ingress Source#

  1. Submit an ingress request ticket

  2. This request must be approved by the Information Security Office.

  3. Northwestern IT will work with your Data Engineer to set up the new Ingress Source.

  4. Once the Ingress source is set up, the Data Engineer can bring the code into the SDE.

  5. The code can then be moved to the appropriate Bucket.

Working with Code Inside the SDE#

You must develop all code directly in your assigned VM. External code repositories (e.g., GitHub) are not accessible from the SDE.

  • You must build and modify your code locally within the VM.

  • Git is available for local version control and should be used to manage your repositories.

  • You can store your repositories on the VM or push them to your project bucket for persistence.

To prevent data loss, you should regularly push your code to a bucket. If your VM is deleted, any code stored in the bucket will remain available for recovery.

Exporting Code from the SDE#

To export code from the SDE, follow the approved egress process below:

  • Move your code to the designated Egress bucket in the Data Lake Project.

  • The Data Engineer must then submit an Egress Request Ticket.

  • The Data Engineer will transfer the code to the Data Egress Project for export.

  • The code can then be downloaded onto your managed endpoint

You can export code from the SDE only after it completes this process.