Object Storage

Object Storage#

Object storage is designed for large volumes of unstructured data such as images, videos, research datasets, and backups. Object storage is commonly used for the following applications:

  • Sharing data with collaborators

  • Publishing datasets for free use to the research community

  • Archiving infrequently used data in lower-cost archival storage classes

Object storage services include Amazon S3 , Azure Blob Storage , and Google Cloud Storage .

These services store data as objects, each with its own identifier and metadata, rather than in a traditional folder structure. This simplifies managing large volumes of data and integrating with cloud workflows.

Storage Classes#

Cloud object storage offers several storage classes. Each one trades storage cost against how quickly and cheaply you can get your data back. The less often you access your data, the less it costs to store, but the more it costs to retrieve.

Class

Best for

Storage cost

Retrieval cost

Minimum storage period

Standard

Data you use regularly

Highest

None

None

Infrequent access

Data you use a few times a year

Lower

Per-GB fee

30 days

Archive

Data you keep but rarely or never use

Lowest

Highest per-GB fee

90–365 days

Each provider uses its own names for these classes:

Class

AWS

Azure

GCP

Standard

S3 Standard

Hot

Standard

Infrequent access

S3 Standard-IA

Cool

Nearline

Archive

S3 Glacier

Cold, Archive

Coldline, Archive

Lower-cost classes are only cheaper when your data stays put. If you delete or move data before the minimum storage period ends, you still pay for the full period. Retrieval fees add up quickly for data you access often, and on AWS and Azure, archived data can take hours to restore before you can use it. Keep active data in Standard and move data you must retain but rarely open to a lower-cost class.

Cost Components#

  • Storage amount: Charged per GB or TB stored each month; cost varies by storage class

  • Data retrieval: Archive tiers are inexpensive to store but incur fees and delays on retrieval

  • Egress fees: Downloading data out of the cloud typically incurs fees. Northwestern’s institutional agreements waive egress fees for most standard research data transfers. Exceptions may include transfers to third-party services outside the provider’s network or unusually high-volume transfers. If you are planning to move large datasets out of the cloud, confirm coverage with the RCDS Team via a Research Cloud Consult before transferring.

  • API requests: Uploading, downloading, listing, and deleting files generate small per-request charges that can add up with heavy usage

Cost Calculator#

Each provider offers a pricing calculator to estimate object storage costs before you store data. The following example configures a standard storage tier with no archival or heavy API usage. Set each field to the value shown; leave any field not listed at its default.

  1. Go to the AWS Pricing Calculator , click Create estimate, then search for and select Amazon S3.

  2. Under Select S3 Storage classes and other features, confirm S3 Standard is selected, then set the following fields:

    Field

    Value

    Choose a Region

    US East (Ohio)

    S3 Standard storage amount

    Your expected storage volume (GB per month)

    How will data be moved into S3 Standard?

    Select the option that matches how you plan to upload data

  3. Optional: to include archival storage, add S3 Standard-Infrequent Access or S3 Glacier under the same panel and set its storage amount.

  4. Optional: to estimate API request costs, expand PUT, COPY, POST, LIST requests to S3 Standard and GET, SELECT, and all other requests from S3 Standard and enter your expected monthly request counts.

  5. Click Save and view summary to review your estimate.

  1. Go to the Azure Pricing Calculator , find Storage Accounts under Products, and click Add to estimate.

  2. Set the following fields:

    Field

    Value

    Region

    North Central US

    Type

    Block Blob Storage

    Performance

    Standard

    Storage Account Type

    General Purpose V2

    Access tier

    Hot

    Redundancy

    LRS

    Capacity

    Your expected storage volume (GB)

  3. Optional: to include archival storage, change Access tier to Cool, Cold, or Archive for a separate estimate line.

  4. Optional: under Operations and Data Transfer, enter your expected Write Operations and Read operations if your workload involves frequent API calls.

  5. Review your estimate in the Your Estimate panel.

  1. Go to the Google Cloud Pricing Calculator , click Add to estimate, and select Cloud Storage.

  2. Set the following fields:

    Field

    Value

    Location type

    Region

    Location

    Iowa (us-central1)

    Storage class

    Standard Storage

    Total amount of storage

    Your expected storage volume (GiB)

  3. Optional: to include archival storage, change Storage class to Nearline, Coldline, or Archive for a separate estimate line.

  4. Optional: toggle Advanced settings to reveal “Class A” operations per month and “Class B” operations per month if your workload involves frequent API calls.

  5. Review your estimated cost in the Cost details panel.

Round your estimate up and account for growth over your project’s timeline. If your estimate seems unexpectedly high or you are unsure how to configure it, request a Research Cloud Consult .