S3

Vocab

Bucket: A globally unique container for objects.
Object: A file and its metadata stored in a bucket.
Key: A unique identifier for an object within a bucket (effectively its “path”).

General Notes

Buckets: Flat storage (no true sub-buckets). Folder-like behavior is simulated using prefixes (e.g., user1/photos/img1.jpg).
Bucket Names: Must be globally unique because S3 uses a single DNS namespace to resolve bucket URLs (e.g., https://mybucket.s3.amazonaws.com), ensuring consistent routing across regions.
Regions: Buckets are region-bound for latency and cost optimization, but names remain globally unique.

Consistency Model

Data Consistency: Refers to how quickly changes (e.g., writes, updates) become visible.
S3 Consistency:
- Strong Consistency: After a successful write or overwrite, all reads reflect the latest version.
- Deleting or overwriting an object ensures immediate visibility of the change.

Access Control

Bucket Policies: JSON documents for bucket-level permissions (e.g., allow public read for specific paths).
Access Control Lists (ACLs): Provide object-level permissions; use sparingly as bucket policies are more scalable.
S3 Access Points: Simplify managing access for shared datasets with unique hostnames and policies.
Presigned URLs: Use a powerful role to temporarily grant its own access to objects without modifying permissions.

Design Patterns

Public and Private Access for a Web App

Public Images:

Store in "public/images/" and allow public access using a bucket policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::mybucket/public/*"
    }
  ]
}

User Images:

Store in "users/{userID}/images/" and restrict access using aws:username in a bucket policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::accountID:user/{username}"
      },
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::mybucket/users/{username}/*"
    }
  ]
}

Multi-Tenant Bucket

Use prefixes (e.g., "team1/data/") for shared datasets.
Manage access via prefixes in the bucket policy.

Bucket-Per-Use

Separate buckets for teams or projects to simplify cost tracking.
Limited to 10,000 buckets per account.

Pricing

add actual charges with links to their sites

Storage Classes:
- Standard: Default, low-latency storage.
- Infrequent Access: Cheaper for rarely accessed data.
- Glacier: Archival storage with retrieval delays.
Request Costs: Charged per operation (e.g., PUT, GET, LIST).
Data Transfer: Costs apply for outbound transfers.
Optimization: Use lifecycle rules to auto-archive unused objects.

Amplify’s Interface

Under the Amplify Hood

References

Amplify’s getURL implementation (for getting pre-signed urls)

Explorer

S3