Architecture note5 lab guides5 lab guides

S3 · Simple Storage Service

Four layers can deny you, read them in order when debugging a 403.

Outcome

Buckets & Objects Bucket Basics: Create, Upload, Download, Delete
- Create a bucket with the correct region and naming rules
- Upload objects with content-type and metadata; use sync for directories
- Explain why S3 has no real folders, keys with / delimiters only
- Use list-objects-v2 with prefix and delimiter to simulate folder navigation
- Delete a specific object and explain why delete-object succeeds on non-existent keys
- Explain the BucketNotEmpty error and how to drain a bucket before deletion
Buckets & Objects Versioning + Lifecycle Policies
- Enable versioning and explain the three states (Unversioned, Enabled, Suspended)
- Explain that versioning cannot be fully disabled once enabled
- Retrieve a specific version of an object by VersionId
- Explain delete markers: what they are, how to create one, how to undelete
- Create lifecycle rules for noncurrent version expiry and incomplete MPU cleanup
Buckets & Objects Access Control: Bucket Policies + Pre-Signed URLs + Public Block
- Explain the four block public access flags and why they act as a master override
- Write a bucket policy that enforces SSL-only access using a Deny on aws:SecureTransport
- Write a prefix-scoped read policy and explain the interaction with IAM policies
- Generate presigned GET and PUT URLs and explain when each is used
- Explain why revoking the signing role immediately invalidates all presigned URLs
Buckets & Objects Multipart Upload + Large Objects + Storage Classes
- Initiate, upload parts, and complete a multipart upload manually
- Explain why the ETag of an MPU object ends in -N
- Identify abandoned MPU sessions via list-multipart-uploads and abort them
- Name the six storage classes and their retrieval characteristics
- Move an object between storage classes without re-uploading via copy-object
- Explain why GLACIER_FLEXIBLE objects cannot be downloaded without restore-object
Buckets & Objects Observability: S3 Metrics + Access Logging + CloudWatch Alarms
- Enable server access logging and name what each log field contains
- Explain why per-request metrics (4xx/5xx) must be explicitly enabled and cost extra
- Create a 4xx error alarm with correct period, threshold, and treatMissingData
- Create a storage size alarm using BucketSizeBytes as a daily metric
- Explain the difference between daily storage metrics and per-minute request metrics
- Diagnose a 4xx spike from access logs and describe what to check first
Bucket Types Lab General Purpose Buckets: Recap + Advanced Patterns
- Explain the three-AZ durability model and why GP is the safe default
- Apply object tags for cost allocation and explain how they differ from metadata
- Write a lifecycle rule that targets objects by tag + prefix combination
- Set up cross-region replication: versioning on both buckets, IAM role, replication config
- Explain why objects uploaded before the replication rule are not replicated automatically
- List at least 4 GP-only features that don't exist on Directory or Table buckets
Bucket Types Lab Directory Buckets (S3 Express One Zone)
- Explain what S3 Express One Zone is and the single-AZ availability trade-off
- Construct a valid Directory bucket name given a region and AZ ID
- Explain what create-session does and why per-request signing is avoided
- Demonstrate that put/get/list work identically to GP at the API surface
- Confirm that versioning, tagging, replication, and storage class selection all fail on Directory buckets
- Explain when you would choose a Directory bucket over a General Purpose bucket
Bucket Types Lab Table Buckets (S3 Tables)
- Create a Table bucket, namespace, and Iceberg table using the s3tables CLI
- Explain the hierarchy: Table bucket → namespace → table → Iceberg metadata → data files
- Read and interpret the Iceberg metadata JSON (format-version, schema, snapshot)
- Explain how S3 Tables handles compaction and snapshot cleanup vs self-managed Iceberg
- Confirm that s3api commands don't work on Table buckets and explain why
- Describe how Athena connects to a Table bucket (via Glue catalog / Iceberg REST catalog)
Vector Buckets Lab Vector Bucket + Index: Create, Inspect, Understand the Config Contract
- Create a vector bucket and explain how its ARN and IAM namespace differ from GP buckets
- Create a vector index with the correct dimension, distance metric, and non-filterable metadata keys
- Explain why dimension, distance metric, and non-filterable keys are immutable after creation
- Choose between cosine and euclidean based on the embedding model's recommendation
- Explain why zero vectors are rejected for cosine distance
Vector Buckets Lab Writing Vectors: Put, Get, List, Delete
- Insert a batch of vectors with typed metadata using put-vectors from a JSON file
- Explain what put-vectors does when the key already exists (upsert semantics)
- Retrieve specific vectors by key using get-vectors with --return-data and --return-metadata
- Perform a segmented parallel scan with list-vectors --segment-count N --segment-index i
- Delete vectors by key and confirm delete-vectors is idempotent on non-existent keys
Vector Buckets Lab Similarity Search: query-vectors, Distance Metrics, Metadata Filtering
- Run a query-vectors call with top-k, --return-distance, and --return-metadata
- Explain why the query vector must come from the same embedding model as the stored vectors
- Apply equality, comparison, $and, and $or metadata filters in a query
- Explain the difference between filtering at query time vs post-filtering
- Confirm that querying on a non-filterable key returns a 400 error
- Demonstrate strong consistency by querying immediately after a put-vectors call
- Explain what happens when top-k exceeds the number of vectors in the index
Vector Buckets Lab IAM, Bucket Policies, Encryption, Tagging
- List the full set of s3vectors:* IAM actions and group them by role (reader, writer, admin)
- Explain why s3vectors:GetVectors is needed in addition to s3vectors:QueryVectors for metadata filtering
- Write a least-privilege IAM policy scoped to a single index ARN
- Apply a vector bucket resource policy and explain its use cases vs IAM identity policies
- Tag a vector bucket and index ARN using tag-resource
- Demonstrate that s3api commands are completely blind to vector buckets
File Systems Lab Mountpoint for S3: Install, Mount, Understand the POSIX Boundary
- Install mount-s3 and mount a General Purpose bucket with --allow-delete and --allow-overwrite
- Explain that FUSE translates file system calls to S3 API calls (ListObjectsV2, GetObject, PutObject)
- Enable the read cache with --cache and explain when it helps vs when it doesn't
- Mount a key prefix with --prefix and confirm objects outside the prefix are invisible
- Mount a Directory bucket with identical syntax and explain why Express latency is lower
- Demonstrate the four POSIX operations that fail (rename, append, symlink, hard link) and explain why each fails at the S3 object model level
- Explain when you'd choose Mountpoint over S3 Files (throughput-optimised reads, lower cost, no POSIX requirement)
File Systems Lab S3 Files: Create, Mount, Full POSIX, Bidirectional Sync
- Explain the two IAM roles required: sync role (assumed by S3 Files service) and compute role (on EC2)
- Create a file system with aws s3files create-file-system linked to a bucket ARN and sync role ARN
- Create a mount target in the correct subnet and wait for it to reach available state
- Mount with sudo mount -t s3files -o mounttargetip=... fs-id:/ /mnt/s3files
- Perform all six POSIX operations that Mountpoint cannot: append, rename, mkdir, symlink, flock, in-place edit
- Demonstrate export sync (file system → S3) within ~60 seconds
- Demonstrate import sync (S3 → file system) triggered by EventBridge within ~15 seconds
- Create an access point scoped to a prefix with a fixed POSIX UID/GID
- Explain the three edge cases: sync window durability, S3-side deletes propagation, 255-char filename limit
File Systems Lab Mountpoint vs S3 Files: Side-by-Side Decision Guide
- Name three workloads for Mountpoint and three for S3 Files
- Explain why S3 Files is faster for repeated small-file reads (high-performance cache) while Mountpoint is faster for large sequential reads (CRT parallelism)
- Explain when to use FSx for Lustre instead of either