PreviewLab8 hours

Buckets & Objects

Create buckets, manage objects and versioning, configure access control, multipart uploads, storage classes, and observability.

Prerequisites

0 of 5 checked

Bucket Basics: Create, Upload, Download, Delete

Goal

Create a bucket from scratch. Upload objects, retrieve them, understand the flat key namespace, and observe what happens when you try to delete a non-empty bucket.

Estimated time: 1–2 hours

Create your first bucket

What's happening here

S3 bucket names are globally unique across all AWS accounts, if someone else has my-bucket: you can't create it. Names must be 3–63 characters, lowercase letters, numbers, and hyphens only; no underscores. us-east-1 is the only region where you omit --create-bucket-configuration (every other region requires it. S3 has no concept of folders) what looks like a folder path (invoices/2024/jan/) is just a key prefix. The / is a regular character. Everything is a flat key-value store.

bash

mkdir s3-lab && cd s3-lab

export BUCKET="${LAB_PREFIX}-main"

# Create in us-east-1 (no LocationConstraint needed)
aws s3api create-bucket \
  --bucket $BUCKET \
  --region $AWS_REGION

echo "Bucket: $BUCKET"

# Verify
aws s3api head-bucket --bucket $BUCKET --region $AWS_REGION && echo "Bucket exists and accessible"

Check bucket location and defaults:

bash

aws s3api get-bucket-location --bucket $BUCKET
aws s3api get-bucket-versioning --bucket $BUCKET
aws s3api get-bucket-encryption --bucket $BUCKET 2>&1 || echo "No encryption config yet"

versioning is empty (disabled by default). Encryption: since April 2023, all new S3 buckets have SSE-S3 encryption on by default: AWS manages the keys, zero config needed.

Checkpoint: bucket exists, region correct.

Upload objects

What's happening here

put-object uploads a single object. The key is the full name including any prefix. S3 does not create "folders", but the AWS console and aws s3 ls use / delimiters to simulate them visually. Content-Type is metadata stored alongside the object; S3 doesn't infer it from the file extension. ETag in the response is an MD5 of the object body for single-part uploads, you can verify integrity client-side. aws s3 sync is a higher-level command that computes diffs by size/last-modified and uploads only what changed.

bash

# Create test files
echo 'Hello from S3 lab' > hello.txt
echo '{"event": "invoice.created", "id": "inv-001", "amount": 9900}' > invoice.json
dd if=/dev/urandom bs=1M count=5 | base64 > medium-file.txt 2>/dev/null

# Upload a single object
aws s3api put-object \
  --bucket $BUCKET \
  --key hello.txt \
  --body hello.txt \
  --content-type text/plain \
  --region $AWS_REGION

# Upload with a prefix (simulated folder structure)
aws s3api put-object \
  --bucket $BUCKET \
  --key invoices/2024/inv-001.json \
  --body invoice.json \
  --content-type application/json \
  --metadata 'client-id=cust-A,source=billing-service' \
  --region $AWS_REGION

# Upload multiple files with sync
aws s3 sync . s3://$BUCKET/uploads/ \
  --exclude '*.py' \
  --region $AWS_REGION

echo "Upload complete"

Check what's in the bucket:

bash

# High-level list
aws s3 ls s3://$BUCKET/ --recursive --human-readable

# Low-level list with metadata
aws s3api list-objects-v2 \
  --bucket $BUCKET \
  --region $AWS_REGION \
  --query 'Contents[*].{Key:Key,Size:Size,LastModified:LastModified}' \
  --output table

Checkpoint: at least 3 objects visible in bucket.

Retrieve objects

What's happening here

get-object streams the object body to a local file. head-object fetches only the metadata (key, size, content-type, ETag, custom metadata) without downloading the body, use this to check object existence and metadata cheaply. presign generates a time-limited URL that grants access to a private object without requiring the caller to have AWS credentials. The URL is signed with your credentials but can be opened by anyone, don't leak it.

bash

# Download an object
aws s3api get-object \
  --bucket $BUCKET \
  --key hello.txt \
  --region $AWS_REGION \
  downloaded-hello.txt

cat downloaded-hello.txt

# Head an object (metadata only, no download)
aws s3api head-object \
  --bucket $BUCKET \
  --key invoices/2024/inv-001.json \
  --region $AWS_REGION | jq '{ContentType, ContentLength, ETag, Metadata}'

# Generate a 10-minute presigned URL
aws s3 presign s3://$BUCKET/hello.txt \
  --expires-in 600 \
  --region $AWS_REGION

the presigned URL contains X-Amz-Signature, X-Amz-Credential, and X-Amz-Expires query parameters. Paste it in a browser, the file downloads without any AWS auth. After 600 seconds, curl on the URL returns AccessDenied.

List with prefix and delimiter (folder simulation)

What's happening here

S3's list-objects-v2 with a delimiter=/ returns two types of results: Contents (objects at this level) and CommonPrefixes ("folders", key segments up to the next /). This is how the console simulates a tree. Without delimiter, list-objects-v2 returns everything flat. Pagination: S3 returns max 1000 objects per call. For buckets with millions of objects, use --page-size and --starting-token (or the SDK paginator).

bash

# List top-level "folders" only
aws s3api list-objects-v2 \
  --bucket $BUCKET \
  --delimiter '/' \
  --region $AWS_REGION | jq '{CommonPrefixes, Contents: [.Contents[]?.Key]}'

# List within a specific prefix
aws s3api list-objects-v2 \
  --bucket $BUCKET \
  --prefix 'invoices/' \
  --delimiter '/' \
  --region $AWS_REGION | jq '{CommonPrefixes, Contents: [.Contents[]?.Key]}'

# Full flat list
aws s3api list-objects-v2 \
  --bucket $BUCKET \
  --prefix 'invoices/' \
  --region $AWS_REGION \
  --query 'Contents[*].Key' --output json

Delete objects and the non-empty bucket trap

What's happening here

S3 returns 204 No Content for delete-object even if the key doesn't exist: it's idempotent by design. delete-bucket fails with BucketNotEmpty if objects remain. There's no atomic "delete bucket and all contents" in the API, you must drain it first. s3 rb --force is the CLI shorthand that does this in two steps: delete all objects, then delete the bucket.

bash

# Delete a single object
aws s3api delete-object \
  --bucket $BUCKET \
  --key hello.txt \
  --region $AWS_REGION

# Delete a key that doesn't exist, succeeds silently (idempotent)
aws s3api delete-object \
  --bucket $BUCKET \
  --key does-not-exist.txt \
  --region $AWS_REGION && echo "No error. S3 delete is idempotent"

# Try to delete non-empty bucket, fails
aws s3api delete-bucket --bucket $BUCKET --region $AWS_REGION 2>&1 | head -5

BucketNotEmpty. The bucket still has objects. Must empty it first.

bash

# Batch delete everything
aws s3 rm s3://$BUCKET/ --recursive --region $AWS_REGION

# Now delete the bucket
aws s3api delete-bucket --bucket $BUCKET --region $AWS_REGION
echo "Bucket deleted"

# Recreate for subsequent sessions
aws s3api create-bucket \
  --bucket $BUCKET \
  --region $AWS_REGION

# Re-upload a few objects
aws s3api put-object --bucket $BUCKET --key hello.txt --body hello.txt --region $AWS_REGION
aws s3api put-object --bucket $BUCKET --key invoices/2024/inv-001.json --body invoice.json --region $AWS_REGION

Checkpoint: bucket recreated with 2 objects.

Break It: Bucket Basics: Create, Upload, Download, Delete

Break 1: Bucket name with uppercase

bash

aws s3api create-bucket --bucket MY-UPPERCASE-BUCKET --region $AWS_REGION 2>&1

InvalidBucketName. Bucket names must be lowercase.

Break 2: Overwrite without version protection

bash

# Upload v1
echo 'version 1' > overwrite-test.txt
aws s3api put-object --bucket $BUCKET --key overwrite-test.txt --body overwrite-test.txt --region $AWS_REGION

# Overwrite with v2, no warning, v1 is gone
echo 'version 2, v1 is permanently gone' > overwrite-test.txt
aws s3api put-object --bucket $BUCKET --key overwrite-test.txt --body overwrite-test.txt --region $AWS_REGION

aws s3api get-object --bucket $BUCKET --key overwrite-test.txt --region $AWS_REGION /dev/stdout 2>/dev/null

you can only see v2. v1 is irretrievably gone without versioning enabled. This is why Versioning + Lifecycle Policies exists.

bash

aws s3api delete-object --bucket $BUCKET --key overwrite-test.txt --region $AWS_REGION

Versioning + Lifecycle Policies

Goal

Enable versioning. Observe how overwrites and deletes work when versioning is on. Create lifecycle rules to expire old versions automatically. Understand delete markers.