PreviewLab6 hours

Topics & Fan-out

Publish to SNS topics, configure subscriptions and filter policies, and build the SNS → SQS fan-out pattern.

Prerequisites

0 of 5 checked

Topic Basics: Create, Subscribe, Publish

Goal

Create a standard SNS topic. Subscribe an email endpoint. Publish a message and observe delivery. Understand the publish-subscribe model and what SNS guarantees.

Estimated time: 1–2 hours

What's happening here

SNS is a push-based pub/sub system: you publish once to a topic and SNS fans out to all subscribers simultaneously. This is the fundamental difference from SQS (pull-based, one consumer per message). A topic is the logical channel: producers publish to it, subscribers receive from it. DisplayName appears as the SMS sender name and email subject prefix. SNS topics are regional, a topic in us-east-1 cannot directly deliver to subscribers registered in a different region.

bash

mkdir sns-lab && cd sns-lab

export TOPIC_ARN=$(aws sns create-topic \
  --name sns-lab-standard \
  --attributes DisplayName=SNSLab \
  --region $AWS_REGION \
  --query 'TopicArn' --output text)

echo "Topic ARN: $TOPIC_ARN"

Verify:

bash

aws sns get-topic-attributes \
  --topic-arn $TOPIC_ARN \
  --region $AWS_REGION \
  --query 'Attributes.{DisplayName:DisplayName,SubscriptionsConfirmed:SubscriptionsConfirmed,SubscriptionsPending:SubscriptionsPending}'

Checkpoint: topic exists, SubscriptionsConfirmed: 0.

What's happening here

SNS subscriptions require confirmation for email, HTTP/HTTPS, and SMS endpoints, a protection against subscribing someone else's address without consent. After subscribe, SNS sends a confirmation email with a unique URL. Until you click it, the subscription is PendingConfirmation and no messages are delivered. SQS, Lambda, and Firehose subscriptions are confirmed automatically, no human action needed. The SubscriptionArn returned before confirmation is pending confirmation: not a usable ARN.

bash

# Replace with your actual email
export MY_EMAIL="your@email.com"

export SUB_ARN=$(aws sns subscribe \
  --topic-arn $TOPIC_ARN \
  --protocol email \
  --notification-endpoint $MY_EMAIL \
  --region $AWS_REGION \
  --query 'SubscriptionArn' --output text)

echo "Subscription ARN: $SUB_ARN"
# Will show "pending confirmation" until you confirm

Check pending status:

bash

aws sns list-subscriptions-by-topic \
  --topic-arn $TOPIC_ARN \
  --region $AWS_REGION \
  --query 'Subscriptions[*].{Protocol:Protocol,Endpoint:Endpoint,SubscriptionArn:SubscriptionArn}'

Now check your inbox and click the confirmation link. Then re-run the list command, SubscriptionArn changes from pending confirmation to a real ARN.

bash

# After confirming, export the real ARN
export SUB_ARN=$(aws sns list-subscriptions-by-topic \
  --topic-arn $TOPIC_ARN \
  --region $AWS_REGION \
  --query 'Subscriptions[0].SubscriptionArn' --output text)

echo "Confirmed subscription ARN: $SUB_ARN"

Verify confirmed count:

bash

aws sns get-topic-attributes \
  --topic-arn $TOPIC_ARN \
  --region $AWS_REGION \
  --query 'Attributes.{Confirmed:SubscriptionsConfirmed,Pending:SubscriptionsPending}'

Checkpoint: SubscriptionsConfirmed: 1, SubscriptionsPending: 0.

What's happening here

publish is a fire-and-forget API, SNS accepts the message, returns a MessageId, and delivers asynchronously to all confirmed subscribers. You don't poll for delivery status (unlike SQS where you poll to receive). The MessageId is SNS's internal ID for the publish event, not per-subscriber. If one subscriber's endpoint is down, SNS retries for that subscriber independently without affecting others. Subject appears as the email subject line; it's ignored for SQS, Lambda, and HTTP subscribers.

bash

# Publish a plain message
aws sns publish \
  --topic-arn $TOPIC_ARN \
  --message 'Hello from SNS lab' \
  --subject 'SNS Lab Test' \
  --region $AWS_REGION

# Publish a JSON payload
aws sns publish \
  --topic-arn $TOPIC_ARN \
  --message '{"event": "invoice.created", "id": "inv-001", "amount": 9900}' \
  --subject 'Invoice Event' \
  --region $AWS_REGION

Check your inbox. You'll receive two emails. Observe the envelope: the message body is wrapped in an SNS JSON envelope, Type, MessageId, TopicArn, Subject, Message, Timestamp, Signature. Your actual payload is inside Message as a string, even if you sent JSON. This is the raw vs structured distinction that matters when SQS is a subscriber (covered in Fan-Out: SNS → Multiple SQS Queues).

Publish with message attributes (metadata on the publish event):

bash

aws sns publish \
  --topic-arn $TOPIC_ARN \
  --message '{"event": "payment.failed", "id": "pay-002"}' \
  --subject 'Payment Event' \
  --message-attributes \
    'source={DataType=String,StringValue=billing-service}' \
    'priority={DataType=Number,StringValue=1}' \
  --region $AWS_REGION

Checkpoint: emails received with SNS envelope structure visible.

What's happening here

publish-batch sends up to 10 messages in one API call, same cost model as SQS batch: charged as one request. Each entry has its own Id (your client-side correlation key), Message, and optional Subject and MessageAttributes. Failures are per-entry, check Failed in the response. A batch partial failure does not roll back successful entries. Unlike SQS batch, there is no MessageGroupId on standard SNS topics (FIFO SNS, covered later, has it).

bash

aws sns publish-batch \
  --topic-arn $TOPIC_ARN \
  --publish-batch-request-entries \
    'Id=e1,Message={"id":"inv-002","amount":5000},Subject=Invoice' \
    'Id=e2,Message={"id":"inv-003","amount":12000},Subject=Invoice' \
    'Id=e3,Message={"id":"inv-004","amount":7500},Subject=Invoice' \
  --region $AWS_REGION | jq '{Successful: [.Successful[].Id], Failed: [.Failed[].Id]}'

Checkpoint: Successful has 3 entries, Failed is empty.

Break 1: Message too large

bash

# SNS max message size is 256 KB
python3 -c "print('x' * 300000)" > big_payload.txt

aws sns publish \
  --topic-arn $TOPIC_ARN \
  --message file://big_payload.txt \
  --region $AWS_REGION

InvalidParameter, Message too long. Same limit as SQS. Same fix: store payload in S3, publish only the S3 key. This is the SNS Extended Client pattern (also available as an AWS library).

Break 2: Publish to a topic that doesn't exist

bash

aws sns publish \
  --topic-arn "arn:aws:sns:$AWS_REGION:$AWS_ACCOUNT:does-not-exist" \
  --message 'test' \
  --region $AWS_REGION

NotFound, Topic does not exist. SNS fails synchronously at publish time if the topic ARN is invalid. Your producer must handle this, it is not retried by SNS.

Topics & Fan-out

Prerequisites

Topic Basics: Create, Subscribe, Publish

Create your first SNS topic

Verify:

Subscribe an email endpoint

Check pending status:

Publish your first message

Publish with message attributes (metadata on the publish event):

Publish batch

Break It: Topic Basics: Create, Subscribe, Publish

Break 1: Message too large

Break 2: Publish to a topic that doesn't exist

Fan-Out: SNS → Multiple SQS Queues