Kinesis · Data Streams

Outcome

• Streams & ShardsStream Basics: Create, Put, Get
  • Create a Kinesis stream in ON_DEMAND mode and explain the shard capacity model
  • Put records with partition keys and explain what partition key controls
  • Read records using the shard iterator model: TRIM_HORIZON, LATEST, AT_SEQUENCE_NUMBER
  • Explain why get-records does not remove records from the stream
  • Replay the full stream from a second independent consumer
  • Handle ExpiredIteratorException and explain when it occurs
• Streams & ShardsShard Mechanics: Ordering, Retention, Resharding
  • Explain ordering guarantees within a shard vs across shards
  • Choose a partition key for a given ordering requirement
  • Extend and reduce stream retention and explain the data loss risk
  • Split a shard using update-shard-count and identify parent/child shard relationship
  • Explain what a hot shard is and how to detect and fix it
• Streams & ShardsLambda Event Source Mapping
  • Explain what Lambda ESM does: polling, checkpointing, retry behavior
  • Configure BisectBatchOnFunctionError and explain what it prevents
  • Set MaximumRetryAttempts and attach a DLQ on an ESM
  • Trace a poison pill from ingestion through bisect retries to DLQ
  • Explain TRIM_HORIZON vs LATEST starting position and when to use each
• Streams & ShardsEnhanced Fan-Out
  • Explain the throughput difference between standard consumers and EFO
  • Register an EFO consumer and use subscribe-to-shard
  • Explain the 20-consumer limit and when EFO is worth the cost
  • Compare polling latency (standard) vs push latency (EFO)
• Streams & ShardsKCL (Kinesis Client Library)
  • Explain what KCL's DynamoDB lease table contains and why it prevents double-processing
  • Write a processor with initialize, process_records, shard_ended, and lease_lost callbacks
  • Checkpoint at the correct granularity and explain the checkpoint/at-least-once tradeoff
  • Observe lease rebalancing when a second worker joins
  • Trace crash recovery from checkpoint through DynamoDB to resumed processing
• Streams & ShardsObservability: CloudWatch Metrics + Alarms
  • Name the five key Kinesis CloudWatch metrics and what each indicates
  • Explain why IteratorAgeMilliseconds is more actionable than record count
  • Create iterator age, write throttle, and read throttle alarms with correct thresholds
  • Use the consumer-stopped vs consumer-slow diagnostic table to identify the right fix
  • Trigger the iterator age alarm by stopping the consumer and verify it clears on restart