Free previewArchitecture note14 min read

EC2 as the Foundation

EC2 is not just a service you use directly. It is the compute substrate that almost every AWS managed service runs on underneath. Understanding this changes how you reason about performance, failure modes, scaling limits, and cost across the entire AWS ecosystem.

The Core Idea

When AWS says a service is "managed", it almost always means:

AWS runs EC2 instances on your behalf, handles the OS patching, the daemon restarts, the disk management, the cluster coordination, and exposes a clean API so you never have to touch any of it.

The EC2 instances are still there. You just don't see them. But their properties (AZ placement, instance type, network throughput, storage IOPS) directly determine the behavior of the service you're using.

javascript

What you see:         RDS endpoint / ElastiCache endpoint / EKS cluster
What actually runs:   EC2 instances in AWS-managed VPC subnets
What still applies:   AZ constraints, instance type limits, EBS/network throughput

Service by Service

RDS (Relational Database Service)

Note

RDS runs your database (PostgreSQL, MySQL, Aurora, etc.) on EC2 instances inside AWS-managed subnets. You pick the instance class (db.t3.medium, db.r6g.xlarge: etc.), this is literally an EC2 instance type with a db. prefix. The database engine runs on that instance. Your data lives on EBS volumes attached to it. What this means practically:

Instance type = your database's CPU and RAM ceiling. A db.t3.medium has 2 vCPU and 4GB RAM. If your query needs more, it waits or OOMs. No amount of RDS configuration fixes an undersized instance.
Storage = EBS. RDS uses gp3 or io1 EBS volumes. IOPS limits on EBS are real, if your workload exceeds provisioned IOPS, queries queue up. This is the most common cause of "RDS is slow" that isn't the query itself.
Multi-AZ = two EC2 instances in different AZs. The standby is a hot replica on a separate instance. Failover = DNS cutover to the standby. Takes 60–120 seconds. The underlying EC2 health check triggers it.
Read replicas = additional EC2 instances with replication lag. Each replica is a separate instance with its own IOPS budget.

javascript

RDS Multi-AZ layout:

AZ-1                          AZ-2
┌─────────────────┐           ┌─────────────────┐
│ db.r6g.xlarge   │──sync────▶│ db.r6g.xlarge   │
│ Primary         │  repl.    │ Standby         │
│ gp3: 3000 IOPS  │           │ gp3: 3000 IOPS  │
└─────────────────┘           └─────────────────┘
         │
         ▼
  RDS endpoint (DNS)
  → points to primary
  → flips to standby on failover

What breaks when you forget this:

Choosing db.t3.micro for a production database because "it's just a managed service", you'll hit CPU credit exhaustion under load (t-series = burstable, same as EC2 t-series)
Not provisioning enough IOPS on the EBS volume, queries slow down under write-heavy workloads
Expecting instant failover. Multi-AZ failover takes 60–120s because it's an EC2 instance being promoted, not a process restart

ElastiCache (Redis / Memcached)

Note

ElastiCache runs Redis or Memcached on EC2 instances. You pick the node type (cache.r6g.large, cache.t3.micro: etc.), again, EC2 instance types with a cache. prefix. The critical thing to understand: Redis is single-threaded for command processing. This means vertical scaling (bigger instance = more RAM and network throughput) matters more than horizontal scaling for most Redis workloads. You're fundamentally constrained by one EC2 instance's network bandwidth and memory. ElastiCache cluster mode spreads data across multiple nodes (sharding), but each shard is still a single EC2 instance as the primary.

javascript

ElastiCache Redis Cluster Mode:

Shard 1              Shard 2              Shard 3
┌──────────┐         ┌──────────┐         ┌──────────┐
│ Primary  │         │ Primary  │         │ Primary  │
│cache.r6g │         │cache.r6g │         │cache.r6g │
└──────────┘         └──────────┘         └──────────┘
     │                    │                    │
┌──────────┐         ┌──────────┐         ┌──────────┐
│ Replica  │         │ Replica  │         │ Replica  │
│ (AZ-2)   │         │ (AZ-2)   │         │ (AZ-2)   │
└──────────┘         └──────────┘         └──────────┘

Each box = one EC2 instance
Each shard = one primary EC2 + N replica EC2s

What breaks when you forget this:

Picking cache.t3.micro for a high-throughput cache, network bandwidth on small instances is throttled, commands queue
Expecting Redis to handle more than ~100k ops/sec on a single node, you're hitting EC2 network limits, not Redis limits
Treating ElastiCache failover as instant, it's an EC2 promotion, similar to RDS, takes 30–60s

EKS (Elastic Kubernetes Service)

Note

EKS is the most transparent about EC2. Kubernetes worker nodes are just EC2 instances in your account, visible in your EC2 console. You manage them (or use managed node groups / Fargate). What AWS manages: the control plane (etcd, API server, scheduler, controller manager), these run on EC2 in AWS-managed accounts. You never see them. What you manage: the worker nodes: EC2 instances that your pods run on. Node groups are essentially ASGs: min/max/desired, launch templates, instance types. This means everything from the EC2 lab applies directly to EKS worker nodes:

Security groups on nodes control pod network access
IAM instance profiles on nodes give pods their AWS permissions (via IRSA or node-level profile)
EBS volumes attach to nodes for persistent volumes
EFS mounts on nodes for shared persistent volumes across pods

javascript

EKS architecture:

AWS-managed (you don't see this)
┌────────────────────────────────┐
│ Control Plane                  │
│ API Server / etcd / Scheduler  │
│ (EC2 in AWS account)           │
└────────────────────────────────┘
            │ kubectl / API
            ▼
Your account (you manage this)
┌──────────────────────────────────────┐
│ Worker Node Group (ASG)              │
│ ┌──────────┐  ┌──────────┐           │
│ │ m5.large │  │ m5.large │  ...      │
│ │ EC2      │  │ EC2      │           │
│ │ Pod Pod  │  │ Pod Pod  │           │
│ └──────────┘  └──────────┘           │
└──────────────────────────────────────┘

What this means practically:

Pod scheduling failures often come down to EC2 instance limits, not enough CPU/memory on the nodes, or too many pods per node (EC2 instance type limits the number of ENIs, which limits pods per node on VPC CNI)
Node group scaling = ASG scaling = same lag as EC2 autoscaling (~2–3 min to launch a new node)
Fargate for EKS removes the node management, each pod gets its own micro-VM (still EC2 underneath, but AWS manages it)

OpenSearch (formerly Elasticsearch)

Note

OpenSearch runs your search/analytics cluster on EC2 instances. You pick instance types (r6g.large.search, m6g.xlarge.search). EC2 instances with a .search suffix. OpenSearch is particularly sensitive to EC2 instance properties because:

JVM heap = half of instance RAM. A r6g.large has 16GB RAM → 8GB heap. Heap exhaustion = OOM = node crash = cluster instability.
Storage = EBS or instance store. UltraWarm nodes use S3-backed storage (cheaper but slower). Hot nodes use EBS. Instance store nodes (i-series) have the fastest IOPS but no persistence.

Shard allocation maps directly to EC2 nodes, a shard is a Lucene index that lives on one node's disk.

javascript

OpenSearch cluster layout:

┌─────────────────────────────────────────┐
│ Domain                                  │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐  │
│ │r6g.large │ │r6g.large │ │r6g.large │  │
│ │ Node 1   │ │ Node 2   │ │ Node 3   │  │
│ │Shard 0P  │ │Shard 1P  │ │Shard 2P  │  │
│ │Shard 1R  │ │Shard 2R  │ │Shard 0R  │  │
│ └──────────┘ └──────────┘ └──────────┘  │
│   (AZ-1)      (AZ-2)       (AZ-3)       │
└─────────────────────────────────────────┘

P = primary shard  R = replica shard
Each node = one EC2 instance

What breaks when you forget this:

Undersizing instance RAM → JVM heap too small → frequent GC pauses → slow queries
Single-AZ cluster → one AZ goes down → cluster unavailable (no replicas in other AZs)
Choosing too few shards for the data volume → shards too large → can't rebalance across nodes

EMR (Elastic MapReduce)

Note

EMR is the most explicit about EC2, you literally pick EC2 instance types for master, core, and task nodes. The Hadoop/Spark cluster runs directly on EC2 instances that appear in your account. Task nodes use Spot instances heavily. Spark tasks are fault-tolerant (they retry on other nodes), so the 2-minute Spot interruption notice is acceptable. This is one of the most cost-effective uses of Spot instances: 70–90% cost reduction on the compute-heavy part of a batch job.

javascript

EMR cluster:

Master node          Core nodes           Task nodes
┌──────────┐    ┌──────────┐         ┌──────────┐
│m5.xlarge │    │r5.2xlarge│ × N     │r5.2xlarge│ × M
│ On-demand│    │ On-demand│         │  SPOT    │
│ YARN RM  │    │ HDFS     │         │ compute  │
│ Spark    │    │ DataNode │         │ only     │
│ Driver   │    └──────────┘         └──────────┘
└──────────┘

Spot interruption → task node lost → Spark retries tasks on surviving nodes
Core node lost → HDFS data loss risk → keep core nodes On-demand

Lambda

Note

Lambda is the furthest abstraction from EC2, you never pick an instance type, you never see a node. But EC2 is still underneath. Each Lambda invocation runs inside a Firecracker microVM: a lightweight virtual machine based on KVM, running on EC2 bare-metal instances in AWS's infrastructure. Firecracker boots in ~125ms, which is how Lambda achieves fast cold starts. What the EC2 substrate means for Lambda:

Memory setting = your only EC2 knob. Lambda allocates CPU proportional to memory. 128MB → 0.08 vCPU. 1769MB → 1 full vCPU. 10GB → ~6 vCPU. If your function is CPU-bound and slow, increase memory, you're getting more vCPU.
VPC Lambda = ENI on EC2. When Lambda runs in a VPC, it creates an ENI in your subnet. ENI creation used to cause cold start delays (fixed with Hyperplane ENIs, but subnet IP exhaustion is still a real failure mode).
Concurrency = number of parallel microVMs. Each concurrent execution is a separate Firecracker VM on some EC2 host somewhere. Account-level concurrency limit = 1000 by default.

javascript

Lambda execution model:

Invocation 1    Invocation 2    Invocation 3
     │                │               │
     ▼                ▼               ▼
┌─────────┐     ┌─────────┐     ┌─────────┐
│Firecrck │     │Firecrck │     │Firecrck │
│microVM  │     │microVM  │     │microVM  │
│ warm    │     │ cold    │     │ cold    │
└─────────┘     └─────────┘     └─────────┘
      └──────────────┴───────────────┘
              EC2 bare-metal host
              (AWS-managed, invisible)

warm = reused execution environment (fast)
cold = new microVM boot (~100-500ms)

What this means practically:

CPU-bound Lambda functions: double the memory, halve the duration, cost stays similar but function is faster
VPC Lambda subnet must have enough free IPs for concurrent executions. IP exhaustion causes invocation failures
Lambda concurrency limit is a regional EC2 capacity constraint, request limit increases proactively for production

ECS on EC2 (vs Fargate)

Note

When you run ECS on EC2 launch type (not Fargate), your containers run on EC2 instances you manage. The ECS agent runs as a daemon on each instance and reports capacity to the ECS control plane. Fargate removes the EC2 layer, each task gets its own Fargate slot (a Firecracker microVM, same as Lambda). You don't manage instances, don't worry about bin-packing containers onto hosts, don't handle OS patches. But Fargate is still EC2 underneath, it's just AWS managing it. The cpu and memory in your task definition map to the Fargate VM's allocated resources, which map to physical EC2 capacity on AWS's hosts.

javascript

ECS on EC2:                     ECS on Fargate:

┌────────────────────┐          ┌──────────────────────┐
│ EC2 instance       │          │ Fargate task         │
│ (you manage)       │          │ (AWS manages)        │
│ ┌────┐ ┌────┐      │          │ ┌──────────────────┐ │
│ │ C1 │ │ C2 │ ...  │          │ │ container(s)     │ │
│ └────┘ └────┘      │          │ └──────────────────┘ │
│ ECS agent          │          │ Firecracker microVM  │
└────────────────────┘          └──────────────────────┘
  You pick instance type           You pick cpu/memory
  You patch the OS                 AWS patches everything
  You manage bin-packing           AWS handles placement

S3

Note

S3 is the one major AWS service that is not EC2-backed in the traditional sense. S3 runs on AWS's custom distributed storage hardware, not general-purpose EC2 instances. However, EC2 still intersects with S3 in a critical way: S3 requests from EC2 go over the network. Without a VPC endpoint, S3 traffic exits your VPC, traverses the internet gateway, and hits S3's public endpoint, consuming internet bandwidth and incurring data transfer costs. With a VPC Gateway Endpoint for S3: traffic stays on AWS's internal network. No internet gateway, no NAT gateway, no data transfer cost for S3 traffic. Free to set up, significant cost impact at scale.

javascript

Without VPC endpoint:           With VPC endpoint:

EC2 → Internet Gateway          EC2 → VPC Gateway Endpoint
    → S3 public endpoint             → S3 (AWS internal network)
    (costs money, slower)            (free, faster)

The Universal Pattern

Across all these services, the same mental model applies:

javascript

Managed service property     →    EC2 reality underneath
─────────────────────────────────────────────────────────
Instance class / node type   →    EC2 instance type (CPU/RAM)
Multi-AZ                     →    EC2 instances in multiple AZs
Storage / IOPS               →    EBS volume (gp3/io2)
Failover time                →    EC2 instance promotion (60-120s)
Scaling speed                →    EC2 launch time (2-3 min)
Network throughput           →    EC2 instance network bandwidth
VPC / subnets / SGs          →    Same EC2 networking model

Why This Matters for Debugging

When something is slow or broken in a managed service, the EC2 mental model gives you the right diagnostic questions:

"RDS is slow"

→ Is it the query (EXPLAIN ANALYZE)? → Is it CPU (instance type too small / burstable credit exhausted)? → Is it IOPS (EBS throughput limit hit, check ReadIOPS, WriteIOPS CloudWatch metrics)? → Is it connections (max_connections = function of instance RAM on PostgreSQL)?

"ElastiCache is slow"

→ Is it network bandwidth on the node type (small cache. instances have low baseline bandwidth)? → Is it memory pressure (evictions happening, check Evictions metric)? → Is it a hot key hitting one shard's single-threaded Redis (EC2 instance at 100% single-core CPU)?

"EKS pods not scheduling"

→ Are nodes at capacity (EC2 instance RAM/CPU exhausted)? → Is the node group at max (ASG max reached, need to increase or add a node group)? → Is it IP exhaustion (VPC CNI assigns IPs from the node's EC2 ENIs, small instance types have few ENIs)?

"Lambda cold starts are slow"

→ Is it VPC Lambda ENI provisioning (switch to Hyperplane ENI or move out of VPC if not needed)? → Is it package size (larger zip = longer microVM init)? → Is it memory too low (more memory = more CPU = faster init code execution)?

Why This Matters for Cost

The EC2 layer is where most AWS cost optimization happens:

Right-sizing: db.r6g.2xlarge costs 4x a db.r6g.large. If your database only uses 20% CPU and 30% RAM, you're paying for idle EC2 capacity. Managed service pricing is EC2 pricing with a management markup.
Reserved Instances / Savings Plans: RDS Reserved Instances, ElastiCache Reserved Nodes, these are all commitments on EC2 instance types for 1 or 3 years. Same mechanism as EC2 Reserved Instances.
Spot for EMR task nodes: 70-90% savings because the EC2 Spot market prices compute at auction. Fault-tolerant workloads (Spark) absorb interruptions.
Fargate vs EC2 for ECS: Fargate charges a ~30% premium over equivalent EC2 capacity in exchange for zero management overhead. For low-density workloads, Fargate is cheaper. For high-density workloads, EC2 launch type wins.
Data transfer: EC2 network egress costs money. Managed services (RDS, ElastiCache) inside the same VPC communicate with each other and with EC2 over private networking (no data transfer cost. Cross-AZ traffic costs $0.01/GB in both directions) invisible until you're moving terabytes.

Summary

EC2 is not one service among many. It is the foundation that AWS's entire managed services portfolio runs on. Every time you interact with RDS, ElastiCache, EKS, OpenSearch, EMR, or Lambda, you are interacting with EC2 at one level of abstraction or another. The managed layer buys you: no OS patching, no daemon management, no cluster coordination, automated backups, multi-AZ failover out of the box. The EC2 layer still gives you: instance type constraints, AZ placement, network throughput limits, EBS IOPS limits, and launch time lag. Knowing both layers makes you significantly better at capacity planning, cost optimization, and production debugging than someone who only knows the managed service API.