VPC Packet Flow Simulator, and where the bill comes from.

An Amazon VPC is a logically isolated virtual network you run inside an AWS region. You choose its IP range, split it into subnets, and decide what reaches the public internet and what stays private. None of it is physical. Every packet inside a VPC is encapsulated, routed, and policed by a fleet of services running on AWS Nitro cards in the hypervisor; there is no cable for you to trace.

The closest mental model is a routed overlay on someone else's data centre. When an EC2 instance sends a packet to another instance in the same VPC, the Nitro card on the source host wraps the packet in an envelope addressed to the Nitro card on the destination host, hands it to the underlying physical network, and unwraps it on arrival. The instances see a normal IP packet. The physical network sees encapsulated traffic between two Nitro cards. AWS's mapping service answers "which Nitro card owns this IP right now?" and is one of the largest pieces of infrastructure that most EC2 users never touch.

That overlay model is why VPCs are cheap to create (the limit per account is 5 by default, raisable to thousands), cheap to delete (no hardware reallocation), and have predictable per-byte costs (no over-provisioned pipes to amortise). It is also why the policy hooks (security groups, NACLs, route tables, flow logs) work at any scale: the control plane never rewires anything physically and only updates a table.

The simulator above shows a small VPC: one public subnet with an IGW and a NAT Gateway, one private subnet with an EC2 instance, plus the policy stops on the packet path. Choose a source and destination, pick a protocol and port, and press Send to follow the packet through the hops. Set a NACL rule to deny and the same packet is dropped. The hop order is always SG → NACL → Route table → Gateway.

A VPC needs a CIDR block. The default suggestion in the AWS console is 10.0.0.0/16: 65,536 addresses, more than most teams need. Take it, since bigger blocks cost nothing and avoid later problems. The worst VPC design mistake is creating two /24 VPCs in different accounts, then trying to peer them later and discovering they overlap.

Inside the VPC, carve out /24 subnets per availability zone. A /24 gives you 251 usable hosts (AWS reserves 5 per subnet for the network address, broadcast, the implicit router, the DNS resolver, and a future-use placeholder). That is plenty for most application workloads. For Lambda-in-VPC or large EKS clusters where every pod gets an ENI IP, jump to /22 or larger per subnet. EKS in particular eats IPs fast; an under-provisioned subnet shows up as pod scheduling failures.

The renumbering problem is the killer. Once instances are launched, processes are bound to IPs, configs reference IPs, and Terraform state knows IPs. Renumbering a live VPC is roughly equivalent to rebuilding it; you spin up a parallel VPC with the new range, migrate workloads across, then decommission the old one. AWS now offers secondary CIDR blocks on VPCs which soften this: you can graft an extra /16 onto an existing VPC if you outgrow the original. Planning for it from the start is still cheaper.

The minimum subnet AWS allows is /28 (16 addresses, 11 usable). The maximum is /16 (65,531 usable). Pick subnet sizes per use case: tiny /28 for a single bastion host, /24 for typical app workloads, /20 or larger for Kubernetes data planes.

Subnets do not carry a public-or-private flag. A subnet is "public" if and only if its route table has an entry sending 0.0.0.0/0 through an Internet Gateway. Drop that entry and the same subnet becomes private. Add it to a previously private subnet and every instance with a public IP is exposed to the internet. Routing is the only difference between the two kinds of subnet.

For an instance in a public subnet to reach the internet, three things must align: the route table sends 0.0.0.0/0 to the IGW; the instance has a public IPv4 address (auto-assigned or via Elastic IP); and security groups permit egress on the relevant port. If any of the three is missing, the packet is lost.

Private subnets reach the internet only through a NAT: either a managed NAT Gateway, a self-managed NAT instance, or in newer designs, AWS Transit Gateway routing through a central egress VPC. The instances themselves never get a public IP. The reverse direction is blocked: nothing on the public internet can initiate a connection to a private instance, because there's no route in to it.

The convention is to put load balancers, NAT Gateways, and bastions in the public subnet, and put all the application servers, databases, and internal services in private subnets. Web traffic enters via the ALB, the ALB forwards to private targets, and outbound traffic from those private targets flows back through the NAT Gateway. Most production VPCs follow this layout, repeated across two or three AZs for availability.

The Internet Gateway (IGW) is a horizontally scaled, redundant component that sits at the edge of a VPC and connects it to the public internet. Conceptually it does two things: it translates between the public IP and the instance's private IP for instances that have a public IP (effectively a 1:1 NAT), and it serves as the route target for the 0.0.0.0/0 entry in public subnets' route tables.

The IGW itself has no throughput cap, no hourly cost, and no per-GB cost beyond standard data-out pricing. It is the cheapest exit available in AWS, but an instance needs a public IP to use it for outbound traffic. That is fine for load balancers and bastions and awkward for fleets of private application servers, which is exactly the use case NAT Gateway exists to handle.

The 1:1 NAT model means inbound packets to your public IP arrive at the IGW, get rewritten to your private IP, and are delivered to the instance. There is no port-to-port mapping like a home router's NAT: your public IP and your private IP have a one-to-one correspondence as long as the instance has the public IP attached. This is why allocating an Elastic IP (a static public IPv4 reserved to your account) gives an instance a stable external identity.

Egress-only Internet Gateways exist for IPv6: they allow outbound IPv6 traffic but block inbound, which is the IPv6 equivalent of "instances in a private subnet behind a NAT." IPv6 doesn't use NAT in AWS; addressing is end-to-end, so the egress-only IGW is how you replicate the inbound-blocked semantics.

A NAT Gateway is AWS's managed network address translator. Instances in a private subnet send outbound traffic through it; the NAT Gateway rewrites the source IP to its own public IP, forwards the packet via the IGW, and tracks the connection so return packets get rewritten back. From the outside world, a fleet of a hundred private instances looks like a single public IP: that of the NAT Gateway.

NAT Gateways scale to 100 Gbps of throughput per NAT and support up to 55,000 simultaneous connections to a single destination IP. They are zonal: one per availability zone for high availability, since a NAT Gateway in us-east-1a doesn't survive an outage of that AZ. Properly designed VPCs run one NAT Gateway per AZ and route each private subnet through the NAT Gateway in its own AZ to avoid cross-AZ data transfer costs.

On price, NAT Gateway costs $0.045 per hour (around $33 a month) plus $0.045 per GB processed. The per-GB charge is on top of normal data-out pricing, so an outbound byte through NAT costs roughly twice what an outbound byte through an IGW costs. For high-egress workloads it becomes the largest line item. A team running a service that pulls 10 TB a day of external API traffic through a NAT Gateway pays $450/day just for the NAT, on top of regular data-out. The same traffic through an IGW with public IPs on the instances would cost half.

This is why mature shops put as much outbound traffic as possible through VPC endpoints (free for gateway endpoints to S3 and DynamoDB), make sure cross-AZ NAT traffic is eliminated, and track NAT Gateway processed GB as a cost metric. Cloud cost engineers have stories of internal-only services routed through NAT Gateway by accident, costing $50k a month before anyone noticed.

VPC endpoints let an instance in a private subnet reach an AWS service without going through the public internet. It needs no NAT, IGW or data-out charge, only an entry in the route table that says "52.218.0.0/16 goes to this endpoint." There are two kinds and their pricing differs by an order of magnitude.

Gateway endpoints exist only for S3 and DynamoDB. They are free. They are implemented as a special route in the subnet's route table that funnels traffic to the service over AWS's internal backbone. They need no ENI and have no hourly charge or per-GB processing fee. Any production VPC that uses S3 should have a gateway endpoint for it, since going without one costs money for no benefit.

Interface endpoints (PrivateLink) exist for everything else: SSM, ECR, KMS, SQS, SNS, the AWS API itself, plus arbitrary third-party services that publish their own endpoints. They are implemented as ENIs in your subnet with private IPs from your CIDR. They cost $0.01 per AZ per hour ($7.20/month/AZ) plus $0.01 per GB processed. The cost is noticeable but usually still lower than NAT-routed access, and security is better because traffic stays on AWS's private network.

The combination (gateway endpoint for S3/DynamoDB, interface endpoints for the AWS APIs the workload needs) lets you build a no-IGW, no-NAT-Gateway VPC where private instances can still reach the AWS services they depend on. Cost-conscious teams use this pattern: a "private" VPC with zero public network exposure and all AWS access via endpoints.

Every subnet is associated with exactly one route table. If you don't explicitly create one, AWS associates the subnet with the VPC's default route table. The table consists of entries mapping CIDR destinations to targets: IGW, NAT Gateway, peering connection, transit gateway attachment, VPC endpoint, network interface, or "local" (the implicit VPC-internal route, which is always present and can't be removed).

The matching rule is longest prefix match, the same rule internet routers use. If the table has entries for 10.0.0.0/16 (local) and 10.0.1.0/24 (custom), a packet to 10.0.1.5 matches both, but the /24 wins because it's more specific. This lets you override the implicit local route for a portion of your CIDR: for instance, to send traffic for a specific subnet through a network appliance for inspection.

Subnets can share a route table; multiple subnets pointing at the same route table is the normal pattern. Most VPCs run with a "public" route table associated with all public subnets and a "private" route table associated with all private subnets. The public table has a 0.0.0.0/0 → IGW entry; the private table has a 0.0.0.0/0 → NAT-GW entry; both have the implicit local route.

A frequently missed point is that peering connections need routes on both sides. Setting up VPC peering only attaches the connection: you still need to add routes in both VPCs' route tables for traffic to flow across. Forgetting this can cost hours of debugging, because peering connections show as ACTIVE while traffic is silently dropped.

Every packet in a VPC passes two firewalls, security groups and NACLs. Security groups are well known, and NACLs tend to be forgotten until something breaks.

Security groups attach to ENIs (the elastic network interface of an EC2 instance, load balancer, RDS endpoint, Lambda-in-VPC pod, anywhere there's an interface). They are stateful: if a rule permits a packet outbound, the return packet is automatically permitted regardless of inbound rules. They are default-deny: no rule, no traffic. Rules are allow-only: you can permit, you can't deny. To "deny" something you remove the permitting rule.

NACLs attach to subnets, not interfaces. They are stateless: every direction must be permitted independently. A connection from 10.0.1.5 to an internet host on port 443 needs an egress rule permitting TCP 443 outbound and an ingress rule permitting TCP on the ephemeral port range (1024-65535 or 32768-60999, depending) inbound. They are default-allow: a fresh NACL permits everything. Rules are numbered and evaluated in order; first match wins. Allow and deny rules both exist.

The evaluation order on egress from an EC2 in a private subnet: SG (at the ENI) → NACL (at the subnet boundary) → route table → next hop. On ingress: NACL → SG → instance. A packet has to pass every check to make it through.

NACLs are a coarse-grained subnet-level perimeter, useful for blocking obviously bad traffic at the boundary (e.g., a known-bad IP range). SGs are the fine-grained per-workload firewall, where most of the actual policy lives. In practice, most VPCs leave NACLs at default-allow and put all the enforcement in security groups. The NACL acts as a second layer of defence.

VPC peering connects two VPCs directly. It supports cross-account, cross-region peering. Traffic flows over AWS's backbone and never the public internet. There is no hourly cost, only standard data-transfer pricing. CIDRs must not overlap, or the connection silently drops packets to the overlapping range.

The main limitation is that peering is non-transitive. If A peers with B and B peers with C, A cannot reach C through B. You'd need a third peering between A and C. For three VPCs that's three peering connections; for ten VPCs that's 45; for a hundred it's 4,950. The number of connections grows quadratically.

Transit Gateway (TGW) solves the combinatorics. It is a regional hub: every VPC attaches to it once, and the TGW handles routing between attachments. Ten VPCs require ten attachments and one TGW. Routes are configured via TGW route tables, which give you fine-grained control over which attachments can talk to which (you can isolate dev from prod even when both attach to the same TGW). On-prem VPN and Direct Connect attach to the TGW too, making it the standard hub for hybrid topologies.

The pricing: TGW costs $0.05 per attachment-hour plus $0.02 per GB processed. For ten attachments that's $0.50/hour, around $360/month, before data transfer. For a small number of VPCs, peering is cheaper. For more than ~6-7 VPCs, TGW typically wins on both cost and operational simplicity. AWS Cloud WAN is the newer (2022+) successor pitched for multi-region, multi-account topologies; it is roughly a TGW of TGWs.

Every VPC ships with an internal DNS resolver, the Route 53 Resolver, reachable at the second IP of the VPC CIDR. For a 10.0.0.0/16 VPC, that's 10.0.0.2. Instances configured with the VPC's DHCP option set use this resolver by default. It answers queries for public DNS names by recursing out to the internet, and answers queries for internal names (private hosted zones, EC2 private DNS names) directly.

The resolver has its own quirks. It rate-limits to 1024 packets per second per ENI; workloads doing huge volumes of DNS lookups (microservices with per-request DNS, certain Java HTTP clients with short-TTL caches, Kubernetes Services without conntrack) can hit the limit and see resolution failures. The standard mitigations: increase application DNS caching, deploy CoreDNS as a per-node cache in EKS, use Route 53 Resolver endpoints if you need higher throughput.

Private hosted zones let you define internal-only names for a VPC. Associate a private hosted zone with a VPC and any query inside that VPC for a name in that zone resolves to whatever you configured. The standard pattern is internal.example.com for internal services, with records pointing at internal ALBs or service discovery endpoints. The same name resolves to nothing from outside the VPC.

For hybrid setups (DNS queries from on-prem needing to resolve VPC private names, or VPC queries needing to resolve on-prem AD names) Route 53 Resolver supports inbound and outbound endpoints. An inbound endpoint exposes the VPC resolver to on-prem via Direct Connect / VPN; an outbound endpoint forwards specific zones from VPC queries to on-prem DNS servers. The two-endpoint pattern is how most hybrid AWS-plus-on-prem shops handle DNS.

Overlapping CIDRs between peered VPCs. The peering attaches successfully; routes get added; traffic to the overlapping range goes nowhere. There is no error or log entry, and traffic is dropped silently. The fix is renumbering one of the VPCs, which is expensive. The prevention is picking non-overlapping CIDRs from the start, ideally with a documented IPAM plan covering everything you might ever peer.

Forgotten NACL deny rules. Someone adds a deny rule to a NACL to block a specific IP. The rule blocks more than intended because NACLs are stateless and the engineer forgot that return traffic on ephemeral ports also passes through the NACL. Health checks fail and deploys break.

NAT Gateway running in a dev subnet. Someone enables a NAT Gateway in dev for a one-off troubleshoot, forgets to remove it. $33/month for the hourly charge plus whatever processed bytes the dev workload generates. Multiplied across forgotten dev environments, this is a five-figure annual line item.

Missing return-route through TGW. Workload A in VPC-A talks to workload B in VPC-B via TGW. Route from A to B exists. Route from B back to A doesn't. A's request goes out, B receives it, B sends a response that has no return path. Connection times out. The fix is symmetric routing on both sides of every TGW attachment.

Public IP without an IGW. Instance auto-assigns a public IP because the subnet is configured to. The subnet's route table doesn't have an IGW entry. The public IP is functionally useless: no traffic enters. The instance still reports as having a public IPv4 in its metadata, which adds to the confusion.

Cross-AZ data transfer through NAT Gateway. Private subnet in us-east-1a routes its 0.0.0.0/0 through a NAT Gateway in us-east-1b. Every outbound byte pays cross-AZ data transfer ($0.01/GB each way) plus NAT Gateway processing ($0.045/GB). Per-AZ NAT Gateways with per-AZ private route tables eliminate the cross-AZ leg.