AWS Cloud WAN series · Companion
The Complete AWS Cloud WAN Reference Architecture: Segments, Inspection, Hybrid, and Every Flow
A complete blueprint for a global, multi-Region (us-east-1 / us-west-2), multi-account core network: segmentation, centralized egress and ingress, east-west inspection, hybrid Direct Connect, and Transit Gateway federation.
TL;DR
- One Cloud WAN core network across two Regions (
us-east-1,us-west-2), one Core Network Edge (CNE) per Region, auto-meshed over the AWS backbone. The hub account owns it and shares it organization-wide with RAM from us-east-1. - Five segments (
production,nonproduction,pci,sharedservices,hybrid) plus two Network Function Groups form a deny-by-default connectivity matrix. - North-south: egress uses
send-toan Egress NFG (firewall + NAT + IGW); ingress is inspected at the VPC boundary. East-west:send-viaan Inspection NFG. Shared-services is reachable non-transitively. - Hybrid: a native Direct Connect Gateway associated to both Regions over primary and backup transit VIFs.
- Federation: the existing Transit Gateway peers to its same-Region CNE, mapping each route table to a segment.
- Symmetry: appliance mode pins every stateful flow to one Availability Zone, so the forward and return packets take the same path.
What we're building — and who owns it
The design is one core network across two Regions — us-east-1 and us-west-2 — with a CNE in each. AWS auto-meshes them over its backbone with eBGP, so there's no inter-Region peering for us to configure or manage. This is a key reason we use Cloud WAN rather than building this on Transit Gateway, where inter-Region peering is static-route-only. We are well within the limits: one CNE per Region per core network, one core network per global network.
Each account owns a clear part of the design. Here is who owns what:
| Account | Owns | Notes |
|---|---|---|
| Network / Hub Account | Global network, core network, core network policy, all CNEs | Bills the CNEs and Connect/VPN attachments; RAM-shares the core network organization-wide from us-east-1 |
| Security / Inspection Account | Inspection VPCs (AWS Network Firewall) and egress VPCs (firewall + NAT) | Holds the attachments that join the Inspection NFG and Egress NFG |
| Shared-Services Account | Shared-services VPC (DNS, AD, CI, logging) | Reachable non-transitively by permitted segments |
| Workload Account · Prod | Production workload VPCs | Attachment tagged into the production segment; pays its own attachment + data |
| Workload Account · Non-Prod | Non-production workload VPCs | Attachment tagged into the nonproduction segment |
Master topology — the whole design on one page
The master shows all five accounts, two Regions with one CNE each in full mesh, the five segments as colored lanes, the Egress and Inspection NFGs, the native DXGW to two on-prem DCs, and the federated TGW. Think of it as the index every other figure zooms into — topology and the segment-to-color legend only, no single traced flow.
How traffic is walled off — segments and the full connectivity matrix
A segment is just a separate routing domain. By default an attachment only reaches others in the same segment — everything else is implicit deny.
Segment budget. Five segments plus two NFGs (each NFG quietly eats one of the 40-segment budget) puts us at 7 of 40 — well within the limit.
A share has three properties. It is:
- bidirectional by default;
- non-transitive;
- carrying only attachment/propagated routes — never static or third-segment routes.
That non-transitivity is the safety property the whole matrix leans on. Share sharedservices with both production and nonproduction and each can reach it — but you never accidentally bridge prod to non-prod.
Here's the full connectivity matrix. Each off-diagonal cell is allow (a share), blocked (no share — implicit deny), or inspected (steered through the Inspection NFG via send-via, never a plain share). The diagonal is the intra-segment call: isolated means isolate-attachments: true (spoke→hub only).
| from ↓ \ to → | production | non-production | pci | shared-services | hybrid |
|---|---|---|---|---|---|
| production | isolated + inspected | inspected | blocked | allow | allow |
| non-production | inspected | allow (intra) | blocked | allow | allow |
| pci | blocked | blocked | isolated | inspected | inspected |
| shared-services | allow | allow | inspected | isolated | allow |
| hybrid | allow | allow | inspected | allow | allow (intra) |
About the diagonals. production, pci, and sharedservices are isolated internally (isolate-attachments: true) — no direct spoke-to-spoke. production also permits specific intra-segment pairs, but only through the Inspection NFG; isolate-attachments is what forces same-segment traffic onto the inspection path instead of letting it mesh directly (Flow 1). nonproduction and hybrid mesh freely.
pci reaches sharedservices and hybrid via inspected send-via paths, never a plain share — so the PCI workloads get centralized DNS, logging, and patching plus on-prem reach, and every PCI border crossing still passes through the Inspection NFG. For maximum isolation instead, set both to blocked.How traffic is inspected — egress, ingress, and east-west service insertion
Service insertion (SI) covers exactly two patterns — and ingress is not one of them:
- north-south internet egress (
send-to); - east-west (
send-via).
A few NFG rules worth knowing.
- An attachment lives in a segment or an NFG, never both.
- An NFG holds at most one attachment per Region.
Centralized egress — send-to the local Egress NFG
Egress is symmetric by design. Both Regions run a full local stack — Inspection, Ingress, and Egress VPC — so send-to installs the default routes (0.0.0.0/0 + ::/0) at the local Egress NFG and every workload exits in its own Region via firewall → NAT → IGW (Flow 5).
Edge-override we don't use here. Every Region has its own egress stack, so there's no egress-less Region to pin to a neighbour — we keep it only as a reference for the asymmetric case (Flow 7, greyed), where the non-geography-aware Region-priority list would otherwise send a stackless Region to a far or random exit.
Centralized ingress — at the VPC boundary, not SI
Ingress isn't service insertion — it's inspected at the VPC/IGW boundary (module type ingress_with_inspection, Flow 6): the IGW gateway route table points at a Network Firewall endpoint fronting a public ALB, which forwards over Cloud WAN. The internet-facing LB is mandatory — a NAT gateway can't accept inbound and Cloud WAN has no L4/L7 LB of its own.
East-west — send-via the Inspection NFG, single-hop
East-west between segments goes through the Inspection NFG via send-via, single-hop (Flow 3 inter-segment, Flow 1 intra-segment). send-via is automatically bidirectional, which keeps the route symmetric. We chose single-hop over dual-hop — dual-hop would need an inspection attachment in every Region of each SI-enabled segment and roughly double inspection data-processing.
Two policy rules. Inter-segment (prod↔non-prod) uses send-via instead of a plain share — a share would create uninspected routes. Intra-segment (within production) requires isolate-attachments, or same-segment traffic bypasses inspection.
Appliance mode is mandatory and separate from SI. On the inspection-VPC attachment it pins a flow to one AZ for its lifetime so the return packet lands on the same firewall — skip it and a cross-AZ flow breaks stateful inspection. VPC-attachments only, and you need it and service insertion.
How we reach the rest of the world — Direct Connect hybrid and TGW federation
Native Direct Connect Gateway, associated to both Regions
A native DXGW attaches straight to the core — no intermediary TGW, automatic bidirectional BGP. One DXGW associates with a single core network and a single segment, here hybrid.
We associate it with both Regions the hybrid segment spans. Each associated CNE advertises only local routes and keeps AS_PATH.
Two transit VIFs — VIF 1 primary, VIF 2 backup (on-prem local-preference picks active/passive) — to two data centers, with local-Region preference keeping each Region's on-prem egress local (Flows 8 and 9). The DXGW ASN sits outside the core ASN range; DX is BGP-only (no static routes) and carries no BGP communities.
Federating the existing Transit Gateway
A single global network can hold both Cloud WAN core networks and Transit Gateways — that's what makes federation possible in the first place.
The existing network stays in place. We peer the existing TGW to its same-Region CNE (us-east-1) and map each TGW route table to a matching segment with route-table attachments — one route-table attachment extends one TGW route table to one segment, dynamic-BGP-only, and the TGW ASN must differ from the CNE ASN.
Migration is non-disruptive. For the same prefix, a static TGW route wins over a dynamically-learned Cloud WAN route — so the core stands up right alongside the TGW with zero traffic impact until you pull the static routes (Flow 10).
The flows, hop by hop
Each figure below stands on its own: the numbered hops trace the VPC route-table decision → the segment route-table lookup at the CNE → the next-hop redirect to the inspection/NFG attachment (where it applies) → appliance-mode AZ pinning (where the flow is stateful) → the return path, with symmetry called out explicitly. Wherever a flow could go asymmetric or hit random-remote-CNE behavior, we say exactly how this design prevents it. Each figure is embedded here in its own section and also ships as a standalone file.
Flow 1 — intra-segment, isolated + inspected (within Production)
How the flow works
- Production VPC · Prod A (us-east-1) → us-east-1 CNE. Prod A’s route to Prod B’s CIDR points at its Cloud WAN attachment. (traffic hop)
- Lookup inside the us-east-1 CNE (pre-inspection). Because the
productionsegment setsisolate-attachments: true, there's no direct spoke-to-spoke route between Prod A and Prod B; the only route resolving Prod B’s CIDR is the same-segment send-via, which steers the flow to the Inspection NFG. (route-table lookup → next hop = Inspection NFG) - us-east-1 CNE → Inspection VPC. The core carries the packet to the firewall once (
single-hop); appliance mode pins the AZ so both directions ride one ENI. (traffic hop) - Inspection VPC → back into the us-east-1 CNE. Once the firewall passes it, the packet is re-injected into the same us-east-1 CNE. (traffic hop)
- Lookup inside the us-east-1 CNE (post-inspection). The route-table now resolves Prod B’s CIDR to Prod B’s attachment in the same Region. (route-table lookup → next hop = Prod B attachment)
- us-east-1 CNE → Production VPC · Prod B. The CNE delivers the inspected packet to Prod B (local-Region delivery). (traffic hop)
Core network policy that drives this flow (// lines are annotations — the real policy JSON has no comments)
// Intra-segment inspection: the segment must be ISOLATED first,
// otherwise attachments in the same segment reach each other directly.
"segments": [
{ "name": "production",
"require-attachment-acceptance": true, // attachments must be approved to join
"isolate-attachments": true } // drop spoke-to-spoke routes; keep only shared / send-via routes
],
// The inspection layer. The Inspection VPC attaches HERE, not to the workload segment.
"network-function-groups": [
{ "name": "InspectionVPC", "require-attachment-acceptance": true }
]"segment-actions": [
{
"action": "send-via", // steer matching traffic THROUGH an NFG, then re-enter the core
"segment": "production", // applies to traffic in the production segment
"mode": "single-hop", // inspect once, at one edge
"when-sent-to": { "segments": ["production"] }, // SAME segment: prod-to-prod is what gets inspected
"via": { "network-function-groups": ["InspectionVPC"] } // route it via this inspection NFG
}
]
// send-via is bidirectional automatically, so the return is inspected too.Flow 2 — prod to PCI is denied (the canonical blocked flow)
How the flow works
- Production VPC → us-east-1 CNE. The prod workload fires a packet for the PCI CIDR at its Cloud WAN attachment. (traffic hop)
- Lookup inside the us-east-1 CNE — denied. The
productionroute-table finds no route to the PCI CIDR: the policy declaresproductionandpcias separate segments with noshareand nosend-viabetween them, so the packet is dropped. (route-table lookup → no route → implicit deny; no delivery hop)
Core network policy that drives this flow (// lines are annotations — the real policy JSON has no comments)
// Both segments exist; segments default to implicit-deny, same-segment only.
"segments": [
{ "name": "production", "require-attachment-acceptance": true },
{ "name": "pci", "require-attachment-acceptance": true }
]"segment-actions": [
//... actions may exist for OTHER segment pairs...
// but there is NO share and NO send-via between production and pci
]
// No share or send-via between production and pci => implicit deny.
// The us-east-1 CNE route-table simply has no route to the pci CIDR.Flow 3 — prod ↔ non-prod, inspected, cross-Region
How the flow works
- Production VPC (us-east-1) → us-east-1 CNE. The prod EC2's route to the non-prod CIDR points at its us-east-1 Cloud WAN attachment; the workload has no idea inspection is about to happen. (traffic hop)
- Lookup inside the us-east-1 CNE (pre-inspection). The
productionroute-table has no share tononproduction, so the only route resolving the non-prod CIDR is a send-via action: it steers the flow to the Inspection NFG, withedge-overridepinning inspection to the us-east-1 edge. (route-table lookup → next hop = Inspection NFG) - us-east-1 CNE → Inspection VPC (Network Firewall). The core carries the packet to the firewall once (
single-hop); appliance mode pins the AZ so both directions ride one ENI and the return lands on the same firewall. (traffic hop) - Inspection VPC → back into the us-east-1 CNE. Once the firewall passes it, the packet is re-injected into the same us-east-1 CNE. (traffic hop)
- Lookup inside the us-east-1 CNE (post-inspection). The non-production CIDR resolves to the non-prod attachment in us-west-2, so the next hop is the us-west-2 CNE across the auto-meshed backbone — this is the step that carries the inspected packet to the other Region. (route-table lookup → next hop = us-west-2 CNE)
- us-east-1 CNE → us-west-2 CNE. The packet crosses Region over the Cloud WAN core — a full-mesh eBGP fabric between every CNE. (traffic hop, cross-Region)
- Lookup inside the us-west-2 CNE. The us-west-2 CNE route-table resolves the destination to the local non-production attachment. (route-table lookup → next hop = local non-prod VPC)
- us-west-2 CNE → Non-Production VPC. The CNE delivers the packet to the non-prod EC2 (local-Region delivery). (traffic hop)
Core network policy — the segment-action that drives this flow
Core network policy that drives this flow (// lines are annotations — the real policy JSON has no comments)
// Two isolated routing domains; by default they cannot reach each other.
"segments": [
{ "name": "production", "require-attachment-acceptance": true }, // attachments must be approved to join
{ "name": "nonproduction", "require-attachment-acceptance": true }
],
// The inspection layer. The Inspection VPC attaches HERE, not to a workload segment.
"network-function-groups": [
{ "name": "InspectionVPC", "require-attachment-acceptance": true }
]"segment-actions": [
{
"action": "send-via", // steer matching traffic THROUGH an NFG (not a plain share)
"segment": "production", // applies to traffic originating in production
"mode": "single-hop", // inspect once, at one edge (not at source and dest both)
"when-sent-to": { "segments": ["nonproduction"] }, // only prod-to-nonprod traffic is inspected
"via": {
"network-function-groups": ["InspectionVPC"], // route it through this inspection NFG
"with-edge-overrides": [
// pin the inspection edge: traffic that would use us-west-2 inspects at us-east-1
{ "edge-sets": [["us-west-2"]], "use-edge-location": "us-east-1" }
]
}
}
]Flow 4 — workloads reach shared-services, non-transitively
How the flow works
- Production VPC → us-east-1 CNE. The prod workload's route to the shared-services CIDR points at its Cloud WAN attachment. (traffic hop)
- Lookup inside the us-east-1 CNE (production). The
productionroute-table resolves the shared-services CIDR through the share linkingproductionandsharedservices; next hop is the Shared-Services attachment. (route-table lookup → next hop = Shared-Services) - us-east-1 CNE → Shared-Services VPC. The CNE delivers the prod packet to the shared-services hub. (traffic hop)
- Non-Production VPC → us-east-1 CNE. Separately, the non-prod workload also routes to the shared-services CIDR via its attachment. (traffic hop)
- Lookup inside the us-east-1 CNE (nonproduction). The
nonproductionroute-table resolves the same CIDR through the share linkingnonproductionandsharedservices. (route-table lookup → next hop = Shared-Services) - us-east-1 CNE → Shared-Services VPC. The CNE delivers the non-prod packet to the same shared-services hub. (traffic hop)
Core network policy that drives this flow (// lines are annotations — the real policy JSON has no comments)
// Three segments; by default none reach each other.
"segments": [
{ "name": "production", "require-attachment-acceptance": true },
{ "name": "nonproduction", "require-attachment-acceptance": true },
{ "name": "sharedservices", "require-attachment-acceptance": true }
]
// Segment names are alphanumeric only; display labels add the hyphens."segment-actions": [
{
"action": "share", // exchange attachment routes between segments
"mode": "attachment-route", // shares the attachment + return routes only
"segment": "sharedservices", // the hub segment
"share-with": ["production", "nonproduction"] // prod and nonprod each pair with sharedservices
}
]
// A share is BIDIRECTIONAL but NON-transitive: prod and nonprod each
// reach sharedservices, but this installs NO prod to nonprod route.Flow 5 — a workload reaches the internet, same Region
How the flow works
- Production VPC (us-east-1) → us-east-1 CNE. The workload EC2's default route (
0.0.0.0/0) points at its us-east-1 Cloud WAN attachment. (traffic hop) - Lookup inside the us-east-1 CNE. A send-to action already put the defaults
0.0.0.0/0and::/0into theproductionsegment with the local Egress NFG attachment as next hop, so0.0.0.0/0resolves to the Egress NFG in this same Region. (route-table lookup → next hop = Egress NFG) - us-east-1 CNE → Egress VPC. The core carries the packet to the Egress VPC firewall; appliance mode pins the AZ so the return lands on the same firewall. (traffic hop)
- Egress VPC → internet. After inspection the NAT gateway source-NATs and the IGW sends the packet out to the public internet. (traffic hop)
The reply (dashed) re-uses the same Egress NFG attachment and firewall AZ, so the whole conversation is symmetric. And because send-to installs the defaults with the local NFG as next hop, every Region with its own Egress VPC exits in-Region — no cross-Region edge here.
Core network policy — the config that drives this flow
Egress is north-south service insertion — one send-to action on the production segment. It takes no mode and installs the defaults 0.0.0.0/0 and ::/0 into the source segment, next hop the local Egress NFG attachment.
Core network policy that drives this flow (// lines are annotations — the real policy JSON has no comments)
// Segments are isolated routing domains; the Egress VPC attaches to an NFG, not a segment.
"segments": [
{ "name": "production", "require-attachment-acceptance": true } // workloads live here
],
"network-function-groups": [
{ "name": "EgressVPC", "require-attachment-acceptance": true } // the egress / NAT inspection layer
]"segment-actions": [
{
"action": "send-to", // north-south service insertion: inspect, then exit
"segment": "production", // applies to traffic leaving the production segment
"via": {
"network-function-groups": ["EgressVPC"] // exit via the local Egress NFG attachment
}
// no "mode" on send-to. It installs default routes 0.0.0.0/0 AND::/0 into
// the production segment, next hop = the Egress NFG attachment in the SAME Region.
}
]Read it as: any production traffic to 0.0.0.0/0 is steered to the local EgressVPC NFG, inspected, then NAT/IGW out — and because the next hop is the local NFG, each Region exits in-Region.
Flow 6 — the internet reaches a workload (ingress)
How the flow works
- Internet → Ingress VPC (us-east-1). Inbound hits the internet gateway. The IGW edge route table forces it to the Network Firewall endpoint first, then a public ALB — a NAT gateway can't accept inbound, so an internet-facing load balancer is mandatory. (traffic hop)
- Ingress VPC → us-east-1 CNE. Behind the ALB the target is a workload in another VPC, so the packet is handed to the local Cloud WAN attachment. (traffic hop)
- Lookup inside the us-east-1 CNE. This is the only Cloud WAN decision in the whole flow: the segment route-table resolves the workload CIDR to the production attachment. (route-table lookup → next hop = production VPC)
- us-east-1 CNE → Production VPC. The CNE delivers to the workload (local-Region delivery). (traffic hop)
So, ingress is not service insertion. SI (send-to / send-via) covers only east-west and internet egress. Inbound gets inspected at the VPC / IGW boundary — IGW route table → Network Firewall endpoint → public ALB — and only then rides Cloud WAN. The core network policy never steers ingress.
Core network policy — the config that drives this flow
There's no ingress action in the Cloud WAN policy at all. What drives this is the VPC-level IGW ingress route table pointing inbound at the firewall endpoint; the core network policy only attaches the Ingress VPC to a segment.
Core network policy that drives this flow (// lines are annotations — the real policy JSON has no comments)
// VPC-level config — NOT part of the Cloud WAN core network policy.
// The internet-gateway "edge association" route table inspects inbound at the boundary:
IGW ingress route table (associated with the Internet Gateway):
destination =
target = // force inbound THROUGH the firewall...
//... the firewall forwards to the public ALB, which load-balances to the workload.
// A NAT gateway cannot accept inbound — the internet-facing ALB is mandatory. // In the CORE NETWORK POLICY the Ingress VPC is just a normal segment attachment.
// Nothing here steers ingress — there is NO send-to / send-via for inbound.
"attachment-policies": [
{
"rule-number": 100,
"condition-logic": "and",
"conditions": [ { "type": "tag-value", "key": "segment", "value": "production" } ],
"action": { "association-method": "constant", "segment": "production" }
}
]
// NOTE: the core network policy does NOT steer ingress; inbound is decided above.Read it as: inbound is inspected and load-balanced at the VPC / IGW boundary (top tab), then handed to Cloud WAN for one ordinary segment lookup to the workload (bottom tab). The policy attaches the VPC; it does not route inbound.
Flow 7 — edge-override for a Region with no local egress stack (reference pattern, not used here)
How the flow works
- Production VPC (us-west-2) → us-west-2 CNE. The workload’s
0.0.0.0/0default points at its us-west-2 Cloud WAN attachment. (traffic hop) - Lookup inside the us-west-2 CNE — no local egress. This Region has no Egress NFG attachment, so the propagated default has nowhere local to go. The
send-toaction’swith-edge-overridesclause pins the exit: edge-override picks us-east-1. (route-table lookup → next hop = us-east-1 CNE) - us-west-2 CNE → us-east-1 CNE. The packet crosses Region over the Cloud WAN core (full-mesh eBGP backbone) to reach the edge that owns the egress stack. (traffic hop, cross-Region)
- Lookup inside the us-east-1 CNE. At the pinned edge the default resolves to the local Egress NFG attachment. (route-table lookup → next hop = Egress NFG)
- us-east-1 CNE → Egress VPC → internet. The Egress VPC firewall inspects, NAT and IGW send it out; the reply (dashed) re-uses the same pinned edge and AZ. (traffic hop)
Again, we don't use this in the symmetric 2-Region design. Every Region here runs its own local egress stack, so egress is just the Flow 5 picture (local send-to, no cross-Region edge). This figure is the asymmetric case — a Region with no egress VPC — that with-edge-overrides patches up, since the default Region-priority list isn't geography-aware.
Core network policy — the config that drives this flow
It's still the egress send-to action, just with a with-edge-overrides clause bolted on. The override keys are edge-sets (the edge with no local egress) and use-edge-location (the edge that inspects and exits instead).
Core network policy that drives this flow (// lines are annotations — the real policy JSON has no comments)
"network-function-groups": [
{ "name": "EgressVPC", "require-attachment-acceptance": true } // ONLY attached in us-east-1 here
],
"segments": [
{ "name": "production", "require-attachment-acceptance": true } // spans both Regions
]
// us-west-2 has NO Egress NFG attachment, so its default route has no local exit."segment-actions": [
{
"action": "send-to", // north-south egress insertion (no "mode")
"segment": "production",
"via": {
"network-function-groups": ["EgressVPC"],
"with-edge-overrides": [
// the default Region-priority list is NOT geography-aware; pin the exit:
{ "edge-sets": [["us-west-2"]], // traffic that would exit at us-west-2...
"use-edge-location": "us-east-1" } //... inspects and exits at us-east-1 instead
]
}
}
]Read it as: traffic that would exit at us-west-2 is forced to inspect and exit at the us-east-1 EgressVPC NFG instead — a fix for the asymmetric case, not part of the symmetric design.
Flow 8 — workload to on-prem over the DXGW, with return symmetry
How the flow works
- Production VPC (us-east-1) → us-east-1 CNE. The prod EC2's route to the on-prem CIDR points at its local us-east-1 Cloud WAN attachment. (traffic hop)
- Lookup inside the us-east-1 CNE. The on-prem prefix lives in the
hybridsegment, whichproductionshares with; because the native DXGW is associated to this Region, the route resolves to the local DXGW attachment — never cross-Region. (route-table lookup → next hop = local DXGW attachment) - us-east-1 CNE → DXGW → Transit VIF 1 → On-prem DC1. The CNE forwards to the native Direct Connect Gateway, which prefers Transit VIF 1 (primary) to DC1; VIF 2 stays passive as backup to DC2. (traffic hop)
Core network policy — what drives this flow
On-prem reachability comes down to two policy lines: the hybrid segment the DXGW lands in, and a share that joins production to hybrid. No send-via here — this is plain routing, not inspection.
Core network policy that drives this flow (// lines are annotations — the real policy JSON has no comments)
// Two routing domains. The native Direct Connect Gateway attaches to ONE of them.
"segments": [
{ "name": "production", "require-attachment-acceptance": true }, // the workload segment
{ "name": "hybrid", "require-attachment-acceptance": true } // carries the on-prem prefixes
],
// A native DXGW attaches DIRECTLY to the core (no intermediary TGW) and to a SINGLE segment.
// Map it into the hybrid segment, and associate it with EVERY Region that segment is in:
// an un-associated Region inherits a propagated remote route and exits via another Region.
"attachment-policies": [
{
"rule-number": 100, // lowest number wins, first match
"condition-logic": "and",
"conditions": [
{ "type": "attachment-type", "operator": "equals", "value": "direct-connect-gateway" }
],
"action": { "association-method": "constant", "segment": "hybrid" } // DXGW lands in hybrid
}
]"segment-actions": [
{
"action": "share", // plain route-exchange (NOT send-via); no inspection here
"segment": "production", // the workload segment...
"share-with": [ "hybrid" ] //...exchanges routes with the hybrid (on-prem) segment
}
]
// share is BIDIRECTIONAL by default and NON-transitive:
// production learns the on-prem prefixes (this egress flow, Flow 8)
// hybrid learns the production prefixes (the inbound flow, Flow 9)
// Nothing else rides through hybrid, because non-transitive sharing is not a bridge.Flow 9 — on-prem reaches a workload (inbound over DXGW)
How the flow works
- On-prem DC1 → Transit VIF 1 → DXGW → us-east-1 CNE. DC1 advertises its prefixes over Transit VIF 1 into the Direct Connect Gateway; because the DXGW is associated to us-east-1, those routes land on the local CNE. (traffic hop)
- Lookup inside the us-east-1 CNE. The CNE route-table resolves the workload prefix to the local Production VPC attachment —
hybridalready learned it through the bidirectionalproduction↔hybridshare. (route-table lookup → next hop = local Production attachment) - us-east-1 CNE → Production VPC. The CNE hands the packet to the prod EC2 in the same Region (local-Region delivery). (traffic hop)
Core network policy — what drives this flow
Inbound rides the exact same two lines as Flow 8: the hybrid segment the DXGW lands in, and the production↔hybrid share. The share is bidirectional, so you need no extra inbound policy.
Core network policy that drives this flow (// lines are annotations — the real policy JSON has no comments)
// Two routing domains. The native Direct Connect Gateway attaches to ONE of them.
"segments": [
{ "name": "production", "require-attachment-acceptance": true }, // the workload segment
{ "name": "hybrid", "require-attachment-acceptance": true } // carries the on-prem prefixes
],
// A native DXGW attaches DIRECTLY to the core (no intermediary TGW) and to a SINGLE segment.
// Map it into the hybrid segment, and associate it with EVERY Region that segment is in:
// an un-associated Region inherits a propagated remote route and exits via another Region.
"attachment-policies": [
{
"rule-number": 100, // lowest number wins, first match
"condition-logic": "and",
"conditions": [
{ "type": "attachment-type", "operator": "equals", "value": "direct-connect-gateway" }
],
"action": { "association-method": "constant", "segment": "hybrid" } // DXGW lands in hybrid
}
]"segment-actions": [
{
"action": "share", // plain route-exchange (NOT send-via); no inspection here
"segment": "production", // the workload segment...
"share-with": [ "hybrid" ] //...exchanges routes with the hybrid (on-prem) segment
}
]
// share is BIDIRECTIONAL by default and NON-transitive:
// production learns the on-prem prefixes (this egress flow, Flow 8)
// hybrid learns the production prefixes (the inbound flow, Flow 9)
// Nothing else rides through hybrid, because non-transitive sharing is not a bridge.Flow 10 — VPC to VPC across the federated TGW
production segment — static beats dynamic, so the cutover is safe and the peering hop costs you no per-GB Cloud WAN data-processing charge.How the flow works
- Legacy VPC → Existing Transit Gateway. The legacy VPC’s route table sends the destination prefix to its existing TGW attachment, exactly as it did before the migration. (traffic hop)
- Lookup at the route-table attachment (TGW route table → segment). The TGW route table maps the Cloud WAN-side prefix to the TGW⇄CNE peering, and the route-table attachment files that peering into the
productionsegment. A static route here beats the dynamically-learned Cloud WAN route. (route-table lookup → next hop = CNE peering) - Existing TGW → CNE ⇄ TGW peering → us-east-1 CNE. The packet crosses the same-Region peering onto the Cloud WAN CNE; that peering hop costs you no per-GB Cloud WAN data-processing charge. (traffic hop)
- Lookup inside the us-east-1 CNE. The segment route-table resolves the destination to the local Cloud WAN Production VPC attachment. (route-table lookup → next hop = local Production attachment)
- us-east-1 CNE → Cloud WAN Production VPC. The CNE delivers the packet to the prod EC2 (local-Region delivery). (traffic hop)
Core network policy — what drives this flow
Federation comes down to one mapping line — a route-table attachment that files the TGW peering into a Cloud WAN segment — plus the cutover rule that keeps coexistence safe.
Core network policy that drives this flow (// lines are annotations — the real policy JSON has no comments)
// Federation by PEERING: a CNE peers with the existing TGW in the SAME Region.
// A transit-gateway-route-table attachment extends ONE TGW route table into ONE segment.
"attachment-policies": [
{
"rule-number": 200, // lowest number wins, first match
"condition-logic": "and",
"conditions": [
{ "type": "attachment-type", "operator": "equals", "value": "transit-gateway-route-table" }
],
"action": { "association-method": "constant", "segment": "production" } // legacy VPCs land in production
}
]
// Dynamic BGP only: a static route on this attachment returns an error.
// The TGW ASN must differ from the CNE ASN, or the peering will not form."segments": [
{ "name": "production", "require-attachment-acceptance": true } // the target segment for migrated VPCs
]
// Cutover safety (migration phase 1):
// for the SAME prefix, a STATIC route on the TGW takes precedence
// over the dynamically-learned Cloud WAN route.
// So you bring Cloud WAN up ALONGSIDE the existing TGW with ZERO traffic impact;
// traffic shifts to the core only when you DELETE the TGW static routes. Reversible.Common pitfalls
Key points to remember before building this:
- Home Region vs RAM-share Region. The Cloud WAN home Region is fixed at
us-west-2; but the core network's RAM share must be initiated fromus-east-1, or the other Regions never see the global resource. If this is wrong, your segments will not appear where you need them. - Attachment tags, not VPC tags. Placement reads tags on the attachment object — tagging the VPC does nothing. Rules evaluate lowest-to-highest, first match wins.
- AZ ID, not AZ name. AZ names are randomized per account, so place every shared, inspection, and attachment subnet using a stable AZ ID like
use1-az1. - Segment names are alphanumeric only (
a–z A–Z 0–9, no hyphens) — the diagrams show hyphenated labels, the policy can't. - An empty NFG drops traffic. Reference an NFG that has no attachments and it deploys fine but drops traffic until you add one.
- Ingress cannot use a NAT gateway. NAT can't accept inbound, so an internet-facing LB is mandatory — and ingress is inspected at the VPC boundary, not via service insertion.
- Appliance mode is mandatory and separate from service insertion. Skip it and a cross-AZ return lands on a different firewall and breaks stateful inspection. VPC attachments only.
- Associate the DXGW with every Region the
hybridsegment spans. An unassociated Region inherits a propagated remote route and exits via another Region — asymmetry, and a random forwarding CNE on equal-attribute routes. - Intra-segment inspection needs
isolate-attachments. Without it, same-segment traffic meshes directly and bypasses the inspection path.
References
De-duplicated AWS sources cited above.
- AWS Docs — What is AWS Cloud WAN?
- AWS Docs — Core network policies (JSON)
- AWS Docs — AWS Cloud WAN quotas
- AWS Docs — Service insertion in the core network policy
- AWS Docs — Attachment peerings (Transit Gateway peering)
- AWS Docs — VPC attachments (appliance mode)
- AWS Docs — Direct Connect gateway attachments
- AWS Docs — Share your Cloud WAN network (AWS RAM)
- AWS Docs — AWS Cloud WAN CloudWatch metrics
- AWS RAM Docs — Working with Availability Zone IDs
- AWS Blog — Achieve optimal routing with AWS Cloud WAN for multi-Region networks
- AWS Blog — Simplify global security inspection with AWS Cloud WAN service insertion
- AWS Blog — Centralized ingress inspection architecture in AWS Cloud WAN
- AWS Blog — AWS Cloud WAN and AWS Transit Gateway migration and interoperability patterns
- AWS — AWS Cloud WAN pricing