AWS Cloud WAN series · Companion

The Complete AWS Cloud WAN Reference Architecture: Segments, Inspection, Hybrid, and Every Flow

A complete blueprint for a global, multi-Region (us-east-1 / us-west-2), multi-account core network: segmentation, centralized egress and ingress, east-west inspection, hybrid Direct Connect, and Transit Gateway federation.

TL;DR

  • One Cloud WAN core network across two Regions (us-east-1, us-west-2), one Core Network Edge (CNE) per Region, auto-meshed over the AWS backbone. The hub account owns it and shares it organization-wide with RAM from us-east-1.
  • Five segments (production, nonproduction, pci, sharedservices, hybrid) plus two Network Function Groups form a deny-by-default connectivity matrix.
  • North-south: egress uses send-to an Egress NFG (firewall + NAT + IGW); ingress is inspected at the VPC boundary. East-west: send-via an Inspection NFG. Shared-services is reachable non-transitively.
  • Hybrid: a native Direct Connect Gateway associated to both Regions over primary and backup transit VIFs.
  • Federation: the existing Transit Gateway peers to its same-Region CNE, mapping each route table to a segment.
  • Symmetry: appliance mode pins every stateful flow to one Availability Zone, so the forward and return packets take the same path.

What we're building — and who owns it

The design is one core network across two Regionsus-east-1 and us-west-2 — with a CNE in each. AWS auto-meshes them over its backbone with eBGP, so there's no inter-Region peering for us to configure or manage. This is a key reason we use Cloud WAN rather than building this on Transit Gateway, where inter-Region peering is static-route-only. We are well within the limits: one CNE per Region per core network, one core network per global network.

Each account owns a clear part of the design. Here is who owns what:

AccountOwnsNotes
Network / Hub AccountGlobal network, core network, core network policy, all CNEsBills the CNEs and Connect/VPN attachments; RAM-shares the core network organization-wide from us-east-1
Security / Inspection AccountInspection VPCs (AWS Network Firewall) and egress VPCs (firewall + NAT)Holds the attachments that join the Inspection NFG and Egress NFG
Shared-Services AccountShared-services VPC (DNS, AD, CI, logging)Reachable non-transitively by permitted segments
Workload Account · ProdProduction workload VPCsAttachment tagged into the production segment; pays its own attachment + data
Workload Account · Non-ProdNon-production workload VPCsAttachment tagged into the nonproduction segment

Master topology — the whole design on one page

The master shows all five accounts, two Regions with one CNE each in full mesh, the five segments as colored lanes, the Egress and Inspection NFGs, the native DXGW to two on-prem DCs, and the federated TGW. Think of it as the index every other figure zooms into — topology and the segment-to-color legend only, no single traced flow.

Master topology · index map Segments:productionnon-productionpcishared-serviceshybridService insertion:Ingress / Egress / Inspection NFGSegments are GLOBAL routing domains — the same segment colour spans every Region it has attachments in. Production and Non-Production span BOTH Regions; this is NOT one segment per Region.◆ Single-Region resources: PCI and Shared-Services exist in us-east-1 ONLY. The security stack (Inspection / Ingress / Egress) and the prod / non-prod segments are present in BOTH Regions.Region · us-east-1 · N. VirginiaRegion · us-west-2 · Oregon (CW home)workload segmentssecurity / NFGProduction VPCproductionNon-Production VPCnon-productionPCI VPCpciShared-Services VPCshared-servicesIngress VPCIngress · inspectedEgress VPCEgress NFG · send-toInspection VPCInspection NFG · send-via◆ Single-RegionPCI + Shared-Servicesus-east-1 onlyProduction VPCproductionNon-Production VPCnon-productionIngress VPCIngress · inspectedEgress VPCEgress NFG · send-toInspection VPCInspection NFG · send-viaCloud WAN core network · one global policyCloud WAN CNEus-east-1Cloud WAN CNEus-west-2auto full-mesh eBGPExisting Transit Gatewayfederated by peeringCNE ⇄ TGW peeringDirect Connect Gatewaynative · direct-to-coreassociate BOTH RegionsOn-premises (outside AWS) · hybrid segmentOn-prem DC1primary siteOn-prem DC2backup siteTransit VIF 1 (primary)Transit VIF 2 (backup)Accounts (compact ownership)Network / Hub — owns 2 CNEs · RAM-shared from us-east-1Security / Inspection — Ingress + Egress + Inspection (both)Shared-Services — Shared-Services VPC (us-east-1 only)Workload — Prod / Non-Prod (both) · PCI (us-east-1 only)
The index map: you list your Regions and AWS meshes the edges; native DXGW and a federated TGW both land on the core; the Ingress, Egress and Inspection NFGs are how traffic gets inserted in each Region. Segments are global routing domains, so the same segment colour repeats across Regions wherever it has attachments — production and non-production span both Regions, while pci and shared-services live in us-east-1 only; this is not one segment per Region. Friendly names: us-east-1 = N. Virginia, us-west-2 = Oregon (CW home Region).

How traffic is walled off — segments and the full connectivity matrix

A segment is just a separate routing domain. By default an attachment only reaches others in the same segment — everything else is implicit deny.

Segment budget. Five segments plus two NFGs (each NFG quietly eats one of the 40-segment budget) puts us at 7 of 40 — well within the limit.

A share has three properties. It is:

That non-transitivity is the safety property the whole matrix leans on. Share sharedservices with both production and nonproduction and each can reach it — but you never accidentally bridge prod to non-prod.

Here's the full connectivity matrix. Each off-diagonal cell is allow (a share), blocked (no share — implicit deny), or inspected (steered through the Inspection NFG via send-via, never a plain share). The diagonal is the intra-segment call: isolated means isolate-attachments: true (spoke→hub only).

from ↓ \ to →productionnon-productionpcishared-serviceshybrid
productionisolated + inspectedinspectedblockedallowallow
non-productioninspectedallow (intra)blockedallowallow
pciblockedblockedisolatedinspectedinspected
shared-servicesallowallowinspectedisolatedallow
hybridallowallowinspectedallowallow (intra)

About the diagonals. production, pci, and sharedservices are isolated internally (isolate-attachments: true) — no direct spoke-to-spoke. production also permits specific intra-segment pairs, but only through the Inspection NFG; isolate-attachments is what forces same-segment traffic onto the inspection path instead of letting it mesh directly (Flow 1). nonproduction and hybrid mesh freely.

Design choice: pci reaches sharedservices and hybrid via inspected send-via paths, never a plain share — so the PCI workloads get centralized DNS, logging, and patching plus on-prem reach, and every PCI border crossing still passes through the Inspection NFG. For maximum isolation instead, set both to blocked.

How traffic is inspected — egress, ingress, and east-west service insertion

Service insertion (SI) covers exactly two patterns — and ingress is not one of them:

A few NFG rules worth knowing.

Centralized egress — send-to the local Egress NFG

Egress is symmetric by design. Both Regions run a full local stack — Inspection, Ingress, and Egress VPC — so send-to installs the default routes (0.0.0.0/0 + ::/0) at the local Egress NFG and every workload exits in its own Region via firewall → NAT → IGW (Flow 5).

Edge-override we don't use here. Every Region has its own egress stack, so there's no egress-less Region to pin to a neighbour — we keep it only as a reference for the asymmetric case (Flow 7, greyed), where the non-geography-aware Region-priority list would otherwise send a stackless Region to a far or random exit.

Centralized ingress — at the VPC boundary, not SI

Ingress isn't service insertion — it's inspected at the VPC/IGW boundary (module type ingress_with_inspection, Flow 6): the IGW gateway route table points at a Network Firewall endpoint fronting a public ALB, which forwards over Cloud WAN. The internet-facing LB is mandatory — a NAT gateway can't accept inbound and Cloud WAN has no L4/L7 LB of its own.

East-west — send-via the Inspection NFG, single-hop

East-west between segments goes through the Inspection NFG via send-via, single-hop (Flow 3 inter-segment, Flow 1 intra-segment). send-via is automatically bidirectional, which keeps the route symmetric. We chose single-hop over dual-hop — dual-hop would need an inspection attachment in every Region of each SI-enabled segment and roughly double inspection data-processing.

Two policy rules. Inter-segment (prod↔non-prod) uses send-via instead of a plain share — a share would create uninspected routes. Intra-segment (within production) requires isolate-attachments, or same-segment traffic bypasses inspection.

Appliance mode is mandatory and separate from SI. On the inspection-VPC attachment it pins a flow to one AZ for its lifetime so the return packet lands on the same firewall — skip it and a cross-AZ flow breaks stateful inspection. VPC-attachments only, and you need it and service insertion.

How we reach the rest of the world — Direct Connect hybrid and TGW federation

Native Direct Connect Gateway, associated to both Regions

A native DXGW attaches straight to the core — no intermediary TGW, automatic bidirectional BGP. One DXGW associates with a single core network and a single segment, here hybrid.

We associate it with both Regions the hybrid segment spans. Each associated CNE advertises only local routes and keeps AS_PATH.

Two transit VIFs — VIF 1 primary, VIF 2 backup (on-prem local-preference picks active/passive) — to two data centers, with local-Region preference keeping each Region's on-prem egress local (Flows 8 and 9). The DXGW ASN sits outside the core ASN range; DX is BGP-only (no static routes) and carries no BGP communities.

Federating the existing Transit Gateway

A single global network can hold both Cloud WAN core networks and Transit Gateways — that's what makes federation possible in the first place.

The existing network stays in place. We peer the existing TGW to its same-Region CNE (us-east-1) and map each TGW route table to a matching segment with route-table attachments — one route-table attachment extends one TGW route table to one segment, dynamic-BGP-only, and the TGW ASN must differ from the CNE ASN.

Migration is non-disruptive. For the same prefix, a static TGW route wins over a dynamically-learned Cloud WAN route — so the core stands up right alongside the TGW with zero traffic impact until you pull the static routes (Flow 10).

The flows, hop by hop

Each figure below stands on its own: the numbered hops trace the VPC route-table decision → the segment route-table lookup at the CNE → the next-hop redirect to the inspection/NFG attachment (where it applies) → appliance-mode AZ pinning (where the flow is stateful) → the return path, with symmetry called out explicitly. Wherever a flow could go asymmetric or hit random-remote-CNE behavior, we say exactly how this design prevents it. Each figure is embedded here in its own section and also ships as a standalone file.

Flow 1 — intra-segment, isolated + inspected (within Production)

Active in this flow:productionInspection NFGFlow 1 send-via, single-hop (forward)symmetric returnGreyed = present in the master map, not used by this flow. Intra-segment inspection requires isolate-attachments; single-hop inspects once at one edge.Region · us-east-1 · N. VirginiaRegion · us-west-2 · Oregon (CW home)workload segmentssecurity / NFGProduction VPCproductionNon-Production VPCnon-productionPCI VPCpciShared-Services VPCshared-servicesIngress VPCIngress · inspectedEgress VPCEgress NFG · send-toInspection VPCInspection NFG · send-viaProduction VPCproductionNon-Production VPCnon-productionIngress VPCIngress · inspectedEgress VPCEgress NFG · send-toInspection VPCInspection NFG · send-viaProduction VPCproduction · Prod AProduction VPCproduction · Prod BInspection VPCInspection NFG · send-viaCloud WAN core network · one global policyauto full-mesh eBGPCloud WAN CNEus-east-1Cloud WAN CNEus-west-2isolate → send-viaappliance-mode AZ pinsingle-hop · inspected onceroute-table → Prod B (production)123456traffic hop — a packet moving on a wire (VPC→CNE, CNE→inspection, inspection→CNE, CNE→VPC)lookup / decision inside a CNE (route-table) — which next-hop the CNE selects; no new wire
Intra-segment traffic is normally direct; isolating the production segment kills that route, so the same-segment send-via is the only path left — it inspects Prod A to Prod B once and returns symmetrically through the same firewall.

How the flow works

  1. Production VPC · Prod A (us-east-1) → us-east-1 CNE. Prod A’s route to Prod B’s CIDR points at its Cloud WAN attachment. (traffic hop)
  2. Lookup inside the us-east-1 CNE (pre-inspection). Because the production segment sets isolate-attachments: true, there's no direct spoke-to-spoke route between Prod A and Prod B; the only route resolving Prod B’s CIDR is the same-segment send-via, which steers the flow to the Inspection NFG. (route-table lookup → next hop = Inspection NFG)
  3. us-east-1 CNE → Inspection VPC. The core carries the packet to the firewall once (single-hop); appliance mode pins the AZ so both directions ride one ENI. (traffic hop)
  4. Inspection VPC → back into the us-east-1 CNE. Once the firewall passes it, the packet is re-injected into the same us-east-1 CNE. (traffic hop)
  5. Lookup inside the us-east-1 CNE (post-inspection). The route-table now resolves Prod B’s CIDR to Prod B’s attachment in the same Region. (route-table lookup → next hop = Prod B attachment)
  6. us-east-1 CNE → Production VPC · Prod B. The CNE delivers the inspected packet to Prod B (local-Region delivery). (traffic hop)

Core network policy that drives this flow (// lines are annotations — the real policy JSON has no comments)

// Intra-segment inspection: the segment must be ISOLATED first,
// otherwise attachments in the same segment reach each other directly.
"segments": [
 { "name": "production",
 "require-attachment-acceptance": true, // attachments must be approved to join
 "isolate-attachments": true } // drop spoke-to-spoke routes; keep only shared / send-via routes
],
// The inspection layer. The Inspection VPC attaches HERE, not to the workload segment.
"network-function-groups": [
 { "name": "InspectionVPC", "require-attachment-acceptance": true }
]
"segment-actions": [
 {
 "action": "send-via", // steer matching traffic THROUGH an NFG, then re-enter the core
 "segment": "production", // applies to traffic in the production segment
 "mode": "single-hop", // inspect once, at one edge
 "when-sent-to": { "segments": ["production"] }, // SAME segment: prod-to-prod is what gets inspected
 "via": { "network-function-groups": ["InspectionVPC"] } // route it via this inspection NFG
 }
]
// send-via is bidirectional automatically, so the return is inspected too.

Flow 2 — prod to PCI is denied (the canonical blocked flow)

Active in this flow:productionpciFlow 2 attempt (forward)DENIED — no route to pci (no share, no send-via)Greyed = present in the master map, not used by this flow. The policy has NO action linking production and pci, so the lookup finds no route — implicit deny.Region · us-east-1 · N. VirginiaRegion · us-west-2 · Oregon (CW home)workload segmentssecurity / NFGProduction VPCproductionNon-Production VPCnon-productionPCI VPCpciShared-Services VPCshared-servicesIngress VPCIngress · inspectedEgress VPCEgress NFG · send-toInspection VPCInspection NFG · send-viaProduction VPCproductionNon-Production VPCnon-productionIngress VPCIngress · inspectedEgress VPCEgress NFG · send-toInspection VPCInspection NFG · send-viaProduction VPCproductionPCI VPCpciCloud WAN core network · one global policyauto full-mesh eBGPCloud WAN CNEus-east-1Cloud WAN CNEus-west-2route-table: no pci routeDENIED — no route to pcino share · no send-via12traffic hop — a packet moving on a wire (VPC→CNE, CNE→inspection, inspection→CNE, CNE→VPC)lookup / decision inside a CNE (route-table) — which next-hop the CNE selects; no new wire
The packet reaches the CNE and the lookup runs, but the production route-table has no entry for the PCI CIDR — no share, no send-via — so it's dropped. There's no delivery hop; the policy denies by omission.

How the flow works

  1. Production VPC → us-east-1 CNE. The prod workload fires a packet for the PCI CIDR at its Cloud WAN attachment. (traffic hop)
  2. Lookup inside the us-east-1 CNE — denied. The production route-table finds no route to the PCI CIDR: the policy declares production and pci as separate segments with no share and no send-via between them, so the packet is dropped. (route-table lookup → no route → implicit deny; no delivery hop)

Core network policy that drives this flow (// lines are annotations — the real policy JSON has no comments)

// Both segments exist; segments default to implicit-deny, same-segment only.
"segments": [
 { "name": "production", "require-attachment-acceptance": true },
 { "name": "pci", "require-attachment-acceptance": true }
]
"segment-actions": [
 //... actions may exist for OTHER segment pairs...
 // but there is NO share and NO send-via between production and pci
]
// No share or send-via between production and pci => implicit deny.
// The us-east-1 CNE route-table simply has no route to the pci CIDR.

Flow 3 — prod ↔ non-prod, inspected, cross-Region

Flow 3 · prod ⇄ non-prod inter-segment, inspected, cross-Region Active in this flow:productionnon-productionInspection NFGFlow 3 send-via single-hop (forward)symmetric returnGreyed = present in the master map, not used by this flow. Single-hop inspects once; edge-override pins the inspection edge to us-east-1.Region · us-east-1 · N. VirginiaRegion · us-west-2 · Oregon (CW home)workload segmentssecurity / NFGNon-Production VPCnon-productionPCI VPCpciShared-Services VPCshared-servicesIngress VPCIngress · inspectedEgress VPCEgress NFG · send-toProduction VPCproductionIngress VPCIngress · inspectedEgress VPCEgress NFG · send-toInspection VPCInspection NFG · send-viaProduction VPCproductionInspection VPCInspection NFG · send-viaNon-Production VPCnon-productionCloud WAN core network · one global policyauto full-mesh eBGPCloud WAN CNEus-east-1Cloud WAN CNEus-west-2appliance-mode AZ pinsingle-hop · inspected onceroute-table → send-viaroute-table → non-prod (us-west-2)cross-Region over CW coredeliver to local non-prod12345678traffic hop — a packet moving on a wire (VPC→CNE, CNE→inspection, inspection→CNE, CNE→VPC)lookup / decision inside a CNE (route-table) — which next-hop the CNE selects; no new wire
Cross-segment east-west is one inspected hop: send-via pulls the flow through a single Inspection NFG (pinned to us-east-1 by edge-override, appliance-mode AZ-pinned), and the return re-uses the same attachment — so it's symmetric and never lands on a random remote CNE.

How the flow works

  1. Production VPC (us-east-1) → us-east-1 CNE. The prod EC2's route to the non-prod CIDR points at its us-east-1 Cloud WAN attachment; the workload has no idea inspection is about to happen. (traffic hop)
  2. Lookup inside the us-east-1 CNE (pre-inspection). The production route-table has no share to nonproduction, so the only route resolving the non-prod CIDR is a send-via action: it steers the flow to the Inspection NFG, with edge-override pinning inspection to the us-east-1 edge. (route-table lookup → next hop = Inspection NFG)
  3. us-east-1 CNE → Inspection VPC (Network Firewall). The core carries the packet to the firewall once (single-hop); appliance mode pins the AZ so both directions ride one ENI and the return lands on the same firewall. (traffic hop)
  4. Inspection VPC → back into the us-east-1 CNE. Once the firewall passes it, the packet is re-injected into the same us-east-1 CNE. (traffic hop)
  5. Lookup inside the us-east-1 CNE (post-inspection). The non-production CIDR resolves to the non-prod attachment in us-west-2, so the next hop is the us-west-2 CNE across the auto-meshed backbone — this is the step that carries the inspected packet to the other Region. (route-table lookup → next hop = us-west-2 CNE)
  6. us-east-1 CNE → us-west-2 CNE. The packet crosses Region over the Cloud WAN core — a full-mesh eBGP fabric between every CNE. (traffic hop, cross-Region)
  7. Lookup inside the us-west-2 CNE. The us-west-2 CNE route-table resolves the destination to the local non-production attachment. (route-table lookup → next hop = local non-prod VPC)
  8. us-west-2 CNE → Non-Production VPC. The CNE delivers the packet to the non-prod EC2 (local-Region delivery). (traffic hop)

Core network policy — the segment-action that drives this flow

Core network policy that drives this flow (// lines are annotations — the real policy JSON has no comments)

// Two isolated routing domains; by default they cannot reach each other.
"segments": [
 { "name": "production", "require-attachment-acceptance": true }, // attachments must be approved to join
 { "name": "nonproduction", "require-attachment-acceptance": true }
],
// The inspection layer. The Inspection VPC attaches HERE, not to a workload segment.
"network-function-groups": [
 { "name": "InspectionVPC", "require-attachment-acceptance": true }
]
"segment-actions": [
 {
 "action": "send-via", // steer matching traffic THROUGH an NFG (not a plain share)
 "segment": "production", // applies to traffic originating in production
 "mode": "single-hop", // inspect once, at one edge (not at source and dest both)
 "when-sent-to": { "segments": ["nonproduction"] }, // only prod-to-nonprod traffic is inspected
 "via": {
 "network-function-groups": ["InspectionVPC"], // route it through this inspection NFG
 "with-edge-overrides": [
 // pin the inspection edge: traffic that would use us-west-2 inspects at us-east-1
 { "edge-sets": [["us-west-2"]], "use-edge-location": "us-east-1" }
 ]
 }
 }
]

Flow 4 — workloads reach shared-services, non-transitively

Active in this flow:productionnon-productionshared-servicesFlow 4 share route (forward)non-transitive — no prod ⇄ non-prod bridgeGreyed = present in the master map, not used by this flow. A share is bidirectional but NON-transitive: each spoke reaches shared-services, never each other.Region · us-east-1 · N. VirginiaRegion · us-west-2 · Oregon (CW home)workload segmentssecurity / NFGProduction VPCproductionNon-Production VPCnon-productionPCI VPCpciShared-Services VPCshared-servicesIngress VPCIngress · inspectedEgress VPCEgress NFG · send-toInspection VPCInspection NFG · send-viaProduction VPCproductionNon-Production VPCnon-productionIngress VPCIngress · inspectedEgress VPCEgress NFG · send-toInspection VPCInspection NFG · send-viaProduction VPCproductionNon-Production VPCnon-productionShared-Services VPCshared-servicesCloud WAN core network · one global policyauto full-mesh eBGPCloud WAN CNEus-east-1Cloud WAN CNEus-west-2route-table → shareshare → shared-servicesno bridge · non-transitive123456traffic hop — a packet moving on a wire (VPC→CNE, CNE→inspection, inspection→CNE, CNE→VPC)lookup / decision inside a CNE (route-table) — which next-hop the CNE selects; no new wire
Two independent flows resolve the same share to shared-services — prod and non-prod both reach the hub, but the red marker is the point: the share installs no prod-to-non-prod route, so the spokes never reach each other.

How the flow works

  1. Production VPC → us-east-1 CNE. The prod workload's route to the shared-services CIDR points at its Cloud WAN attachment. (traffic hop)
  2. Lookup inside the us-east-1 CNE (production). The production route-table resolves the shared-services CIDR through the share linking production and sharedservices; next hop is the Shared-Services attachment. (route-table lookup → next hop = Shared-Services)
  3. us-east-1 CNE → Shared-Services VPC. The CNE delivers the prod packet to the shared-services hub. (traffic hop)
  4. Non-Production VPC → us-east-1 CNE. Separately, the non-prod workload also routes to the shared-services CIDR via its attachment. (traffic hop)
  5. Lookup inside the us-east-1 CNE (nonproduction). The nonproduction route-table resolves the same CIDR through the share linking nonproduction and sharedservices. (route-table lookup → next hop = Shared-Services)
  6. us-east-1 CNE → Shared-Services VPC. The CNE delivers the non-prod packet to the same shared-services hub. (traffic hop)

Core network policy that drives this flow (// lines are annotations — the real policy JSON has no comments)

// Three segments; by default none reach each other.
"segments": [
 { "name": "production", "require-attachment-acceptance": true },
 { "name": "nonproduction", "require-attachment-acceptance": true },
 { "name": "sharedservices", "require-attachment-acceptance": true }
]
// Segment names are alphanumeric only; display labels add the hyphens.
"segment-actions": [
 {
 "action": "share", // exchange attachment routes between segments
 "mode": "attachment-route", // shares the attachment + return routes only
 "segment": "sharedservices", // the hub segment
 "share-with": ["production", "nonproduction"] // prod and nonprod each pair with sharedservices
 }
]
// A share is BIDIRECTIONAL but NON-transitive: prod and nonprod each
// reach sharedservices, but this installs NO prod to nonprod route.

Flow 5 — a workload reaches the internet, same Region

Flow 5 · workload → internet egress, same-Region (send-to)Active in this flow:productionEgress NFGFlow 5 send-to egress (forward)symmetric returnGreyed = present in the master map, not used by this flow. send-to inspects then exits at the LOCAL Egress NFG — no cross-Region edge.Region · us-east-1 · N. VirginiaRegion · us-west-2 · Oregon (CW home)workload segmentssecurity / NFGNon-Production VPCnon-productionPCI VPCpciShared-Services VPCshared-servicesIngress VPCIngress · inspectedInspection VPCInspection NFG · send-viaProduction VPCproductionNon-Production VPCnon-productionIngress VPCIngress · inspectedEgress VPCEgress NFG · send-toInspection VPCInspection NFG · send-viaCloud WAN CNEus-west-2Production VPCproductionEgress VPCEgress NFG · send-toCloud WAN core network · one global policyauto full-mesh eBGPCloud WAN CNEus-east-1Cloud WAN CNEus-west-2Internetroute-table → send-to → Egress NFGappliance-mode AZ pininspect → NAT → IGWleaves AWS1234traffic hop — a packet moving on a wire (VPC→CNE, CNE→Egress NFG, NAT→IGW→internet)lookup / decision inside a CNE (route-table) — which next-hop the CNE selects; no new wire
Same-Region egress is two CNE-local steps: the production segment route-table carries a default route a send-to action installed, hands the packet to the local Egress NFG, which inspects then NAT/IGW out — and the reply re-uses the same path, so it's symmetric.

How the flow works

  1. Production VPC (us-east-1) → us-east-1 CNE. The workload EC2's default route (0.0.0.0/0) points at its us-east-1 Cloud WAN attachment. (traffic hop)
  2. Lookup inside the us-east-1 CNE. A send-to action already put the defaults 0.0.0.0/0 and ::/0 into the production segment with the local Egress NFG attachment as next hop, so 0.0.0.0/0 resolves to the Egress NFG in this same Region. (route-table lookup → next hop = Egress NFG)
  3. us-east-1 CNE → Egress VPC. The core carries the packet to the Egress VPC firewall; appliance mode pins the AZ so the return lands on the same firewall. (traffic hop)
  4. Egress VPC → internet. After inspection the NAT gateway source-NATs and the IGW sends the packet out to the public internet. (traffic hop)

The reply (dashed) re-uses the same Egress NFG attachment and firewall AZ, so the whole conversation is symmetric. And because send-to installs the defaults with the local NFG as next hop, every Region with its own Egress VPC exits in-Region — no cross-Region edge here.

Core network policy — the config that drives this flow

Egress is north-south service insertion — one send-to action on the production segment. It takes no mode and installs the defaults 0.0.0.0/0 and ::/0 into the source segment, next hop the local Egress NFG attachment.

Core network policy that drives this flow (// lines are annotations — the real policy JSON has no comments)

// Segments are isolated routing domains; the Egress VPC attaches to an NFG, not a segment.
"segments": [
 { "name": "production", "require-attachment-acceptance": true } // workloads live here
],
"network-function-groups": [
 { "name": "EgressVPC", "require-attachment-acceptance": true } // the egress / NAT inspection layer
]
"segment-actions": [
 {
 "action": "send-to", // north-south service insertion: inspect, then exit
 "segment": "production", // applies to traffic leaving the production segment
 "via": {
 "network-function-groups": ["EgressVPC"] // exit via the local Egress NFG attachment
 }
 // no "mode" on send-to. It installs default routes 0.0.0.0/0 AND::/0 into
 // the production segment, next hop = the Egress NFG attachment in the SAME Region.
 }
]

Read it as: any production traffic to 0.0.0.0/0 is steered to the local EgressVPC NFG, inspected, then NAT/IGW out — and because the next hop is the local NFG, each Region exits in-Region.

Flow 6 — the internet reaches a workload (ingress)

Flow 6 · internet → workload ingress, inspected at the VPC boundaryActive in this flow:productionIngress VPCFlow 6 ingress (forward)symmetric returnIngress is decided at the VPC / IGW boundary (NOT a Cloud WAN segment-action). A NAT gateway cannot accept inbound — an internet-facing load balancer is mandatory.Region · us-east-1 · N. VirginiaRegion · us-west-2 · Oregon (CW home)workload segmentssecurity / NFGNon-Production VPCnon-productionPCI VPCpciShared-Services VPCshared-servicesEgress VPCEgress NFG · send-toInspection VPCInspection NFG · send-viaProduction VPCproductionNon-Production VPCnon-productionIngress VPCIngress · inspectedEgress VPCEgress NFG · send-toInspection VPCInspection NFG · send-viaCloud WAN CNEus-west-2Production VPCproductionIngress VPCIGW → NFW → ALBCloud WAN core network · one global policyauto full-mesh eBGPCloud WAN CNEus-east-1Cloud WAN CNEus-west-2InternetIGW route table → NFW endpoint → public ALBroute-table → production attachmentover Cloud WAN1234traffic hop — a packet moving on a wire (internet→Ingress VPC, Ingress VPC→CNE, CNE→workload VPC)lookup / decision inside a CNE (route-table) — which next-hop the CNE selects; no new wire
Ingress is decided at the VPC boundary, not by Cloud WAN: the IGW edge route table sends inbound through the Network Firewall to a public ALB, and only the final delivery is a normal Cloud WAN segment lookup. A NAT gateway can't accept inbound, so the load balancer is mandatory.

How the flow works

  1. Internet → Ingress VPC (us-east-1). Inbound hits the internet gateway. The IGW edge route table forces it to the Network Firewall endpoint first, then a public ALB — a NAT gateway can't accept inbound, so an internet-facing load balancer is mandatory. (traffic hop)
  2. Ingress VPC → us-east-1 CNE. Behind the ALB the target is a workload in another VPC, so the packet is handed to the local Cloud WAN attachment. (traffic hop)
  3. Lookup inside the us-east-1 CNE. This is the only Cloud WAN decision in the whole flow: the segment route-table resolves the workload CIDR to the production attachment. (route-table lookup → next hop = production VPC)
  4. us-east-1 CNE → Production VPC. The CNE delivers to the workload (local-Region delivery). (traffic hop)

So, ingress is not service insertion. SI (send-to / send-via) covers only east-west and internet egress. Inbound gets inspected at the VPC / IGW boundary — IGW route table → Network Firewall endpoint → public ALB — and only then rides Cloud WAN. The core network policy never steers ingress.

Core network policy — the config that drives this flow

There's no ingress action in the Cloud WAN policy at all. What drives this is the VPC-level IGW ingress route table pointing inbound at the firewall endpoint; the core network policy only attaches the Ingress VPC to a segment.

Core network policy that drives this flow (// lines are annotations — the real policy JSON has no comments)

// VPC-level config — NOT part of the Cloud WAN core network policy.
// The internet-gateway "edge association" route table inspects inbound at the boundary:
IGW ingress route table (associated with the Internet Gateway):
 destination = 
 target =  // force inbound THROUGH the firewall...
 //... the firewall forwards to the public ALB, which load-balances to the workload.
 // A NAT gateway cannot accept inbound — the internet-facing ALB is mandatory.
// In the CORE NETWORK POLICY the Ingress VPC is just a normal segment attachment.
// Nothing here steers ingress — there is NO send-to / send-via for inbound.
"attachment-policies": [
 {
 "rule-number": 100,
 "condition-logic": "and",
 "conditions": [ { "type": "tag-value", "key": "segment", "value": "production" } ],
 "action": { "association-method": "constant", "segment": "production" }
 }
]
// NOTE: the core network policy does NOT steer ingress; inbound is decided above.

Read it as: inbound is inspected and load-balanced at the VPC / IGW boundary (top tab), then handed to Cloud WAN for one ordinary segment lookup to the workload (bottom tab). The policy attaches the VPC; it does not route inbound.

Flow 7 — edge-override for a Region with no local egress stack (reference pattern, not used here)

Flow 7 · egress via edge-override (asymmetric reference)Active in this flow:productionEgress NFGFlow 7 edge-override egress (forward)symmetric returnREFERENCE / asymmetric case — NOT used in this symmetric 2-Region design where every Region has local egress. Shown for a Region with no egress stack.Region · us-east-1 · N. VirginiaRegion · us-west-2 · Oregon (CW home)workload segmentssecurity / NFGus-west-2 drawn WITHOUT a local egress stack (reference only)Production VPCproductionNon-Production VPCnon-productionPCI VPCpciShared-Services VPCshared-servicesIngress VPCIngress · inspectedInspection VPCInspection NFG · send-viaNon-Production VPCnon-productionIngress VPCIngress · inspectedInspection VPCInspection NFG · send-viaEgress VPCEgress NFG · send-to✕ no local egress (asymmetric case)Production VPCproductionEgress VPCEgress NFG · send-toCloud WAN core network · one global policyauto full-mesh eBGPCloud WAN CNEus-east-1Cloud WAN CNEus-west-2Internetno local egressedge-override → us-east-1cross-Region over CW coredeliver → local Egress NFGsend-to → NAT → IGW12345traffic hop — a packet moving on a wire (VPC→CNE, CNE→CNE cross-Region, CNE→Egress NFG, NAT→IGW→internet)lookup / decision inside a CNE (route-table) — which next-hop the CNE selects; no new wire
Reference case only: a Region with no local egress stack falls back to edge-override, which pins its internet exit to the us-east-1 Egress NFG. Our symmetric 2-Region design never hits this — every Region has local egress.

How the flow works

  1. Production VPC (us-west-2) → us-west-2 CNE. The workload’s 0.0.0.0/0 default points at its us-west-2 Cloud WAN attachment. (traffic hop)
  2. Lookup inside the us-west-2 CNE — no local egress. This Region has no Egress NFG attachment, so the propagated default has nowhere local to go. The send-to action’s with-edge-overrides clause pins the exit: edge-override picks us-east-1. (route-table lookup → next hop = us-east-1 CNE)
  3. us-west-2 CNE → us-east-1 CNE. The packet crosses Region over the Cloud WAN core (full-mesh eBGP backbone) to reach the edge that owns the egress stack. (traffic hop, cross-Region)
  4. Lookup inside the us-east-1 CNE. At the pinned edge the default resolves to the local Egress NFG attachment. (route-table lookup → next hop = Egress NFG)
  5. us-east-1 CNE → Egress VPC → internet. The Egress VPC firewall inspects, NAT and IGW send it out; the reply (dashed) re-uses the same pinned edge and AZ. (traffic hop)

Again, we don't use this in the symmetric 2-Region design. Every Region here runs its own local egress stack, so egress is just the Flow 5 picture (local send-to, no cross-Region edge). This figure is the asymmetric case — a Region with no egress VPC — that with-edge-overrides patches up, since the default Region-priority list isn't geography-aware.

Core network policy — the config that drives this flow

It's still the egress send-to action, just with a with-edge-overrides clause bolted on. The override keys are edge-sets (the edge with no local egress) and use-edge-location (the edge that inspects and exits instead).

Core network policy that drives this flow (// lines are annotations — the real policy JSON has no comments)

"network-function-groups": [
 { "name": "EgressVPC", "require-attachment-acceptance": true } // ONLY attached in us-east-1 here
],
"segments": [
 { "name": "production", "require-attachment-acceptance": true } // spans both Regions
]
// us-west-2 has NO Egress NFG attachment, so its default route has no local exit.
"segment-actions": [
 {
 "action": "send-to", // north-south egress insertion (no "mode")
 "segment": "production",
 "via": {
 "network-function-groups": ["EgressVPC"],
 "with-edge-overrides": [
 // the default Region-priority list is NOT geography-aware; pin the exit:
 { "edge-sets": [["us-west-2"]], // traffic that would exit at us-west-2...
 "use-edge-location": "us-east-1" } //... inspects and exits at us-east-1 instead
 ]
 }
 }
]

Read it as: traffic that would exit at us-west-2 is forced to inspect and exit at the us-east-1 EgressVPC NFG instead — a fix for the asymmetric case, not part of the symmetric design.

Flow 8 — workload to on-prem over the DXGW, with return symmetry

Flow 8 · workload → on-prem via native DXGW, exit + return symmetric Active in this flow:productionhybridFlow 8 egress via DXGW (forward)symmetric returnGreyed = present in the master map, not used by this flow. The DXGW associates to BOTH Regions, so us-east-1 exits locally; AS_PATH is retained so the return lands in the same Region.Region · us-east-1 · N. VirginiaRegion · us-west-2 · Oregon (CW home)workload segmentssecurity / NFGNon-Production VPCnon-productionPCI VPCpciShared-Services VPCshared-servicesIngress VPCIngress · inspectedEgress VPCEgress NFG · send-toInspection VPCInspection NFG · send-viaProduction VPCproductionNon-Production VPCnon-productionIngress VPCIngress · inspectedEgress VPCEgress NFG · send-toInspection VPCInspection NFG · send-viaCloud WAN CNEus-west-2Existing Transit Gatewayfederated by peeringOn-prem DC2backup siteCNE peering (unused)Cloud WAN core network · one global policyauto full-mesh eBGPProduction VPCproductionCloud WAN CNEus-east-1Direct Connect Gatewaynative · direct-to-coreOn-premises (outside AWS) · hybrid segmentOn-prem DC1primary siteassociate BOTH RegionsTransit VIF 1 (primary)segment RT → DXGW (local)local-Region preferred · AS_PATH retainedVIF 1 primary · VIF 2 backup123traffic hop — a packet moving on a wirelookup / decision inside a CNE (route-table) — which next-hop the CNE selects; no new wire
The DXGW is associated to both Regions, so the us-east-1 CNE owns a local on-prem route and exits over Transit VIF 1; AS_PATH is kept, so on-prem comes back to the same Region — exit and return ride one CNE and one VIF, never a random remote edge.

How the flow works

  1. Production VPC (us-east-1) → us-east-1 CNE. The prod EC2's route to the on-prem CIDR points at its local us-east-1 Cloud WAN attachment. (traffic hop)
  2. Lookup inside the us-east-1 CNE. The on-prem prefix lives in the hybrid segment, which production shares with; because the native DXGW is associated to this Region, the route resolves to the local DXGW attachment — never cross-Region. (route-table lookup → next hop = local DXGW attachment)
  3. us-east-1 CNE → DXGW → Transit VIF 1 → On-prem DC1. The CNE forwards to the native Direct Connect Gateway, which prefers Transit VIF 1 (primary) to DC1; VIF 2 stays passive as backup to DC2. (traffic hop)

Core network policy — what drives this flow

On-prem reachability comes down to two policy lines: the hybrid segment the DXGW lands in, and a share that joins production to hybrid. No send-via here — this is plain routing, not inspection.

Core network policy that drives this flow (// lines are annotations — the real policy JSON has no comments)

// Two routing domains. The native Direct Connect Gateway attaches to ONE of them.
"segments": [
 { "name": "production", "require-attachment-acceptance": true }, // the workload segment
 { "name": "hybrid", "require-attachment-acceptance": true } // carries the on-prem prefixes
],
// A native DXGW attaches DIRECTLY to the core (no intermediary TGW) and to a SINGLE segment.
// Map it into the hybrid segment, and associate it with EVERY Region that segment is in:
// an un-associated Region inherits a propagated remote route and exits via another Region.
"attachment-policies": [
 {
 "rule-number": 100, // lowest number wins, first match
 "condition-logic": "and",
 "conditions": [
 { "type": "attachment-type", "operator": "equals", "value": "direct-connect-gateway" }
 ],
 "action": { "association-method": "constant", "segment": "hybrid" } // DXGW lands in hybrid
 }
]
"segment-actions": [
 {
 "action": "share", // plain route-exchange (NOT send-via); no inspection here
 "segment": "production", // the workload segment...
 "share-with": [ "hybrid" ] //...exchanges routes with the hybrid (on-prem) segment
 }
]
// share is BIDIRECTIONAL by default and NON-transitive:
// production learns the on-prem prefixes (this egress flow, Flow 8)
// hybrid learns the production prefixes (the inbound flow, Flow 9)
// Nothing else rides through hybrid, because non-transitive sharing is not a bridge.

Flow 9 — on-prem reaches a workload (inbound over DXGW)

Flow 9 · on-prem → workload inbound, mirror of Flow 8 Active in this flow:productionhybridFlow 9 inbound over DXGW (forward)symmetric returnGreyed = present in the master map, not used by this flow. Inbound is the mirror of Flow 8: the same local us-east-1 CNE and the same Transit VIF 1 carry the reply, so the round trip stays in-Region.Region · us-east-1 · N. VirginiaRegion · us-west-2 · Oregon (CW home)workload segmentssecurity / NFGNon-Production VPCnon-productionPCI VPCpciShared-Services VPCshared-servicesIngress VPCIngress · inspectedEgress VPCEgress NFG · send-toInspection VPCInspection NFG · send-viaProduction VPCproductionNon-Production VPCnon-productionIngress VPCIngress · inspectedEgress VPCEgress NFG · send-toInspection VPCInspection NFG · send-viaCloud WAN CNEus-west-2Existing Transit Gatewayfederated by peeringOn-prem DC2backup siteCNE peering (unused)Cloud WAN core network · one global policyauto full-mesh eBGPProduction VPCproductionCloud WAN CNEus-east-1Direct Connect Gatewaynative · direct-to-coreOn-premises (outside AWS) · hybrid segmentOn-prem DC1primary siteassociate BOTH RegionsTransit VIF 1 (primary)segment RT → productionlocal-Region preferred · AS_PATH retainedVIF 1 primary · VIF 2 backup123traffic hop — a packet moving on a wirelookup / decision inside a CNE (route-table) — which next-hop the CNE selects; no new wire
Inbound is just Flow 8 run backwards: DC1 reaches the workload over the same Transit VIF 1 and the same local us-east-1 CNE, and the reply retraces it — so the round trip never leaves the Region.

How the flow works

  1. On-prem DC1 → Transit VIF 1 → DXGW → us-east-1 CNE. DC1 advertises its prefixes over Transit VIF 1 into the Direct Connect Gateway; because the DXGW is associated to us-east-1, those routes land on the local CNE. (traffic hop)
  2. Lookup inside the us-east-1 CNE. The CNE route-table resolves the workload prefix to the local Production VPC attachment — hybrid already learned it through the bidirectional productionhybrid share. (route-table lookup → next hop = local Production attachment)
  3. us-east-1 CNE → Production VPC. The CNE hands the packet to the prod EC2 in the same Region (local-Region delivery). (traffic hop)

Core network policy — what drives this flow

Inbound rides the exact same two lines as Flow 8: the hybrid segment the DXGW lands in, and the productionhybrid share. The share is bidirectional, so you need no extra inbound policy.

Core network policy that drives this flow (// lines are annotations — the real policy JSON has no comments)

// Two routing domains. The native Direct Connect Gateway attaches to ONE of them.
"segments": [
 { "name": "production", "require-attachment-acceptance": true }, // the workload segment
 { "name": "hybrid", "require-attachment-acceptance": true } // carries the on-prem prefixes
],
// A native DXGW attaches DIRECTLY to the core (no intermediary TGW) and to a SINGLE segment.
// Map it into the hybrid segment, and associate it with EVERY Region that segment is in:
// an un-associated Region inherits a propagated remote route and exits via another Region.
"attachment-policies": [
 {
 "rule-number": 100, // lowest number wins, first match
 "condition-logic": "and",
 "conditions": [
 { "type": "attachment-type", "operator": "equals", "value": "direct-connect-gateway" }
 ],
 "action": { "association-method": "constant", "segment": "hybrid" } // DXGW lands in hybrid
 }
]
"segment-actions": [
 {
 "action": "share", // plain route-exchange (NOT send-via); no inspection here
 "segment": "production", // the workload segment...
 "share-with": [ "hybrid" ] //...exchanges routes with the hybrid (on-prem) segment
 }
]
// share is BIDIRECTIONAL by default and NON-transitive:
// production learns the on-prem prefixes (this egress flow, Flow 8)
// hybrid learns the production prefixes (the inbound flow, Flow 9)
// Nothing else rides through hybrid, because non-transitive sharing is not a bridge.

Flow 10 — VPC to VPC across the federated TGW

Flow 10 · VPC → VPC across the federated TGW, cutover-safe Active in this flow:productionExisting TGWFlow 10 VPC→VPC via federated TGW (forward)symmetric returnGreyed = present in the master map, not used by this flow. During coexistence a STATIC route on the TGW beats the dynamic Cloud WAN route, so the cutover is safe and reversible.Region · us-east-1 · N. VirginiaRegion · us-west-2 · Oregon (CW home)workload segmentssecurity / NFGNon-Production VPCnon-productionPCI VPCpciShared-Services VPCshared-servicesIngress VPCIngress · inspectedEgress VPCEgress NFG · send-toInspection VPCInspection NFG · send-viaProduction VPCproductionNon-Production VPCnon-productionIngress VPCIngress · inspectedEgress VPCEgress NFG · send-toInspection VPCInspection NFG · send-viaCloud WAN CNEus-west-2Direct Connect Gatewaynative · direct-to-coreOn-premises (outside AWS) · hybrid segmentOn-prem DC1primary siteOn-prem DC2backup siteCloud WAN core network · one global policyauto full-mesh eBGPProduction VPCCloud WAN · productionCloud WAN CNEus-east-1Existing Transit Gatewayfederated by peeringLegacy VPCbehind Existing TGWCNE ⇄ TGW peeringTGW route-table → productionsegment RT → production (deliver)static beats dynamic (cutover-safe)12345traffic hop — a packet moving on a wirelookup / decision at a CNE or TGW route-table attachment — which next-hop is selected; no new wire
During coexistence the legacy TGW points a static route at the Cloud WAN peering, and a route-table attachment files that peering into the production segment — static beats dynamic, so the cutover is safe and the peering hop costs you no per-GB Cloud WAN data-processing charge.

How the flow works

  1. Legacy VPC → Existing Transit Gateway. The legacy VPC’s route table sends the destination prefix to its existing TGW attachment, exactly as it did before the migration. (traffic hop)
  2. Lookup at the route-table attachment (TGW route table → segment). The TGW route table maps the Cloud WAN-side prefix to the TGW⇄CNE peering, and the route-table attachment files that peering into the production segment. A static route here beats the dynamically-learned Cloud WAN route. (route-table lookup → next hop = CNE peering)
  3. Existing TGW → CNE ⇄ TGW peering → us-east-1 CNE. The packet crosses the same-Region peering onto the Cloud WAN CNE; that peering hop costs you no per-GB Cloud WAN data-processing charge. (traffic hop)
  4. Lookup inside the us-east-1 CNE. The segment route-table resolves the destination to the local Cloud WAN Production VPC attachment. (route-table lookup → next hop = local Production attachment)
  5. us-east-1 CNE → Cloud WAN Production VPC. The CNE delivers the packet to the prod EC2 (local-Region delivery). (traffic hop)

Core network policy — what drives this flow

Federation comes down to one mapping line — a route-table attachment that files the TGW peering into a Cloud WAN segment — plus the cutover rule that keeps coexistence safe.

Core network policy that drives this flow (// lines are annotations — the real policy JSON has no comments)

// Federation by PEERING: a CNE peers with the existing TGW in the SAME Region.
// A transit-gateway-route-table attachment extends ONE TGW route table into ONE segment.
"attachment-policies": [
 {
 "rule-number": 200, // lowest number wins, first match
 "condition-logic": "and",
 "conditions": [
 { "type": "attachment-type", "operator": "equals", "value": "transit-gateway-route-table" }
 ],
 "action": { "association-method": "constant", "segment": "production" } // legacy VPCs land in production
 }
]
// Dynamic BGP only: a static route on this attachment returns an error.
// The TGW ASN must differ from the CNE ASN, or the peering will not form.
"segments": [
 { "name": "production", "require-attachment-acceptance": true } // the target segment for migrated VPCs
]
// Cutover safety (migration phase 1):
// for the SAME prefix, a STATIC route on the TGW takes precedence
// over the dynamically-learned Cloud WAN route.
// So you bring Cloud WAN up ALONGSIDE the existing TGW with ZERO traffic impact;
// traffic shifts to the core only when you DELETE the TGW static routes. Reversible.

Common pitfalls

Key points to remember before building this:

References

De-duplicated AWS sources cited above.

  1. AWS Docs — What is AWS Cloud WAN?
  2. AWS Docs — Core network policies (JSON)
  3. AWS Docs — AWS Cloud WAN quotas
  4. AWS Docs — Service insertion in the core network policy
  5. AWS Docs — Attachment peerings (Transit Gateway peering)
  6. AWS Docs — VPC attachments (appliance mode)
  7. AWS Docs — Direct Connect gateway attachments
  8. AWS Docs — Share your Cloud WAN network (AWS RAM)
  9. AWS Docs — AWS Cloud WAN CloudWatch metrics
  10. AWS RAM Docs — Working with Availability Zone IDs
  11. AWS Blog — Achieve optimal routing with AWS Cloud WAN for multi-Region networks
  12. AWS Blog — Simplify global security inspection with AWS Cloud WAN service insertion
  13. AWS Blog — Centralized ingress inspection architecture in AWS Cloud WAN
  14. AWS Blog — AWS Cloud WAN and AWS Transit Gateway migration and interoperability patterns
  15. AWS — AWS Cloud WAN pricing