AWS owns one end: building Direct Connect from the on-prem side, step by step

The clicks that lie — where AWS's job ends and yours begins

The Direct Connect console makes it look like a few clicks: request a connection, create a virtual interface, done. Then the port comes up and BGP won't establish — or it establishes and drops an hour later — and none of it is in the console. It's your VLAN tag, your MD5 key, your prefix count, your MTU. AWS owns exactly one end of this circuit: from its Direct Connect device inward. Everything from the cross-connect patch outward is yours to order, cable, and configure. This guide walks that on-prem build end to end, with the router config and the failures that actually bite.

What you're building, and what you need in place first

Before the phases, hold the whole picture. A Direct Connect deployment is a chain, and your team owns most of the links in it: an edge router with the right optics, a cross-connect through a colo meet-me room, an AWS DX port, then a band of virtual interfaces — each a VLAN with its own BGP session — reaching a VPC, the Transit Gateway, or AWS's public service prefixes.

One detail the corridor above hides: your edge router usually doesn't sit in your data center — it lives in the Direct Connect colo, and a backhaul circuit carries traffic the rest of the way to where your servers actually are. That last mile is yours too, and it is often the real bottleneck and the single point of failure.

Prerequisites — have these before you order

The phases below run in bring-up order, and each one names who does the work — because the split between your team, the colo, and the AWS console is exactly where hand-offs drop.

Phase 1 — Dedicated or hosted: the choice that sets everything

This is the one decision that reshapes the whole build, so make it first. A dedicated connection is a physical port that's yours alone. You order it from AWS at 1, 10, 100, or 400 Gbps and you supply the router, the optics, and the cross-connect into the Direct Connect location. You get the full physical build — and the full VIF budget: up to 50 private/public VIFs plus 4 transit VIFs on one port. Provisioning realistically takes weeks, most of it circuit and cross-connect lead time, not AWS.

A hosted connection is capacity a partner carves off their own port for you. The Direct Connect Partner has already done the physical work — their gear, their cross-connect — so you contact them, they provision it, and it appears in your console to accept. Hosted speeds run 50 Mbps–25 Gbps (sub-1G is hosted-only). The trade: a hosted connection carries exactly one VIF, and AWS traffic-polices it, so bursts above your rate get dropped.

Phases 2–3 — Order the port, pull the LOA-CFA, run the cross-connect

Goal: a provisioned AWS port, the one document that unlocks the physical build, and a lit fiber path from your panel to AWS's.

In the console you request a dedicated connection: pick the Direct Connect location, the port speed, and whether you're reaching it through a partner. AWS reviews the request — up to 72 business hours — and may email for detail (you have 7 days to respond or it's deleted). You can't change the speed later; to change it you delete and recreate. What you're waiting for is the LOA-CFA, the Letter of Authorization and Connecting Facility Assignment: a signed, watermarked PDF naming the AWS patch panel and port assignment that authorizes the facility to run a cross-connect to it. That single artifact turns an AWS order into a physical build.

The cross-connect itself is a fiber patch inside the facility's meet-me room (MMR), joining your patch panel to AWS's. You hand the LOA-CFA to the colo (or your partner) and they run it — AWS keeps physical separation between its cage and yours through the MMR's structured cabling. If you have no equipment in the location, you cannot order this yourself; that's the whole reason presence-or-partner is a prerequisite.

① Order the dedicated port in the AWS console.   ② AWS issues the LOA-CFA — the authorization to patch.   ③ The colo runs the cross-connect in the meet-me room.   ④ Light appears on your interface.

Validation: you should see light on the interface once the patch is complete. Ask the colo for an optical signal report and check the receive level is in spec. One practitioner troubleshooting write-up cites roughly −14.4 to +2.50 dBm for 1–10 Gbps and −4.3 to +4.5 dBm for 100 Gbps (awsforengineers.com — practitioner figures; treat as a sanity range and confirm against your optic's datasheet and your colo). No light means the patch isn't done, the strands are swapped (Tx/Rx), or the optic is wrong — the next phase. Rollback: nothing to undo on your router yet; if the location or speed was wrong, delete the AWS connection within the 90-day window and re-order.

Phase 4 — Cable and light the router: fiber, optics, port settings

Goal: a clean Layer-1/Layer-2 link — interface up, 802.1Q trunking, no auto-negotiation surprises. The optic has to match the speed tier exactly. Direct Connect is single-mode fiber only, with a specific transceiver per speed; get the wavelength or reach wrong and the link stays dark.

(AWS User Guide — network requirements; version-sensitive, verify the current table.)

Two more physical requirements from the same doc. First, 802.1Q VLAN encapsulation must work across the entire path, including any intermediate switch — Direct Connect delivers your VIFs as tagged VLANs on a trunk, so a device that strips tags kills it. Second, auto-negotiation is endpoint-dependent: AWS says it "might need to be enabled or disabled" depending on the DX endpoint, and the practical rule practitioners land on is that above 1 Gbps you disable auto-negotiation and hard-set speed and full-duplex on both the port and any transport in between.

interface TenGigE0/0/0
 description AWS-Direct-Connect-dxcon-xxxx
 no negotiation auto          ! hard-set above 1G; do not auto-negotiate
 mtu 9100                     ! headroom for jumbo VIFs + VLAN tag + FCS
 no shutdown
!
! 802.1Q comes from the per-VIF subinterfaces in Phase 5 —
! this physical interface is the dot1q trunk that carries them.

set interfaces xe-0/0/0 description "AWS-Direct-Connect-dxcon-xxxx"
set interfaces xe-0/0/0 gigether-options no-auto-negotiation
set interfaces xe-0/0/0 mtu 9192
set interfaces xe-0/0/0 flexible-vlan-tagging
set interfaces xe-0/0/0 encapsulation flexible-ethernet-services

Validation: interface line protocol up, the correct optic reported, receive light in spec. Rollback: shut the interface — nothing downstream exists yet.

Phase 5 — Stand up each VIF: VLAN, peer IP, ASN, MD5

Goal: an established eBGP session per virtual interface. A VIF is one VLAN carrying one eBGP session. You create it in the console (or the partner does, for hosted) and AWS hands back the parameters; your job is to mirror them exactly on a dot1q subinterface. Three VIF types, and which you build depends on what you're reaching:

The per-VIF parameters you configure on-prem:

Here's a private VIF on both a Cisco and a Juniper edge — the heart of the on-prem config: a subinterface, the peer, the ASN, the key.

interface TenGigE0/0/0.100
 description AWS-PRIVATE-VIF
 encapsulation dot1Q 100
 ip address 169.254.100.2 255.255.255.252   ! AWS side = .1
!
router bgp 65001
 neighbor 169.254.100.1 remote-as 64512      ! AWS VGW/DXGW ASN
 neighbor 169.254.100.1 password 7hisMustMatchExactly
 address-family ipv4
  neighbor 169.254.100.1 activate
  network 10.20.0.0 mask 255.255.0.0         ! advertise your on-prem prefix
 exit-address-family

set interfaces xe-0/0/0 unit 100 vlan-id 100
set interfaces xe-0/0/0 unit 100 family inet address 169.254.100.2/30
set protocols bgp group AWS-DX type external
set protocols bgp group AWS-DX peer-as 64512
set protocols bgp group AWS-DX neighbor 169.254.100.1 authentication-key "7hisMustMatchExactly"
set policy-options policy-statement ADV-ONPREM term 1 from route-filter 10.20.0.0/16 exact
set policy-options policy-statement ADV-ONPREM term 1 then accept
set protocols bgp group AWS-DX export ADV-ONPREM

A public VIF differs in three ways: the AWS peer ASN is 7224, you peer over public IPs you own (not link-local/private), and you must advertise at least one public prefix you're registered for. Everything else — VLAN, MD5, dot1q — is the same shape.

Phase 6 — Shape the routing: prefixes, limits, communities, MTU

Goal: your prefixes advertised, AWS's return path where you want it, and no self-inflicted session drops.

Return-path control uses BGP communities and prefix specificity. AWS won't honor arbitrary knobs, but it publishes a specific set. On private/transit VIFs, tag your advertised prefixes with a local-preference community — 7224:7100 (low), 7224:7200 (medium), or 7224:7300 (high) — to steer AWS→on-prem return traffic; these are evaluated before AS_PATH. On public VIFs, use a scope community — 7224:9100 (local Region), 7224:9200 (continent), or 7224:9300/untagged (global) — to control how far AWS propagates your prefixes. Longest-prefix and AS_PATH still work normally.

router bgp 65001
 address-family ipv4
  neighbor 169.254.100.1 send-community
  neighbor 169.254.100.1 route-map SET-LP-HIGH out
 exit-address-family
!
ip prefix-list ONPREM seq 5 permit 10.20.0.0/16
route-map SET-LP-HIGH permit 10
 match ip address prefix-list ONPREM
 set community 7224:7300           ! high local-pref: AWS prefers this VIF for return
! backup VIF: same shape, set community 7224:7100 (low)

route-map PUBLIC-SCOPE permit 10
 match ip address prefix-list MY-PUBLIC
 set community 7224:9200            ! continent scope (7224:9100 = Region, 9300 = global)
!
router bgp 65001
 address-family ipv4
  neighbor  send-community
  neighbor  route-map PUBLIC-SCOPE out
 exit-address-family

MTU is part of routing hygiene. A private VIF supports 1500 or 9001; a transit VIF supports 1500 or 8500 (Transit Gateway caps jumbo at 8500). Both ends must agree — if two VIFs advertise the same route with different MTUs, AWS falls back to 1500. Set your subinterface MTU to match the VIF, and remember that flipping a VIF to jumbo can bounce the underlying physical link for up to ~30 seconds, affecting every VIF on it. (AWS User Guide — version-sensitive.) Validation: route counts under the limit (show bgp summary), your prefixes visible from AWS, and a jumbo-frame ping with the DF bit set succeeding end-to-end if you enabled it.

Phase 7 — Make it resilient, and encrypt it

Goal: survive a device, fiber, or location failure — and protect the traffic on the wire. AWS's Resiliency Toolkit is a console wizard, but what it's really choosing is how many connections you build and where. The models map to SLAs:

The wizard enforces the useful part — it won't let you terminate both connections on the same AWS DX device, which is the failure mode people accidentally build. To bundle multiple ports at one location into one logical link, use a LAG (Link Aggregation Group): same speed on every member, up to 4 members below 100 Gbps and 2 at 100 Gbps. A LAG gives you more bandwidth and member-level redundancy, but it's still one location — not a substitute for a second site.

① A second device in a second location is what buys the 99.99% SLA.   ② A LAG bundles ports at one location — more bandwidth, not a second site.   ③ BFD on your router is the fast-failover trigger.   ④ MACsec encrypts the cross-connect; IPsec over a public VIF encrypts end-to-end.

BFD: the piece you have to enable yourself

Redundant links are useless if failover is slow. Asynchronous BFD is automatically enabled on the AWS side for every VIF — but it does nothing until you configure it on your router. AWS uses a minimum interval of 300 ms and a multiplier of 3, so a dead link is detected in roughly 900 ms instead of waiting out BGP's default hold timer. (AWS re:Post, enable BFD for Direct Connect — timers version-sensitive.)

interface TenGigE0/0/0.100
 bfd interval 300 min_rx 300 multiplier 3
!
router bgp 65001
 neighbor 169.254.100.1 fall-over bfd

set protocols bgp group AWS-DX bfd-liveness-detection minimum-interval 300
set protocols bgp group AWS-DX bfd-liveness-detection multiplier 3

Then actually test it. The Toolkit has a failover test that brings the BGP peering down on AWS's side so you can watch traffic move to the standby VIF. Run it before you depend on the redundancy — an untested failover path is a guess.

Encryption: MACsec on the port, or IPsec over a public VIF

Direct Connect is private, but it is not encrypted by default. Two options, both needing on-prem work:

• MACsec — IEEE 802.1AE Layer-2 encryption over the cross-connect, between your router and the AWS DX device. Available on dedicated 10/100/400 Gbps connections at selected locations (not hosted), and your edge port must support it. You generate a CKN/CAK pair (hex, 64 characters, AES-256), associate it with the connection, and provision the same pair on your router. SCI must be on; 100/400 Gbps require Extended Packet Numbering (GCM-AES-XPN-256). Default mode is should_encrypt (falls back to clear if negotiation fails); must_encrypt is stricter but drops all traffic if MACsec can't establish. (AWS MACsec docs — availability/cipher details version-sensitive.)
• IPsec VPN over a public VIF — the portable option, and the only one for hosted or sub-1G. You bring up a public VIF, then run an AWS Site-to-Site VPN (or a self-managed tunnel) to an AWS VPN endpoint's public IP over that VIF. It's true end-to-end encryption terminating on your customer gateway, at the cost of tunnel MTU/throughput overhead. MACsec, by contrast, only protects the point-to-point cross-connect, not multiple segments end-to-end.

Phase 8 — Validate the whole chain, and test failover

Work Layer 1 up, because a failure at a lower layer masks everything above it. Interface and optic first, then the tag, then BGP, then reachability, then the failover you can't assume works until you've watched it.

show interfaces TenGigE0/0/0 transceiver detail   ! optic present, Rx light in spec
show interfaces TenGigE0/0/0.100                   ! subinterface up/up, dot1q 100
show bgp ipv4 unicast summary                      ! neighbor state = Established
show bgp ipv4 unicast neighbors 169.254.100.1 advertised-routes
show bgp ipv4 unicast neighbors 169.254.100.1 routes   ! received count under 100
show bfd neighbors

show interfaces diagnostics optics xe-0/0/0        ! Rx power in spec
show bgp neighbor 169.254.100.1                    ! state Established, prefix counts
show route receive-protocol bgp 169.254.100.1
show route advertising-protocol bgp 169.254.100.1
show bfd session

1. L1: interface up, correct optic, receive light in spec (optical report from the colo).
2. L2: 802.1Q tag correct, ARP resolves to the AWS peer's MAC on the right VLAN.
3. L3/BGP: session Established, expected routes in and out, received count under the limit.
4. Reachability: traceroute shows the Direct Connect hop; a test EC2 instance in the attached VPC answers a ping (security group allowing ICMP).
5. Resiliency: run the Toolkit failover test and confirm traffic moves to the standby VIF and back.

Where bring-up dies — the on-prem gotchas

Every one of these lives on your side of the cross-connect. They're roughly in bring-up order, which is also the order they'll bite you.

References

1. AWS Direct Connect User Guide — What is Direct Connect? (network requirements, VIF types): https://docs.aws.amazon.com/directconnect/latest/UserGuide/Welcome.html
2. AWS Direct Connect User Guide — Dedicated and hosted connections: https://docs.aws.amazon.com/directconnect/latest/UserGuide/WorkingWithConnections.html
3. AWS Direct Connect User Guide — Dedicated connections (LOA-CFA, 90-day rule, port speeds): https://docs.aws.amazon.com/directconnect/latest/UserGuide/dedicated_connection.html
4. AWS Direct Connect User Guide — Hosted connections (speeds, policing): https://docs.aws.amazon.com/directconnect/latest/UserGuide/hosted_connection.html
5. AWS Direct Connect User Guide — Virtual interfaces (VLAN, peer IP, ASN, MD5, MTU): https://docs.aws.amazon.com/directconnect/latest/UserGuide/WorkingWithVirtualInterfaces.html
6. AWS Direct Connect User Guide — Routing policies and BGP communities (7224:xxxx, local pref, scope): https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html
7. AWS Direct Connect User Guide — Quotas (prefix limits, LAG members, VIF counts): https://docs.aws.amazon.com/directconnect/latest/UserGuide/limits.html
8. AWS Direct Connect User Guide — Resiliency Toolkit (Max/High/Dev-test, SLAs): https://docs.aws.amazon.com/directconnect/latest/UserGuide/resilency_toolkit.html
9. AWS Direct Connect User Guide — High resiliency walkthrough (VIF params, RFC 1918/3927): https://docs.aws.amazon.com/directconnect/latest/UserGuide/high_resiliency.html
10. AWS Direct Connect User Guide — MACsec (CKN/CAK, AES-256, XPN, SCI, modes): https://docs.aws.amazon.com/directconnect/latest/UserGuide/MACsec.html
11. AWS Networking Blog — Direct Connect Layer 1 explained (MMR, cross-connect, optics): https://aws.amazon.com/blogs/networking-and-content-delivery/aws-direct-connect-layer-1-explained-from-data-centers-to-cloud-connectivity/
12. AWS re:Post — Enable BFD for a Direct Connect connection (300 ms × 3, Cisco/Juniper): https://repost.aws/knowledge-center/enable-bfd-direct-connect
13. AWS re:Post — Troubleshoot BGP issues over Direct Connect: https://repost.aws/knowledge-center/troubleshoot-bgp-dx
14. AWS Well-Architected Hybrid Networking Lens — Direct Connect and IPsec VPN: https://docs.aws.amazon.com/wellarchitected/latest/hybrid-networking-lens/aws-direct-connect-and-ipsec-vpn.html
15. awsforengineers.com — Troubleshooting AWS Direct Connect step-by-step (light levels, bring-up failures): https://awsforengineers.com/blog/troubleshooting-aws-direct-connect-step-by-step/
16. Orixcom — Direct Connect Dedicated vs Hosted comparison: https://blog.orixcom.com/aws-direct-connect-dedicated-vs-hosted-detailed-comparison