Managing Hundreds of Amazon EKS Clusters at Fleet Scale · Part 9 of 18

Connecting an EKS Fleet: Load Balancers, Pod Security Groups, and Service Mesh

Once you run clusters by the hundred, "how does traffic reach this service?" splits into four separate decisions — getting traffic in, isolating a single pod, walling off tenants, and identity between clusters. This piece wires all four, and argues against reaching for a service mesh before you actually need one.

Why connecting a fleet is four decisions, not one

On a single cluster, "connectivity" feels like one question: put a load balancer in front, maybe add a NetworkPolicy, done. At fleet scale that question fractures. You are now answering four separate ones, and each has a different tool, a different failure mode, and a different owner.

They are worth naming up front, because the rest of this article is one per section:

• How does outside traffic reach a service — and can one load balancer serve more than one cluster?
• How do you give a single pod its own network identity so it can reach a locked-down database?
• How do you stop tenants in a shared cluster from talking to each other?
• When — if ever — do you need encrypted identity between services across clusters?

This builds directly on the networking foundation laid earlier in the series (the VPC, subnet, and IP-planning article): private nodes, a NAT gateway per Availability Zone, dedicated cluster subnets sized for pod ENIs, and subnet tags that let controllers auto-discover where to place load balancers.³ If that layer isn't in place, none of what follows lands cleanly. The platform shape itself — the GitOps hub-and-spoke every cluster is cut from — comes from Part 1.

The through-line: reach for the smallest tool that solves the decision in front of you. A load balancer is not a mesh; a security group is not a NetworkPolicy. Most connectivity problems on a fleet are solved without ever installing a mesh — and the last section makes that call explicit.

How one load balancer fronts many clusters — the LB Controller and cross-cluster TargetGroupBinding

The AWS Load Balancer Controller is the piece that turns Kubernetes objects into real AWS load balancers. An Ingress resource provisions an Application Load Balancer; a Service of type LoadBalancer provisions a Network Load Balancer.¹ That much is table stakes. Two capabilities are what make it a fleet tool.

First, since v2.6.0 the controller auto-manages the NLB's security groups. Before that, NLB security-group wiring was a manual, error-prone step teams got wrong repeatedly; now the controller owns it.¹ Second — and this is the fleet-scale unlock — v2.10+ supports cross-cluster, multi-cluster TargetGroupBinding: you can register targets from multiple EKS clusters into one NLB target group.¹

To see why that matters, you need what TargetGroupBinding does normally. It binds a Kubernetes Service to a target group you already created in AWS, so the controller keeps that group's targets in sync with the Service's pods — the load balancer itself lives outside Kubernetes. The cross-cluster variant lets a single target group hold pod IPs from cluster A and cluster B at the same time (Figure 1).

That single capability answers several fleet problems at once. You can blue/green a workload from an old cluster to a new one behind one stable DNS name. You can fan a shared public entry point across many small clusters without standing up an NLB per cluster. And you sidestep the per-cluster load-balancer sprawl that otherwise scales linearly with cluster count.

The wiring detail that trips people: the controller only knows where to put load balancers if your subnets are tagged for it. Public subnets need kubernetes.io/role/elb, private subnets kubernetes.io/role/internal-elb; that subnet tagging is part of the networking foundation, not an afterthought here.³ Cross-cluster target registration also assumes the pod IPs are routable to the load balancer — the clusters share a VPC or are peered, which is exactly the kind of decision the IP-planning article settles before you get here.

Controller version numbers move; the v2.6.0 and v2.10 behaviours above are current as of 2026-06-30 and worth re-checking against the controller release notes at publish time, because feature gates and defaults shift release to release.¹

The ingress you're standing on is being retired — plan the Gateway API move

There is a deadline you cannot ignore on a large fleet: ingress-NGINX, one of the most widely deployed ingress controllers in the Kubernetes world, is being retired upstream in March 2026.⁴ On a fleet of hundreds of clusters, that is not a footnote — it is a migration program, because the controller is probably running nearly everywhere.

The successor is the Kubernetes Gateway API: a role-oriented, more expressive replacement for the aging Ingress object, split into a GatewayClass, Gateway, and route objects like HTTPRoute so platform owners and application teams manage different layers cleanly.⁵ The AWS Load Balancer Controller and other implementations already support it, so the destination is stable even while the source is on a clock.

The fleet-scale move is to stop provisioning new ingress on ingress-NGINX now, standardize new services on the Gateway API through your golden-path template, and schedule the migration of existing ingress as part of routine day-2 upgrade work rather than a separate fire drill — the upgrades article later in the series covers how that folds into the normal version cadence. The cost of ignoring it is a fleet-wide unsupported dependency discovered the hard way. (Retirement timing current as of 2026-06-30; re-verify against the upstream project and the EKS version notes.)

How you give a single pod its own network identity — security groups for pods

Sometimes the isolation you need is below Kubernetes entirely. A pod has to reach an RDS database or an ElastiCache cluster that only accepts traffic from a specific VPC security group — and you do not want to open that database to every node in the fleet. Security groups for pods solve exactly this: they attach a VPC security group directly to a pod.²

Mechanically, the VPC CNI gives the pod a dedicated branch network interface carved from a trunk interface on the node (a Nitro feature), and the security group rides on that branch ENI. The pod now sits inside your VPC security-group model as a first-class member, so an RDS security group can allow just that pod's SG and nothing else.²

One knob governs how strictly those rules apply. POD_SECURITY_GROUP_ENFORCING_MODE has two settings: strict (the default), where the pod's security group governs all of its traffic, and standard. The rule to remember is that when you combine security groups for pods with Kubernetes network policies, you must use standard mode — otherwise the two enforcement paths collide and traffic you expected to allow gets dropped.²

Where L4 stops and L7 begins — NetworkPolicy versus a service mesh

The last two decisions are about traffic between services, and this is where teams most often over-buy. The honest split is by layer (Figure 2). NetworkPolicy operates at L3/L4; a service mesh operates at L7. They are not substitutes.

NetworkPolicy is your L3/L4 tool. It allows or denies traffic by pod selector, namespace, and port — the native way to wall tenants off inside a shared cluster. It needs a CNI that enforces it (Calico or Cilium, and the Amazon VPC CNI has enforced network policies since v1.14). What it does not give you is identity or encryption; it only decides who can reach whom.³

That is the tenant-isolation lever from the many-vs-few and policy articles: if you run soft multi-tenancy — several teams sharing a cluster by namespace — NetworkPolicy is how you keep their pods from talking across the boundary. Hard isolation (a cluster or account per tenant) is the other path, decided earlier in the series.³

A service mesh is your L7 tool, and you buy it for capabilities NetworkPolicy structurally cannot provide: mutual-TLS identity between services, retries and timeouts, traffic-splitting for canaries, and service identity that spans clusters. Istio and Linkerd are the common choices; AWS App Mesh is the managed option.³

Here is the opinion this section exists to deliver: do not reach for a mesh if NetworkPolicy plus security groups for pods already covers your isolation needs. A mesh is real, ongoing operational cost — a control plane to run, sidecars or an ambient data plane to manage, certificates to rotate — and on a fleet that cost multiplies by cluster count. Adopt it when you have a concrete L7 requirement, not as a default.

What to reach for, by need

Put together, the four decisions collapse into a short decision table. Match the tool to the need in front of you, and notice that only the last one carries fleet-wide operational weight (Figure 3).

The connectivity you should actually build

Here is the decision to carry out of this piece. For nearly every fleet, the connectivity stack is three tools, not four: the AWS Load Balancer Controller for ingress and egress — with cross-cluster TargetGroupBinding when one NLB should serve many clusters — security groups for pods where a workload needs its own VPC identity, and NetworkPolicy for tenant isolation inside shared clusters. That covers the overwhelming majority of what "connect the fleet" actually means.

Add a service mesh only when a concrete L7 requirement appears: mutual-TLS between services as a compliance mandate, traffic-shaping for progressive delivery, or service identity that has to span clusters. Until then, a mesh is operational weight multiplied across every cluster in the fleet, buying capabilities you are not using.

Two dates belong on the roadmap now. Start any new ingress on the Gateway API rather than ingress-NGINX, which retires upstream in March 2026, and re-verify the Load Balancer Controller version behaviours — the v2.6.0 NLB security-group management and the v2.10 cross-cluster target groups — against the current release notes before you rely on them, because these move faster than the docs suggest. (All version and retirement facts current as of 2026-06-30.)

The next decisions in the series turn inward: how identity works for the pods themselves — EKS Pod Identity versus IRSA — and how you keep the whole fleet inside standard support as versions roll forward.