Design Patterns · Principal Track

Designing DNS for a Kubernetes Estate: One Architecture at Five Resolutions

CoreDNS is per cluster and the resolution edge is per estate; the ceiling that ends most DNS incidents is a per-interface packet allowance shared with two other services. Checked against the Kubernetes, CoreDNS and AWS documentation · August 2026.

← → to step

The verdict

  1. Two zones, two owners, and CoreDNS writes to neither. cluster.local is answered by CoreDNS from the API server and never appears in Route 53. corp.example is a Route 53 private hosted zone and never appears in the Corefile. Records reach that zone from ExternalDNS or your IaC — CoreDNS is a resolver, not a registrar, and holds no AWS credentials.
  2. CoreDNS is per cluster; the resolution edge is per estate. CoreDNS answers from the API server of the cluster it runs in, so it cannot be shared. Endpoints, forwarding rules and private zones are the opposite — built once and associated outward. Sizing and ownership follow that split, and so does failure.
  3. The ceiling that ends most DNS incidents is a packet allowance, not a QPS number. Each network interface allows 1024 packets per second to link-local services, shared with the instance metadata and time services, and packets over the limit are rejected rather than queued. It is the one limit teams rarely budget for, and the only control that removes packets rather than moving them is a node-local cache.
  4. Size the endpoint against what you measure. A resolver endpoint IP is documented at 10,000 UDP queries per second, but as low as 1,500 where connection tracking is enforced or queries arrive through a load balancer. Sizing against the headline number and landing in the tracked case is a factor of six, discovered under load.
  5. On EKS, editing the Corefile directly is silently temporary. The add-on fully manages the ConfigMap, and the documentation’s own example of what gets overwritten is adding conditional forwarders for on-premises resolution — exactly the change this design needs. Set it through the add-on’s configuration instead.
  6. Negative answers are a design decision, not a detail. Denials cache for up to 1800 seconds with a five-second floor. A pod that resolves a Service before it exists keeps failing long after you create it, and looks healthy the whole time.
  7. Two replicas is a starting point, not a capacity decision. EKS ships two regardless of node count. Autoscaling is opt-in and tracks max(nodes/16, cores/256).

How to read the five resolutions

EnvironmentThe estate as it already is — a VPC resolver in every VPC, corporate zones run by another team, on-premises resolvers nobody here owns. No solution internals.
RequirementsThe bar this is graded against, as a board. Every non-functional line states how it is verified; every constraint names an owning team.
ConceptualCapabilities stacked by dependency, no product named. Swap CoreDNS for another server and this picture has to survive.
LogicalNamed components and the flows between them — CoreDNS, the node cache, endpoints, rules, private zones — but no counts, subnets or TTLs.
PhysicalWhat an SRE stands up: replica counts, endpoint addressing, cache lifetimes, add-on versions. Values are illustrative and marked as such.
Resolution
—
Flows traffic & connectivity control & config identity & trust telemetry structural, not configured Borders a real container the boundary, or outside it
One name-resolution architecture for a Kubernetes estate On-premises DNS and the public internet sit outside a dashed boundary that holds the private namespace. A shared-services account owns the resolution edge: its VPC resolver at the VPC plus two address, an outbound endpoint that carries queries to on-premises, an inbound endpoint that carries them back in, the private hosted zones and the resolver rules. A workload account holds an EKS cluster: its own VPC resolver, CoreDNS in kube-system with the add-on autoscaler beside it, and nodes running a NodeLocal DNS cache in front of the pods. A pod resolves through the cache, then CoreDNS, then its own VPC resolver, and only then across to the shared account. The stepper lights the elements and flows each design decision touches. On-prem DNS Public DNS authoritative elsewhere — this design forwards to them, it does not host them PRIVATE NAMESPACE · CORP.EXAMPLE AWS · SHARED SERVICES az-a az-b az-c VPC 10.0.0.0/16 THE VPC RESOLVER VPC + 2the Route 53 resolver RESOLVER ENDPOINTS outbound inbound WHAT THIS ACCOUNT OWNS ASSOCIATED TO EVERY VPC private zonecorp.example resolver rulesone per domain AWS · WORKLOAD ACCOUNT az-a az-b az-c VPC 10.20.0.0/16 THE VPC RESOLVER VPC + 2 EKS · WORKLOAD CLUSTER KUBE-SYSTEM CoreDNS autoscaler PLATFORM ADD-ONS ExternalDNSwrites records NODES your pod NodeLocal 1 · ndots search 2 · a hit stops here 3 · cluster.local 4 · forward . resolv.conf 5 · a resolver rule matches 6 · out to on-prem corp.example on-prem asks us private zones answer everything else nodes / 16 creates + updates records the query path answered from here the reverse direction scale + fallback

Decision register — every step on this canvas

#ResolutionDecisionRecommendationRef

Capability → component: how the conceptual tab lands

Conceptual and Requirements deliberately break the shared topology, because a bar and a capability model have no deployment shape. That is what this table owes the reader — and the mapping is many-to-many.

CapabilityWhat implements itWhere it is configured
Who owns a nameThe cluster domain vs the corporate zone — a written boundarykubernetes cluster.local · Route 53 private hosted zones
Where it is registeredService objects; zone records held elsewhereService.metadata.name · namespace
Who is asked firstThe node-local cache, reached at a link-local addressnode-local-dns DaemonSet · 169.254.0.0/16
Who is asked nextCoreDNS, then the node’s own resolverforward . /etc/resolv.conf
When to stop askingAn authoritative plugin, or an explicit default routekubernetes · forward FROM TO
Where an answer may restThree layers: node cache, CoreDNS, the VPC resolver’s own cachecache 30 · success / denial
How long, including denialsPositive and negative lifetimes, set deliberatelycache: 3600 max · 1800 denial · 5s floor
Who publishes a name outwardExternalDNS, or your IaC for static records — never CoreDNSService / Ingress annotations · --txt-owner-id
Who may read the logQuery logging, scoped like security telemetryResolver query log configurations

The spine: one concern, five resolutions

Follow the hardest one — how a corporate name resolves from inside a pod — and watch it resolve from a constraint into something an SRE could stand up.

ResolutionThe same concern at this levelThe decision captured
EnvironmentCorporate zones are authoritative somewhere this team does not run, and reaching them needs a path that may not exist yet.Build the path deliberately in both directions, and date the dependency
RequirementsNot a component — a bar: a pod resolves cluster, corporate and public names, each with a stated owner and a test.Three paths, three owners, three alerts — not one line saying “DNS works”
ConceptualTwo capabilities: decide who is authoritative, then decide where a query goes when nobody here is.Per-domain rules with an explicit default — one-way door
LogicalCoreDNS forwards to the node’s own resolver, and a rule matches the domain. It never points at on-premises directly.Forward to the VPC resolver, so one rule set governs the whole estate
PhysicalAn outbound endpoint with addresses sized against measured peak, and the Corefile carried as add-on configuration so it survives an upgrade.Measure before trusting 10,000 QPS; never edit the ConfigMap directly

Rollout — the order that keeps the doors open

PhaseWhat shipsGate before moving on
1Baseline measurement before any change: query rate per node, and the interface allowance counter.You know the current number, so later you can tell whether anything improved
2The resolution edge in a shared-services account — outbound and inbound endpoints, zones, rules.Both directions tested, not just outbound
3Rules associated to one non-production VPC. Corporate names resolve from a pod there.No query for a private name reaches public recursion
4The node-local cache as a DaemonSet on the same non-production cluster.The interface allowance counter measurably drops
5CoreDNS autoscaling enabled with a floor above two, and cache lifetimes set deliberately.A denial window short enough that a late Service recovers on its own
6Production, with the Corefile carried as add-on configuration from the first commit.An add-on upgrade rehearsed, and the conditional forwarder observed to survive it

Who owns what

TeamOwnsThe thing they get paged for
PlatformCoreDNS, the node cache, the add-on version and its configurationResolution failing inside a cluster
NetworkEndpoints, forwarding rules, zone associations, the private linkCorporate names failing estate-wide
Whoever runs ExternalDNSThe records published into the private zone, and the owner id per clusterA name that resolves to the wrong cluster’s load balancer
Application teamsThe names their code asks for, and whether they are absoluteTheir own latency, when the fix is a trailing dot
SecurityQuery-log access, and what leaves the estate when a rule is missingInternal hostnames appearing in a log nobody here owns
Corporate DNS ownersThe zones this design forwards to, and the rule on their resolverAn outage that breaks every cross-boundary call

What is most likely to go wrong

FailureWhy it happensWhat blunts it
A conditional forwarder disappearsOn EKS the add-on fully manages data.Corefile, and the documentation’s own example of an overwritten edit is a conditional forwarder for on-premises resolution.Carry it in the add-on’s configurationValues, and rehearse an upgrade before production
DNS times out under load with nothing obviously wrongThe link-local allowance is 1024 packets per second per interface, shared with the metadata and time services. Excess packets are rejected silently.Watch linklocal_allowance_exceeded; run a node-local cache so hits never become packets
The endpoint saturates at a sixth of its rated capacity10,000 QPS per IP drops to as low as 1,500 where connection tracking is enforced or queries arrive via an NLB.Measure the real rate; add an address once peak passes half of any one interface
A pod stays broken after its Service existsDenials cache for up to 1800 seconds. The pod asked too early and the negative answer outlived the fix.Shorten the denial window deliberately, and treat it as a design value rather than a default
One call becomes five queriesndots:5 counts dots, not intent, so a complete external hostname is treated as a fragment and the search list runs first.A trailing dot, or a lower ndots per workload via dnsConfig
Private names leak to public recursionA forwarding chain with no explicit default sends unmatched queries out to the internet.Per-domain rules with a stated default, reviewed as a set
A DNS controller can rewrite any zone in the accountThe EKS community add-on ships with AmazonRoute53FullAccess, which is far wider than publishing records for one cluster.Reduce it to route53:ChangeResourceRecordSets, ListHostedZones and ListResourceRecordSets, scoped to the zones that cluster may manage
Two clusters fight over one recordExternalDNS uses a TXT record to mark ownership, and the owner id must be unique per deployment. Two clusters sharing a zone with the same id will each try to own the same name.Set --txt-owner-id per cluster, and treat it as immutable once set
A hosted zone stops accepting associationsA private hosted zone associates to 300 VPCs, and an estate reaches that quietly.Move to Route 53 Profiles before the wall, not after

References

Behaviour, defaults and limits checked against these sources in August 2026. Recommendations are marked as such: where the documentation prescribes nothing — how many resolution edges to run, whether to deploy a node-local cache — the page says so rather than implying a rule exists. Sizing figures on the canvases are illustrative.