nmadiraju.com
Zero-to-expert guides for cloud, infra & GenAI engineers — primers, interviews, design patterns, and implementation.
Gen AI
16 guidesAgent Cost & TCO: Token Economics and Latency Budgets
A deep dive on what an agent actually costs to run: why cheaper per-token prices can still produce bigger bills, because the agent loop re-reads its whole growing context every step — so cost scales with the square of the conversation, not the token rate. The dominant driver (the quadratic re-read) and how prompt caching and compaction bend it; the price surface (model tier, input/output ratio, caching, batch); the biggest lever — routing by difficulty so a cheap model handles the easy majority and an expensive one only the hard tail; the latency-vs-accuracy budget; infrastructure vs. token cost; and a cost-control checklist. With diagrams and worked token math.
Multi-Agent Orchestration
The seventh article in the agentic series: when one agent's context gets too crowded with tools and instructions, you split it into many — but the agents were never the hard part, the handoffs are. Why you split at all (context isolation, specialized tool sets, parallelism) and why you shouldn't until one agent genuinely breaks; the three topologies on one spectrum (supervisor/worker as the default, hierarchical supervisors-of-supervisors for scale, swarm peers with no boss); why the handoff — what state transfers, what's summarized, what's dropped — is the real system; containing the blast radius when one sub-agent fails or loops; A2A handoffs across team/org boundaries; and how to choose a topology. With flow diagrams and runnable LangGraph code.
Human-in-the-Loop Approval for Autonomous Agents
The sixth article in the agentic series: autonomy removed the human checkpoint that used to sit between intent and consequence, and HITL puts a calibrated one back — without it, an agent's first catastrophic mistake is also its last. Why approval is an architecture decision, not a UX afterthought; gating by risk class rather than by step (reversible/cheap actions run free, irreversible/expensive ones pause); the mechanism — pause and resume on the checkpointer, so a paused graph survives a restart; the failure mode on the safe-looking side (approval fatigue, where too many prompts train humans to rubber-stamp); HITL vs HOTL (does the gate block, or just observe?); designing the wait itself (timeouts, escalation, what the human sees); and a pre-ship checklist. With flow diagrams and runnable LangGraph interrupt() code.
Agent Security & Prompt Injection
The fifth article in the agentic series, and a design pattern rather than a primer: prompt injection isn't a bug you patch, it's an architecture an attacker hijacks with a sentence hidden in a web page. Why it's an architecture problem, not a prompt problem; the threat model (the lethal trifecta — untrusted content + private data + the ability to act, and why any one missing leg defuses it); the root cause (the confused deputy — the model can't tell your instructions from the attacker's); the defense patterns in priority order (least-privilege tools, separate read from act, sandbox the execution, human-in-the-loop for high-risk actions), each traced as vulnerable-vs-hardened wiring in code; layering them as defense-in-depth; and a pre-ship security checklist. With flow diagrams and runnable LangGraph code.
Agent Evaluation: Trajectories, Tool-Calls, Task Completion
The fourth primer in the series: how do you know the agent is any good? You stop grading it the way you graded a plain LLM app. A model hands you an output to check against a reference; an agent hands you a trajectory, and the output is only its last step — so you grade the path. Why final-answer eval is a trap for agents; the three layers worth measuring (final response, trajectory, single tool-call) and why most teams measure only the first; the two scorer families (reference-based vs reference-free / LLM-as-judge) and matching the scorer to the question; the two concrete ways to grade a trajectory from the open-source agentevals package (when you know the right path vs when you don't); pass^k — the reliability metric that exposes non-determinism that pass@1 hides; offline (pre-release, gate the PR) vs online (post-release, every production failure becomes a new dataset case) eval on the lifecycle; and a pre-ship checklist. With diagrams and runnable code.
Agent Memory Architectures
The third primer in the series: a stateless model forgets everything between calls, so the entire feeling of an agent that 'knows you' is an illusion you construct by deciding what text to re-send each turn. The two layers you must never conflate — short-term thread state (the checkpointer, keyed by thread_id) vs long-term cross-session knowledge (the store, scoped by user_id); the context window as a budget, not a bucket, and the compaction that keeps it honest; long-term memory's three jobs (semantic, episodic, procedural) and why one bucket for all three breaks retrieval; the recall and save nodes that bracket the loop; resuming a thread by its transcript vs greeting a fresh session with the user's facts; the line between memory and RAG; and a 17-row pre-ship checklist. With flow diagrams and runnable LangGraph code.
Primer
15 guidesSLOs for Infrastructure Teams Whose Users Are Other Teams
The sequel to Article 17: that one taught the mechanics; this one handles the organizational reality that makes infrastructure SLOs genuinely harder than product SLOs. Infrastructure SLOs fail not because the math is hard but because the user is ambiguous — you're not measuring a service, you're measuring an interface between teams. Why product-SLO thinking breaks (fifty consumers, no single golden signal, their SLOs stacked on yours), identifying the actual consumer (define the SLI at your interface, one per capability — not an outcome three layers up), the responsibility boundary as the heart (measure at the Resolver endpoint, not the client; looser than your dependency composite; the SLI that turns 'the network is slow' into data), the infra SLI menu (availability / latency / correctness — plus the provisioning time-to-ready SLO nobody publishes), the negotiation ('what do you assume about us?' → usually 100%; publish it; tier at a chargeback price), error budgets when a consumer causes the burn (per-tenant SLIs, quotas, and why you don't freeze for their fault), and the anti-patterns. One shared capability threaded through all six steps into a published one-page contract.
SLOs from Scratch: Picking Your First SLI for a Network Service
Your first SLO should measure what consumers experience, be based on current performance — not aspiration — and ship with a budget policy, or it's a dashboard, not an SLO. The four terms kept surgically separate (SLI / SLO / error budget / SLA), the hard part for network services (what counts as an 'event' — flows, packets, queries), a worked first SLI for hybrid DNS (Route 53 Resolver + on-prem forwarders: valid = reaches the endpoint, good = non-SERVFAIL within 100 ms), setting the number from measurement and dependency math rather than vibes, error-budget arithmetic and the budget policy everyone skips, and why you alert on burn rate. Every claim sourced.
GitHub Actions, One Layer at a Time: From Six Lines to a Hardened Pipeline
Build one pipeline from six lines up, adding each concept only when the previous version hits a wall — steps vs jobs, why jobs are isolated (and how data actually crosses via artifacts / cache / outputs), the action as the reusable unit, the GITHUB_TOKEN and OIDC, Environments and approvals, the fork trust line and the three ways repos get owned, and matrix / concurrency / cost. The concepts arrive already motivated, not dumped. Every claim sourced.
Leading Without Authority: The Platform Tech Lead's Playbook
You own the outcome but can't order anyone to do anything — and for a platform team that gap isn't a bug in your title, it's the whole job. Why mandates backfire (resentment, shadow workflows, dead feedback), the operating model that works instead (credibility + Cohen & Bradford's currencies of influence + platform-as-product), and 11 concrete plays a platform tech lead actually runs — each tagged with who advocates it and whether it's evidence-based or named opinion. Attributed to Fournier, Larson, Reilly, Team Topologies, and DORA/DevEx research.
Agent Evaluation: Trajectories, Tool-Calls, Task Completion
The fourth primer in the series: how do you know the agent is any good? You stop grading it the way you graded a plain LLM app. A model hands you an output to check against a reference; an agent hands you a trajectory, and the output is only its last step — so you grade the path. Why final-answer eval is a trap for agents; the three layers worth measuring (final response, trajectory, single tool-call) and why most teams measure only the first; the two scorer families (reference-based vs reference-free / LLM-as-judge) and matching the scorer to the question; the two concrete ways to grade a trajectory from the open-source agentevals package (when you know the right path vs when you don't); pass^k — the reliability metric that exposes non-determinism that pass@1 hides; offline (pre-release, gate the PR) vs online (post-release, every production failure becomes a new dataset case) eval on the lifecycle; and a pre-ship checklist. With diagrams and runnable code.
Agent Memory Architectures
The third primer in the series: a stateless model forgets everything between calls, so the entire feeling of an agent that 'knows you' is an illusion you construct by deciding what text to re-send each turn. The two layers you must never conflate — short-term thread state (the checkpointer, keyed by thread_id) vs long-term cross-session knowledge (the store, scoped by user_id); the context window as a budget, not a bucket, and the compaction that keeps it honest; long-term memory's three jobs (semantic, episodic, procedural) and why one bucket for all three breaks retrieval; the recall and save nodes that bracket the loop; resuming a thread by its transcript vs greeting a fresh session with the user's facts; the line between memory and RAG; and a 17-row pre-ship checklist. With flow diagrams and runnable LangGraph code.
Design Patterns
10 guidesThe One-Page ADR: Context, Decision, Consequences
If a decision isn't written down with its context and consequences, it will be relitigated — or worse, silently eroded by someone who never knew it was load-bearing. The Architecture Decision Record is the cheapest tool in architecture, and one page is a forcing function, not a constraint. What earns an ADR (the one-way-door test: expensive to reverse, crosses team boundaries, constrains the future), the anatomy section by section (Status as a lifecycle, Context written for a reader two years out, Decision in one active sentence, Consequences with costs not just benefits, Options considered as the section that stops relitigation), a real one-pager written in full (a dedicated network pipeline split from AFT), why one page, the operational side most advice skips (in-repo, PR-reviewed, superseded not edited), and ADRs as an influence instrument — how a decision log scales your judgment to teams you'll never meet. Template + a two-question test included.
The Paved Road Problem: Driving Adoption Across Teams You Don't Control
You built something better — a platform, a workflow, a golden path — but you can't make anyone use it, and "it's better, trust me" has never once been enough. Why the mandate reflex backfires (resentment, shadow workflows, adoption theater), the pattern that works instead — the paved road / golden path (Netflix, Spotify, Google) run as a product whose only success metric is voluntary adoption — and 12 concrete plays that win adoption by pull: make it the path of least resistance, self-service scaffolding, lighthouse teams, migrate-by-default, strangle the old road, and mandate only the safety floor. Measure usage, not installs. Every claim sourced.
Shift-Left, Explained: Catching Failures Where They're Cheapest
Move testing, security, and validation from post-merge toward pre-commit, so the machine tells the author it's broken while the fix still costs minutes. What shift-left actually means beyond the buzzword; why the famous "10x cost of a defect" number is folklore (and what's true instead); designing a tiered pipeline — a fast required pre-merge tier vs a deep nightly one; twelve concrete practices enterprises actually run (pre-commit hooks, merge queues, SAST, secret + dependency scanning, policy-as-code, contract tests, preview envs, test-impact analysis, feature flags) with real tools and where each shifts; the metrics that prove it (escape rate, time-to-first-signal, DORA); and the culture shift that makes or breaks it. Every claim sourced.
Multi-Agent Orchestration
The seventh article in the agentic series: when one agent's context gets too crowded with tools and instructions, you split it into many — but the agents were never the hard part, the handoffs are. Why you split at all (context isolation, specialized tool sets, parallelism) and why you shouldn't until one agent genuinely breaks; the three topologies on one spectrum (supervisor/worker as the default, hierarchical supervisors-of-supervisors for scale, swarm peers with no boss); why the handoff — what state transfers, what's summarized, what's dropped — is the real system; containing the blast radius when one sub-agent fails or loops; A2A handoffs across team/org boundaries; and how to choose a topology. With flow diagrams and runnable LangGraph code.
Human-in-the-Loop Approval for Autonomous Agents
The sixth article in the agentic series: autonomy removed the human checkpoint that used to sit between intent and consequence, and HITL puts a calibrated one back — without it, an agent's first catastrophic mistake is also its last. Why approval is an architecture decision, not a UX afterthought; gating by risk class rather than by step (reversible/cheap actions run free, irreversible/expensive ones pause); the mechanism — pause and resume on the checkpointer, so a paused graph survives a restart; the failure mode on the safe-looking side (approval fatigue, where too many prompts train humans to rubber-stamp); HITL vs HOTL (does the gate block, or just observe?); designing the wait itself (timeouts, escalation, what the human sees); and a pre-ship checklist. With flow diagrams and runnable LangGraph interrupt() code.
Agent Security & Prompt Injection
The fifth article in the agentic series, and a design pattern rather than a primer: prompt injection isn't a bug you patch, it's an architecture an attacker hijacks with a sentence hidden in a web page. Why it's an architecture problem, not a prompt problem; the threat model (the lethal trifecta — untrusted content + private data + the ability to act, and why any one missing leg defuses it); the root cause (the confused deputy — the model can't tell your instructions from the attacker's); the defense patterns in priority order (least-privilege tools, separate read from act, sandbox the execution, human-in-the-loop for high-risk actions), each traced as vulnerable-vs-hardened wiring in code; layering them as defense-in-depth; and a pre-ship security checklist. With flow diagrams and runnable LangGraph code.
Implementation
7 guidesRelease Engineering from Scratch: Build Once, Prove Origin, Version the Mesh
A hands-on build of a real release system from an empty repo. Branch for continuous delivery (trunk-based, not git-flow — its own creator says so); build one immutable artifact and promote the same digest dev to prod (never rebuild); prove where it came from with signing AND provenance (a signature isn't provenance — SolarWinds shipped a validly-signed backdoor), via cosign + SLSA; and the part everyone underestimates — versioning interdependent components without shipping a diamond-shaped disaster (SemVer is an estimate, Changesets/release-trains coordinate it). Real cosign / SLSA-provenance / Changesets / CI config. Every claim sourced.
Account Factory for Terraform (AFT): A Step-by-Step Build
Turn AWS account creation into a git push. This guide stands up Control Tower Account Factory for Terraform in seven phases — deploy the framework from one module (run with Control Tower management-account credentials), wire up the four repos, vend your first account from a verbatim account_request .tf, then baseline it with global and per-account customizations. With the real module inputs, the numbered end-to-end provisioning flow, version pinning, and the black-box gotchas (apply-only-from-default-branch, batches of 5, invoke-success ≠ pipeline-success). Every claim sourced.
GitHub Actions → AWS via OIDC — Implementation (Terraform)
The hands-on build of keyless CI/CD from GitHub Actions to AWS — no long-lived access keys. The trust model (GitHub mints a short-lived JWT per job; AWS STS exchanges it via AssumeRoleWithWebIdentity), the account-boundary rule (one IAM OIDC provider per account, same account as the role), the exact config that makes it work: the provider for token.actions.githubusercontent.com with audience sts.amazonaws.com, an IAM role whose trust policy conditions on aud and a tightly-scoped sub (branch or GitHub Environment) plus a least-privilege permissions policy, the workflow's id-token: write and aws-actions/configure-aws-credentials@v6, the sub-wildcard footgun and fork-PR risk, debugging 'Not authorized to AssumeRoleWithWebIdentity' denials, and the Terraform module topology (shared provider + per-repo role) across multiple accounts. 27 build steps with a diagram for each.
Centralized Egress Inspection — Build It (Terraform, multi-account)
The hands-on build of the centralized-egress pattern in Terraform across accounts: a dedicated network account owns the Transit Gateway (shared to spokes via AWS RAM, auto-accepted through Organizations) and the inspection/egress VPC. The exact resources and their wiring — TGW route tables with the spoke-to-spoke blackhole, the three subnet tiers per AZ, AWS Network Firewall (rule groups → policy → firewall with a per-AZ endpoint), reading firewall_status.sync_states for the endpoint id, the full route chain (TGW subnet → firewall → NAT → IGW), HOME_NET across every spoke CIDR, end-to-end validation, plus day-2 logging and the module topology with a spoke-vending template. 27 build steps with a diagram for each.
Build an Agent Factory on Amazon Bedrock AgentCore (AWS, CLI/CDK)
The same factory built on AgentCore's GA runtime, not classic Bedrock Agents: ownership & IAM boundaries (the bedrock-agentcore.amazonaws.com execution-role trust + iam:PassRole), a reusable AgentCore project template (agentcore.json + app/<agent>/main.py + a least-privilege execution role) deployed to a Runtime endpoint, governed tools via Gateway, Identity (JWT-only) + Memory + Guardrails, observability & endpoint rollback, and the control-plane/per-agent 4-stack split — each step with validation and the real pitfalls (wrong service principal, iam:PassRole, the >53-layer/non-numeric-USER container, GetWorkloadAccessTokenForUserId escalation, LTM provisioning). Anchored to the AWS AgentCore CLI + runtime-permissions docs; Container-build, code-based-agent variant.
Build an AI Agent Factory on AWS (Bedrock Agents + Terraform)
How to build a thin AI Agent Factory: ownership & IAM boundaries, a reusable Terraform agent-template module (agent + action-group Lambda + scoped service role + Guardrail + alias), a vending pipeline that Prepares → versions → aliases each agent, governance gates, observability & alias-rollback, and the control-plane/template module split — each step with validation and the real pitfalls (the Prepare→alias race, the mandatory Lambda resource-based policy). Anchored to AWS's Terraform agent-lifecycle reference; classic Bedrock Agents variant.
AWS Cloud
25 guidesShaping the SD-WAN Fabric on AWS, Part A: The Regional Hub and Keeping Traffic Symmetric
One regional-hub design — where the SD-WAN appliance pair attaches, how it survives losing a node without becoming a single-flow bottleneck, and how every return packet finds its way back to the same firewall — built once on Transit Gateway and once on Cloud WAN, so you see the same design in two realizations (only the resource names change). The hub is a routing statement, not a product: active/active means both appliances advertise the same prefixes with the same AS-PATH so the fabric installs ECMP; active/standby is a longer AS-PATH. ECMP is per-flow, so '20 Gbps aggregate' never means 20 Gbps for one connection — a single flow pins to one Connect peer at ≤5 Gbps; size for flow count. Return-path symmetry is a per-attachment flag (TGW appliance mode / Cloud WAN NFG) and forgetting it is the classic silent asymmetric drop; GWLB is the other way to build the pair, with fail-open and rebalance defaults you must set on purpose. Part A of the design tier of the SD-WAN-on-AWS series; every number sourced against current AWS docs.
SD-WAN Meets AWS: Five Ways Into the Cloud, One Fabric Riding BGP
The AWS side of an SD-WAN rollout isn't one integration — it's five, and they look alike on a slide but diverge hard on bandwidth, encryption, and cost. What SD-WAN actually is beneath the marketing (a control/data-plane split that turns any transport into one encrypted overlay), the BGP every option rides (eBGP between your AS and AWS's, with ECMP across paths as the only way past a single tunnel's ceiling), and the complete map of the five doors: IPsec VPN → Transit Gateway (encryption built in, ~5 Gbps/tunnel), TGW Connect (GRE + BGP, 20 Gbps, trust the underlay), Cloud WAN Connect (global reach + segmentation, tunnel-less up to 100 Gbps/AZ), the vendor appliance on EC2 (full parity, but you run and scale it), and Direct Connect as the private underlay that carries the others rather than terminating anything itself. The real axis is encapsulation versus bandwidth. Article 1 of an SD-WAN-on-AWS series; every number sourced against current AWS docs.
IPv6 in AWS: It's Decided by Routing and Security Groups, Not the Address
Turning on IPv6 in AWS isn't 'IPv4 with longer addresses' — five reflexes break. There's no NAT and no private range, so a subnet is public or private purely by where you route ::/0 (::/0 → internet gateway is public-by-default; ::/0 → egress-only gateway is the outbound-only NAT stand-in that translates nothing); IPv6-only subnets still reach IPv4 via DNS64 + NAT64 (64:ff9b::/96, automatic on the NAT gateway); security groups are per-address-family, so your IPv4 rules do nothing for IPv6 and you fail open or closed; and the reason to do it now is cost — public IPv4 bills $0.005/hr since Feb 2024. Dual-stack to migrate, IPv6-only to stop paying. The whole model in one VPC diagram, then each break greyed-in. Every claim sourced.
AWS Firewall Manager: A Control Plane That Isn't a Firewall
It never inspects a packet — it's the service that stamps AWS WAF, Shield Advanced, security groups, Network Firewall, and DNS Firewall (plus Palo Alto / Fortinet) across every account in your organization and keeps them there. The three stores of truth it rides (Organizations owns membership, AWS Config owns detection, FMS owns intent), why the management account can't be the policy admin, every policy type and what it deploys, the Network Firewall building-blocks chain (IP sets to rule groups to firewall policy to the FMS wrapper to endpoints), scope-detect-remediate, the Config-everywhere cost that dwarfs the fee, and a full setup walkthrough. Every claim sourced.
AWS Owns One End: Building Direct Connect From the On-Prem Side
From the cross-connect patch outward — router, optics, VLANs, BGP, routing policy, encryption — Direct Connect is your build, and that's exactly where every bring-up fails. The demarcation at the cross-connect, dedicated vs hosted, standing up each VIF (VLAN / BGP / ASN / MD5), the 100-route limit and MTU traps, and resiliency (two locations, BFD, LAG) plus encryption (MACsec vs IPsec over a public VIF) — with verbatim Cisco IOS-XE and Juniper Junos config. Every claim sourced.
AWS Control Tower, End to End: The Managed Landing Zone and Where It Bites
One managed service wires AWS Organizations, IAM Identity Center, Config, CloudTrail and StackSets into a governed multi-account landing zone — here's every moving part, and the honest limits. The fixed account skeleton, the three kinds of control (preventive / detective / proactive), how accounts get vended, how drift is caught, Region governance and the 'opting out isn't blocking' misconception, what AWS Config actually costs, and when you outgrow it into Landing Zone Accelerator. Every claim sourced.
Amazon EKS
28 guidesYour Compute Scaled Out, Now DNS Is Timing Out: Taming the CoreDNS Bottleneck
A traffic surge scales your pods flawlessly, then the logs fill with 'i/o timeout' and CoreDNS pins at 100% CPU. The culprit is rarely CoreDNS itself — it's the default ndots:5 quietly turning one lookup into eight, and UDP resolution racing the kernel's conntrack table into flat 5-second stalls. Here's the amplification math, and the fix: NodeLocal DNSCache plus the cluster-proportional-autoscaler, then cutting the fan-out at the source with ndots:2, FQDNs, and autopath. Every claim sourced.
Decoupling Network Ingress: Architecting Cross-Account ALBs for Amazon EKS
Your cluster lives in a locked-down Compute account; your internet-facing ALBs, certs, and DNS live in a monitored Edge account. Here's how a developer deploys a plain Kubernetes Ingress and the platform wires up a secure ALB across the account boundary — a credential-free STS chain (EKS Pod Identity role chaining or IRSA), a cross-account TargetGroupBinding so the cluster only registers pod IPs, and an IngressClassParams that hides the whole thing. Not a single static IAM key. Every claim sourced.
DiskPressure Evicted Your Database to Reclaim a Few Gigabytes of Logs
When a worker node's root disk fills with container logs and image layers, the kubelet flips DiskPressure, fails image garbage collection, and starts killing pods to save itself — and left to defaults it can kill your most critical stateful pod. Here's the eviction algorithm (the nodefs/imagefs signals and thresholds), the hardening fix (isolate /var/lib/containerd and /var/lib/kubelet onto a dedicated volume; Bottlerocket does it by default), and the two levers the ranker sorts on: PriorityClass and ephemeral-storage limits. Every claim sourced.
Why Your P99 Latency Spikes on EKS While CPU Sits at 40%
Your service is slow, P99 latency spikes into seconds, yet average CPU is flat at 40% — no restarts, no OOM, no exceptions. It's being silently strangled at the Linux kernel level by the mechanism meant to protect it: the Completely Fair Scheduler. Here's how a CPU limit becomes a per-100ms budget the kernel enforces at a cliff, the throttle-ratio metric that exposes it, and why dropping CPU limits (while keeping requests) is usually the fix. Every claim sourced.
GitOps Drift Control That Survives a 2 a.m. Incident
Strict ArgoCD self-heal and live incident response pull in opposite directions — the 2 a.m. kubectl fix that reverts in ten seconds. The answer isn't loosening enforcement; it's shrinking the reconciled surface (secrets to ESO, toggles to a flag store, HPA fields to ignoreDifferences) and adding one scoped break-glass lever, so self-heal can stay strict because there's nothing legitimate left for it to fight. Every claim sourced.
The Custom Networking Trap: Asymmetric Routing Across a Transit Gateway
You enabled custom networking to stop running out of IPs. Pod-to-pod works, egress works — then a pod tries to reach RDS across a Transit Gateway and the connection just hangs, packets vanishing though the routes look right. Here's how splitting a node into two networks makes replies return on the wrong path, why strict rp_filter silently drops them, and the two fixes: mask the pod IP with default SNAT, or make the round trip symmetric (return routes + TGW appliance mode). Every claim sourced.