Gen AI
Everything generative-AI on AWS — Bedrock, agents, and agentic platforms — across every format: primers that build the mental model, interview question banks, and phase-by-phase implementation guides.
Agent Cost & TCO: Token Economics and Latency Budgets
A deep dive on what an agent actually costs to run: why cheaper per-token prices can still produce bigger bills, because the agent loop re-reads its whole growing context every step — so cost scales with the square of the conversation, not the token rate. The dominant driver (the quadratic re-read) and how prompt caching and compaction bend it; the price surface (model tier, input/output ratio, caching, batch); the biggest lever — routing by difficulty so a cheap model handles the easy majority and an expensive one only the hard tail; the latency-vs-accuracy budget; infrastructure vs. token cost; and a cost-control checklist. With diagrams and worked token math.
Multi-Agent Orchestration
The seventh article in the agentic series: when one agent's context gets too crowded with tools and instructions, you split it into many — but the agents were never the hard part, the handoffs are. Why you split at all (context isolation, specialized tool sets, parallelism) and why you shouldn't until one agent genuinely breaks; the three topologies on one spectrum (supervisor/worker as the default, hierarchical supervisors-of-supervisors for scale, swarm peers with no boss); why the handoff — what state transfers, what's summarized, what's dropped — is the real system; containing the blast radius when one sub-agent fails or loops; A2A handoffs across team/org boundaries; and how to choose a topology. With flow diagrams and runnable LangGraph code.
Human-in-the-Loop Approval for Autonomous Agents
The sixth article in the agentic series: autonomy removed the human checkpoint that used to sit between intent and consequence, and HITL puts a calibrated one back — without it, an agent's first catastrophic mistake is also its last. Why approval is an architecture decision, not a UX afterthought; gating by risk class rather than by step (reversible/cheap actions run free, irreversible/expensive ones pause); the mechanism — pause and resume on the checkpointer, so a paused graph survives a restart; the failure mode on the safe-looking side (approval fatigue, where too many prompts train humans to rubber-stamp); HITL vs HOTL (does the gate block, or just observe?); designing the wait itself (timeouts, escalation, what the human sees); and a pre-ship checklist. With flow diagrams and runnable LangGraph interrupt() code.
Agent Security & Prompt Injection
The fifth article in the agentic series, and a design pattern rather than a primer: prompt injection isn't a bug you patch, it's an architecture an attacker hijacks with a sentence hidden in a web page. Why it's an architecture problem, not a prompt problem; the threat model (the lethal trifecta — untrusted content + private data + the ability to act, and why any one missing leg defuses it); the root cause (the confused deputy — the model can't tell your instructions from the attacker's); the defense patterns in priority order (least-privilege tools, separate read from act, sandbox the execution, human-in-the-loop for high-risk actions), each traced as vulnerable-vs-hardened wiring in code; layering them as defense-in-depth; and a pre-ship security checklist. With flow diagrams and runnable LangGraph code.
Agent Evaluation: Trajectories, Tool-Calls, Task Completion
The fourth primer in the series: how do you know the agent is any good? You stop grading it the way you graded a plain LLM app. A model hands you an output to check against a reference; an agent hands you a trajectory, and the output is only its last step — so you grade the path. Why final-answer eval is a trap for agents; the three layers worth measuring (final response, trajectory, single tool-call) and why most teams measure only the first; the two scorer families (reference-based vs reference-free / LLM-as-judge) and matching the scorer to the question; the two concrete ways to grade a trajectory from the open-source agentevals package (when you know the right path vs when you don't); pass^k — the reliability metric that exposes non-determinism that pass@1 hides; offline (pre-release, gate the PR) vs online (post-release, every production failure becomes a new dataset case) eval on the lifecycle; and a pre-ship checklist. With diagrams and runnable code.
Agent Memory Architectures
The third primer in the series: a stateless model forgets everything between calls, so the entire feeling of an agent that 'knows you' is an illusion you construct by deciding what text to re-send each turn. The two layers you must never conflate — short-term thread state (the checkpointer, keyed by thread_id) vs long-term cross-session knowledge (the store, scoped by user_id); the context window as a budget, not a bucket, and the compaction that keeps it honest; long-term memory's three jobs (semantic, episodic, procedural) and why one bucket for all three breaks retrieval; the recall and save nodes that bracket the loop; resuming a thread by its transcript vs greeting a fresh session with the user's facts; the line between memory and RAG; and a 17-row pre-ship checklist. With flow diagrams and runnable LangGraph code.
Tool Use & the Agent-Computer Interface
The sequel to The Agent Loop: how to design the tools an agent calls so it picks the right one, fills the arguments correctly, and recovers when they fail. Why a tool is model-facing documentation, not an API; the brain-in-a-jar mechanism of a single tool call traced on the wire (and why it's two API calls); curating a narrow-but-consolidated tool layer with the ~15-tool cap; schema hygiene (enums over naked strings, flat objects, additionalProperties:false, Pydantic validation at both ends); errors as instructions for self-correction; idempotent writes and stateless tools; routing instead of dumping past ~20 tools (progressive discovery, a single meta-tool, a hierarchical router, semantic retrieval); and evaluating tools against a golden set. With diagrams and runnable Anthropic-SDK and LangGraph code.
The Agent Loop
An in-depth primer on the one primitive every agent reduces to: a model in a loop with tools — think, act, observe, repeat. Walks ReAct as the base loop, when Plan-and-Execute and Reflection earn their place, how production layers all three into a hierarchical architecture (an outer planner, ReAct sub-agents per step, a reflection gate), and the deterministic guardrails that actually keep it safe — strict termination, context pruning, execution sandboxing, and human-in-the-loop interrupts. With diagrams and runnable Anthropic-SDK and LangGraph code.
The Agentic System Design Map
An interactive blueprint board for designing a production AI agent — 34 architectural decisions grouped into five leverage tiers and ordered by blast radius, not lifecycle. Map your Tier 1 framing (agent vs. workflow, autonomy ceiling and reversibility, success criteria, failure-mode severity) and the board cascades common downstream defaults across architecture, capability, production, and sustaining — each with the reasoning attached and live diagnostics when your choices contradict each other. Spectrums are mutually exclusive; stackable capabilities combine. Compile an exportable spec manifest of the architecture you draft. The thesis: the highest-leverage decision in the set is autonomy-and-reversibility, and nearly every expensive agent failure is an early-tier decision made late.
Embeddings & Vector Search — a Primer
An in-depth, engineer-leaning explainer (not Q&A): the one mental model — an embedding turns meaning into coordinates, so search-by-meaning becomes find-nearest-points. Cosine vs dot product vs Euclidean and the magnitude trap, why on normalized vectors all three rank identically, why brute-force k-NN is exact but O(N) and breaks at scale, approximate nearest neighbor and the recall-vs-latency dial (recall@k), and how the HNSW graph index actually works (layered proximity graph, greedy descent, M/ef) alongside IVF and product quantization. With diagrams and a runnable zero-dependency notebook that makes cosine, dot, and L2 disagree then agree.
Evals for LLM Apps — a Primer
An in-depth, engineer-leaning explainer (not Q&A): the one mental model — an eval is a test suite for a non-deterministic system, so you measure quality with a grader over a dataset, not exact equality. The anatomy (dataset → task → grader → score), the three grading methods (code, human, LLM-as-judge) and their cost-vs-nuance trade, running LLM-as-judge well (rubrics, structured verdicts, reason-then-discard, a different model) and its real biases (position, verbosity, self-enhancement), plus offline vs online, regression testing as CI-for-prompts, and guardrail testing. With diagrams and a runnable eval-harness notebook that exposes the judge flipping under position bias.
Model Context Protocol (MCP) — a Primer
An in-depth, engineer-leaning explainer (not Q&A): the one mental model — MCP is a universal port that turns M×N custom integrations into M+N — then the host/client/server architecture (one client per server, 1:1), the JSON-RPC data layer inside the transport layer, the three server primitives (tools the model calls, resources the app loads, prompts the user picks), the lifecycle traced in real messages (initialize → negotiate capabilities → list → call → notify), stdio vs Streamable HTTP, and client primitives (sampling, elicitation). With diagrams and a runnable, zero-dependency protocol-handshake notebook.
RAG, End to End — a Primer
An in-depth, engineer-leaning explainer (not Q&A): the one mental model — RAG is a funnel that selects the model's input, not its weights — traced stage by stage (chunk → embed → retrieve → rerank → generate). Why 'similar' isn't 'relevant' (with a runnable zero-dependency demo where the answer-bearing chunk loses to a vocabulary match), retrieve-then-rerank (bi-encoder recall vs cross-encoder precision), the four-stage failure cascade where the right chunk gets dropped before the model sees it, and managed RAG on Bedrock Knowledge Bases — plus when RAG beats a bigger context window or fine-tuning. With diagrams, a runnable notebook, and a hands-on exercise.
Build an Agent Factory on Amazon Bedrock AgentCore (AWS, CLI/CDK)
The same factory built on AgentCore's GA runtime, not classic Bedrock Agents: ownership & IAM boundaries (the bedrock-agentcore.amazonaws.com execution-role trust + iam:PassRole), a reusable AgentCore project template (agentcore.json + app/<agent>/main.py + a least-privilege execution role) deployed to a Runtime endpoint, governed tools via Gateway, Identity (JWT-only) + Memory + Guardrails, observability & endpoint rollback, and the control-plane/per-agent 4-stack split — each step with validation and the real pitfalls (wrong service principal, iam:PassRole, the >53-layer/non-numeric-USER container, GetWorkloadAccessTokenForUserId escalation, LTM provisioning). Anchored to the AWS AgentCore CLI + runtime-permissions docs; Container-build, code-based-agent variant.
Build an AI Agent Factory on AWS (Bedrock Agents + Terraform)
How to build a thin AI Agent Factory: ownership & IAM boundaries, a reusable Terraform agent-template module (agent + action-group Lambda + scoped service role + Guardrail + alias), a vending pipeline that Prepares → versions → aliases each agent, governance gates, observability & alias-rollback, and the control-plane/template module split — each step with validation and the real pitfalls (the Prepare→alias race, the mandatory Lambda resource-based policy). Anchored to AWS's Terraform agent-lifecycle reference; classic Bedrock Agents variant.
AI Agent Factory — a Primer
An in-depth, mixed-audience explainer (not Q&A): the production-line mental model — from artisanal one-off agents to a governed factory — the moving parts (templates, orchestration, tools/connectors, governance, lifecycle), the agent lifecycle with governance gates, the four governance pillars, supervisor orchestration and define-once agent specs, and when a factory earns its weight vs an agent framework. With diagrams, a governance checklist, an agent-spec template, and a blueprint exercise.
No posts match — try a different keyword or clear the filters.