Agentic Coding

COMPLETED May 19, 2026
Summary

Briefing: Agentic Coding

Key Insights

Emerging Patterns

  1. Generator-evaluator architecture produces dramatically better output than solo agent loops — but requires adversarial prompt design. Anthropic's internal experiments showed the same prompt with the same model producing a non-functional app in a solo loop and a production-ready application with a generator-evaluator harness. The key mechanism: the generator and evaluator agents negotiate "what done means" before any code is written, creating a contract the evaluator grades against. Critically, Claude out of the box is a "really bad QA agent" due to a generosity bias — the evaluator's system prompt needs explicit adversarial framing, a concrete rubric (design, originality, craft, functionality), and ideally a tool like Playwright to test live rather than reading its own output. The Cloudflare security harness independently arrived at the same design: an adversarial review stage using a different model with no generation capability cuts noise dramatically.
  2. Build Agents That Run for Hours (Without Losing the Plot) — Ash Prabaker & Andrew Wilson, Anthropic

  3. Project Glasswing: what Mythos showed us

  4. Grep and direct search outperform vector retrieval for coding agent tasks — and the token cost difference is 8x. A latent.space aggregation of current practitioner experimentation found that grep-style text search, wrapped in the right harness, matches or beats embedding-based retrieval on coding agent tasks. A separate comparison of SDK vs. MCP for a GraphQL API showed 1 step and 15k tokens for SDK versus 4 steps and 158k tokens for real MCP — an 8.4x token cost difference for identical output. These are harness-level decisions, not model-level ones, and they're compounding: at scale (Vercel reports 35 models in regular use at 10M+ request organizations), routing decisions made at the harness layer determine whether costs are manageable or catastrophic.

  5. [AINews] Cerebras' $60B IPO: Slowly, then All at Once
  6. [AINews] Everything is Conductor

  7. AI Gateway production index

  8. Flat peer-to-peer agent architectures are outperforming hierarchical orchestration for specific task types — but require deliberate context engineering. Pi's peer-to-peer agent talk demonstrates that flat, bidirectional communication between equal agents produces "best information wins" dynamics that hierarchical orchestration misses — valuable information gets stuck in orchestrator-to-worker flows. However, the Pi practitioner explicitly warns: sloppy prompts cause infinite loops, token costs scale linearly with agent count, and you must handle edge cases explicitly. This creates a meaningful tension with standard "use Claude Code / Codex for everything" advice — the practitioners achieving outsized results are building custom harnesses, not using vendor tools as-is. The tradeoff is real: custom harnesses unlock peer-to-peer patterns and compounding capabilities, but require substantial engineering investment that vendor tools avoid.

  9. Pi to Pi: Two-Way Agent Orchestration with the Pi Coding Agent
  10. [AINews] Everything is Conductor

Dissenting Views

Read & Act

What to read

  • Build Agents That Run for Hours (Without Losing the Plot) — Ash Prabaker & Andrew Wilson, Anthropic — The most complete practitioner-level treatment of long-running agent architecture available: generator-evaluator harness design, file system state management, adversarial evaluation rubrics, the model's self-assessment sycophancy problem, and the data showing run duration improved from 1 hour to 12 hours across model generations. The trace-reading debugging methodology alone is worth the time — it's the primary loop for finding and fixing harness issues, not prompt iteration.

  • Ship Real Agents: Hands-On Evals for Agentic Applications — Laurie Voss, Arize — The frameworks here — capability vs. regression evals, the five-part LLM judge rubric, Swiss cheese layering, meta-evaluation with precision/recall, the 200-400 sample sizing guidance — require the full context to apply correctly. If you're currently "vibe checking" your agent, this is the session that tells you what to build instead and in what order.

  • Three Kinds of Software Survive: Tasklet's Andrew Lee on Competing to be a Horizontal Platform — Andrew Lee is operating an agent harness in production with real cost constraints and live model switching decisions. The 30% cost increase from Opus 4.6→4.7 tokenizer change, the smaller-model-calling-larger-model pattern, the migration script as human-approvable artifact, and the file-system-as-context-store implementation details are all specific enough to apply directly. Read this alongside the Anthropic harness talk for complementary architectural perspectives.

  • How Intuit, DoorDash, and Atlassian are adopting AI coding — Hard quantitative data combined with specific architectural patterns (code craft context injection, alpha team feedback loops, prototype-before-discussion mandate, multi-agent SDLC pipelines). If you're trying to make the case internally for where to invest next — beyond raw code generation — this is the evidence base.

What to do

  • Instrument your agent loops with traces before touching any prompt. Spend 15 minutes reading actual execution traces from your last 5 agent runs before your next prompt iteration. The Anthropic practitioners' primary debugging loop is manual trace reading — not experiments, not A/B tests. If you don't have trace infrastructure, add OpenTelemetry instrumentation this week (the Arize Phoenix talk shows exactly how to set this up). You cannot improve what you cannot see, and the failure modes that matter (context rot, premature completion claims, QA bias) are invisible without traces.

  • Add a cache refresh mechanism to any agent loop that runs longer than 60 minutes. The Claude cache invalidates at 62.5 minutes regardless of cache size. If your agent loop spans that boundary without a refresh, you're paying full price to rebuild context that should be cached. Set up a heartbeat or checkpoint event at 55-minute intervals to keep cache alive. This is a mechanical fix that costs 30 minutes to implement and reduces costs materially on any long-running agent.

  • Route all model calls through an abstraction layer before your next production deployment. Given the Anthropic programmatic usage restriction events, treat any direct coupling to a single provider's subscription model as a risk. Set up OpenRouter, Vercel AI Gateway, or your own routing layer now — before a pricing event forces you to do it under pressure. The Vercel production index shows organizations running 35 models at scale; the routing architecture that makes that manageable is the same one that protects you from single-provider dependency. This week's task: move one production agent's model calls behind an abstraction layer with a fallback configured.

Source Articles

← More from Agentic Coding