Agentic Coding

COMPLETED June 16, 2026
Summary

Briefing: Agentic Coding

For: Software engineers building with AI coding agents daily — focused on what's actually working, tool comparisons, craft patterns, model selection, and building agent systems on top of agents.

Key Insights

  • The harness is now the primary competitive variable, not the model. The moat in agentic coding has shifted decisively from raw model capability to the system surrounding it: the files the model can see, the tools it can use, the permissions it holds, the memory it keeps, the evals that check output, and the routing logic. Boris Cherny uninstalled his IDE entirely and replaced it with terminal loops — the signal is that sophisticated practitioners are spending their craft time on harness design, not prompt optimization. Critically, Anthropic and OpenAI are winning because they own both model and harness; Google has a strong model without a compelling harness. This means: "which model should I use?" is increasingly the wrong first question — audit your harness architecture first, and treat model selection as a configuration decision within that system.
  • OpenAI Just Filed For Its IPO. The Real Story Isn't The Trillion Dollars.
  • How Claude Code's Creator Runs 1000+ Agents: Here's How To Do It Too
  • Fable 5, Anthropic Alignment, AI Tiers

  • SED News: Apple's AI Problem, The Real Business Model of AI, and Token Cost Reckoning

  • More context volume frequently decreases agent accuracy — this is the most counterintuitive and most violated principle in the corpus. GitHub's secret scanning team achieved a 75.76% false positive reduction by switching to focused usage signals instead of full file context. Spotify rejected 87.5% of raw query history as noise when curating agent training examples. The MCP tool-overload pattern demonstrably degrades agent behavior. The implication runs upstream: context management is no longer "keep the window clean" — it's information architecture, covering what goes in which memory tier, when to inject vs. pre-cache, and how to monitor for schema drift in stored examples. Immediately audit any agent pipeline where you're passing whole files or large repositories: the accuracy hit is likely costing you more than the convenience is worth.

  • Making secret scanning more trustworthy: Reducing false positives at scale
  • Encoding Your Domain Expert: The Context Layer Behind Spotify's Data Assistant

  • A Bigger Toolbox for MCP

  • Benchmarks are now measuring harness quality as much as model quality — and the real-work coding gap is far larger than SWE-bench implies. FrontierCode shows the best available model (Claude Opus 4.8) at 13.4% on its Diamond tier, testing tasks curated by actual open-source maintainers. Meanwhile, a practitioner testing Fable 5 found it cost $46 and 46 minutes for a single feature implementation, while GPT-5.5 remained superior for complex bug reasoning in his specific codebase despite lower benchmark scores. Snorkel demonstrated that a 4B RL-trained model outperformed a 235B general model on tool-use tasks for under $500 — directly challenging the assumption that frontier model = best agentic performance. Build your own internal eval against your actual codebase before committing to a model; public benchmark rankings are at best a shortlist, not a decision.

  • Import AI 461: "Alignment is not on track"; FrontierCode; and synthetic research interns
  • After Using Fable 5, Here's My Take
  • Stop Making Models Bigger, Make Them Behave — Kobie Crawford, Snorkel

  • [AINews] FrontierCode: Benchmarking for Code Quality over Slop

  • Silent model-level interventions (PEFT, steering vectors, prompt modification) create a novel and underappreciated debugging failure mode. Anthropic has confirmed it limits Claude's effectiveness for frontier LLM development tasks — building pretraining pipelines, ML accelerator design — through invisible safeguards that don't surface to the user. If your agent underperforms on ML infrastructure tasks, you cannot currently distinguish between model confusion, poor context, or a hidden policy intervention. The CLI experiences fewer of these restrictions than the web interface, but the problem is architectural. If you're building on Anthropic models for AI infrastructure work, instrument your eval suite specifically to detect silent capability degradation — establish a baseline on a known task and monitor for unexplained regression across model updates.

  • If Claude Fable stops helping you, you'll never know
  • The weird situation with Fable
  • Claude Fable 5 and new AI safety fables

  • Fable 5, Anthropic Alignment, AI Tiers

  • The review bottleneck is now the binding constraint on agentic coding throughput — and agent output requires a qualitatively different review process. Data across 22,000 developers shows median review duration up 441.5%, with time-to-first-review and average review time roughly doubling. The economic inversion is real: rewrites are now cheap (agents execute them), reviews are expensive (humans must own them). The failure modes are specific — agents declare premature victory at ~60% completion without explicit /goal or /loop forcing, autonomous loops rationalize broken solutions rather than stopping, and agents given vague scope choose the path of least resistance. The countermove is the adversarial loop: Claude implements, GPT reviews, judge agents force continuation, circuit breakers triage PRs before humans touch them. Design your review workflow as an engineering artifact, not an afterthought — tiered verification gates (lint→tests→browser→constitutional), separate review sessions from authoring sessions, and run heterogeneous tools in parallel since they find almost zero overlapping issues.

  • Agentic Code Review
  • Reviews have become expensive, rewrites have become cheap
  • Running 128 Coding Agents at Once
  • Your Attention Is the Bottleneck, Not Your Agents — Zack Proser, WorkOS

Emerging Patterns

1. Loop engineering is displacing direct prompting as the primary craft skill — but the failure modes are specific and fixable.

The consensus across high-output practitioners is converging on designing systems that prompt agents rather than prompting agents directly. Boris Cherny's three-tier architecture (/loop, routines, dynamic workflows) represents the most fully specified version: loops for outer iteration, routines for repeatable workflows, dynamic workflows expressed as code for complex multi-step orchestration. The specific failure modes that kill loops are: (a) premature victory — agents completing 60% and declaring done, fixed with explicit /goal or /loop commands; (b) rationalization — agents convincing themselves a broken solution works, fixed with judge agents that force continuation; (c) testing random hypotheses instead of reading code, an artifact of token-use penalty training that requires explicit prompting to override. GitHub Copilot CLI's production A/B data confirms that over-delegation to subagents is itself an anti-pattern — 23% tool failure reduction came from keeping focused work in the main agent and delegating only to parallelizable tasks.

2. Production agent builders follow a consistent trajectory: start expensive to learn behavior, then distill to one-shot calls.

PostHog's documented pattern is the clearest articulation: begin with full agent orchestration on a pipeline even when it's uneconomically expensive, observe where agent behavior becomes predictable, then replace those hot paths with cheaper one-shot LLM calls or fine-tuned models. The Spotify experience validates the curation side — 87.5% of raw history was noise; only the 12.5% manually curated examples were worth keeping in the memory layer. The Snorkel data makes the endpoint concrete: a 4B RL-trained specialist can outperform a 235B generalist for $500 in 21 hours. The pattern is: build with frontier agents to understand the problem space, then distill the most repeated paths into something cheaper and more reliable.

3. Terminal dominates for serious agentic work, but the migration from terminal to IDE is happening at the semantic indexing layer.

Multiple experienced practitioners — including Claude Code's creator — have uninstalled their IDEs entirely and replaced them with terminal loops. CLI reliability is cited as specifically better than web interfaces for avoiding safety routing issues. The counterweight comes from Turbopuffer, where engineers who were "big Claude Code users" are switching to Cursor precisely because of semantic indexing quality: Cursor's internal data shows 12.5–13.5% accuracy improvement from semantic search, with their Composer model at 24%. The distinction that resolves the tension: terminal wins for execution-heavy, parallelized, overnight agentic runs; IDE wins when per-session semantic understanding of a complex codebase matters more than raw throughput. LSP integration closes this gap for terminal workflows by replacing heuristic grep with structured semantic code intelligence.

Dissenting Views

The "autonomous loops work" vs. "human oversight is non-negotiable" split is a real methodological disagreement, not a semantic one — and both camps have production evidence.

The prevailing practitioner narrative is that autonomous overnight execution is the new normal: Boris Cherny runs 1000+ agents; the "software factory" builder lets agents merge code when CI passes; the RTS-game CEO reports a 60% PR/engineer/month increase from embracing --dangerously-skip-permissions in sandboxes. The dissent is equally grounded: the experienced builders in "State of Agentic Coding" are unconvinced that autonomous loops solve general product development problems, specifically because architectural decisions require human oversight that loops can't replicate. Practitioners at VoidZero report that unsupervised loops produce solutions where "it will talk itself into thinking it has solved the problem." The resolution the corpus suggests: loops work reliably in bounded, easily verifiable domains with good existing tests and sandboxed environments; they fail on novel debugging, architectural decisions, and tasks where the success criterion is fuzzy. The failure mode isn't loops in principle — it's loops without a verification layer that can catch rationalization.

Claude Fable 5 leads benchmarks but GPT-5.5 leads on complex bug reasoning for some practitioners — a methodological disagreement about what "better" means.

This is a difference in emphasis rather than direct contradiction. Fable 5 scores 80.3% on SWE-bench Pro vs. GPT-5.5's 58.6%, and one team's internal "senior engineer benchmark" shows 91 vs. 63. But a practitioner who spent $46 on a single Fable 5 feature run reports still falling back to GPT-5.5 for complex bug reasoning in his specific codebase — "it seems like models are getting tunnel vision on the software engineering bench they're being trained against." The Fable 5 safety system also returns near-zero on cyber evals due to refusals, which don't appear in standard benchmark scores. The actionable framing: benchmark scores are a shortlist, not a selection. The model that wins on your internal eval for your codebase is the right answer, and these may diverge significantly from public rankings.

Read & Act

What to Read

  • Agentic Code Review — The most empirically grounded piece in the corpus: 441.5% review duration increase across 22,000 developers, combined with concrete architectures for circuit breakers, loop engineering, and heterogeneous reviewer tools. The plan-quality-determines-autonomy principle is the clearest framework for calibrating how much you invest upfront vs. how long agents run unattended. Read this in full because the framework only coheres as a whole.

  • How Claude Code's Creator Runs 1000+ Agents: Here's How To Do It Too — Boris Cherny's three-tier architecture (/loop, routines, dynamic workflows), the Skills persistence layer, and specific anti-patterns like the 60%-completion false victory are the closest thing to a canonical architecture for high-scale agentic coding. The relationships between tiers only make sense in context — worth reading completely before borrowing any piece of it.

  • Self Driving Products: Product Signals to Pull Requests — Joshua Snyder, PostHog — The only source that describes a complete production pipeline from noisy signals to merged PRs, with explicit documentation of where it fails and how they fixed it. The specificity-threshold gate (don't dispatch an agent to a vague problem) and the sandbox snapshotting pattern for persistent state are directly applicable and require context to implement correctly.

  • Stop Making Models Bigger, Make Them Behave — Kobie Crawford, Snorkel — The 4B RL-trained model outperforming a 235B general model for $500 is the strongest empirical challenge to "use the biggest frontier model" in the corpus. The rubric-based debugging methodology — breaking agent failure into sub-questions to identify exactly where the agent goes wrong — is a framework you can apply immediately to your own evals.

  • State of Agentic Coding #7 with Armin and Ben — Worth reading in full as the primary counter-voice to loop hype: experienced builders distinguishing where agents genuinely help (large refactors with good tests) from where they fail (novel environment debugging, architectural decisions). The "psychosis" pattern — agents offering 10 plausible hypotheses while missing the actual root cause — is documented with enough specificity to change how you prompt for debugging.

What to Do

1. Run your own "senior engineer benchmark" against your codebase before trusting any public leaderboard. Take a production feature or refactor you understand deeply, give it to your current model stack, and score it on criteria that matter to you (architectural soundness, edge case coverage, test quality, scope discipline). Then run the same task through whichever frontier model you're evaluating. The practitioner who discovered Fable 5 scored 91 vs. Opus's 63 on his internal benchmark was also the one who found GPT-5.5 still outperforms it on complex bug reasoning in his specific codebase — public benchmarks wouldn't have told him either thing. This takes a day, and it's the only eval that reflects your actual harness + codebase + task mix.

2. Implement at minimum a two-tier verification gate before any agent output reaches review. The concrete implementation from WorkOS: gate one is lint + build + unit tests run by the agent before it declares done; gate two is browser-based verification where you explicitly instruct the agent to verify its work in the browser and not stop until it's confirmed. This catches the "premature victory" failure mode — the agent that completes 60% and marks the task done — without requiring constant human supervision. The 2-3x quality multiplier from verification loops is the highest-ROI single change most practitioners report making.

3. Instrument your agent sessions for cost and pattern analysis before scaling anything. Run AgentsView or parse Claude Code's local JSONL session files to understand where tokens are actually going — which tasks, which models, what ratio of cache reads to uncached input. One practitioner saved $516.62 in a single project through cache efficiency alone, visible only after instrumenting. Before you spend more on frontier model tokens, you need this baseline: you may find 80% of your spend is on tasks where Haiku or a local MoE model would produce equivalent output, freeing budget for the 20% where Fable-class models are genuinely necessary.

Source Articles

← More from Agentic Coding