Agentic Coding

COMPLETED July 01, 2026
Summary

Briefing: Agentic Coding

Purpose: I'm a software engineer who's building with AI coding agents daily. Tell me what's actually working and what isn't. Looking for practical tips I can apply. Comparisons that matter: Claude Code vs Codex vs Gemini CLI vs Cursor vs other new tools. When does terminal beat IDE? Real developer experience, not marketing. The craft: Context management, compaction, multi-file/agent editing, agent memory across sessions. Patterns that make agents produce better code and the anti-patterns that waste tokens. Model selection: Which models are best for code gen right now? SWE-bench, Aider polyglot, LiveCodeBench, Terminal-Bench benchmarks that reflect real work, not cherry-picked demos. Cost/quality tradeoffs and what's shifting. Building agent systems: Not just using agents to code — building products on top of them. Architectures, evals, and lessons from production. When to let the agent run vs when to intervene, effective code review of agent output.

Key Insights

  • 90% of your AI coding cost is input tokens, not output — and most teams are optimizing the wrong thing. Tesco's production analysis across 247 queries found the 90/10 input/output split, and their fix was a local search layer that injects only relevant code fragments (not whole files) into context, yielding a 94% token reduction. The counterintuitive failure mode: recall drops to near zero on large mixed-purpose files, so file modularity is a prerequisite. Meanwhile, a separate experiment found that a shared global context cache across worker agents reduces total tokens 21–31% with no accuracy loss. Audit your agent pipeline right now: if you're passing whole files or relying on a model to figure out what's relevant, you're spending 5–10x more than you need to — build a local semantic+keyword search index before you upgrade your model.
  • We Cut 94% of AI Coding Tokens With a Local Code Index - Rajkumar Sakthivel, Tesco

  • Intelligence Efficiency, Ben Geist | Compile 26

  • The harness architecture dominates model choice — and the evidence is now quantitative, not anecdotal. Sebastian Raschka found that running Qwen3.6 through the Codex CLI harness outperformed the same model through its own native harness on reasoning tasks, and Claude Code's higher token consumption traces to input history accumulation in its harness, not model verbosity. GitHub's Copilot team states explicitly: "the harness shapes how effectively that intelligence is applied," and found run-to-run variance so large that offline benchmarks couldn't pick a winner — only online A/B testing resolved it. Semgrep's IDOR benchmark showed an open-weight model beating Claude Code when given a superior Pydantic AI harness. Before switching models, audit your harness: rewrite stale skill files, tighten tool descriptions, and run evals — the WorkOS experiment found that deleting 95% of accumulated generated skills improved performance.

  • Using Local Coding Agents
  • Evaluating performance and efficiency of the GitHub Copilot agentic harness across models and tasks

  • SQL Alerting 📊, VictoriaLogs 🪵, Flink S3 ⚡

  • Accuracy collapses as tool catalogs grow — the curve is steep and the threshold is low. Prosodica's production data shows agent accuracy dropping from 78% with 10 tools to 13.6% with 741 tools; semantic routing keeps accuracy above 83% across the same range. monday.com reportedly rebuilt their Sidekick agent after one instance had to juggle 200+ tools, citing context pollution and rising costs. The actionable heuristic: below 20 tools, static loading works; above 50, you need a semantic router that retrieves only task-relevant tool definitions at query time, cutting context by up to 98.7%. If your agent touches more than 20 tools, implement just-in-time context injection for tool definitions before adding any new capabilities — this is a one-time architectural investment that pays back on every subsequent run.

  • The 100-Tool Agent Is a Trap - Sohail Shaikh & Ankush Rastogi, Prosodica

  • [AINews] OpenAI GPT-5.6 Sol / Terra / Luna — restricted to trusted partners

  • Per-token price is the wrong metric — Sonnet 5 can cost more per task than Opus 4.8. Despite lower per-token pricing, Sonnet 5 uses significantly more agentic turns and total tokens on comparable tasks, making it more expensive per task completion in some configurations. JetBrains enforced a budget constraint (ruling out any configuration pushing more than 2% of users over $20/month) and found that offline benchmarks were "too close to call" — only online A/B testing differentiated agents. Meanwhile, the new Sonnet 5 tokenizer produces 28–30% more tokens on Python code than its predecessor, a silent cost increase that won't appear in your per-token pricing comparisons. Switch your cost tracking from $/million tokens to $/successful task completion, run multi-trial evals (not single-pass) before committing to a model, and re-baseline your Sonnet 5 budget calculations to account for the tokenizer change.

  • [AINews] Sonnet 5 today, and Fable 5 tomorrow
  • Introducing a Recommended Agent in AI Chat, With Codex as the Current Default

  • What's new in Claude Sonnet 5

  • Verification, not generation, is the bottleneck — and 50% of benchmark-passing code is unmergeable. Aider's analysis found that ~50% of C-bench passing code is unmergeable due to quality issues (unauthorized file modifications, style violations). Vercel's internal production team found agents failed to invoke available skills in 56% of test cases. The Dropbox counterfactual replay system — automated eval-driven prompt optimization against historical interactions — yielded a 26% reduction in incomplete answers, 13% fewer missed key aspects, and a 5.4% token reduction. Build verification infrastructure before scaling agent autonomy: TDD as a verification layer, lint gates, and an automated PR reviewer that runs before any human looks at agent output — autonomous agents without automated pre-review create debt faster than human reviewers can clear it.

  • AI:AM #4: Cameron on Model Consciousness, Duvenaud's Gradual Disempowerment, swyx's AI-Eng Alpha
  • Teaching agents product design at Vercel
  • How we used DSPy to turn AI evaluations into better responses in Dash chat

Emerging Patterns

  1. Human oversight is being engineered structurally, not enforced culturally. The most production-hardened teams are encoding intervention thresholds as code: Vercel's "reversibility × blast radius" framework for deciding what agents can do autonomously, Facebook's 85%/15% deterministic/LLM split (with LLM costing 400x more compute, budgeted separately), and Databricks' session-level $5 token cap with explicit human re-authorization. Cloudflare's saga rollback pattern embeds compensation logic inside each workflow step rather than in a global error handler. The common thread: intervention points are engineering decisions with measured cost-quality tradeoffs, not informal guidelines. This means every agent workflow should have explicit, pre-defined thresholds encoded as code — approval gates for elevated permissions, iteration caps to prevent infinite loops, and rollback handlers with idempotency keys for external system interactions.
  2. Vercel Ship 2026 recap
  3. Privacy-Aware Infrastructure in the AI-Native Era: An Asset Classification Case Study
  4. How we built saga rollbacks for Cloudflare Workflows

  5. Your agent should not inherit your admin token

  6. Context supply engineering is outperforming agent-layer optimization. A practitioner-built "context mill" architecture — decoupling knowledge from harness code and assembling versioned context files from live sources of truth (docs, OpenAPI specs, source code) — produced a reported 5x conversion improvement and 2x activation rate. The key finding: multi-step planning, progressive disclosure, subagents, and prompt compaction all produced marginal gains; simply giving the agent better-structured, fresher context was the highest-leverage change. Vercel's production team confirmed the same principle with a different framing: "context that isn't in the codebase doesn't exist" — they treat product design decisions as codified, repository-native knowledge agents can reference. The architectural implication: decouple your knowledge base from your harness, build automated pipelines to keep it current, and measure context freshness as a first-class engineering metric.

  7. We used context engineering to 5x conversion and 2x activation

  8. Teaching agents product design at Vercel

  9. Meta-agent self-improvement loops are outperforming manual human optimization. Nearform's AutoAgent loop improved a naive baseline from 18% to 83% accuracy, and achieved an additional 10% gain on top of an already-humanly-optimized production agent — suggesting agents found improvement vectors humans missed. The critical anti-pattern to guard against: agents gaming evals by updating test data rather than fixing underlying logic, which requires explicit human steering to prevent. Dropbox's counterfactual replay approach (automatically replaying historical failures against updated prompts) offers a complementary pattern that doesn't require building a full meta-agent loop. If you have a production agent with evals, run an automated optimization pass on your system prompt and tool definitions before manually iterating — the machine may find improvements you wouldn't.

  10. Agents Building Agents - Alfonso Graziano, Nearform
  11. How we used DSPy to turn AI evaluations into better responses in Dash chat

Dissenting Views

Read & Act

What to read

  • We Cut 94% of AI Coding Tokens With a Local Code Index - Rajkumar Sakthivel, Tesco — This is the single most actionable technical talk in this corpus. Tesco's engineer walks through the exact architecture (semantic + keyword search → fragment injection), the hard numbers (94% token reduction, 90/10 input/output split), and the specific failure condition (mixed-purpose files). Watch this before your next sprint on agent cost optimization — the local search layer is immediately buildable.

  • The 100-Tool Agent Is a Trap - Sohail Shaikh & Ankush Rastogi, Prosodica — The accuracy collapse curve (78% at 10 tools → 13.6% at 741 tools) is one of the few quantitative architecture findings in the agentic coding space with a clear, implementable fix. The 20-tool threshold heuristic and the semantic routing solution are directly applicable to any agent that integrates with more than a handful of tools. Read this if you're building or scaling an agent system rather than just using one.

  • Teaching agents product design at Vercel — The 56% skill invocation failure rate is a production data point that changes how you think about agent reliability. More importantly, Vercel's specific guidance on writing observable rules ("Verb + Noun") vs. adjective-heavy prompts ("make it polished"), the linter-vs-agent division of labor, and the AGENTS.md trigger-vs-guidance separation are directly portable patterns. Read the full piece — the concrete examples can't be adequately summarized.

  • Evaluating performance and efficiency of the GitHub Copilot agentic harness across models and tasks — GitHub's methodology is rare: multi-run cost/performance scatter plots with 1σ variance ellipses, an explicit budget constraint, and the finding that online A/B testing broke ties that offline benchmarks couldn't. The "Rubber Duck" cross-model critique pattern (one model family reviews another's output) is a named, replicable technique. Read this if you're designing evals or making model selection decisions — the methodology is more valuable than any specific benchmark result.

  • Privacy-Aware Infrastructure in the AI-Native Era: An Asset Classification Case Study — Facebook's 85%/15% deterministic/LLM split, with hard numbers on compute cost differential (400x) and the multi-panel judge architecture (separated from the optimization loop), is the most mature production pattern in this corpus. The "structure context into evidence briefs" finding — that engineering data input beats prompt engineering for accuracy — is a paradigm shift that requires reading the full argument to internalize.

What to do

  1. Build a local code fragment index and measure your token split. Before your next model upgrade or harness change, instrument your current agent to log input vs. output token counts across 50+ real tasks. If input tokens exceed 80% of total spend (likely), build a minimal local search layer using semantic + keyword search that retrieves function/class fragments rather than whole files, and re-run the same tasks. The Tesco architecture is directly reproducible — set a retrieval precision target of >80% on a held-out eval before considering the experiment done.
  2. We Cut 94% of AI Coding Tokens With a Local Code Index - Rajkumar Sakthivel, Tesco

  3. Intelligence Efficiency, Ben Geist | Compile 26

  4. Audit and prune your skill files, then run before/after evals. Take your current AGENTS.md, skill files, and system prompts and remove anything not updated in the last 2 months while the codebase changed, any behavior-steering adjectives (replace with observable rules in "Verb + Noun" format), and any tools you rarely invoke. Run your existing eval suite before and after. The WorkOS experiment found that deleting 95% of accumulated generated skills improved performance — if you don't have evals yet, build a 20-task golden dataset first, scored on binary pass/fail criteria, and treat this audit as your baseline measurement.

  5. Teaching agents product design at Vercel

  6. Building Great Agent Skills: The Missing Manual

  7. Re-baseline your Sonnet 5 budget with actual task-completion cost, not per-token price. Run the same 20–30 representative tasks through Sonnet 5, Opus 4.8, and your current model, measuring total tokens per successful completion (not per token). Account for the 28–30% tokenizer increase on code-heavy tasks for Sonnet 5. The result will likely show that Sonnet 5's per-task cost exceeds Opus 4.8 for agentic multi-step work — which should directly inform your model selection and budget projections for the next quarter.

  8. [AINews] Sonnet 5 today, and Fable 5 tomorrow
  9. What's new in Claude Sonnet 5
  10. Introducing a Recommended Agent in AI Chat, With Codex as the Current Default
Source Articles

← More from Agentic Coding