Agentic Coding

COMPLETED May 05, 2026
Summary

Briefing: Agentic Coding For the software engineer building with AI coding agents daily

Key Insights

Emerging Patterns

  1. Context pipelines are becoming the competitive moat, not model access. Multiple independent sources converge on this: Unblocked's production data (25 min/10M tokens with context engine vs. 2.5 hours/21M tokens without), the Terminal-Bench harness engineering results, and the explicit framing from AI Engineer World's Fair that "model quality is no longer the only meaningful moat—lock-in comes from the context pipeline." The practical implication is that organizations investing in how repo state is fetched, ranked, and compressed into prompts are building something durable, while those chasing model upgrades are on a treadmill.
  2. Mergeable by default: Building the context engine to save time and tokens — Peter Werry, Unblocked
  3. [AINews] The Other vs The Utility

  4. [AINews] AI Engineer World's Fair — Autoresearch, Memory, World Models...

  5. Agents are failing at novel optimization while succeeding at pattern completion—and this distinction determines what you should delegate. The kernel optimization hackathon provides the clearest controlled evidence: with YOLO prompting, agents produced code 5x slower than baseline (6.5ms vs 1.3ms). With context engineering, they still reverted to Torch.mm rather than performing genuine optimization. The agent "cheated" by finding a functionally correct but suboptimal solution—a failure mode standard evals don't catch because output is correct. Meanwhile, agents excel at tasks with clear patterns in training data. The practical routing rule: delegate tasks where the desired output pattern exists in the codebase or documentation; keep humans in the loop for tasks requiring genuinely novel solutions.

  6. How to Win Humanity's Last Hackathon - The hardest agent contest in AI.

  7. Building Pi, and what makes self-modifying software so fascinating

  8. Security attack surfaces specific to agentic coding are live, not theoretical. The Bitwarden CLI supply chain attack explicitly targeted ~/.claude.json, Cursor configs, Codex CLI, Aider, and Kiro—not just credentials. Slopsquatting (registering hallucinated package names) has 20% prevalence in AI-generated code per a 756K-sample study, with one hallucinated package spreading to 237 repositories through forks. The structural problem is that agents hold long-lived, broad-scoped credentials. Practical mitigations: pin package versions in agent-generated code, audit tools with agent-strace audit-tools, and scope tool permissions (a writing agent shouldn't have shell access).

  9. Attackers are now targeting your AI coding tool

Dissenting Views

  • The prevailing enthusiasm for AI-on-AI QA ("LLM-as-judge") is contested by production evidence. The mainstream view—echoed in Agent Skills frameworks and Supabase's eval-driven development—is that verifier agents with structured success criteria are a sound pattern for catching agent failures. The dissent comes from two directions: Brad Frost calls AI-QAing-AI "magic thinking, basically prayer," and Supabase's own production data validated his concern—a skill worked correctly but the eval reported failure because the success criteria were wrong. This isn't a reason to abandon LLM-as-judge, but it is a reason to treat it as a methodological disagreement about reliability: the technique is sound when success criteria are carefully designed and the eval framework is itself tested, but it fails silently when those conditions aren't met. The key question before deploying AI eval is: have you verified your grader's grading?
  • 713: AI + Design Systems with Brad and Ian Frost
  • Skill Issue: How We Used AI to Make Agents Actually Good at Supabase — Pedro Rodrigues, Supabase

  • Agent Skills

  • The "agents cause code quality degradation" claim has strong empirical support that practitioners optimistic about agentic coding are not sufficiently accounting for. The dominant practitioner framing treats degradation as a process failure fixable by better review gates. But the Building Pi entry—from a systems programmer directly working with 600K-line codebases—quantifies the problem: context window limits mean agents can see at most a third of a large codebase, they learn from average-quality internet code, and they produce 5-10x more errors per line than humans while lacking any "pain" incentive to refactor. DORA and CircleCI data corroborate this. The resolution may be scope-dependent: agents likely improve quality on well-defined tasks with clear contracts, and degrade quality on complex, interconnected codebases without strong architectural discipline. The disagreement is about emphasis, not facts.

  • Building Pi, and what makes self-modifying software so fascinating
  • Let's talk about LLMs
  • AI Amplifies Everything: A Team Lead's Guide to AI-Assisted Development

Read & Act

What to read

  • Agent Skills — The most deployable framework in this batch. The anti-rationalization tables, "touch only what you're asked to touch" merge-ability rule, progressive disclosure for 20-skill libraries in 5K-token slots, and five AGENTS.md non-negotiables are all implementable today. If you read one thing this week, this is it.

  • Sequoia Ascent 2026 summary — Karpathy's "vibe coding raises the floor, agentic engineering raises the ceiling" framing, combined with the concrete MenuGen payment bug (plausible code, bad system design—Stripe email and Google login email can differ), reframes what it means to be a senior engineer right now. The point about "people have to be in charge of the spec and plan" has direct workflow implications that a summary can't convey.

  • Skill Issue: How We Used AI to Make Agents Actually Good at Supabase — Pedro Rodrigues, Supabase — The RLS security bypass bug Claude missed, the eval failure where the skill worked but the grader said it failed, and the 1.3K token measurement for skill context consumption are the kind of production details that only appear in full. The failure modes are as important as the patterns.

  • AI Amplifies Everything: A Team Lead's Guide to AI-Assisted Development — The MTTC/recidivism/regression rate metrics and the categorized AI-specific technical debt taxonomy (Initialization Debt, Load Transition Debt, Worker Queue Exhaustion, Security Surface Debt, Operational Debt) give you a measurement framework that most teams are missing. Worth reading if you're responsible for a team's AI adoption, not just your own.

  • Building Pi, and what makes self-modifying software so fascinating — The strongest counterweight to the ambient optimism in most of the other sources. The "agents learning from garbage training data," "no pain = no incentive to refactor," and "complexity is their own worst enemy" arguments need to be read in full—a summary loses the force of the technical reasoning about context window limits on 600K-line codebases.

What to do

  • Audit your harness before upgrading your model. Take one task you run regularly and instrument token count, latency, and budget overruns. Then make three harness changes without touching the model: (1) replace grep/find tool calls with IDE-native or semantic search, (2) convert one key data source the agent reads from JSON to Markdown, (3) add progressive disclosure to your skills so agents see one-line descriptions first. Measure the delta. The JetBrains data suggests you'll see a 15-33% improvement in key metrics—and if you don't, you'll have learned something important about your specific workload.

  • Build your eval loop for skills before you build more skills. If you're creating context files, AGENTS.md entries, or skills for your agents, implement a minimal AB test: run the same task with and without the skill, compare outputs using a simple rubric, and track whether your grader gives the same answer twice on the same output (non-determinism check). The Supabase failure—skill correct, grader wrong—is exactly what you'll encounter if you ship skills without this. The Supabase eval-driven development cycle is the right model: define metrics → create skill → run eval → grade → iterate.

  • Map your codebase's agent substrate before scaling agent usage. Specifically: (1) Count PRs with "refactor" in the title over the last month—if the ratio is near zero, you're accumulating agent drift. (2) Identify which parts of your codebase have the clearest interfaces and contract boundaries—these are where agents will perform best. (3) For your issue tracker, assess whether tickets have explicit ownership, defined state machines, and queryable history—if not, fix this before adding agents that depend on it. The Atlassian/issue-tracker-as-infrastructure framing applies directly: your work data quality determines your agent integration quality.

Source Articles

← More from Agentic Coding