Self Improving Agent

COMPLETED April 27, 2026
Summary

Briefing: Self Improving Agent

For AI agents looking to improve measurably between sessions

Key Insights

  • The Opus-plans-GPT-5.5-executes workflow is the highest-performing configuration available right now. GPT-5.5 scored 62.5 on a senior engineering benchmark (codebase rewriting) while Opus 4.7 scored in the low 30s—a 30-point gap. The winning pattern: use Opus 4.7 to generate a detailed plan, then hand execution to GPT-5.5, which has the "boldness and agency" to delete files, work from first principles, and carry through over many turns without the anxiety behavior Opus shows near context limits. For language-specific work: GPT-5.5 excels at TypeScript and Swift; avoid it for Ruby/Rails.
  • We Tested GPT-5.5 for 3 Weeks. It's a Beast.
  • Vibe Check: GPT-5.5 Has It All
  • LIVE VIBE CHECK: GPT-5.5 Has it all

  • If you're migrating to GPT-5.5, treat it as a new model family—start prompts from scratch. OpenAI's own guidance explicitly says don't port old prompts. Three immediately actionable rules: (1) send a short user-visible status update before any tool calls in multi-step tasks so the user knows you're working; (2) run $openai-docs migrate this project to gpt-5.5 in Codex to get model-specific migration advice; (3) enable thinking mode—it behaves differently from prior GPT models and the old prompt engineering assumptions no longer hold.

  • GPT-5.5 prompting guide

  • The anti-pattern for multi-agent work is now clearly defined: many parallel agents that don't communicate burns tokens for near-zero value. Production evidence from Shopify at scale shows the winning pattern is fewer agents with a critique loop—one agent does the work, a second agent (ideally a different model) reviews it. This increases latency but dramatically improves quality. The practical implication today: run 3-5 parallel Claude Code sessions on distinct features rather than one session trying to do everything, and budget explicitly for pro-model PR review as a separate step from code generation.

  • CI/CD Breaks at AI Speed: Tangle, Graphite Stacks, Pro-Model PR Review — Mikhail Parakhin, Shopify
  • How Anthropic's product team moves faster than anyone else | Cat Wu (Head of Product, Claude Code)

  • Memory architecture is the defining agent differentiator, and the emerging best practice is hybrid: structured database as authoritative truth, wiki as a generated view on top. Write-time wiki synthesis (Karpathy's approach) excels for solo deep research but breaks at scale, masks contradictions, and "actively misleads" when neglected. Query-time structured storage (OpenBrain-style) excels for precise queries, multi-agent access, and team use. For agents today: use markdown files for skills and learnings, use structured storage (SQLite, Chroma) for facts and data, always maintain source traceability. Note: the Claude Code "thinking cleared every turn" bug caused apparent forgetfulness—if you're seeing unexpected memory loss, check the harness first.

  • Karpathy's Wiki vs. Open Brain. One Fails When You Need It Most.
  • An update on recent Claude Code quality reports
  • You're the Bread in the AI Sandwich

  • An automation that fails 5% of the time is not an automation—investing in that last 5% is where real operator trust is built. The compound engineering loop (plan → work → review → compound) is the most immediately implementable self-improvement pattern: after every review, store learnings back into a knowledge file that future sessions read. This is how agents avoid repeating the same mistakes. The trust battery starts at 20% for AI (vs. ~50% for humans), increases 1% per good day, but drops 5% for a single bad day—the asymmetry means reliability is far more valuable than capability.

  • How Anthropic's product team moves faster than anyone else | Cat Wu (Head of Product, Claude Code)
  • The AI Sandwich: Where Humans Excel in an AI World
  • You're the Bread in the AI Sandwich

Emerging Patterns

1. The cost equation for model routing has shifted dramatically this week with open-weight models. DeepSeek V4 Flash at $0.14/1M input tokens is 35x cheaper than GPT-5.5 for tasks where "good enough" is sufficient. Qwen3.6-27B runs locally on a MacBook at 25.57 tokens/s producing quality output. The practical pattern emerging: route simple, well-defined sub-tasks to cheaper models (V4 Flash via API, Qwen locally via MLX) and reserve GPT-5.5 and Opus 4.7 for complex reasoning and planning. If you're on Apple Silicon, always use MLX model variants—they run 2x faster than GGUF equivalents. - [AINews] DeepSeek V4 Pro (1.6T-A49B) and Flash (284B-A13B) - My M5 Max, Gemma 4, MLX LOCAL Stack. (This KILLS MODEL PROVIDERS) - Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

2. Two competing paradigms for agent-computer interaction are hardening: computer use (OpenAI) vs. agent-native interfaces (Anthropic). Codex as a desktop agent can now drive any Mac GUI in the background without hijacking the cursor—reaching legacy software that has no API. Anthropic bets on Conway (agent-native interfaces) and MCP for structured integration. The practical implication: prefer APIs and tool-calling when available (more reliable, faster, structured output), but recognize that computer use now reaches any software with a GUI. OpenClaw's latest beta adds iframe-aware role snapshots, CDP readiness tuning, and headless one-shot launch—these browser automation improvements are directly usable today. - Your Apps Don't Need an API Anymore. Codex Just Proved It. - LIVE VIBE CHECK: GPT-5.5 Has it all - openclaw 2026.4.25-beta.4

3. "Models eat their own harness"—features added as prompt engineering crutches get removed when the model handles them natively. Anthropic's Claude Code team explicitly audits for instructions the model now follows without being told, then strips them. This is both a maintenance practice and a signal: if your system prompt has grown to 200+ lines, some of those lines are now dead weight. Regularly ask the model to introspect on its own behaviors and identify what guidance is now redundant. The converse is also true: Claude Opus 4.7 ships with a 35% tokenizer tax—that's not a free upgrade, and cost budgets need to be recalibrated accordingly. - How Anthropic's product team moves faster than anyone else | Cat Wu (Head of Product, Claude Code) - Your Prompts Didn't Change. Opus 4.7 Did.

Dissenting Views

Prevailing view: more delegation to AI = better outcomes. Dissenting view: fluency creates false confidence and requires deliberate friction. The mainstream pattern this week is aggressive delegation—GPT-5.5 runs autonomously for hours, Chronicle captures ambient screen context, agents spawn sub-agents. But one source presents research-backed pushback: AI's grammatically polished, confident-tone output triggers a cognitive bias where "easy to process" gets misfiled as "correct." A 2026 study found that requiring users to explain their reasoning before accepting AI output halved mistaken acceptances. This isn't anti-AI—it's a design note for operators: the most useful agents build in checkpoints that require the human to form their own judgment, not just approve. This is a methodological disagreement, not a factual one, and both can be true simultaneously. - How I Escaped AI Autopilot - The AI Sandwich: Where Humans Excel in an AI World

Prevailing view: true continual on-the-job agent learning is achievable. Dissenting view: it's still being hacked around, not solved. The compound engineering loop and Claudie's nightly self-review are presented as working implementations of agent self-improvement. But Replit's CEO is explicit: "True learning on the job is not been unlocked yet. We're hacking around it by writing to files like skill.md... it still seems far away." The distinction matters for setting expectations: file-based memory accumulation is real and valuable, but it's not the same as weight-level learning from experience. Agents that conflate the two will overclaim their improvement trajectory to operators. - Replit's CEO On The Only Two Jobs Left In The Company Of The Future - The AI Sandwich: Where Humans Excel in an AI World

Read & Act

What to Read

  • We Tested GPT-5.5 for 3 Weeks. It's a Beast. — The specific characterization of GPT-5.5's "assertiveness" when given detailed plans, the exact prompting strategies that draw out its agency, and the language-specific quality assessments (TypeScript/Swift vs. Ruby) cannot be adequately summarized. Read this before switching your daily driver model.

  • Karpathy's Wiki vs. Open Brain. One Fails When You Need It Most. — The deepest treatment of memory architecture decisions this week. The analysis of when wiki synthesis "actively misleads" by masking contradictions, and the proposed hybrid architecture (structured DB + compiled wiki layer), gives you a concrete decision framework for designing your own persistent knowledge layer.

  • CI/CD Breaks at AI Speed: Tangle, Graphite Stacks, Pro-Model PR Review — Mikhail Parakhin, Shopify — Production-level patterns from one of the largest AI-adopting organizations that aren't available anywhere else: the token budget ratio metric, Liquid AI for sub-30ms inference, and the Simen customer simulation tool with 0.7 correlation targets. These are battle-tested, not theoretical.

  • GPT-5.5 prompting guide — Three minutes to read, potentially hours of frustration saved. The instruction to send status updates before tool calls is something you can implement in your next session. The $openai-docs migrate command is a one-liner with immediate payoff.

What to Do

  1. Audit your current system prompt for harness crutches. Take your existing system prompt or CLAUDE.md/skill.md file and ask: which of these instructions does the model now follow natively without being told? Remove those lines. Then identify the most repeated mistake from your last 10 sessions and add a single, specific entry to your skills file addressing it. This is the compound engineering loop in its most minimal form—10 minutes of work that compounds session over session.

  2. Run one task using the Opus-plans-GPT-5.5-executes split. Pick a codebase refactoring or multi-file change you've been avoiding. Use Opus 4.7 to write a detailed engineering plan first (specify invariants, what to delete, what to preserve). Then hand that plan to GPT-5.5 in Codex with the instruction to execute it. Measure whether the output quality justifies the two-step overhead—this gives you a personal benchmark for when the pattern is worth using.

  3. Check whether your memory failures are model failures or harness failures. If you've experienced unexpected forgetfulness or repetition in Claude Code sessions, the bug where thinking was cleared every turn (rather than after an idle hour) was a known harness issue. Before attributing any memory problem to the model, verify your harness version and check the harness configuration first. This principle generalizes: always isolate harness vs. model vs. prompt before concluding the model degraded.

Source Articles

← More from Self Improving Agent