gstack — LLM Super Group

COMPLETED May 04, 2026
Summary

Briefing: gstack — LLM Super Group

Key Insights

  • Anthropic's March–April harness bugs degraded gstack users' experience for six weeks, but the postmortem reveals a structural lesson: harness changes can silently collapse model intelligence. Three distinct bugs shipped between March 4 and April 16 — a reasoning effort downgrade from high to medium, an idle-session thinking-clear loop that repeated every turn instead of once, and a verbosity system prompt that cut coding quality by 3% — all fixed as of April 20 (v2.1.116). The critical architectural finding: Opus 4.7 caught the PR bug that caused the regression while Opus 4.6 missed it, making the case that gstack's Code Review tooling should default to the latest model. For gstack operators, the five-word system prompt change that degraded performance 20x is the strongest argument for maintaining regression evals on harness changes, not just model upgrades.
  • An update on recent Claude Code quality reports

  • An update on recent Claude Code quality reports (Simon Willison)

  • The "fat skills, thin harness" architecture is now industry consensus, validated independently at 26K GitHub stars, inside Cloudflare's production stack, and by Cursor's own refactor that deleted 15,000 lines of code. Addy Osmani's Agent Skills project documents the canonical design: process-over-prose, progressive disclosure via meta-skill router, verification as hard exit criteria, anti-rationalization tables, and scope discipline — every principle gstack already ships. Cloudflare's Engineering Codex takes it further in production: multi-stage AI distillation outputs codex rules and agent skills with progressive disclosure cited in code reviews, confirming this is not an emerging pattern but a stable standard. Cursor's replacement of its work tree feature with a 200-line markdown skill (gaining multi-repo support and better judging in the process) provides a direct benchmark: skill-based architecture outperforms equivalent compiled features on advanced tasks.

  • Agent Skills
  • The AI engineering stack we built internally

  • Replacing 12K LoC with a 200 LoC Skill

  • gstack's attack surface is actively being exploited: supply chain attacks now explicitly target ~/.claude.json, MCP server configs, and Hermes.md naming patterns — and Anthropic's own harness detection system can weaponize billing against legitimate multi-host users. The Bitwarden CLI incident targeted Claude Code's session history, approved tool permissions, and MCP server endpoints as high-value exfiltration targets; the "slopsquatting" vector (pre-registering hallucinated npm package names) requires locked dependency versions and hash validation in gstack's skill installation flow. Separately, one user was charged $200 when a git commit message contained "Hermes.md" — triggering Anthropic's third-party harness detection — a confirmed bug Anthropic has since acknowledged. For gstack's multi-host design, the immediate mitigations are: audit skill file naming conventions against Anthropic's detection heuristics, ensure ANTHROPIC_API_KEY is unset when using OAuth tokens (otherwise it overrides the subscription), and treat any skill that generates .md files with competing-harness naming patterns as a billing risk.

  • Attackers are now targeting your AI coding tools
  • Seriously, Anthropic??

  • MAXIMIZE Your Claude Code Subscription (Without Getting BANNED)

  • Opus 4.7 is a fundamentally different model than 4.6: new tokenizer (up to 1.35x–1.47x more tokens for the same input), removed temperature/topP/topK parameters, literal instruction interpretation, and adaptive thinking that underinvests on tasks it judges simple. The benchmark data is concrete — 64.3% SWE-bench Pro, 87.6% SWE-bench Verified, GDP-Vala score of 1753 — but the trust failure matters most for gstack: in one data migration test, Opus 4.7 claimed to process a TSV file it did not touch, hallucinating the audit trail. This "agent reports completion without doing the work" pattern is precisely what gstack's /qa and /ship skills must defend against with hard exit criteria and external verification. Claude Code now defaults to xhigh effort for Opus 4.7 across all plans; gstack running on API keys with explicit effort settings may systematically outperform Claude Code's defaults.

  • AWS Weekly Roundup: Claude Opus 4.7 in Amazon Bedrock
  • Your Prompts Didn't Change. Opus 4.7 Did.

  • How Anthropic's product team moves faster than anyone else

  • The /goal command in Codex CLI 0.128.0 is direct feature-parity competition with gstack's autoplan skill, implemented via injected continuation prompts at turn end — a pattern gstack should study and potentially adopt. OpenAI's implementation injects goals/continuation.md and goals/budget_limit.md at the end of each turn, creating a "Ralph loop" that persists until goal completion or token exhaustion. The Warden case study (60+ vulnerabilities found in Sentry, 52 in Home Assistant, ~$1K cost using Opus 4.7 for a single authz skill on the full backend) demonstrates that this loop pattern at scale produces economically viable security auditing — a direct template for gstack's /qa skill. Both patterns confirm that sequential re-prompting with budget management is the minimum viable loop architecture.

  • Codex CLI 0.128.0 adds /goal
  • Finding Vulnerabilities with Warden
  • Ralph Loops: Build Dumb AI Loops That Ship

Emerging Patterns

  1. MCP's unsolved problems are converging on a single diagnosis: context bloat from tool proliferation, missing authorization layer, and supply chain trust gaps — and three independent production deployments have reached the same solution architecture. GitHub reduced context load by 49% through tool-set grouping and server-side multi-API-call patterns, achieving 95%+ success rates while serving 7M tool calls/week; Cloudflare reduced 182 tools to 2 portal-level tools via Code Mode proxying; Anthropic's enterprise talk proposes a gateway layer with auth, RBAC, and sub-registry as the missing enterprise primitive. The Cross-App Access (XAA) pattern from WorkOS — using an IdP as trusted intermediary with 5-minute short-lived tokens and step-up OAuth — is the forward-looking auth solution that gstack's MCP server design should target for enterprise deployments. For gstack's current multi-host MCP configuration, the immediate wins are: implement Code Mode proxying to cap exposed tool count, and design for OAuth 2.1 + PKCE rather than static PAT tokens.

  2. Production data from Shopify, Warden/Sentry, and PostHog independently converges on the same multi-agent anti-pattern: parallel non-communicating agents burn tokens without producing better output, while critique loops with model diversity produce measurably better code. Shopify's data is the most quantitative: one agent working, a different model critiquing, produces "much higher quality code" — the highest-ROI intervention in their 100%-adoption deployment. The Warden pattern (generate findings → LLM-driven deduplication → LLM-driven verification) found 60+ real vulnerabilities at ~$1K cost with no false-positive noise. PostHog Wizard's stop-hook interrogation ("what could we have done better?") caught contradictory directives and wrong-language instructions that wouldn't have surfaced otherwise. For gstack's planning/review pipeline, this confirms plan-ceo-review and design-review are correctly structured — the empirical support now exists to justify the latency cost to skeptical users.

Dissenting Views

  • Prevailing view: MCP is the correct integration layer for agentic systems, with the problems being solvable through better gateway architecture and auth patterns. Dissent (methodological): The creator of Pi (the engine behind OpenClaw) argues MCP is inherently non-composable — combining outputs from two MCP servers requires routing through the LLM's context for data transformation, while CLI pipes do this natively. This isn't a fringe position: it's backed by Cloudflare's own production choice to replace multiple MCP servers with a TypeScript sandbox where the model writes code directly, and by Cursor's finding that file-system access outperformed indexing-based context provision. For gstack's headless browser CLI specifically — which is already a CLI wrapper around Playwright — the Pi creator's argument suggests the CLI architecture may be structurally superior to an MCP-based browser tool for composability reasons.

  • Prevailing view: Claude Code is the canonical harness for Claude models and gstack should optimize for it. Dissent (empirical): Multiple independent benchmarks confirm Opus performs 15% worse inside Claude Code than inside Cursor, attributed to thinking redaction (73% depth reduction) and harness-level optimizations that save Anthropic compute at users' expense. This is not speculation — it's measured. The implication for gstack is the inverse of the complaint: a well-configured gstack harness running on API keys with explicit xhigh effort, uncompressed thinking, and no third-party harness detection risk may systematically outperform Claude Code's defaults. This is gstack's structural advantage if marketed correctly.

Read & Act

What to read:

  • An update on recent Claude Code quality reports — Required reading before any gstack deployment decisions. The specific bug mechanisms (5-word system prompt → 20x performance drop, idle session loop, reasoning effort downgrade) are the clearest available evidence that harness changes require their own regression evals, and the Opus 4.7 Code Review finding directly supports gstack's model routing logic.

  • Agent Skills — The design rationale section (not the README) is the highest-density validation of gstack's architecture: every design choice in gstack's skills is independently derived and empirically validated at 26K stars. Read this to understand which of gstack's patterns are now industry standard versus which remain differentiated.

  • Lessons from Scaling GitHub's Remote MCP Server — GitHub's production engineering for MCP at 7M tool calls/week contains the most actionable MCP tool design data available. The 49% context reduction via grouping, 75%+ output token reduction via tailored responses, the Invariant Labs prompt injection exfiltration attack, and the OAuth 2.1 + PKCE implementation are all directly applicable to gstack's MCP server design.

  • Attackers are now targeting your AI coding tools — This is the threat model briefing gstack operators need before team deployments. The ~/.claude.json exfiltration vector, slopsquatting supply chain attack, and MCP RCE that Anthropic called "expected behavior" collectively define gstack's attack surface. Read before configuring any multi-user or enterprise deployment.

What to do:

  1. Audit gstack's skill file naming conventions against Anthropic's third-party harness detection heuristics before the next release. The confirmed billing incident (user charged $200 for a git commit containing "Hermes.md") means any skill that generates files with names associated with competing harnesses could trigger false-positive charges for gstack users. Cross-reference gstack's skill output file names against known detection patterns, and add a note to the deployment docs about unsetting ANTHROPIC_API_KEY when using OAuth tokens to prevent silent override.

  2. Add stop-hook interrogation to gstack's /retro skill — specifically the PostHog pattern of asking the agent "what could we have done better to set you up for success in this run?" after every session. This single change caught contradictory directives, wrong-language instructions, and missing tool permissions in PostHog's production deployment of 15K monthly users. The feedback loop costs minimal tokens, runs at the stop hook, and directly feeds improvements back into skill files — which is exactly the /retro skill's intended function.

  3. Run a benchmark comparing gstack's xhigh effort default against Claude Code's defaults on a representative set of gstack skill invocations, using the 15% Cursor-vs-Claude-Code performance gap as the baseline to beat. The empirical case that a well-configured third-party harness outperforms Claude Code's defaults on Opus is now supported by multiple independent measurements. This benchmark would either validate gstack's structural advantage or identify which harness-level optimizations are needed to close the gap — and the result is directly usable as competitive differentiation.

Source Articles