Agentic Coding

Header Briefing: Generative AI Insights A briefing for software engineers at early-stage startups on building reliable and effective LLM-based systems.

Key Insights

1. Treat LLMs as Probabilistic Components, Not Decision-Makers. The most critical shift for building reliable AI systems is to separate the LLM's role of generating options from the system's role of making decisions. The LLM should produce drafts, suggestions, or structured data, but the final authority—what is true, safe, or executed—must reside in deterministic code, a workflow, or a human operator. This prevents the model from being the final arbiter of truth or the system's security boundary. (Source)
2. A New "Skill Tree" is Required for Production AI. Building robust LLM applications requires a new set of engineering disciplines beyond simple prompting. Key skills include: Intent Specification (reducing ambiguity), Context Engineering (managing the context window precisely), Constraint Design (enforcing schemas and output formats), Verification (using deterministic checks and unit tests), and Provenance (ensuring outputs are traceable to their sources). (Source)
3. Structure Your Startup Around "Tiger Teams" Empowered by AI. For early-stage startups, AI delivers the most value as a "power pack" that multiplies the productivity of small, autonomous teams, not as a top-down tool for organizational visibility. Over-investing in AI for reporting risks creating "fake legibility"—dashboards and metrics that look empirical but are un-debugged and misleading. The effective model is to let small teams execute quickly and use AI as a "cheap historian" to synthesize and report on their work afterward. (Source)
4. Evaluation Harnesses are Essential for Iteration and Safety. To safely evolve prompts, models, or retrieval strategies, you must build evaluation harnesses. These include golden example sets, regression tests, and scorecards with defined thresholds. Without robust evaluation, you cannot compound improvements reliably. This moves debugging from ad-hoc prompt fiddling to a systematic process of classifying failure modes (e.g., missing context, retrieval error, hallucination) and addressing them at the correct layer of the stack. (Source)

Emerging Ideas / Undercurrents

• Tool-Mode vs. Infrastructure-Mode: A clear debate is emerging between using LLMs as conversational "tools" (like a chatbot) versus integrating them as reliable, governed "infrastructure" components. The infrastructure approach, while more complex, is presented as the only viable path for production systems that require reliability, security, and traceability.
• Visibility vs. Execution: There is a fundamental tension in how organizations apply AI. The default tendency is toward using AI for top-down visibility and control. However, sources argue this is a trap that can stifle innovation. The counter-current advocates for prioritizing AI tools that accelerate bottom-up execution by small, agile teams, even if their work is initially messy and less "legible" to management.

Actionable Steps ("Header Actions")

1. Implement a "Constraint Design" Pattern: For your next LLM feature, enforce a strict JSON schema for the output. In your application code, add a deterministic validation step that parses the LLM's output and fails loudly if it doesn't conform to the schema. This turns the stochastic generator into a more predictable component.
2. Create a Minimal Evaluation Harness: Identify one critical prompt in your system. Create a "golden set" of 5-10 diverse inputs with their expected, ideal outputs. Before deploying any change to that prompt, model, or its context retrieval, run it against this set as an automated regression test.
3. Separate Generation from Decision: Review a feature where an LLM's output triggers an action (e.g., sending an email, updating a database). Refactor the code to ensure the LLM only proposes the action and its content. A separate, deterministic piece of code should make the final decision to execute based on predefined rules.
4. Audit Your AI-Driven Metrics: If you use AI to generate performance or risk metrics, ensure you can trace each metric back to a concrete, observable event (e.g., a specific API call trace, a document chunk). Reject any "vapor metrics" that cannot be debugged or explained.

Source Highlights

• Tool-Mode vs Infrastructure-Mode: The Bet 90% of Companies Are Losing: This video provides a dense, actionable framework for engineers building production-grade LLM systems. It introduces a "skill tree" covering context engineering, constraint design, verification, observability, and evaluation, offering a clear vocabulary and mental model for moving beyond basic prompting.
• The Fork Most Leaders Don’t See: Visibility vs. Execution: This source offers a strategic perspective for startup leaders and engineers on how to structure teams and processes to leverage AI effectively. It warns against the allure of AI-driven dashboards ("fake legibility") and advocates for a "tiger team" model where AI serves to accelerate execution rather than enforce top-down control.

Next Directions

• Focus on implementing robust Observability for your LLM calls. Ensure you are logging the full context, prompts, tool calls, intermediate outputs, and validation results. This is a prerequisite for systematically debugging failures and building effective evaluation loops.
• Explore practical Agent Design Patterns that incorporate security and reliability from the start, such as using deterministic permission envelopes (allow lists, scoped tools, approval steps) rather than letting the model decide what actions it can take.