Product Design

Briefing: Product Design Purpose: Advice for a first-time founder building an AI-native consumer product pre-PMF, seeking early customers, determining ICP, iterating on feedback, and building a product where knowledge workers learn about AI through action-based experiments.

Key Insights

• Your demo is not your product — the 90% that remains is. Every company that has shipped a production AI product reports the same pattern: a functional prototype takes days, but getting it to work reliably across real users, edge cases, and configuration variations takes months. Intercom's account of building Operator describes "months of engineering" just to handle performance questions correctly across thousands of customer workspaces, and explicitly warns that "the gap between a working demo and a production system is where most of the engineering investment lives." For a pre-PMF founder, this has an immediately actionable implication: stop iterating on features and start identifying the single, narrow workflow your target user does every week. Build that one thing to production quality before expanding. The AI Engineering role definition crystallizes this into a memorable frame: "The agent is the easy part. The loop is the job." Your product is the evaluation and improvement system, not the chat interface.
• Operator: A look under the hood

• AI Engineer Is a New Role

• Your product's central design tension is the most important question you haven't answered yet. Most AI tools are optimized to remove friction and complete tasks faster. But your stated mission — teaching knowledge workers about AI through action-based learning — requires the opposite: preserving friction as a pedagogical mechanism. Addy Osmani's research is the clearest data point available: engineers who used AI to copy-paste code scored under 40% on comprehension tests, while those who used it to ask conceptual questions scored above 65%. The insight isn't just that passive consumption degrades learning — it's that the order in which AI is introduced to a task matters more than the total amount of AI used. If AI frames the problem at the start, users anchor to that framing even when doing the rest of the work themselves. This means your product design must actively structure a hypothesis-first loop: prompt users to form a belief before querying the AI, then surface the AI's reasoning, then require active selection rather than passive receipt. No other entry in this batch provides research-backed, quantitative evidence for a design principle this central to your product thesis.

• Don't Outsource the Learning

• Your first architecture assumption about ICP might be wrong — and that's expected. Every.to gave every employee an AI agent and discovered that "one AI assistant per employee was the wrong starting point." The product that actually got adoption was one structured around shared team resources — agents with defined jobs that benefit the whole team when updated once. This is a direct pre-PMF hypothesis invalidation story from a knowledge-worker-focused company that looks a lot like yours. The structural question it raises: is your product building toward individual AI companions or shared organizational tools? These require different architectures, different onboarding flows, and attract different ICPs. The answer determines whether you're competing with Notion or with GitHub Copilot. You should treat the first three months less as "building the product" and more as running cheap experiments to eliminate that hypothesis, using every customer conversation to test whether your users think of AI learning as a personal journey or a team capability.

• We Gave Every Employee an AI Agent. Here's What We're Doing Differently Now.

• The ICP discovery loop has a known pattern: expand, then compact ruthlessly. Andrew Lee's account of building Tasklet is the most granular pre-PMF product iteration story in this batch. He rebuilt the product from scratch twice: first adding synchronous chat when users wanted it, then rearchitecting the entire context management system when conversation length broke economics. Jake Cooper at Railway describes the same rhythm explicitly — "periods of expansion where we test use cases, then periods of compaction where we strip out features that don't fit our ICP anymore." Paul Graham's "initial version is to be rewritten" reinforces this from first principles. The actionable synthesis: in your first three months, every feature decision should be framed as "which ICP hypothesis does this eliminate?" not "does this express our vision?" The specific context management solution Andrew Lee developed — storing conversation history in the file system instead of feeding it to the LLM directly — emerged from customer feedback that the product broke at scale, not from a design document.

• Three Kinds of Software Survive: Tasklet's Andrew Lee on Competing to be a Horizontal Platform
• The Agent-Native Cloud: 3M Users, 100K Signups/Wk, Data Centers, & Death PRs — Jake Cooper, Railway

• What I Did this Summer

• Session continuity and memory persistence are not features — they are your core trust mechanism. Every AI product that has reached production at scale reports the same class of user experience failure: the user loses progress on disconnection, the agent forgets context between sessions, the product feels unreliable in ways that have nothing to do with the model's intelligence. The Ably entry on durable sessions makes the architectural cause explicit: when the stream's health is tied to a single client connection, every network hiccup destroys the user's session. The AI Memory entry demonstrates that persistent memory across conversations changes the entire user relationship — the difference between "what should I have for dinner" answered generically versus answered with remembered preferences is the difference between a tool and a collaborator. For a learning product specifically, session continuity is even more critical: if a user is mid-experiment and the session drops, the learning moment is lost. Architect for state persistence from day one, even with simple SQLite-based memory, because retrofitting this later requires rewriting the entire user model.

• Why Your AI UX Is Broken (and It's Not the Model's Fault) — Mike Christensen, Ably

• AI Memory: Stop Building Stateless Agents

• Anthropic's eight design principles for agents are immediately deployable as your product spec. The Agent Builder's Playbook distills internal Anthropic practices into a set of frameworks that are specifically actionable for pre-PMF builders. The Gather-Act-Verify loop structures every agent interaction. The Reversibility Filter — use reversibility as a signal for what to trust — maps directly onto a learning product where users should be able to undo experiments without consequence. The "locked-in-a-room test" for tool design asks whether a tool is understandable and manageable in a constrained environment, which is exactly the cognitive load question your users face when experimenting with unfamiliar AI capabilities. The most immediately deployable recommendation is the transcript reading mandate: "Read the transcripts every time." For a pre-PMF consumer product, this isn't optional — it's the primary source of ICP signal. Every conversation your users have with your product is a data point about what they're actually trying to learn, which is almost certainly different from what you think they want to learn.

• The Agent Builder's Playbook: 8 Lessons from Anthropic

• Pricing is a research problem you should start now, not a decision you make at launch. Intercom's process for developing Fin's pricing is the most structured approach to this question in the batch, and its central finding is directly translatable: they discovered through qualitative buyer research that buyers don't want to pay for usage, they want to pay for results — which led to outcome-based pricing where you only pay when Fin resolves a query. For a knowledge-worker learning product, the equivalent question is: what does "delivering learning value" look like in a measurable way? Is it the number of AI experiments completed? The self-assessed capability improvement after a session? The answer matters enormously right now — not because you need to charge customers immediately, but because the metric you choose defines the product's success condition and acts as an ICP filter simultaneously. Users who care about your pricing metric are your ICP. Intercom ran qualitative buyer interviews before any quantitative willingness-to-pay testing, and that sequence matters: you can't ask the right WTP questions until you've done the qualitative work to understand how buyers define value.

• How we develop pricing and packaging at Fin

• Ben Horowitz's "right product, right time" framing cuts through all other noise. Everything else in this briefing — architecture decisions, pricing research, ICP iteration loops — is in service of one thing: delivering the right product at the right time. Ben Horowitz is explicit that this is "the only thing that determines whether a company works" and that all other activities, including hiring and fundraising, are secondary and supportive. The corollary he draws is that product strategy is not a one-time document but a continuous learning process where initial assumptions are always wrong, and the founders who survive are those who can distinguish between "this assumption was wrong, which I need to adjust" versus "this whole thing is unviable." For a first-time founder, this framing is useful as a filter: when you're unsure whether to invest time in a feature, a pricing experiment, or a customer conversation, ask which one is most likely to tell you whether you have the right product. The answer almost always points toward the customer conversation.

• Ben Horowitz - "Your ONLY job is Right Product, Right Time"

Emerging Patterns

• The demo-to-production gap is universally underestimated, and the gap is growing as AI makes demos easier. Multiple production engineering teams converge on the same observation from different angles: building an AI prototype is now trivially easy (Intercom, Every.to, Anthropic), which paradoxically makes the demo-to-production gap more dangerous because it creates false confidence. Every.to rebuilt their AI product architecture after discovering that the individual-agent model that demoed beautifully created unmaintainable maintenance burdens in production. Intercom explicitly states that their Operator prototype "demonstrates the 10% of the system that's straightforward to build." The AI Engineering role definition frames the entire discipline as being about what comes after the prototype: "Building an agent is easy. You can vibe code an agent. There are SDKs that let you do it in five lines. The part that matters is everything that comes after." The pattern that emerges across all three is that founders who mistake a working demo for product-market fit spend their engineering resources on features when they should be building reliability infrastructure. For a learning-oriented product, this is particularly acute: the learning moments users need are high-trust, continuous interactions — exactly the scenarios where production reliability matters most.
• Operator: A look under the hood
• We Gave Every Employee an AI Agent. Here's What We're Doing Differently Now.

• AI Engineer Is a New Role

• The architecture shapes the experience: harness, memory, and session continuity matter as much as model selection. Multiple engineering-focused sources converge on the finding that the decisions most founders spend the least time on — session persistence, memory architecture, context management — determine the actual user experience more than model selection does. The Ably entry demonstrates that stream architecture determines whether users lose their work on network drops. The AI Memory entry demonstrates that stateless agents feel fundamentally less capable than stateful ones, regardless of underlying model quality. Tasklet's Andrew Lee found that the switch from LLM-fed chat history to file-system-stored context was the architectural decision that unlocked scale — not a model upgrade. These sources collectively argue that "which LLM should I use" is the wrong question at the pre-PMF stage; the right question is "what is the minimum architecture that allows users to have a continuous, trustworthy relationship with this product?" For a learning product, where users need to build on previous experiments and return to earlier hypotheses, this is especially decisive.

• Why Your AI UX Is Broken (and It's Not the Model's Fault) — Mike Christensen, Ably
• AI Memory: Stop Building Stateless Agents

• Three Kinds of Software Survive: Tasklet's Andrew Lee on Competing to be a Horizontal Platform

• The role of domain expertise is the most underrated variable in AI product quality, and you have it by virtue of your mission. The Langfuse case study, the Tasklet story, and the Abridge healthcare AI discussion all converge on a finding that applies directly to your situation: winning in vertical AI is an organizational problem, not a model problem. The system for incorporating domain expertise into the product matters more than model selection. For Langfuse, this meant embedding product skills directly into coding agents through "skills" — structured instructions that give agents access to up-to-date documentation and best practices. For Abridge, it meant hiring "clinician scientists" embedded in engineering teams who serve simultaneously as oracle, evaluator, and architect. Your product's domain is "learning about AI through action," and your unfair advantage is that you are building the product by using it — Every.to's explicit strategy is to build "AI-native ways of working out of our own experience using these tools every day." This is a genuine moat if you systematize it: use your own product to learn AI, capture what works, and encode those insights into the product itself.

• How to Leverage Domain Expertise — Chris Lovejoy, Notius Labs
• We Gave Every Employee an AI Agent. Here's What We're Doing Differently Now.
• Inside Abridge: The AI Listening to 100 Million Doctor Visits

Dissenting Views

• Tension: Should your product remove all friction, or deliberately preserve some? The prevailing view across most AI product engineering content — including Railway's Jake Cooper, who says "we will swim to the bottom of the swimming pool to get the experience right" — is that friction is an enemy: eliminate it, automate it, abstract it. The Ably durable sessions work, the memory architecture research, the agent maturity maturity models all optimize for reducing user friction. The dissenting view, supported by the only research-backed quantitative data in this batch, is that friction is where learning lives, and a product explicitly designed for learning must actively resist the friction-elimination default. This is not a semantic disagreement — it is a fundamental product philosophy question. The resolution may be ICP-dependent: if your users are experienced knowledge workers who already understand AI and want to go faster, friction-removal is right. If your users are knowledge workers who are AI novices trying to build genuine capability, friction preservation is a feature. You almost certainly do not yet know which of these is your ICP, which means this tension is the most important thing your first customer conversations should resolve.
• Don't Outsource the Learning
• The Agent-Native Cloud: 3M Users, 100K Signups/Wk, Data Centers, & Death PRs — Jake Cooper, Railway

• AIE Singapore Day 1

• Tension: Does technical foundation matter before PMF, or should you move as fast as possible? The vibe coding debate surfaces a genuine and unresolved disagreement that affects a first-time founder directly. One camp argues that foundational technical knowledge is prerequisite: "Vibe coding is not okay under any circumstance if you don't have a foundation in software development and architecture. This is not a tooling problem, it's a knowledge problem." The opposing view — represented by Teresa Torres and Petra Wille — is that product discovery skills transfer directly to AI engineering, that the willingness to learn matters more than existing technical background, and that "the most important skill right now isn't coding." This is a difference in emphasis rather than pure contradiction: both agree that learning is essential, but they disagree about whether you need to build foundations before using AI tools or can build them concurrently by learning through AI. For a non-technical founder building a learning product, the Torres/Wille framing is more practically actionable and aligns with your product's thesis. For a founder who already has technical background, the vibe coding caution about foundation-first is a useful guardrail.

• I've Changed My Opinion On DEVELOPERS Vibe Coding
• AI Engineering - All Things Product Podcast with Teresa Torres & Petra Wille

Read & Act

What to read

• Operator: A look under the hood — This is the most honest account available of the engineering reality behind a production AI product, including an explicit build-vs-buy framework and the "10% demo / 90% engineering" framing. Read it specifically to calibrate your intuition for where your product investment should actually go, and to use the "action layer" safety/reversibility/auditability framework as a checklist for your own product.

• Don't Outsource the Learning — This is the only piece in the batch with research-backed quantitative evidence for a design principle that is central to your product thesis. The specific patterns it offers — form a hypothesis before asking, treat AI output like a junior engineer's PR, the "order of operations matters more than total AI used" finding — are immediately deployable as UX constraints in your product. Read it to internalize the design implications, not just the principle.

• Three Kinds of Software Survive: Tasklet's Andrew Lee on Competing to be a Horizontal Platform — Andrew Lee's account of two complete product rebuilds, including the specific context management solutions he developed and the framework for surviving AI disruption (horizontal platforms, headless/API-first, solutions companies), is the most detailed pre-PMF iteration story in the batch. The technical decisions he made — file system as context store, synchronous chat as a customer-driven addition — provide a template for how to translate early customer signals into architectural choices.

• How we develop pricing and packaging at Fin — Read this now, not when you're ready to charge customers. The methodological sequence (qualitative buyer research → pricing model → pricing metric → quantitative WTP) reveals what questions to ask before you have a pricing strategy, and the "outcomes" metric example shows how the right pricing unit can double as your product's ICP filter.

What to do

• Run a friction experiment in your next five customer conversations. Before your next user session, decide in advance whether you will show the product in friction-elimination mode (AI completes the task) or friction-preservation mode (AI supports the user's own reasoning). Run both versions with different users. After each session, ask directly: "Did you feel like you learned something, or did you feel like you got something done?" The answer will tell you which ICP you're actually building for, and it will resolve the most important product philosophy tension in this briefing before you've made irreversible architectural decisions.

• Read every conversation transcript from your first ten users, and annotate them for what the user was actually trying to learn. The Agent Builder's Playbook mandates transcript reading as a core practice, and Every.to discovered their wrong architecture assumption through exactly this kind of observation. For each transcript, note: (1) what the user said they wanted, (2) what they actually tried, (3) what they did when the product didn't give them what they expected. These three data points will surface your real ICP signal faster than any survey.

• Define your pricing metric before building your next feature. Following Intercom's sequencing: schedule two to three qualitative conversations with the kind of knowledge worker you want as your customer, ask how they'd define "getting value" from a product like yours, and listen for whether they describe an outcome ("I understand how to use AI for X") or an activity ("I used the product Y times"). The answer is your pricing metric candidate. Make that metric your north star for every feature decision for the next 90 days — build only things that move it.