Briefing: AX Research

For a postdoctoral researcher in Experimental/Cognitive Psychology targeting industry AX research roles — tracking evaluation frameworks, human-agent interaction design, and safety/alignment challenges.

Key Insights

• Agent evaluation is an architecture, not a judgment call. The field has moved beyond single-score performance metrics toward a layered design stack: an eval harness runs fixed scenarios against model checkpoints; reward signals are then classified by type (verifiable vs. learned) and density (sparse vs. dense); and rubrics decompose those rewards into weighted dimensions that can be combined using structured logic like weighted sums or sequential gates. For an AX researcher, the practical implication is that choosing how to measure an agent's behavior is as consequential as the behavior itself — each design choice in this stack shapes which agent behaviors get reinforced and which go unmeasured. Fluency in this architecture, not just familiarity with the vocabulary, is a genuine differentiator when entering industry research teams that are actively building these systems.

• Harness, Scaffold, and the AI Agent Terms Worth Getting Right

• Rubric-based evaluation is the most immediately actionable tool for AX researchers coming from experimental backgrounds. The rubric model — breaking performance into explicit, weighted dimensions rather than collapsing to a single score — translates directly to the dependent-variable logic that experimental psychologists already use: outcomes can be decomposed, independently weighted, and recombined in principled ways. This is a concrete bridge between lab-trained methodology and industry evaluation practice, and it addresses an active design problem in AX: how do you capture nuanced agent behavior (appropriateness, efficiency, safety, user experience) without losing the signal that a single satisfaction score would bury?

• Harness, Scaffold, and the AI Agent Terms Worth Getting Right

• Context engineering reframes human-agent interaction design as an ongoing architectural problem. Rather than treating the agent's input as a fixed prompt, context engineering involves actively managing what the agent sees at every step — system prompts, tool descriptions, conversation history, retrieved knowledge — with the harness dynamically updating this across the run. Paired with the short-term/long-term memory distinction (in-context vs. externally retrieved), this means the user experience is partly determined before the model responds, by upstream decisions about what gets surfaced and retained. Researchers with a cognitive psychology background should recognize an analog here: this is working memory manipulation by design, and that conceptual hook is a genuine asset for bridging experimental expertise to agentic interaction research.

• Harness, Scaffold, and the AI Agent Terms Worth Getting Right

• Terminological instability in AX is not just a communication nuisance — it is a structural opportunity. The field is currently in a state where vocabulary evolves faster than shared understanding, with the same terms used differently across frameworks and teams. For an industry researcher entering cross-functional teams — where product, engineering, and research must coordinate around agent behavior — the ability to impose definitional clarity is itself a valued research contribution, not a soft skill. A postdoc grounding in psychological constructs (mental models, distributed cognition, cognitive load) combined with fluency in emerging agentic glossaries positions them as a conceptual bridge-builder at a moment when most teams are still negotiating what words mean.

• Harness, Scaffold, and the AI Agent Terms Worth Getting Right

Emerging Patterns

1. Evaluation architecture and interaction design are converging on the same problem: what the agent attends to. The eval harness, reward design, and rubric structure all address how agent behavior is measured after the fact; context engineering and memory architecture determine what information shapes behavior in the first place. These are two sides of the same design challenge — and for AX researchers, the connection matters because what gets measured and what gets surfaced to the agent are not independent choices. Interaction design decisions about context management directly constrain what evaluation frameworks can detect, meaning AX researchers need fluency in both layers simultaneously.
2. Harness, Scaffold, and the AI Agent Terms Worth Getting Right

Dissenting Views

No meaningful dissent detected in this content batch. All entries originate from a single-source glossary with internally consistent framing. This is worth noting as a limitation: the insights above reflect one team's definitional choices, not cross-source convergence. Treat them as a strong starting vocabulary, not an empirically triangulated consensus.

Read & Act

What to read

• Harness, Scaffold, and the AI Agent Terms Worth Getting Right — Read in full, but prioritize the Reward and Rubrics sections. The reward taxonomy (verifiable vs. learned, sparse vs. dense) and the rubric decomposition model contain design logic that cannot survive summary — you need to see the combinable object structure to understand how it maps to experiment design. This is the most direct vocabulary investment you can make before entering conversations with industry AX teams.

What to do

• Draft a rubric for a human-agent interaction scenario you already know. Pick a task from your existing research — a judgment task, a recall paradigm, a decision task — and reconstruct it as an agent evaluation rubric: identify 3–4 performance dimensions, assign relative weights, and specify whether they should be combined additively or sequentially (i.e., must one gate unlock before another is scored). This exercise forces you to apply the framework concretely and will surface where your experimental intuitions transfer cleanly versus where agentic systems introduce new constraints. The output is also a tangible artifact you can bring into interviews or portfolio work.

• Map your existing psychological constructs onto the agentic architecture vocabulary. Take three constructs you use fluently — working memory, mental models, cognitive load, divided attention, or similar — and write a one-paragraph translation of each into the agentic interaction stack: where does working memory show up in context engineering? Where does cognitive load appear in scaffold design? This is not an academic exercise; industry AX teams are actively trying to import psychological frameworks but often lack the disciplinary grounding to do it rigorously. Arriving with pre-built translations is a positioning move, not background preparation.