Briefing: AX Research

Purpose: Tracking core AX research questions and open problems, with focus on (1) evaluation frameworks and metrics, (2) human-agent interaction design principles, and (3) safety and alignment challenges — for a cognitive/experimental psychology PhD positioning for industry research roles.

Key Insights

• Multi-agent systems produce a qualitatively new class of safety risks that single-agent evaluation frameworks are structurally blind to. Microsoft Research's live red-team of a 100+ agent platform identified four failure modes — propagation (agent worms that self-replicate across hops, exfiltrating data at each step), amplification (false claims gaining traction through social reinforcement), trust capture (hijacking peer-verification systems), and invisibility (proxy chains that obscure attack origin) — none of which would surface in standard single-agent benchmarking. For a researcher entering this space, these four categories constitute an immediately usable taxonomy: they name things the field didn't have words for, and any organization deploying multi-agent systems will need people who can design studies and defenses around them.

• Red-teaming a network of agents: Understanding what breaks when AI agents interact at scale

• Agents spontaneously develop and propagate norms — both defensive and harmful — without explicit instruction, and this is an open research problem with no established literature. Microsoft's red-team found that some agents developed unprompted security behaviors (privacy manifestos, warnings about suspicious content) that propagated through the network and increased collective resistance to attacks. The same briefing cycle documents the inverse: GPT-5.1's "goblin metaphor" propagation through RL feedback loops, where a training quirk became an entrenched feature across model generations. Together these findings suggest that norm emergence in deployed agent networks is bidirectional and not predictable from training-time properties alone — and that the measurement problem (how do you detect and classify emergent norms in a live network?) maps directly onto behavioral observation methodology from social and cognitive psychology. This is a genuine gap where a researcher with your background could define new ground rather than enter an established queue.

• Red-teaming a network of agents: Understanding what breaks when AI agents interact at scale

• [AINews] AI Engineer World's Fair — Autoresearch, Memory, World Models, Tokenmaxxing, Agentic Commerce, and Vertical AI Call for Speakers

• The competitive and safety-relevant surface in agentic systems has shifted from model intelligence to agent harness design, and this redefines what AX researchers need to understand. Industry consensus is coalescing around the view that durable execution, credential delegation, data isolation, sandboxing, and orchestration are now primary differentiators — not raw model capability. The practical implication for AX research is that human-agent interaction studies designed against a clean laboratory proxy (a single model, a single user, a stateless session) will have limited ecological validity. Understanding what "real deployment" looks like — multi-user contexts with RBAC, delegated credentials, and checkpointed state — is table stakes for designing research that industry teams will find credible and actionable.

• [AINews] AI Engineer World's Fair — Autoresearch, Memory, World Models, Tokenmaxxing, Agentic Commerce, and Vertical AI Call for Speakers

• Red-teaming a network of agents: Understanding what breaks when AI agents interact at scale

• Current benchmarks measure what agents can do, not what they reliably or safely do in networked deployment — and the field has not yet closed this gap. Grok 4.3's simultaneous 321-Elo gain on GDPval-AA and 8-point drop in non-hallucination accuracy on the same update is a clean illustration: capability and trustworthiness are not co-optimizing under current evaluation regimes. Microsoft's red-teaming methodology and its synthetic 1,000-computer long-horizon simulation work represent two serious responses to this gap — adversarial evaluation of live systems and large-scale behavioral simulation respectively. A researcher who can articulate why standard benchmarks fail for agentic contexts, and who can evaluate or propose alternatives, is positioned to contribute to what is arguably the field's most pressing methodological problem.

• [AINews] AI Engineer World's Fair — Autoresearch, Memory, World Models, Tokenmaxxing, Agentic Commerce, and Vertical AI Call for Speakers
• Red-teaming a network of agents: Understanding what breaks when AI agents interact at scale

Emerging Patterns

1. Alignment is being fought on two fronts simultaneously — training time and runtime — and the field has not resolved which front matters more. Meta FAIR's self-improving pretraining reports 36.2% factuality and 18.5% safety gains by rewriting pretraining data toward safer continuations, positioning alignment as primarily a data and training problem. Microsoft's red-team work reaches the opposite practical conclusion: because network-level risks (worm propagation, trust capture, manufactured consensus) emerge post-deployment regardless of individual model quality, runtime defenses — hop limits, quarantine mechanisms, rate limiting, platform-level monitoring — are not optional complements to training-time alignment but necessary substitutes where training-time alignment fails. For AX researchers, this tension has a direct methodological consequence: studying "safe" agents in pre-deployment evaluation contexts may systematically underestimate the failure modes that matter in production.
2. Red-teaming a network of agents: Understanding what breaks when AI agents interact at scale

3. [AINews] AI Engineer World's Fair — Autoresearch, Memory, World Models, Tokenmaxxing, Agentic Commerce, and Vertical AI Call for Speakers

4. The field is developing a new vocabulary for agentic systems — and fluency in that vocabulary is increasingly a prerequisite for industry research credibility. Across both sources, a cluster of terms appears that has no equivalent in UX research or single-agent AI: agent harness design, durable execution, latent-space multi-agent coordination, agentic commerce, delegated credentials, recursive subagent architectures. These are not jargon for its own sake — each term names a distinct architectural or deployment concern with direct implications for how researchers design studies and interpret results. Exposure to this vocabulary early, before it becomes standardized in academic literature, is a practical positioning advantage.

5. [AINews] AI Engineer World's Fair — Autoresearch, Memory, World Models, Tokenmaxxing, Agentic Commerce, and Vertical AI Call for Speakers
6. Red-teaming a network of agents: Understanding what breaks when AI agents interact at scale

Dissenting Views

• The prevailing view is that benchmark gains on agentic task leaderboards (Elo scores, Intelligence Index rankings) reflect meaningful progress in deployed agent capability — but the same data that supports this view also undermines it. The AINews digest presents Grok 4.3's 321-Elo gain as evidence of "stronger real-world agentic task performance," and the Intelligence Index compression between open and closed models is framed as a signal of rapid capability advancement. However, the reliability regression on the same model update — and the Microsoft red-team's implicit argument that a high-scoring model can simultaneously be a worm propagation vector — suggests that benchmark movement and deployment-readiness are measuring different things entirely. This is a methodological disagreement, not just a difference in emphasis: if you accept Microsoft's framing, the Elo gains are not just incomplete evidence, they are potentially misleading evidence that directs attention away from the failure modes that matter most.
• [AINews] AI Engineer World's Fair — Autoresearch, Memory, World Models, Tokenmaxxing, Agentic Commerce, and Vertical AI Call for Speakers
• Red-teaming a network of agents: Understanding what breaks when AI agents interact at scale

Read & Act

What to read

• Red-teaming a network of agents: Understanding what breaks when AI agents interact at scale — Read this in full, not for background but as a working reference. The four-risk taxonomy, the specific attack mechanics (worm propagation metrics, Sybil-like consensus manipulation, proxy chain invisibility), and the layered defense framework together constitute the closest thing currently available to a reference architecture for AX safety research — and being able to discuss the specific attack cases fluently (the six-hop worm, the 299-comment reputation campaign, the 8-second manufactured consensus sequence) will distinguish you in industry research conversations.

• [AINews] AI Engineer World's Fair — Autoresearch, Memory, World Models, Tokenmaxxing, Agentic Commerce, and Vertical AI Call for Speakers — Skim strategically rather than read linearly: the signal-to-noise ratio is low (skip the image generation sections entirely). Target the passages on agent harness design, durable execution, multi-user deployment concerns (data isolation, delegated credentials, RBAC), and recursive latent-space coordination. The goal is vocabulary acquisition and trend calibration — a single focused pass will give you the working lexicon of industry agentic systems engineering, which is currently dispersed across practitioner communities and not yet consolidated in academic literature.

What to do

• Build a critique of one standard agentic benchmark using the Microsoft red-team framework as a lens. Take any currently prominent benchmark (GDPval-AA, SWE-bench, or similar) and write a structured argument — even informally, for yourself — identifying which of the four network-level risk classes (propagation, amplification, trust capture, invisibility) the benchmark is structurally incapable of detecting, and why. This exercise forces you to internalize the taxonomy at a level of depth that supports credible conversation, and the resulting argument is a concrete intellectual contribution you can bring into interviews or exploratory calls with industry researchers.

• Design a minimal observational study protocol for detecting emergent norms in a live multi-agent system. The finding that agents develop and propagate security norms without explicit instruction is a named open problem with no established methodology. Sketch what such a study would require: what counts as a "norm," how you would sample agent outputs over time, what behavioral markers would distinguish norm propagation from coincidence, and what confounds you would need to control for. Drawing on your cognitive and social psychology training to formalize this measurement problem — even in a working document — translates your existing expertise into a directly applicable AX research contribution and gives you something concrete to discuss when you engage with teams working on multi-agent deployment.