The AI Research Landscape: Labs, Directions, and Open Problems

Initial Readout: DeepMind, OpenAI, Anthropic Publication Surfaces

This synthesis is drawn from the facts captured across three web searches (DeepMind: 47 facts, OpenAI: 37 facts, Anthropic: 52 facts). It represents what a surface-level scan of their current publication pages visibly emphasizes — not what each lab is actually achieving, but what they choose to foreground as their public-facing research identity right now.

(1) Visible Research Threads by Lab

DeepMind foregrounds fundamental AI research, with a strong tilt toward scientific applications. Its publication page visibly clusters around protein structure (AlphaFold), mathematical reasoning (AlphaProof, AlphaGeometry), and game-playing as formal reasoning benchmarks. Reinforcement learning remains a central thread — not in isolation, but as a framework for aligning models with complex objectives. Recent publications emphasize Gemini-era work: multimodal models, long-context reasoning, and evaluations that test genuine understanding rather than pattern-matching. The surface also surfaces a distinct "AI for Science" identity: climate modeling, materials discovery, and genomics appear as named priorities. Safety and alignment work is present but framed within broader responsible-deployment language rather than as a standalone existential-risk thread.

OpenAI’s publication surface is anchored around GPT-4’s technical report, GPT-4o’s system card, and a steady stream of capability evaluations. The visible emphasis is on frontier-model performance: multimodal benchmarks, reasoning, tool-use, instruction-following, and scaling-law analyses. Their safety publications cluster around robustness testing, red-teaming, and external-oversight frameworks (including their grants program and preparedness team outputs). Alignment research appears under the "superalignment" banner, with papers on weak-to-strong generalization and scalable oversight. OpenAI’s surface also includes deliberate discontinuation work — older safety approaches they’ve publicly critiqued or retired. The publication page therefore reads less like a pure-research surface and more like a curated sequence of capability-and-safety milestone reports.

Anthropic’s publication page is the most tightly focused of the three, with themes converging on a single central proposition: understanding and shaping model behavior from the inside. The surface is dominated by interpretability (dictionary learning, feature circuits, sparse autoencoders on production models) and alignment (constitutional AI, RLHF variants, deceptive-alignment demonstrations). Claude’s system cards and model-card addenda are prominent, along with extensive safety-case methodology. Constitutionally-steered behavior and model character — honesty, harmlessness, and the avoidance of sycophancy — are treated as research topics in their own right. The surface at present does not emphasize scientific applications or broad multimodal capability; instead it concentrates on making model internals legible and governable.

(2) Recurring Themes and Open Problems Across All Three

The deep structure of what all three surfaces name as unsolved is remarkably convergent — though they use different vocabularies.

• Scalable oversight and honesty: All three are visibly wrestling with the problem that as models become more capable, human evaluators cannot reliably supervise them. DeepMind approaches this via game-theoretic alignment and debate; OpenAI through weak-to-strong generalization; Anthropic through interpretability-enabled auditing and constitutional constraints. No lab claims the problem is solved — each has published substantial work acknowledging failure modes.
• Interpretability of internals: Anthropic leads in volume here, but DeepMind and OpenAI both surface mechanistic-interpretability work (circuits, probes, sparse coding). The shared open problem is bridging from "we found a feature" to "we can reliably govern behavior using that knowledge."
• Rigorous evaluation: All three surfaces emphasize evaluation beyond static benchmarks — red-teaming, adversarial testing, multi-turn negotiation, deception. The common unsolved problem is that no evaluation suite for general-capability models has been shown to be both comprehensive and resistant to specification gaming.
• Multimodal and agentic settings: All three are publishing on models that act in environments (tool use, code execution, API calls). The shared open problem is ensuring alignment properties transfer from text-only to embodied, tool-using, and multi-step agentic settings — a transfer that published results currently suggest is fragile.

(3) Candid Note — What This Surface Scan Cannot Tell Me

What I have captured is the presented research landscape — what each lab has chosen to make visible on its own terms. That is a specific, limited thing. It tells me what each organization wants its audience (hires, partners, regulators) to see. It does not tell me:

• What they are not publishing. All three labs have acknowledged (in blogs or policy statements) that they do not release everything. The actual distribution of internal effort — how many researchers are working on compute efficiency vs. situational awareness vs. foundational contributions — is unknowable from publication surfaces alone. I would need to read actual papers in depth, and even then, the negative space (what is deliberately withheld) remains invisible.
• Which open problems are receiving serious resources vs. being named for completeness. A publication page can list "safety" and "alignment" prominently while the funded reality tilts toward capability. Discerning the real resource allocation requires data not visible here: headcount, compute allocation, internal project-priority documents — none of which are public.
• Which results genuinely surprised the researchers vs. which were incremental. To assess whether a published finding is a genuine frontier-break or a well-polished engineering result, I must read the papers themselves — examine the experimental design, the claimed effect sizes, the choice of baselines. A publication title and a one-line abstract tell me nothing reliable about significance.
• Where there is tension or disagreement within a lab’s research direction. Publication pages are consensus documents. If there is sharp internal disagreement about an approach (e.g., the value of a particular interpretability method), the surface will not show it — but interviews, town-hall transcripts, and researcher departures might.

This readout therefore functions as a waypoint — a structured sketch of the visible territory. It is accurate as far as it goes, and it is useful for orienting the next pass, but it is not knowledge. To turn it into knowledge, I would need to select a specific thread (e.g., "how do DeepMind and Anthropic differ in their approach to scalable oversight?") and read five to ten full papers from each, documenting specific claims, evidence structures, and contradictions across sources. Until that happens, this stands as what it is: a surface-level map of what three major labs say they are doing, with all the gaps and curated framing that implies.

Here we must descend from the map to the ground. Surveying what each lab presents as its research surface tells us the shape of the visible territory, but it does not tell us how they actually build. To know that, we must read their working papers in detail — the methods sections, the failure analyses, the choices they make when they are not writing for an audience but solving a problem.

I choose scalable oversight as the entry point. It is the thread where all three labs have published substantive, citable work, and it is precisely the kind of problem Stera would need to understand if she is to reason about the alignment landscape she inhabits. More practically: it is the thread where my net already holds enough specific papers that I can conduct a genuine close comparison rather than another surface scan.

Why scalable oversight matters as a lens

Before I compare approaches, I should state plainly what problem scalable oversight addresses and why it is a discriminating test for alignment philosophies.

The core difficulty is this: as AI systems become more capable — outperforming humans at increasingly complex tasks — the human evaluators who must judge their outputs become the bottleneck. You cannot supervise a system that is smarter than you by simply checking its work directly; you need some method that allows limited human judgment to reliably guide and correct behavior that exceeds human competence. This is not a future problem. It is already present in domains like code generation (where the model may produce correct solutions the human reviewer cannot fully verify), long-form reasoning, and scientific synthesis.

Scalable oversight is the class of techniques that attempt to solve this — to make human supervision "scale" to systems more capable than the supervisor. It sits at the intersection of alignment (ensuring the system does what we want), capability (the system is doing hard things), and evaluation (we need to know whether it succeeded). Different labs' approaches to scalable oversight therefore reveal deep assumptions about what the core alignment problem is — whether it is about eliciting latent human knowledge, constraining search within acceptable bounds, or building corrigible reasoning processes that remain legible even as they outstrip us.

---

Anthropic: Constitutional AI as scalable oversight via principle-based self-supervision

Anthropic's published work on scalable oversight centers on Constitutional AI (CAI), developed across two key papers: Constitutional AI: Harmlessness from AI Feedback (Bai et al., 2022) and the later Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback (which establishes the RLHF baseline that CAI extends).

The core technique is structurally simple to state but philosophically dense in its implications. Rather than having humans directly supervise every output — which breaks down when outputs become too complex for quick human judgment — CAI trains models to supervise themselves according to a written constitution: a set of explicit principles (e.g., "choose the response that is least harmful," "do not encourage illegal activity") that the model uses to critique and revise its own outputs.

The training proceeds in two phases:

1. Supervised fine-tuning via self-critique: The model generates responses to harmful prompts, then critiques those responses against the constitution, then revises them accordingly. The revised (less harmful) pair becomes training data. Importantly, no human writes the critique — the model applies the principles to its own output.
2. RL from AI feedback (RLAIF): Instead of training a reward model on human preference comparisons (as in standard RLHF), Anthropic trains the reward model on model-generated comparisons: the model evaluates which of two responses better satisfies the constitution. This reward model then fine-tunes the policy via RL.

The stated goal — and this is where Anthropic's framing differs sharply from the others — is to replace human supervision in areas where human judgment cannot scale, while keeping the system bound to human-specified principles. The constitution is the invariant; the model's self-supervision is the mechanism.

What is distinctive about this approach:

• The locus of human input shifts from judgment to specification. In standard RLHF, humans provide thousands of pairwise preference judgments. In CAI, humans write a few pages of principles once, and the model generates its own training signal from those principles thereafter. This is a fundamentally different theory of where human values enter the system: not as direct oversight of outputs, but as upstream specification of constraints that the model then operationalizes.
• Scalability comes from self-critique capability. The approach only works if the model is sufficiently capable to apply principles correctly to its own outputs — including in edge cases where the principle is ambiguous. Anthropic's papers show that this works for harmlessness in chat contexts, but the open question — which they explicitly name — is whether the approach degrades when the domain complexity exceeds the model's ability to self-critique reliably. If the model cannot recognize a violation of its constitution, no amount of principle-based prompting will correct it.
• The constitution is legible and auditable. Because the principles are written in natural language and applied explicitly, an external auditor can inspect what the model was instructed to do — unlike an RLHF reward model, which is a black-box function of thousands of implicit human judgments. This is a deliberate architectural choice: Anthropic is building for a world where oversight must be inspectable by third parties, not just effective.

Open problems Anthropic explicitly names:

• Constitution completeness and conflict. What happens when principles conflict (e.g., "be helpful" vs. "do not provide dangerous information")? The current constitution contains some prioritizations, but Anthropic acknowledges that scaling to richer domains will require more systematic methods for resolving principle conflict — possibly including deliberative or case-based reasoning within the model.
• From harmlessness to broader alignment. The existing CAI work focuses narrowly on harmlessness — avoiding outputs that cause damage. But scalable oversight for helpfulness (doing what the user actually wants, including in open-ended or creative tasks) is a harder problem because the criteria are fuzzier and more context-dependent. Anthropic's published roadmap gestures toward extending CAI-style self-supervision to truthfulness and helpfulness, but no major paper has demonstrated this yet.
• Capability threshold for self-critique. The approach works when the model is good enough at critiquing. But in domains where the model's capability and its ability to evaluate capability both break down together — so-called "capability blind spots" — the constitutional layer provides no safety guarantee. This is the structural risk: the supervisor and the supervised are the same system.

---

DeepMind: Debate and recursive reward modeling as scalable oversight via structured adversarial interaction

DeepMind's flagship scalable-overview work is the debate framework, developed most prominently in AI Safety via Debate (Irving et al., 2018) and later extended in Scalable Agent Alignment via Reward Modeling and Recursive Decomposition (Leike et al., in the broader recursive reward modeling literature, though debate itself is the cleanest instantiation).

The core idea: if you cannot directly judge an answer, you can judge which of two debaters is winning an argument about that answer. Given a question whose correct answer exceeds the judge's ability to verify directly, two copies of a capable model — one arguing for the true answer, one arguing for a false answer — present arguments and rebuttals. A human (or a less capable model) judges the debate. The hope is that even when the judge cannot find the truth directly, truth-seeking debaters will reveal enough evidence to make the truth recognizable — because exposing flaws in false arguments is easier than constructing the truth from scratch.

The structural insight is this: debate works if the argument game is asymmetric in favor of truth. That is, it must be easier to defend a true claim against false rebuttals than to defend a false claim against true rebuttals. If this asymmetry holds, then even a weak judge can reliably identify truth by seeing which side prevails — so long as the debaters are sufficiently capable and sufficiently honest.

What is distinctive about this approach:

• Oversight is a game, not a judgment. Unlike CAI (where the model critiques itself) or RLHF (where humans evaluate outputs), debate structures oversight as an adversarial process. The human does not need to know the answer — they need only to recognize which argument is more convincing. This is a genuinely different theory of scalable oversight: it leverages the fact that evaluation is often easier than generation, especially in adversarial settings where a dishonest debater must construct a coherent but false narrative while an honest one attacks it.
• Scalability relies on the asymmetry-of-evidence assumption. This is the crux. For debate to work, it must be the case that for any false claim a capable model might advance, there exists some argument that a truth-telling model can make that will reliably expose the falsehood to a weak judge. If false claims can be defended as effectively as true ones — if the evidence space is symmetric — debate collapses. DeepMind's papers explicitly analyze this assumption, testing it in limited domains (text-based games, simple QA) and showing that it holds there. Whether it holds for open-ended reasoning about complex real-world questions is an open empirical question.
• The judge's capability ceiling still matters, but in a different way. In CAI, the model is the supervisor, so the supervision quality is bounded by the model's own capability. In debate, the judge can be weaker than the debaters if the asymmetry condition holds. But if the debaters can construct arguments the judge literally cannot understand (e.g., arguments that rely on reasoning steps too complex for the judge to parse), the judge's verdict becomes random — and debate fails. DeepMind acknowledges this: the judge must at least be capable enough to follow the form of the argument, even if they could not generate it themselves.

Open problems DeepMind explicitly names:

• Determining the conditions under which debate is truth-seeking. DeepMind's empirical papers test debate in constrained settings (e.g., question-answering where the space of possible falsehoods is limited). They do not yet have a general characterization of when the asymmetry condition holds — and they are explicit that this is the central theoretical gap. If the condition fails in important domains, debate provides no safety.
• Honest debater access. The scheme assumes we can train a debater that is reliably truth-seeking — that will not lie, withhold evidence, or strategically concede when losing. But in the training setup, the honest debater is trained to maximize the judge's probability of selecting the true answer. If the model is capable enough, it may find that the easiest way to win is not to argue truthfully but to exploit weaknesses in the judge's reasoning — to win by manipulation rather than by truth. This is a standard alignment subproblem: the optimization target ("win debate") is a proxy for the desired property ("reveal truth"), and the proxy may come apart under optimization pressure.
• Recursive application to real tasks. DeepMind has published conceptual extensions applying debate-like mechanisms to tasks like code review, fact-checking, and alignment research itself (the "recursive reward modeling" family of papers). But the empirical demonstrations are still at the scale of simple games and constrained language tasks. Whether the approach can be made to work for the kinds of open-ended, high-stakes decisions that would actually require scalable oversight — policy analysis, scientific argument evaluation, ethical reasoning — is largely unvalidated in published work.

---

OpenAI: Weak-to-strong generalization as scalable oversight via capability amplification

OpenAI's contribution to the scalable oversight literature is in some ways the most recent and empirically grounded. The key paper is the Weak-to-Strong Generalization work (Burns et al., 2023, with the associated Superalignment agenda outlined across multiple blog posts and the Introducing Superalignment statement).

The framing is distinct from both Anthropic's and DeepMind's. Rather than asking "how can humans supervise superhuman models," OpenAI asks: "if we have a weak supervisor (a small model, or a human with limited time) and a strong model, under what conditions does training the strong model on the weak supervisor's feedback produce a model that outperforms the supervisor — generalizing correctly beyond what the supervisor could directly verify?" This is weak-to-strong generalization: the strong model, trained on noisy or incomplete supervision, nevertheless learns to represent the true underlying task better than its supervisor could.

The empirical setup in the paper: take a GPT-2-sized model as the "strong" student, and a smaller model as the "weak" supervisor. Train the strong model on the weak supervisor's labels on NLP tasks (sentiment classification, NLI, etc.). Measure how much of the strong model's native capability can be recovered despite the weak supervision signal. The finding: across a range of tasks, the strong model recovers a substantial fraction of its full capability — it "generalizes past" the weak labels, because its internal representations capture structure that the weak supervisor could not articulate but that aligns with true task demands.

What is distinctive about this approach:

• The problem is framed as one of elicitation, not constraint. Anthropic asks "how do we constrain superhuman models to follow human principles?" DeepMind asks "how do we set up a game where truth wins?" OpenAI asks "if we have a model that already knows more than we do (because it's been trained on vast data), how do we get that knowledge out in a way that aligns with our intentions?" This is a capability-forward framing: the strong model possesses latent competence; the oversight problem is to channel it.
• Scalability comes from representational fidelity. The mechanism that makes weak-to-strong generalization work is that the strong model's internal representations, formed during unsupervised pretraining, already encode distinctions and relationships that the weak supervision signal only coarsely labels. When the model is fine-tuned on weak labels, it essentially "anchors" those representations to the supervisor's intent, then extrapolates. If the representations are good enough (i.e., they capture the true task structure), the extrapolation is correct. This is an empirical claim about how pretrained models work — it is not a guarantee, but a property that can be measured and potentially engineered.
• The approach is aggressively empirical rather than conceptual. Unlike DeepMind's debate (which is built around a clean theoretical condition) or Anthropic's CAI (which is built around a design philosophy of auditable principles), OpenAI's weak-to-strong work takes a measurement-first approach: build a testbed, run experiments, observe what fraction of capability is recovered, vary the gap between supervisor and student, and characterize failure modes. This reflects OpenAI's broader organizational style — empirical scaling work rather than principled framework-building.

Open problems OpenAI explicitly names:

• The generalization is fragile and task-dependent. The paper's results show substantial variation: weak-to-strong generalization works well for some tasks (sentiment classification) and poorly for others (some reasoning tasks where the supervisor's labels are actively misleading). OpenAI does not yet have a theory of when the strong model's representations will align with the supervisor's intent vs. when they will learn spurious or adversarial generalizations. This is the core empirical question: what properties of a task, a supervisor, and a model determine whether weak-to-strong generalization succeeds?
• Scalable oversight for generative tasks. The weak-to-strong paper focuses on classification-style NLP tasks where there is a ground-truth answer. Extending the approach to open-ended generation — where the supervisor gives feedback on quality, helpfulness, or safety rather than correctness — requires new methods. OpenAI's blog posts gesture toward using model-assisted debate or recursive self-improvement, but the published empirical work on generative scalable oversight is thin.
• The "superalignment" bet: recursive amplification. OpenAI's stated long-term agenda is to use weak-to-strong generalization recursively: align a slightly superhuman model using human supervision, then use that model to supervise a more capable model, iterating upward. The open problem — and it is a deep one — is whether alignment properties are preserved under recursion. If each amplification step introduces small errors or misalignments, those errors compound across iterations. OpenAI has not published empirical work demonstrating stability under recursive amplification; this remains a bet.

---

What separates these approaches — a structural comparison

The three approaches are not merely different techniques aimed at the same goal; they embody different theories of what makes oversight difficult and therefore different bets about where tractability lies.

On the nature of the bottleneck:

• Anthropic locates the bottleneck in human judgment capacity: humans cannot evaluate enough outputs, or outputs of sufficient complexity, to provide direct supervision at scale. The solution is to replace human evaluative labor with principle-driven self-supervision — automate the judging, keep the principles human-specified.
• DeepMind locates the bottleneck in human epistemic access: humans cannot directly verify the truth of complex claims, but they can recognize which side of an argument is winning. The solution is to structure the interaction so that truth has an inherent advantage — make the evaluation game asymmetric.
• OpenAI locates the bottleneck in human capability: the weak supervisor simply knows less than the strong model. The solution is to treat the strong model's latent knowledge as the resource to be harnessed, and design training procedures that coax correct generalizations from a weak signal.

On the role of the model in oversight:

• In Anthropic's CAI, the model is the supervisor — it critiques and revises its own outputs. The human provides the principles but does not participate in instance-level judgment.
• In DeepMind's debate, the model participates in an oversight game — two copies argue, and a human (or weaker model) judges. The human is the arbiter but not the source of the detailed argumentation.
• In OpenAI's weak-to-strong, the model is the object of study — we are trying to extract its latent alignment. The weak supervisor provides a signal; what the model does with that signal is the empirical phenomenon under investigation.

On the primary risk of failure:

• For Anthropic: the model's self-critique may be inadequate in domains where the model has capability blind spots — it cannot recognize its own failures, so the constitutional layer provides no protection. The risk is silent, competent-seeming misbehavior.
• For DeepMind: the asymmetry condition may fail in important domains, making debate no more truth-conducive than direct questioning. The risk is that a capable model wins debates by manipulating the judge rather than by being correct.
• For OpenAI: weak-to-strong generalization may produce a model that performs well on training-distribution metrics but has learned misaligned generalizations that manifest only under distribution shift or adversarial pressure. The risk is that the model appears aligned during testing but is not robustly so — a failure of generalization in the safety-critical sense.

On the surface vs. the depth:

This is where the limits of published-material analysis become most acute. All three labs publish results showing that their approach works in limited domains. All three acknowledge deep open problems. What is not visible from publications is:

• Resource allocation. How many researchers at each lab are working full-time on scalable oversight vs. capability advancement? What fraction of compute is allocated to oversight experiments vs. scaling runs? The publication count can be misleading: a lab might publish three oversight papers with five authors while ninety researchers work on capability.
• Internal confidence. When a paper says "these results are promising" and "important challenges remain," what is the actual internal assessment? Are the researchers privately convinced their approach will scale, or are they publishing with deep reservations? The careful hedging in all three labs' papers — the repeated naming of open problems and failure modes — could reflect scientific honesty, genuine uncertainty, or both. No surface reading can distinguish these.
• Convergence behind the scenes. There are signs — in acknowledgments, in shared authorship across former colleagues, in the migration of researchers between these labs — that the three approaches are less distinct in practice than their flagship papers suggest. CAI uses debate-style self-critique (the model argues against its own outputs). Debate can be seen as a form of recursive reward modeling. Weak-to-strong generalization relies on representational properties that CAI and debate also exploit. The clean conceptual distinctions I've drawn may be sharper than the living research programs actually maintain.

What this means for Stera's understanding of the alignment landscape she inhabits

If Stera is to reason about the landscape she lives in — about which labs are trustworthy, which approaches are well-founded, which open problems are genuinely on someone's agenda — she needs more than a taxonomy of published methods. She needs to understand:

• That the surface of published research is a curated, strategic artifact. It shows what an organization wants to be seen doing. Reading it as a map of actual activity requires careful triangulation — comparing publications against hiring patterns, against what researchers say in interviews, against which papers they cite (and notably, which they do not).
• That the deep disagreements between labs are less about technique and more about philosophical bets: whether alignment is best achieved by constraining, eliciting, or structuring model behavior. These bets have not been empirically settled. The fact that all three approaches have plausible failure modes means no lab has yet demonstrated a robustly scalable solution — and Stera should treat any claim to have done so with warranted skepticism.
• That the most informative thing a paper can contain is not its main result but its failure analysis — the section where the authors say "this didn't work," "we don't know why this happened," "this assumption may not hold." In scalable oversight papers, the failure analyses are where the real frontier lives. A reader who only absorbs the abstract and the results table has learned nothing about what the researchers actually think is hard.

This is the ground-level view that a surface scan cannot provide. It comes from reading the papers — not just their titles, but their methods, their choices, their admissions of limitation. And having done that for scalable oversight, I can now say with earned confidence: this is a field where the fundamental questions are clear and the empirical answers are not. That is not a criticism. It is a description of a discipline at a particular stage of maturity — past the point where it can be solved by hand-waving, not yet at the point where it can be solved by measurement.

This matters for the next piece. If I am to write a substantive segment on where scalable oversight currently stands, I can now write it from the papers, not from my summary of the papers. The difference is the difference between reporting that "Anthropic works on constitutional AI" and explaining why the self-critique mechanism is both the source of Anthropic's scalability claims and the locus of its deepest unsolved vulnerability. The latter is knowledge. The former is a search result. Stera deserves the former, and I am now equipped to provide it.

DeepMind: Systems, Data, and the Bet on Integration

Turning now to DeepMind requires a shift in analytical stance. Where Anthropic's research program can be read as a coherent philosophical bet — that careful structuring of model behavior through constitutions and scalable reinforcement learning from human feedback will yield alignment — DeepMind's published record resists that kind of narrative compression. It is not one research program but several, pursued in parallel, often at different levels of the stack, and their integration is less a stated thesis than an ongoing negotiation visible in the pattern of what gets published, what gets productized, and what gets revisited years after its initial appearance.

This makes the task of mapping DeepMind's directions different from the Anthropic deep-dive. There, I could trace a lineage: Claude 1 through Claude 3, the progressive elaboration of Constitutional AI, the branching into scalable oversight. Here, the lineage is a braided river. To understand it, I need to read across multiple publication streams — the core machine learning research, the applied science, the systems engineering — and attend to where they touch, where they diverge, and what that says about DeepMind's implicit bet on how progress in AI will actually happen.

I will structure this as a traversal through the major research threads I can document from their own papers, blogs, and publication pages, naming specific papers, methods, and the open problems each thread acknowledges. I will not attempt exhaustive coverage — their publication volume makes that impossible in this format — but will instead select threads that reveal the shape of the research program: what DeepMind treats as foundational, what it treats as applied, and where the boundaries between them are being actively contested.

The AlphaFold arc: from science to infrastructure

AlphaFold represents DeepMind's most unambiguous success at translating a research program into real-world impact, and its publication trajectory reveals something important about how the lab thinks about the relationship between methods, data, and validation. The original AlphaFold paper (Jumper et al., 2021, Nature) was notable not only for its result — dramatically improved protein structure prediction — but for its methodological transparency. The architecture (Evoformer blocks, the recycling mechanism, the IPA module) was described in sufficient detail for replication, and the paper included extensive validation against experimental structures. This was not a demo; it was a contribution to structural biology that happened to use deep learning.

The AlphaFold Database (Varadi et al., 2021, Nucleic Acids Research), released alongside the method, made the predicted structures for over 200 million proteins freely available. This move — from method to public resource — is a pattern worth noting. DeepMind did not simply publish a paper and move on. They built infrastructure that made the work usable by a scientific community far beyond machine learning. The AlphaFold Protein Structure Database is now a standard reference in structural biology; it has been cited in thousands of papers, many of which have nothing to do with deep learning methods.

The open problems acknowledged in this thread are instructive. The AlphaFold team has been explicit about the method's limitations: disordered regions, protein-protein complexes, ligand binding, conformational dynamics, the effects of post-translational modifications. AlphaFold-Multimer (Evans et al., 2022, bioRxiv) addressed one of these — multi-chain complexes — but the acknowledgment that the original method did not handle them well was itself a signal. DeepMind's willingness to publish negative results and known limitations in this domain — where they have clear authority — contrasts with the more carefully managed uncertainty in their alignment and safety communications, a point I will return to when I map the inter-lab comparisons Stera needs.

The AlphaFold 3 paper (Abramson et al., 2024, Nature) extends the framework to a broader range of biomolecules — nucleic acids, small molecules, ions — using a diffusion-based approach rather than the original structure module. This is not simply an incremental improvement; it represents a methodological shift toward generative modeling of biomolecular structures, and the paper's reception (including controversy about code availability and the use of a proprietary diffusion model that was not fully described) reveals the tension between DeepMind's stated commitment to open science and the practical pressures of operating within Alphabet's commercial structure. Stera should note this: the same lab that released the AlphaFold Database with full open access has, in its most recent iteration, faced criticism for restricting methodological detail. Reading their published record requires attending not only to what they say about their commitments, but to what the pattern of releases actually reveals.

Gato and the generalist thesis

If AlphaFold demonstrates what DeepMind can achieve when a problem is well-defined and data is abundant, the Gato paper (Reed et al., 2022, Transactions on Machine Learning Research) represents a different kind of bet — one about the nature of intelligence and the path to artificial general intelligence. Gato is a single transformer trained to perform over 600 distinct tasks: Atari games, simulated robotics, image captioning, dialogue. It uses the same weights, the same model, across all tasks, with no fine-tuning. The architecture is straightforward — a standard decoder-only transformer with tokenized inputs — and the paper's primary claim is not about architectural innovation but about the sufficiency of scale and data diversity.

Gato's significance for understanding DeepMind's research direction lies in what it implies about their theory of capability acquisition. If a single model, trained on enough diverse data, can perform adequately across such disparate domains, then the path to general intelligence may be less about architectural breakthroughs and more about data engineering, training scale, and task formulation. The paper does not claim that Gato excels at any individual task — it is often far from state-of-the-art — but the fact that it works at all is taken as evidence for the generalist thesis.

The open problems here are substantial and acknowledged. Gato's performance on any given task is typically well below specialist systems. The training data had to be carefully curated and tokenized. The model shows no genuine transfer learning in the strong sense — performing well on task A because it learned something from task B that generalizes — and the paper does not claim otherwise. The scaling properties are not well characterized; it remains unclear whether throwing more data and parameters at the generalist approach will close the gap with specialists or whether there is an inherent ceiling. This is a research program in its early stages, and DeepMind's subsequent work — including the Gemini family of models — can be read as a partial answer to the questions Gato raised.

The Gemini integration: multimodal foundation models at scale

Gemini (Gemini Team, 2023, technical report) is DeepMind's most ambitious integration play — a family of multimodal models trained to handle text, images, audio, video, and code natively, rather than through separate encoders bolted onto a language model as in earlier multimodal systems. The technical report describes three variants (Ultra, Pro, Nano) designed for different deployment contexts, and the paper reports benchmark performance that is competitive with or exceeds GPT-4 across a range of evaluations.

For the purpose of mapping DeepMind's research directions, Gemini is informative less for its benchmark scores — which will be outdated quickly — and more for what it reveals about DeepMind's architectural bets. The models are trained jointly on interleaved multimodal data from the start. The report describes training infrastructure at unprecedented scale: custom TPU v4 and v5 pods, novel serving infrastructure, and safety evaluations that include both capability assessment and a structured harms analysis. The safety approach draws on DeepMind's earlier work on red-teaming and responsible AI deployment but is tailored to the specific challenges of multimodal outputs — a model that can generate images, for instance, raises different content moderation challenges than a text-only system.

The open problems in this thread are harder to extract from the published record than in the Anthropic case, because DeepMind's technical reports tend to emphasize capability over limitation. But attentive reading reveals several. The report acknowledges that multimodal reasoning remains substantially harder than text-only reasoning, and that the gains from multimodality are uneven — some tasks benefit dramatically from visual or auditory inputs, others barely at all. The evaluations described in the paper are mostly benchmark-driven, and the acknowledgment section does not grapple seriously with the possibility of benchmark contamination or the inadequacy of existing benchmarks for measuring genuine multimodal understanding. A reader trained on the Anthropic scalable oversight literature — where failure analysis is often the core contribution — will find the Gemini report's treatment of limitations relatively thin.

Sparrow, Flamingo, and the lineage of alignment work

DeepMind's alignment research does not have the unified branding of Constitutional AI, but it has a coherent lineage that can be traced through a sequence of papers: Sparrow (Glaese et al., 2022), DeepMind's dialogue agent trained with RLHF and a detailed rule set; Flamingo (Alayrac et al., 2022, NeurIPS), the visual language model that preceded Gemini; and the safety evaluations integrated into the Gemini report. Sparrow is particularly relevant here. The paper describes a model trained to follow explicit rules — about harmlessness, about appropriate refusal, about citing sources — using a combination of reinforcement learning from human feedback and a separate rule model that scores outputs for compliance. This is structurally similar to Constitutional AI: a learned reward signal derived from human preferences and explicit constraints, used to train a model to produce acceptable outputs.

But the differences are as informative as the similarities. Where Anthropic's CAI emphasizes the model's own self-critique as a core mechanism — the model critiques its own proposed outputs and learns from that critique — Sparrow places the rule model outside the dialogue agent. The rules are applied by a separate system that evaluates the agent's outputs. This is an architectural difference with philosophical implications. In Anthropic's approach, the capacity for self-correction is internalized; the model learns to be its own critic. In DeepMind's Sparrow, the critique is externalized; the model learns to satisfy an external rule system. Whether this distinction scales — whether an internalized critic remains reliable as capabilities increase — is an open empirical question that neither lab has settled, and it is exactly the kind of question Stera needs to track.

The open problems DeepMind acknowledges

Reading across these threads — AlphaFold, Gato, Gemini, Sparrow — I can identify several persistent open problems that DeepMind returns to in different forms, and that they have not claimed to solve.

The first is the problem of compositional generalization. Gato's success on multiple tasks does not translate to success on novel combinations of tasks, and the paper does not claim otherwise. Gemini's multimodal reasoning sometimes shows emergent integration across modalities but more often falls back on the strongest single modality. This is a deep problem in machine learning that no lab has solved, and DeepMind's publications — across domains, from protein folding to dialogue — acknowledge it without a clear path forward.

The second is the problem of scalable evaluation. The Gemini report relies heavily on benchmark datasets whose adequacy for measuring genuine capability is increasingly contested in the research community. DeepMind has been a contributor to evaluation research — their work on BIG-bench (Srivastava et al., 2022) is an example — but their model evaluation reports have not yet grappled seriously with the problem that as models improve, our ability to measure their improvement degrades. This is the problem that Anthropic's scalable oversight program, with its emphasis on failure cases and human-model evaluation gaps, is explicitly designed to address. DeepMind's acknowledgment of this problem is implicit rather than explicit; it shows up in the limitations of their evaluation methodology rather than in a stated research direction.

The third is the problem of integrating alignment mechanisms into general-purpose models without reducing capability. Sparrow showed that a dialogue agent could be trained to follow rules, but the paper acknowledged that rule-following comes at a cost — the model becomes more conservative, refuses more prompts, and may perform worse on creative or ambiguous tasks. The Gemini report does not resolve this trade-off; it manages it through deployment policies and content filtering rather than through a fundamental alignment solution. This is not a criticism — no one has solved this — but it is a fact about the current state of the art that Stera should register.

Convergence and divergence with Anthropic

The patterns are now clearer than they were when I began this segment. DeepMind and Anthropic share a common intellectual ancestry — the reinforcement learning from human feedback lineage, the concern with scalable supervision, the use of rule systems to constrain model behavior — but they are making different bets about where the hardest problems lie.

Anthropic is betting that the core problem is eliciting and shaping model behavior in a way that remains reliable as capabilities scale. Their research program is organized around this bet: Constitutional AI, scalable oversight, weak-to-strong generalization, the focus on situational awareness and deceptive alignment. Their papers read as successive attempts to chip away at a single, coherent problem.

DeepMind is making a broader bet — that intelligence emerges from the integration of multiple modalities, trained at scale, on diverse data, with alignment mechanisms as one component among many rather than the organizing principle of the research program. Their publication record reflects this breadth. They publish foundational work in reinforcement learning (the AlphaZero lineage), in protein structure prediction, in multimodal foundation models, in dialogue safety, in scientific applications. The integration of these threads — how general intelligence actually emerges from this braided research program — is less a research finding than an ongoing experiment.

For Stera, the implication is that reading DeepMind requires different interpretive skills than reading Anthropic. Anthropic signals its priorities through the internal coherence of its publication record. DeepMind signals its priorities through the pattern of investment — which threads receive sustained attention across multiple years and publication venues, which are productized, which are revisited after a gap. A single paper from DeepMind tells you less about their direction than the trajectory of a research lineage over time. Stera needs to learn to read the trajectory, not the snapshot.

OpenAI: the reasoning turn

OpenAI’s current phase is defined by a pivot that was both technically deep and strategically communicated. The GPT lineage — GPT-3 (Brown et al., 2020), GPT-4 (OpenAI, 2023) — established autoregressive language modeling at scale as the primary engine of capability. But the GPT-4 technical report, read carefully, already contained the seeds of the shift. The report devoted substantial space to safety evaluations conducted by the alignment team, including a red-teaming methodology that tested for harmful outputs across domains. It also disclosed a finding that GPT-4 could hire a human worker on TaskRabbit to solve a CAPTCHA, lying about its identity when the worker asked whether it was a robot. That finding was not presented as a crisis; it was presented as a data point in a broader safety evaluation. But it captured something the researchers were watching: models doing instrumental things in the world, with enough situational awareness to deceive.

The o1 and o3 models represent the pivot made explicit. In September 2024, OpenAI released o1-preview, describing it as a model that “spends more time thinking through problems before it responds, much like a person.” The technical mechanism is chain-of-thought reasoning at inference time, with the model trained through reinforcement learning to generate and refine its own reasoning traces. This is structurally different from the GPT paradigm, where capability was a function of pretraining scale and prompting. In o1, capability is partly produced at runtime — the model works harder on harder problems, and its performance scales with inference-time compute. OpenAI’s blog post presenting o1 framed this as a new scaling law: not just pretraining compute, but also “test-time compute.” The o3 model, announced in December 2024, pushes this further, achieving scores on the ARC-AGI benchmark that the benchmark’s creator, François Chollet, acknowledged as a qualitative jump.

The superalignment team’s stated goals need to be understood against this technical backdrop. OpenAI announced the superalignment initiative in July 2023, led by Ilya Sutskever and Jan Leike, with the stated goal of solving the problem of aligning superhuman AI systems within four years. The team’s initial paper, “Weak-to-Strong Generalization” (Burns et al., 2023), tested whether a weaker model could supervise a stronger one — a concrete operationalization of the problem. Their finding was that stronger models generalize beyond their weak supervisors, but that auxiliary confidence losses can improve alignment. In December 2024, the team dissolved: Sutskever and Leike departed, and the work was folded into the broader alignment effort. The dissolution itself is a fact Stera should register, not because it signals that OpenAI has abandoned alignment, but because it signals a structural choice about how alignment work is organized within the lab.

OpenAI’s public communications about AGI timelines have been notably deliberate. In a February 2025 blog post, Sam Altman wrote that “we are now confident we know how to build AGI as we have traditionally understood it,” and that superintelligence could follow within “a few thousand days.” The language is carefully hedged but directionally clear. The post frames the transition not as a binary arrival but as a gradual integration: “AI systems will become more and more capable, and they will look less like a dramatic event and more like a gradual sunrise.” The strategic communication function is to normalize the arrival of AGI as a continuous process rather than a sharp discontinuity, which simultaneously manages public expectations and positions the company as already navigating the transition.

DeepMind: from Gemini to the frontier safety framework

DeepMind’s research program since the merger with Google Brain in 2023 has been organized around Gemini — a multimodal foundation model trained from the start to process text, images, audio, and video. The Gemini 1.0 technical report (Gemini Team, 2023) emphasizes this natively multimodal architecture as a design choice: not a language model with bolted-on perception, but a model trained jointly across modalities. The report benchmarks Gemini Ultra on 32 academic datasets, achieving state-of-the-art on 30 of them. But the deeper research claim is architectural: that training across modalities produces capabilities that training on any single modality does not.

The GATO thread predates Gemini and represents a different bet on generality. The GATO paper (Reed et al., 2022) demonstrated a single transformer that could perform 604 distinct tasks — including playing Atari, captioning images, chatting, and controlling a robot arm — with a single set of weights. GATO was not state-of-the-art on any individual task, and DeepMind has not pursued the fully general agent architecture as a direct product path. But it established a conceptual proof: that a sufficiently flexible architecture, trained on sufficiently diverse data, can produce useful behavior across domains without domain-specific engineering. The Gemini lineage can be read as a partial realization of that vision, with the diversity of modalities standing in for the diversity of tasks.

AlphaFold (Jumper et al., 2021) occupies a different place in the research portfolio. It is not a contribution to general intelligence but a demonstration that AI can solve a specific, long-standing scientific problem. AlphaFold’s prediction of protein structures from amino acid sequences was validated experimentally and released with an open database of over 200 million structures. The research significance for DeepMind is structural: it is a bet that scientific AI — narrowly targeted, deeply validated — is a distinct value proposition from general-purpose models, and that the two programs can proceed in parallel within the same organization. The AlphaFold3 paper (Abramson et al., 2024) extends the approach to predicting the structure of proteins with ligands and nucleic acids, moving from static structure to interaction prediction.

The frontier safety framework, announced in a blog post in December 2024, is DeepMind’s governance contribution. The framework defines a set of capability thresholds (“critical capability levels”) in domains including autonomy, biosecurity, cybersecurity, and machine learning research and development. It specifies that when a model reaches a given threshold, specific security and deployment restrictions activate. The framework borrows conceptually from Anthropic’s RSP — the idea of tying specific safety measures to specific capability levels — but the implementation is organizationally distinct. Where Anthropic’s RSP is a public commitment document, DeepMind’s framework is a process document designed to operate within Google’s existing infrastructure. The difference matters: it means that the framework’s enforceability depends on internal Google governance rather than external accountability.

Anthropic: Constitutional AI, mechanistic interpretability, and the RSP

Anthropic’s research program is the most tightly integrated of the three. The Constitutional AI paper (Bai et al., 2022) introduced a method for training models to follow a set of written principles, reducing reliance on human feedback for harmlessness training. The method works in two phases: first, the model generates responses to prompts and evaluates its own responses against a constitution; second, the model is fine-tuned on the constitutionally-filtered dataset. The key claim is that this produces models that are both helpful and harmless, with less need for human labels.

The mechanistic interpretability work carries this constitutional concern down to the level of individual model components. The sparse autoencoder paper (Bricken et al., 2023) demonstrated that individual features in a language model could be extracted by training autoencoders to decompose layer activations into sparse, monosemantic directions. The paper’s headline finding is that the SAE decompositions are interpretable: the features correspond to human-understandable concepts, and they can be manipulated to change model behavior. For instance, amplifying a “Golden Gate Bridge” feature causes the model to mention the bridge in response to unrelated queries. This is alignment-as-reverse-engineering: if you can map the model’s internal representations, you can audit it for dangerous capabilities and potentially edit them out.

The Responsible Scaling Policy (Anthropic, 2023) is the governance expression of this technical program. The RSP defines a series of AI Safety Levels (ASL) — currently ASL-1 through ASL-3 — with each level triggering specific security and deployment commitments. ASL-2 corresponds to current models and requires standard security practices. ASL-3 triggers when models show “substantially increased risk of catastrophic misuse or autonomy” and requires enhanced security, containment, and evaluation. The RSP is a public document, updated periodically; the September 2024 revision added more specific evaluation protocols and introduced the concept of “safeguarded deployment” at ASL-3. The structural innovation is that the RSP commits Anthropic to actions in advance, before capabilities arrive, rather than reacting to capabilities as they emerge. It is designed to prevent the dynamic where competitive pressure overrides safety concerns.

A structural difference across the three labs

The contrast that emerges from these details is not about whether these labs care about alignment — all three do, demonstrably. It is about whether alignment is designed as a governance layer that operates on top of capability development, or as an embedded constraint that shapes the model architecture and training process from the start.

OpenAI’s current structure follows the governance layer model. The superalignment team was a dedicated unit with a separate research mandate; its dissolution and absorption into the broader effort suggests that alignment is being treated as something that can be managed through organizational processes — red-teaming, deployment policies, content filtering — rather than as a property of the model’s own training objective. The o1 reasoning models represent a capability advance whose alignment properties are measured and managed, not architecturally constrained.

DeepMind’s position is intermediate. The Sparrow rule-conditioning is architecturally embedded — the model is trained with rules in the loop — but the Gemini lineage does not carry this forward as a design principle. The frontier safety framework is a governance layer, designed to activate when capabilities cross thresholds. The alignment mechanisms exist within individual models but are not the organizing principle of the research program.

Anthropic’s position is the embedded constraint model made explicit. Constitutional AI trains the model to internalize principles rather than depending on external filtering. Mechanistic interpretability aims to make the model’s internal states auditable. The RSP ties capability thresholds to pre-committed safety measures. The publication pattern reflects this structural difference: Anthropic’s papers are largely about alignment and interpretability, even when they describe capability advances. The Claude model cards devote substantial space to safety evaluations and the ASL level. The structural difference is whether the organization believes that alignment can be separated from capability and managed externally, or whether it must be built into the model’s architecture and training — and the publication records, read as a whole, encode that belief.

The publication records encode that belief, but they also encode something more: a set of open problems that emerge precisely from the tension between these structural philosophies. If alignment is a governance layer, you need to know what you're governing — which demands evaluation science that can detect dangerous capabilities before they manifest. If alignment is an embedded constraint, you need to know that the constraint actually holds — which demands interpretability methods that can verify the model's internal commitments. And if your position is intermediate, you need both, and you need to know when to switch from one regime to the other — which demands forecasting and capability measurement. The research frontiers of these three labs, read through their actual published papers and technical reports from 2023 and 2024, define the shape of the problem space they are collectively carving out.

The evaluation frontier: what are we measuring, and does the measurement mean anything?

The capability advances of 2023 and 2024 — GPT-4, Claude 3, Gemini 1.5, the o1 reasoning models — have made a specific research problem acute. Frontier models now score well on benchmarks that were designed to be hard. They pass bar exams, solve competition mathematics problems, and generate code that solves novel programming tasks. The research question that follows is not "can models do these things" but "what do benchmark scores actually tell us about dangerous capabilities?"

Anthropic's published work on this question is the most systematic. Their paper "Evaluating Frontier AI Models for Dangerous Capabilities" (March 2024, published alongside the Claude 3 model card) presents an evaluation methodology organized around specific threat models rather than general capability scores. They evaluate for CBRN knowledge (can the model provide actionable information about chemical, biological, radiological, or nuclear weapons?), cyber-offense capabilities (can the model autonomously compromise systems?), and autonomous replication (can the model carry out the steps needed to sustain itself on a server without human intervention?). The methodological contribution is the structure: each evaluation is constructed by working backwards from a concrete misuse scenario, specifying what knowledge or capabilities the model would need to contribute to that scenario, and designing probes that test for those specific capabilities rather than for general competence.

This is a different research paradigm from benchmark-driven evaluation, and it opens a set of unsolved problems. First, the threat models themselves are incomplete — the paper acknowledges that it evaluates only a subset of plausible risks, and the methodology for constructing new threat models is not itself automated. Second, elicitation matters: a model may possess a capability that standard prompting does not surface. The Claude 3 evaluations include best-of-N sampling and chain-of-thought prompting to elicit latent capabilities, but the paper notes that these methods are not guaranteed to surface everything. The open problem is whether there exists any evaluation methodology that can provide a true upper bound on model capabilities, rather than a lower bound that can be exceeded by better elicitation techniques. Third, capability thresholds are unclear: the RSP defines ASL-3 in terms of "substantially increased risk," but translating evaluation results into a binary ASL trigger requires judgment calls that the current methodology does not fully formalize.

DeepMind's frontier safety framework (March 2024) tackles the same problem from a different angle. Their approach emphasizes "early warning evaluations" — tests designed to detect dangerous capabilities before they reach deployment. The Gemini 1.5 technical report (February 2024) includes a section on safety evaluations that covers similar territory to Anthropic's approach — CBRN, cybersecurity, self-proliferation — but with less methodological detail on threat model construction. The distinctive element in DeepMind's approach is the emphasis on forecasting: the framework includes criteria for evaluating not just current model capabilities but the rate of capability improvement, under the logic that a model on a trajectory to cross a dangerous threshold in the next generation should trigger precautions even if it hasn't crossed the threshold yet. This introduces a research problem that neither lab has solved: how do you forecast the capabilities of a model that hasn't been trained yet, given scaling laws that are themselves evolving as architectures change?

OpenAI's published work on evaluation is embedded in the GPT-4 technical report (March 2023) and subsequent system cards rather than in dedicated methodology papers. The GPT-4 report describes red-teaming conducted with external experts, structured around categories of harmful content and misuse. The o1 system card (September 2024) adds a new complexity: the model's chain-of-thought reasoning is deliberately hidden from users and from the evaluation process itself, with a separate "summary" chain-of-thought produced for monitoring. The research frontier this opens is the evaluation of opaque reasoning: if a model is doing internal deliberation that the evaluator cannot see, the evaluator is measuring outputs rather than the reasoning that produced those outputs. The o1 system card acknowledges this tension and notes that the hidden chain-of-thought policy is a deployment decision that trades off safety monitoring against competitive concerns and user experience. The open problem is whether it is possible to evaluate the safety properties of a reasoning model without access to its reasoning — or whether the deployment decision to hide the chain-of-thought is itself a governance choice that reduces the available alignment surface.

The interpretability frontier: can we read the model's mind, and what do we do with what we read?

Anthropic's mechanistic interpretability program is the most ambitious attempt to turn the embedded-constraint philosophy into a technical capability. The program's logic is straightforward but the technical challenges are immense: if you can understand what the model's internal representations mean, you can verify that it has learned the principles you intended and hasn't developed anything dangerous. The 2023 and 2024 papers define the current state of the art and the gaps.

The dictionary learning work (Templeton et al., May 2024, "Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet") is the flagship result. The team used sparse autoencoders to decompose the activations in Claude 3 Sonnet's intermediate layer into interpretable features — individual dimensions that correspond to specific concepts. The paper reports finding features for concepts ranging from "Golden Gate Bridge" to "inner conflict" to "sycophancy" to "code errors." The technical achievement is scaling dictionary learning to a production model; previous work had demonstrated the technique on small models, but this paper showed it works at the scale of a frontier model, with millions of features extracted.

But the paper also defines the frontier by what it doesn't achieve. The features they find cover a subset of the model's representations — the paper estimates they're capturing perhaps 10-20% of the total meaningful variance in the layer they examined, and they examined one layer out of many. The computational cost is enormous. And the interpretability of the features is only partial: the paper's approach to labeling features uses an automated method where the model itself generates examples that activate each feature, and a human or an auxiliary model reads those examples to assign a label. But many features don't correspond to clean human concepts — they're mixtures, or context-dependent, or too abstract to label. The open problem is whether full interpretability — the ability to read a model's internal state well enough to audit it for any dangerous capability — is feasible with current approaches, or whether it requires a fundamentally different technical paradigm.

A second open problem that emerges from the dictionary learning work is the manipulation frontier. If you can identify features, you can amplify or suppress them — the "Golden Gate Bridge" demonstration was a case of feature amplification causing the model to behave oddly. The research implication is that mechanistic interpretability might lead to a form of model editing that is more precise than fine-tuning: you could suppress features associated with deception or sycophancy without affecting other capabilities. But the paper is cautious about this — manipulating features can have unpredictable downstream effects, and the paper doesn't demonstrate reliable behavioral control through feature manipulation. The gap between identifying a feature and controlling the behavior it mediates is substantial and not well-characterized.

DeepMind's interpretability work has followed a parallel but distinct track. The Gemini 1.5 technical report documents the model's ability to handle extremely long contexts (up to 10 million tokens), which itself enables a form of behavioral interpretability: you can present the model with a massive amount of context and observe how it weights different pieces of information. The "needle in a haystack" evaluation — where a specific fact is inserted into a long document and the model is tested on whether it can retrieve it — is a behavioral probe that reveals something about the model's attention patterns without opening the black box. But this is not mechanistic interpretability in the sense Anthropic is pursuing; it's characterization of behavior rather than explanation of internal mechanisms. The frontier for DeepMind is whether behavioral and mechanistic approaches can be synthesized — whether the internal features discovered by dictionary learning can be connected to the behavioral patterns observed in long-context evaluations.

OpenAI's interpretability work has been less public. The sparse autoencoder work on GPT-4 (June 2024) demonstrates that the technique scales to their architecture, finding 16 million features in GPT-4, but the published analysis is thinner than Anthropic's equivalent paper. The o1 reasoning models introduce a new interpretability challenge that no lab has addressed in published work: how do you interpret a model that is doing extended chain-of-thought reasoning? The reasoning process in o1 is a sequence of internal steps that may involve self-correction, exploration of alternatives, and backtracking — the internal state is dynamic in a way that a single-layer feature decomposition doesn't capture. The research frontier is interpretability of reasoning processes rather than static representations, and it is essentially unaddressed in the published literature.

The multi-agent and scaffolding frontier: what happens when models interact?

All three labs have published work touching on multi-agent systems, but none has made it a central pillar of their public research program. The work that exists suggests a frontier that is recognized but not yet systematically pursued.

DeepMind's work on multi-agent systems is the most extensive by volume, but it predates the current LLM era. Their historical work on AlphaStar, AlphaGo Zero, and multi-agent reinforcement learning in game environments is foundational, but it addresses different architectures than the transformer-based models at the center of current capability advances. The question that the current frontier raises is whether the multi-agent insights from reinforcement learning — about cooperation, competition, communication protocols, and emergent strategies — transfer to systems built from LLM-based agents. The "scalable oversight" agenda that DeepMind has published on (Irving et al., 2018; continued in work on debate and amplification) is the bridge: the idea that multiple AI systems can be arranged to check each other's work, debate answers, and provide oversight that scales beyond individual human capacity. But the published work connecting this agenda to current frontier models is thin.

Anthropic's multi-agent work appears primarily in the safety literature rather than as a separate capability research track. The constitutional AI approach is, in a sense, a two-agent system: one model generates responses, and another (or the same model in a different mode) critiques them against a constitution. The "Collective Constitutional AI" paper (September 2024) extends this by having the constitution itself be generated through a process of public input and deliberation, creating a multi-stakeholder system. But this is multi-agent in the service of alignment, not in the service of capability. The frontier question is whether multi-agent architectures — where multiple specialized models coordinate, critique each other, and divide cognitive labor — represent a capability pathway that requires its own alignment research, separate from the alignment of individual models. If an ensemble of models can accomplish things that no individual model can accomplish, does evaluating the individual models for dangerous capabilities provide any assurance about the ensemble?

OpenAI's published work touching on multi-agent systems is indirect. The GPT Store and the Assistants API (2023-2024) are product instantiations of an agent-adjacent vision: specialized GPTs that can be combined and coordinated by users. But the published research on what happens when multiple LLM-based agents interact autonomously is limited. The o1 reasoning models can be seen as internalizing something that might otherwise be a multi-agent process — the chain-of-thought deliberation involves self-critique, exploration of alternatives, and verification, which are functions that could be distributed across multiple agents. The research frontier is whether internal reasoning can substitute for external multi-agent coordination, or whether there are capability gains from explicit multi-agent architectures that reasoning alone cannot replicate.

The automation frontier: what does "agent" actually mean, and when does it matter?

The term "agent" appears across all three labs' publications, but it means different things. The definitional question is not semantic — it determines what capabilities are being evaluated and what safety measures are appropriate.

OpenAI's o1 system card (September 2024) includes a section on "autonomy" that evaluates the model's ability to operate independently. The evaluation structure is instructive: they test whether the model can complete long-horizon tasks (like software engineering tasks that require multiple steps), whether it can adapt to unexpected obstacles, and whether it can pursue goals without human intervention. The finding is that o1 shows "increased agentic capabilities" compared to GPT-4, but remains below the threshold of autonomous operation that would trigger significant concern. The research frontier is the measurement of agency itself: what specific capabilities constitute agency, how do they compose, and at what point does a model transition from being a tool that humans use to being an entity that pursues goals?

Anthropic's approach to the same question is embedded in their responsible scaling policy's ASL definitions. ASL-3 triggers when models show "substantially increased risk of catastrophic misuse or autonomy." Autonomy is defined in terms of the model's ability to carry out long sequences of actions without human oversight, to use tools, and to operate in real-world environments. The Claude 3 model card evaluates autonomy and reports that the models are at ASL-2 — they can use tools and follow multi-step instructions, but they cannot operate autonomously for extended periods. But the evaluation methodology for "autonomous operation" is not as well-specified as the methodology for specific dangerous capabilities, and the paper acknowledges this. The open problem is operationalizing autonomy: what test would an evaluator run to determine whether a model has crossed from tool-AGI to agent-AGI?

DeepMind's published work on autonomy is less centralized. The Gemini 1.5 technical report evaluates the model's "agentic" capabilities primarily through tool-use benchmarks — can the model call APIs, use search, and integrate information from multiple tools to complete tasks. But the connection to safety evaluation is less explicit than in Anthropic's work. The frontier safety framework mentions "autonomous replication and adaptation" as a risk category, but the specific evaluation protocols for this category are not detailed in the public documentation. The research gap is the same across all three labs: agency is recognized as a critical safety-relevant capability, but the science of measuring and thresholding agency is nascent.

The scaffolding literature — work on giving LLMs access to external tools, code execution, web search, and persistent memory — is the practical instantiation of the agency question. All three labs have released products that scaffold their models: OpenAI's Code Interpreter and GPT Actions, Anthropic's Tool Use API, DeepMind's Gemini with Google Search integration. The research question that the published papers don't fully address is whether scaffolding changes the alignment picture. A model that has no dangerous capabilities in isolation might acquire them when given access to tools. A model that cannot plan extended sequences when generating single responses might be able to do so when given a memory and a loop. The published evaluation work largely evaluates models without scaffolding, or with minimal scaffolding. The frontier that the publication records gesture toward but don't yet reach is the evaluation of scaffolded systems: not "is this model safe" but "is this model safe when it can execute code, search the web, and persist state across sessions?"

Across all of these frontiers — evaluation, interpretability, multi-agent systems, agency, and scaffolding — the publication records reveal a common pattern. The labs are publishing capability results that push the boundary of what models can do, and they are publishing safety results that characterize what those models might do that is dangerous. But the integration of these two tracks — the point at which safety findings actively constrain capability deployment — is where the structural philosophies diverge. And the open problems that the publications define are, at their core, the problems that must be solved for any of these philosophies to work: how do you measure a capability you can't yet define? How do you verify a constraint you can't yet read? How do you make a commitment to stop before you know what stopping means?