METR says AI may have doubled Anthropic’s research pace

METR says Anthropic is likely improving model capabilities on well-specified tasks at roughly twice the pace seen during the previous comparison period. The independent evaluator says it would not be surprised if the true acceleration were closer to 1.3 times or 3 times, so the headline estimate should be read as a range rather than a precise measurement.

The group received unusually deep access to internal Anthropic data, including model benchmarks, training-compute figures, researcher interviews and estimates of AI-assisted speedups. Its clearest public metric is Anthropic’s version of the Epoch Capabilities Index, whose trend accelerated by an estimated 1.9 times. METR says faster growth in training compute does not explain the change.

METR’s subjective judgment is that a majority — perhaps about 80% — of the current acceleration comes from AI assisting AI research. Researchers described AI as speeding up work more often than completing the most important research autonomously, though METR saw signs that autonomous contributions could grow.

The report carries a major transparency caveat: Anthropic redacted most of the evidence METR found most informative, citing competitive sensitivity. METR also found no clear near-term bottleneck in algorithms, compute, experiments or data, but repeatedly stresses that its view could change as more evidence becomes available.