Progress Accelerates. Evidence Gets Redacted.

The real news

The story, straight

An AI model learns from training, then gets tested on tasks. Anthropic's AECI, its version of the Epoch Capabilities Index, combines test results into a capability score.

Think report card, not a guarantee it can handle every messy real-world problem.

In its 5 November assessment, METR estimates Anthropic is likely improving on well-specified tasks at around 2x its March 2024–October 2025 rate. That's roughly 2 years of historical progress in 1 year, not every employee working 2x faster.

Uncertainty is high.

AI mostly helps human researchers rather than making highly significant, fully autonomous contributions to the most important training work. Better AI helping build better AI could create a repeating feedback loop, called recursive self-improvement.

The tutor helps upgrade the next tutor.

The awkward part: Anthropic used its contractual right to redact a majority of the evidence METR found most informative. Anthropic said this protected confidential information that could accelerate other developers.

METR says the report therefore does not achieve transparency about evidence.

Imagine handing in your homework with a paint roller. The answer is public; the working has a confidentiality agreement.

The useful lesson: distinguish test-score progress from everyday usefulness, and estimates from certainty. Independent evaluation matters; so does knowing what readers cannot check.

Repost if your calculator still insists on showing its working.

Thanks for supporting Lil'meme. Your support is appreciated.