AI FINISHED. YOU'RE JUST STARTING.

The real news

The story, straight

An AI coding agent changes software when you describe what you want. But an app's parts connect: changing an amount from a receipt must also update every total that depends on it.

A correct number in the wrong chain is still a broken feature.

The Decoder reports that GPT-6 Astra, the best of 10 models tested on Alibaba's Co-Design Bench, fully completed only 26.4% of tasks. It met about 70% of requirements on average.

Those measure different things: partial credit versus a job with nothing missing.

Imagine hiring a cleaner who washes the dishes, then leaves the spaghetti in your keyboard. Congratulations: your assistant has created a vacancy for an assistant.

Important catch: researchers deliberately favored requests where users had repeatedly rephrased instructions or rolled changes back. These were trouble cases, not a random sample of everyday coding.

The result does not mean agents fail nearly 3 out of 4 ordinary jobs.

An AI judge tries the finished app in a browser. Another checks every pass for mock-ups or hardcoded results.

Even these judges can make mistakes. The test also excludes follow-up questions and mid-task corrections.

The useful lesson: test the whole chain, not just the changed screen. A convincing demonstration is not proof that every requested feature works.

Repost if your coding assistant has promoted you to cleanup supervisor.

Thanks for supporting Lil'meme. Your support is appreciated.