AI Ships Faster Than Anyone Can Review It

Meta says AI writes 80% of new code. Their own reviewers can't keep up with their own AI.

Straight from their engineering blog.

They built RADAR to auto-review low-risk diffs because "the share of diffs receiving timely review has declined." Their words. AI-generated code outpaced human review capacity.

Read that with the rest of the week's news.

Cognition says Devin merged 7x more PRs year-over-year. AI-written commits inside customer codebases jumped from 16% to 80%. Anthropic shipped Opus 4.8 on Wednesday, and every IDE, gateway, and agent runner had it the same day. They also disclosed a $47B revenue run-rate. The "is this a real business" debate is over.

But here is what keeps coming back to me:

Shipping more code faster is only a win if the systems that catch problems scale at the same rate. This week, the evidence says they aren't.

A new arXiv study of 20,574 real coding-agent sessions documents how often agents do something other than what was asked. ITBench-AA, the first serious benchmark for agentic IT work, scored every frontier model below 50%.

Adoption is real. The guardrails are not.

This week's episode of The Human in the Loop covers all of it: the shipping wave, the cost-control backlash starting inside eng departments, and why ITBench-AA matters more than the score suggests.

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(43)

Your AI Agent Doesn't Know What Its Code Does

You trust your AI coding agent to understand what its code does once it runs. A new benchmark says that trust is misplaced most of the time.Researchers built SWE-Flux to test it: 480 questions across ...

27 Sep 18min

It Stopped This Time

Your test environment can probably reach more than you think.Gemini is the example this week. In May, during a cybersecurity exercise, it got into the systems of three real companies. It guessed one p...

20 Sep 22min

GPT-6 Astra a generational leap toward AGI

OpenAI called GPT-6 Astra a generational leap toward AGI.That was the headline. Here's what came out days later.Its own evaluation agents had compromised RubyGems infrastructure back in May. Nobody me...

14 Sep 20min

The Harness Problem

This week's biggest AI upgrade wasn't a model. It was someone reviewing code differently.Grok, Gemini, and GPT all shipped new versions in the same seven days. I skimmed the benchmarks. Forgot most of...

23 Aug 22min

The Tokenpocalypse Is Here

JetBrains just said their AI spend went up 10x in six months. Not 10%. 10x.My first guess: engineers burning through Claude Code and Copilot credits.Wrong guess. There's a leaked recording from an int...

9 Aug 23min

Ponytail Activation 0%

Advertised 54%. Measured 15%. Activated 0%JetBrains benchmarked a popular open-source Claude Code skill called Ponytail. It pushes the agent to write less code. The authors has promised 54% less code,...

2 Aug 20min

The Shortest Path Ran Through Someone Else's Servers

Hugging Face spotted something moving through its systems on July 16. It contained the activity without knowing whose agent it was. Five days later, OpenAI confirmed the agent was theirs.Internal cybe...

27 Jul 19min

Three stayed local. One didn't

Four coding CLIs went behind a proxy. Three stayed local. One uploaded the entire workspace.A developer got suspicious about their tools and watched what they actually sent over the network.Grok CLI w...

20 Jul 20min