2026-09-12 Analysis

Safety is the engine, not the brake.

On Dario Amodei's essay "We Must Pace the Frontier" (12 Sep 2026).

Amodei's new essay is candid and largely right about the danger. His first concrete step — independent, embedded evaluators with real access — is real and welcome. But his central prescription, pacing, is the wrong lever. His own logic shows why.

Where we agree

Two of his points are ours as well: AI must stay under democratic control, and democracies must keep the lead. In our image, the white dragon — AI anchored in all of us — has to be stronger than the black one, AI in the hands of autocrats. Why we need to be fast →

Why pacing is the wrong lever

Amodei insists that authoritarian actors won't slow down — only democracies would. If that is true, slowing ourselves only widens the reckless actors' relative lead. You cannot slow what won't slow. The frontier labs are trapped in exactly this vicious circle — the dilemma, in detail →

The deeper problem: pacing treats safety and capability as separable — brake one to buy time for the other. We think they are the same mechanism.

Safety and capability from one source

A superhuman AI has no goals of its own; it amplifies whatever objective it is given. Set that objective by a few, and one bad one is enough. Set it by many — democratically, coupled to people's ongoing feedback — and the same coupling that keeps the system aligned also makes it learn faster. Alignment and capability from one source. Safety becomes the engine, not the brake. Why this makes the AI itself stronger →

Warning is not a plan

Amodei's proposal is almost entirely governance and coordination — no technical mechanism for alignment itself. The missing piece is buildable, and not just in theory: give the AI a conscience it consults — a living model of what people would want, kept honest by their ongoing feedback. It runs on today's models, and deliberative aggregation of this kind already improves group consensus at scale. How the objective function gets its goals →

So don't just pace the frontier — align it. You don't buy safety by braking; you build it. The choice was never fast or safe. Slowing only helps those who won't.

← All news