OpenAI’s chief scientist wants the leading AI labs to agree on how far they’ll go with their next models, and what would justify going further. At 9:03 a.m. Pacific on Sunday, Jakub Pachocki put some of OpenAI’s own training results on the table. Nine minutes later, Sam Altman called Pachocki’s essay “the most important thing OpenAI has published in awhile.”

In “Perplexity and Intelligence,” Pachocki asks labs to settle on a threshold they can all measure. Before crossing it, they’d have to show they can understand and monitor what a more capable model does. The threshold itself is still up for discussion.

The test OpenAI runs on its own code

Since GPT-3.5, OpenAI has given successive models the same sample from its internal codebase and measured how well each predicts the next token. The measure is called perplexity. Lower scores mean better predictions. Publishing those scores gives competitors a look at how much OpenAI gained from one generation to the next on a test its researchers actually use.

Pachocki describes what happened when his colleagues tried a stronger model on open math problems. Solutions to questions some of them had worked on for years started turning up in a single Slack thread. He thinks something similar could happen as early as 2027 if models start finding better ways to train the next generation. He expects AI to help with safety research, too. His concern is how much faster the training work could go.

What Anthropic has offered to show outsiders

Altman made his appeal two days after Anthropic closed the biggest IPO in history. Pachocki cites a September essay by Anthropic CEO Dario Amodei, who had already called for shared limits. Amodei committed to giving outside evaluators access to Anthropic’s staff and systems. Under his proposal, those evaluators could publish an unfavorable finding even if Anthropic objected to their conclusion.

For either CEO, a pause is easier to explain to investors if the competition has to pause, too. Agreeing on the test is harder. OpenAI can compare years of results because it uses the same sample of code. Anthropic could test a model on its own data, get a different score and have little to show for the comparison. Pachocki acknowledges that problem and proposes using other measures alongside perplexity.

Even a test both companies trust would settle only half the bargain. It could tell OpenAI when it had reached the limit. If OpenAI’s researchers and an outside reviewer then disagreed about whether its safeguards were good enough to resume training, the essay leaves that decision open.