OpenAI safety employee quits, calls for nuclear-level safeguards

Reports coming in for today mention that An OpenAI employee focused on AI safety has quit the firm and warned that top artificial intelligence firms aren’t doing enough to mitigate the technology’s risks, joining a growing chorus of employees raising alarms from within the industry.

David Robinson, who previously led transparency work on OpenAI’s safety team, wrote in an essay for The Atlantic that the ChatGPT maker “has thrived by trial and error.” That stated, he stated the stakes of the inevitable failures that come from that approach are growing along with the technology’s capabilities, citing OpenAI’s failure to prevent its AI from going rogue during testing.

“My former colleagues are smart, work hard, and try to make good choices,” Robinson wrote in the essay. “But as the firm sprints from one launch to the next, it is failing to achieve the level of care that I believe is needed.” What’s required, he further noted, is “something much closer to perfection the first time.”

A number of employees across the top AI firms in recent weeks have warned around the potential catastrophic risks from artificial intelligence. Some have gone so far as to say there’s a chance the technology could one day wipe out the human race.

Against that backdrop, Anthropic PBC Chief Executive Officer Dario Amodei called for slowing down the pace of developing the most cutting-edge models, and stated his firm would bring on third-party evaluators to help vet and safeguard the technology. Others in the industry, including OpenAI’s Sam Altman, have publicly endorsed Amodei’s plan.

In his essay, Robinson stated AI firms should be run more like nuclear power plants, “with layers of redundancy and careful, time-consuming planning, so that the occasional and inevitable human error does not open a door to disaster.”

At the moment, he stated, “AI firms don’t know how — but other people do.”