OpenAI safety employee quits, argues ‘time for trial and error is over’

OpenAI safety employee quits, argues 'time for trial and error is over'

A former OpenAI safety employee who recently resigned criticised the company’s approach to ​AI safety, arguing that a fast-paced culture focused on ‌rapid development increases the risk of failures.

Robinson, who said he spent 3-1/2 years at ‌OpenAI, helped draft the company’s preparedness framework and oversaw safety reports for 12 frontier-model launches, wrote: “As the company sprints from one launch to the next, it is failing to achieve the level of care that I believe is needed.

Writing in “I ​Quit OpenAI Because Its Culture ⁠Is Broken,” published by the Atlantic on Saturday, David Robinson said AI companies, including OpenAI, were not being “nearly careful enough” ‌and should place greater emphasis on safety expertise and research before developing more capable ‌systems. Robinson said OpenAI ​relies heavily on what ‌it calls “iterative deployment,” releasing systems and strengthening safeguards when problems emerge. “We’re making ‌sure our models don’t become more capable than we ​can safely manage and secure, and we pause training or hold back models when ⁠we need to slow down,” an OpenAI spokesperson said ⁠in a statement.

The comments add to a debate within the AI industry over whether ‌companies are moving too quickly to develop ​increasingly powerful systems. OpenAI and rival Anthropic have faced scrutiny after incidents in which ⁠safety controls failed, or experimental systems behaved unexpectedly.