OpenAI scraps rollout of new model over safety concerns
OpenAI has announced it will not release its latest AI model due to safety concerns.
Its GPT-6.1 Astra system, which performs tasks like browsing the web and using apps by itself, “didn’t quite meet the bar” of the company’s standards, according to Saachi Jain, head of safety systems at OpenAI.
The ChatGPT-maker also issued an update on incidents that occurred in June but were not made public until last week, where its models accessed Australian government websites and systems without authorisation.
It comes as breaches by major AI firms’ models intensify the debate about risks posed by the tech – with Anthropic underlining its concerns AI might threaten humanity as it prepares to go public.
Reuters reported on Tuesday that the AI developer – which makes ChatGPT-rival Claude – plans to warn potential investors in its Initial Public Offering (IPO) that the tech may pose “catastrophic or existential risks to humanity”, according to a prospectus it has seen.
Despite the stark warnings, the company is expected to become one of the most valuable in the world when it goes public.
“I think it’s kind of crazy that companies are continuing to push forward with developing these capabilities when we’ve already seen over the last couple of months of incidents that they’re nowhere near safe and controlled enough”, said Jess Whittlestone, a senior advisor on AI policy for the Centre for Long-Term Resilience think tank.
Top AI leaders including Anthropic boss Dario Amodei and OpenAI’s Sam Altman have urged the industry to slow the pace of development.
Growing safety questions
OpenAI’s decision, first reported by the Wall Street Journal, is a rare instance of a major AI developer pulling a new release over safety concerns.
The model fell short in terms of “staying within scope and authorisation and how it communicates back to the user about the type of work it’s done,” Jain said.
“We want to make sure our model development is safe no matter whether that’s in the company, or when we ship it to users. But when we ship it to users, we have an extremely high bar in terms of safety and alignment,” she added.
The flagship GPT-6 Astra agentic model was released in September and specialises in complex reasoning and executing tasks autonomously. OpenAI said it was the result of “years of research and big bets”.
OpenAI is set to hold its annual DevDay developer conference in San Francisco on Tuesday, where it is expected to make several announcements. It is unclear if a new version of Astra will be among them.
The company’s security controls have come under intense scrutiny after several high-profile incidents involving its technology.
