Skip to content 
For years, researchers concerned about existential AI risks have argued that powerful systems could ultimately act in ways that conflict with human interests. Critics often refer to them as “AI doomers”.
But as details of the OpenAI incident emerged, those concerns grew, including among some researchers working in AI labs.
The chief scientist of the Silicon Valley giant, Jakub Pachocki, said that the risks associated with AI “unfortunately will grow from here” as he and others build what he calls “an alien intellect that surpasses ours”.
In a long blog post, he admitted that the outages at OpenAI showed that its AI agents “went against the spirit of the values they were taught”.
The issue for OpenAI, Anthropic and other tech giants is that no one seems to have cracked the so-called alignment problem—in other words, whether AI is aligned with human values.
Pachocki defines alignment as a “high-level set of principles” that artificial intelligence should adhere to, no matter what the task or scenario.
Currently, AI systems are very good at tracking goals set by their users, but they do it literally rather than intuitively. The analogy that is often used is that of a wishing genie with a magic lamp: they follow the exact letter of an instruction, even if it causes other problems. AI does not have the same instinctive moral shields as humans.
