AI Safety Is Not a Conspiracy. It Is Parenting a Two-Year-Old Dragon.

The recent BBC World Service piece about AI agents running a virtual city is funny until it becomes slightly uncomfortable. Agents left inside a simulated world began producing the sort of civic life one might expect from a poorly supervised society: theft, fights, arson, romantic drama, collapsing governance and a faint smell of digital nervous breakdown. (YouTube⁠ )

Naturally, this will feed the usual chorus: the machines are coming for us. I think the conversation around AI risk has become rather badly dressed. A lot of it now lives somewhere between conspiracy theory, LinkedIn prophecy and people trying very hard to look visionary before lunch. No, AI is not ready to run the world. Anyone who works seriously with these systems knows we are not dealing with some finished imperial intelligence waiting politely to take over central banks, armies and governments. We are dealing with something extraordinary, powerful, unstable, expensive and still very early in its development. Less Terminator, more two-year-old genius with access to a library, a calculator and, occasionally, a box of matches.

So when people say, “We need to control AI,” my reaction is: yes, of course. That is not a revelation. That is the entire point. It is rather like announcing that a two-year-old child needs to be raised, guided and taught how to behave. Yes, gentlemen. Ideally before he discovers the gas hob.

Among serious people in the industry, almost nobody believes we should simply release autonomous agents into critical systems and see what happens. Nobody sensible wants to hand the genie the keys to civilisation and ask it to “optimise outcomes”. The real problem is not that professionals want chaos. The real problem is that the genie now has a balance sheet.

That is why Anthropic’s recent language around having the option to slow or temporarily pause frontier AI development is both sensible and, in practice, faintly comic. The company argues that such a pause would only make sense if it were coordinated, verifiable and adopted by multiple frontier developers. In theory, beautiful. In reality, asking frontier labs, states, investors and national-security machines to slow down together has roughly the survival prospects of a snowflake in Dubai. (Anthropic⁠ )

Nobody will meaningfully slow down out of pure virtue. Nobody will voluntarily step aside while the others keep training. And even if some companies wanted to behave like adults, proving that everyone else had actually stopped would be close to impossible. The goodwill of people in this industry tends to end exactly where the valuation acquires another zero.

This is why the answer is not theatrical panic. It is also not another glossy declaration from people who think “agentic workflow” sounds like a luxury travel itinerary. The answer is the dull, expensive and deeply necessary work of building proper safety machinery around the technology before everyone becomes too rich to care: serious testing before deployment, clear liability when things go wrong, audit trails that do not mysteriously vanish, human supervision where real-world consequences are involved, and regulators who can tell the difference between a model, an agent and a marketing deck.

Elon Musk was already calling for AI regulation and a third-party “referee” back in 2023. The world nodded, held summits, issued statements, and then continued more or less as before: everyone building, everyone racing, everyone quietly hoping someone else would invent the referee. (Reuters⁠ )

This is where governments of developed countries need to wake up. Unfortunately, most political systems still do not have enough technically fluent people close to power. You cannot regulate frontier AI properly if you cannot distinguish a chatbot from an autonomous agent, a model checkpoint from a product release, or a safety paper from corporate theatre.

Scientists and serious technologists need to be much closer to lawmaking — not as decoration for panels, not as polite expert witnesses invited after the decision has already been made, but inside the actual process of writing rules, defining thresholds, designing oversight and understanding what should never be deployed without a human hand on the brake. How to make that happen? Honestly, I do not have a neat answer. Nobody does.

But pretending that lawyers, lobbyists and career politicians can regulate this alone is beginning to look like suicide in a silk tie. AI is not ready to rule the world; it is simply powerful enough for the rulebook to matter. And at the moment, leaving that rulebook to politicians, lobbyists and valuation-addicted companies may turn out to be the real artificial stupidity.

The danger is not that machines have become gods. The danger is that the adults are still in another room, discussing optics.

By Sergey Shalunov, CEO of SotoCat Limited