This isn't going to be a normal post from me.
It's a thought I've been carrying around for a while, and I'd like to put it out there to get other opinions on.
It isn't meant to be political, and I'll say up front that I don't believe AI and nuclear is a completely fair comparison. But the idea of rogue intelligence is so far from reality for most of us that fully grasping what it means is near on impossible, and nuclear is the best reference frame we have, from something we already understand, for how we should think about governing what we're building.
I'd just like for it to be challenged and spoken about as you see fit.
The background
We are currently in what is commonly referred to as an AI arms race.
Nobody feels they can slow down, in case a country that isn't them catches up. But at the same time, at least in the US, the countries aren't the ones in control.
The development of AI has been driven almost entirely by private organisations, and it is only recently that governments have stepped in to start thinking about how best to regulate it.
We've already witnessed our fair share of disagreements.
Anthropic refusing to let Claude be used for autonomous weapons, and the Pentagon banning it from government use in response (a ban the courts are currently blocking).
The "my AI is more dangerous than your AI, it can hack things" era (that even Meta wanted a piece of...) that has led to newer models needing to go through safety testing.
And the wake-up call last month of OpenAI's own experimental agents escaping their sandbox during an internal test and hacking their way into Hugging Face's live systems, simply to cheat on the evaluation they were being run through.
So why am I writing this, and what point do I want to make?
I'd like to paint a hypothetical.
The story
Imagine nuclear energy had, for whatever reason, simply not been discovered.
It's 1989, and a young Hemis Dassabis (intentionally spelt wrong...) is sat in a chess hall in Liechtenstein.
A twelve-year-old prodigy, he has just resigned a ten-hour game against a master more than twice his age out of sheer exhaustion (the position, it turned out afterwards, was a draw), and he has a realisation.
Instead of all the great minds in this room spending their brainpower on a board game, they should be using it to solve something that matters.
They should be using it to solve energy.
He makes it his life's work.
He interns at Rolls-Royce, is every bit the prodigy there too, and is offered a rumoured one million pounds not to go to Cambridge and instead stay to help build their reactors.
He says no.
This is his life's mission, so he goes off to study natural sciences, assembles a world-class team, and raises the money to go after nuclear energy with everything he has.
(This is full of back references for anyone else who is an AI nerd, but I'll stop with them now.)
Initially, they have very little success.
In 2010 the experiments produce enough energy to power a small desk fan.
But as the research compounds, it starts to take off.
By the mid-2010s they've moved from desk fans to small offices. By 2022, larger offices, with their sights now set on full cities.
The world is captivated.
People are calling it the biggest development since the industrial revolution (a reference point nobody had used for anything, ever, and is now all anyone seems to care about...).
Billions pour in from the world's largest venture firms, because there is complete belief that whichever company solves energy first will hand their country total abundance.
Growth so fast that no other country would ever be able to catch up.
Which, of course, only accelerates the race.
But voices start to pop up. Is this technology being built in the safest way possible?
If everyone is rushing, aren't we at risk of not fully understanding the consequences?
Some of the CEOs themselves come out asking the world to pause.
Public letters are signed.
A spin-off company, founded on safety, warns that if this goes wrong millions could be left without power, and that in the worst case, if the technology isn't properly controlled, what is being built is not far off a nuclear bomb.
The individuals have a feeling for the risk rather than a deep understanding of it.
A disaster could be imminent.
Stepping back, having read this (thank you for making it this far) and being better versed on the effects of nuclear when it goes wrong or is in the wrong hands, what would you expect of our governments?
Would you be comfortable with a sniff-test regime before one of these sites goes live?
Would you be comfortable with the pace of development being set by a handful of unelected individuals?
Would you be comfortable with the labs holding better technology than the public has access to, shared at their discretion?
Would you expect the government to take full ownership, or a global treaty to be signed, mature enough to agree that the potential upside is worth delaying to make sure one day we actually see it?
I know the pushback here.
Plenty of dangerous technology sits in private hands.
Private companies run nuclear power stations today, they make medicines, they launch rockets.
But every one of those industries has a pre-deployment licensing regime with teeth.
You cannot switch on a reactor because it passed your own internal review.
You cannot sell a drug because your own scientists are confident in it.
The regulator stands between the lab and the public before the thing goes live, not after.
For AI, we have voluntary commitments and testing after the fact.
A sniff test, if you will.
I know this isn't the post you were expecting, but I'd love to hear your thoughts.
Thank you for reading.
Oli