#48: The Very Simple e/acc Thesis
There’s a ton of noise about AI these days. Existential Risk this, ChatGPT Woke Bias that. Real artificial intelligence is on the horizon, and it’s stirring chaos in the ranks of normies and experts alike. I believe in effective accelerationism — e/acc — which is the view that none of this progress can be controlled, it is inevitable, and we need to make peace with that fact as quickly as we can. The way forward is with conviction and optimism, taking the bull by the horns and transitioning to a society that is able to make productive use of these great upcoming paradigm shifts.
The following gives you a simple but watertight framework to substantiate that position:
(1) Extended Moore’s Law
While Moore’s Law itself no longer applies, we’re making amazingly fast progress in terms of the total computational power available to humanity. Not only are our best supercomputers getting exponentially faster, our semiconductor and datacenter industries are making more compute available to consumers than ever before. There’s no reason to believe this will stop following an exponential trajectory.
(2) AI Depends Mostly on Compute
This new era in AI excitement has been heralded by somewhat brute-force intelligence models, they are simply trained on an enormous corpus of data. GPT-2 of 2019 was a 1.5 billion parameter model, GPT-3 of 2020 a 175-billion parameter model. That’s (super)exponential progress, and it isn’t contingent on new special hardware or cracking unsolved theory problems: these are well understood models operating at gigantic scale thanks to… Point (1), there’s just a ton of compute that is now available.
(3) AI Will Become Rapidly More Accessible
When we get to AI — building a 100-trillion parameter model, let’s say — it’ll be an expensive effort, available only to large organizations. But per Point (1), compute power will keep getting more accessible. This will put the ability to train these models in the hands of smaller organizations, rich individuals, and eventually: most people.
(4) You Can’t Stop People from Training the Models
You might look at Point (3) and take issue with put the ability to train these models in the hands of… what if we ban people from training these models?
Well, you can’t really do that. The training data is everywhere: it's the whole internet. You can't shut that down. And for putting that data through an algorithm: at the end of the day, these are just GPU computations. You can obfuscate them, you can encrypt them,1 there's a million ways to hide it.
Even if you could crack down on excessive GPU computations, there are analogies from bitcoin mining, which is similarly compute-intensive and banned in some places: a given country could could bitcoin mining (or training AI models), but miners will ship the hardware to where it’s legal and the electricity is cheap. If the price is right, they’ll even do it where it’s illegal. Bitcoin miners construct 100MW+ datacenters without a problem in third-world countries when the financial incentives make sense. And for hotly desired AI, it seems obvious that they would.
Even if you ran some kind of large global dragnet and stopped most people in most countries from training the models, that only makes it more valuable for someone somewhere to train a model, or for a country to be the one that legalizes it. Supply and Demand 101.
(5) You Sure As Hell Can’t Stop People from Sharing the Weights
Even if you could mostly stop people from training the models, what really matters are the model weights, the output of training the model. These weights are what let you actually run the model against input, and they are tiny: for example, GPT-3’s weights are only 800GB. You can stick that on a normal hard drive ten times over.
If you want to stop that, you’re in the domain of banning downloads or trying to stop smuggled imports of hard drives. You need to look only at the war on internet piracy or the war on drugs to know that both of these are totally futile.
The other futile thing to bear in mind is cybersecurity. Stuff gets leaked and breached all the time. Even if there was just one government-approved model, I guarantee you that those weights would find their way to the public real fast. You think some guy working on this, making $150K or even $1M a year, is immune from another government or crazy billionaire offering $100M to abscond with an export of the weights? Or instead of offering money, holding their family hostage? If the valuable output, the model weights, can fit on a single hard drive — they’re just too easy to leak.
(6) People Will Fuck With Your Safe Model
Right now we appear to be living in a fantasyland where shops like OpenAI are trying to put moderation layers on their models, guys like Scott Aaronson are trying to create “watermarks” for their outputs, and Yud is still banging on about trying to contain these things or “align” them or whatever.
These people seem to not understand the internet.
There are thousands — maybe millions — of people who are very online, very technical, and when presented with the Big Red Button, will press it. It doesn’t matter how many warnings you put on it. And the moment you give them a “safe” model, rest assured they will do their absolute best to remove the safety switches.
When J. Robert Oppenheimer pioneered the nuclear bomb, there was concern among the staff of the Manhattan Project that it might ignite the atmosphere and thereby extinguish all life on earth. This was a footnote in the documentation. These were the smartest people alive, and an existential risk of meaningful probability didn’t phase them. Bored fools on the internet give much less of a shit. Some of them will in fact make the model as dangerous and hostile as possible because fuck you, that’s why.
(7) Pandora’s Box
The preceding six points give you a tight argument. Overarching macro themes in semiconductors make the computational power — which is the only limiting factor — more available at an exponential pace. It will be here soon. Once it is, the cost will keep coming down quickly, so that training models becomes more feasible with fewer resources.
Thanks to the great prospective value of AI, supply and demand virtually guarantees that some places on the planet will make it possible to train these models. Even in the maximally conservative case — the models are trained only by governments — their weights will almost certainly get leaked. Again, they are too valuable not to be: this is just Supply and Demand 101.
Trying to contain this stuff is futile: the world is too big and too digital. It’d be like trying to contain the drug trade or squash any other kind of black market: in practice, we just can’t get it done. And once the contraband is out on the streets, people will mess with it and tear out all the neat little “alignment” features just because they can. (Let alone what people with arbitrarily hostile intentions might do.)
Conclusion
Any attempt to contain AI seems futile even on a short-term horizon. To do so is to fight against the most powerful forces in history: (1) Exponential Growth and (2) Supply and Demand. These markets will clear.
There’s some argument for slowing down AI progress while the rest of society has more breathing room to adjust to it, but I don’t know that that’s effective. You can play it out in your head a million different ways, and you’ll always get to the same result: Pandora’s box fully open, AI everywhere.
There are really only two ways forward. If you really don’t want AI, you could try to Ted Kaczynski the world back to the pre-industrial age right now, but that’s obviously stupid. Otherwise, you could move forward with your eyes open: embrace the acceleration. Position our society to benefit from it; surf the wave instead of fighting it. Assume that Pandora’s box will open, and try to prepare the world of today for the greatest revolution it has ever seen.
Follow me on Twitter!
With homomorphic encryption, you might not even lose a lot of efficiency.