As the New York Times reported last night:
The chief executive of Anthropic, the company behind the Claude chatbot, called for a global slowdown of artificial intelligence development in a 3,800-word essay on Saturday, just days after one of the company's employees quit over concerns about the safety of the technology.
Dario Amodei, who co-founded Anthropic to focus on securely and carefully building A.I., wrote that while he believed the technology could bring many benefits, it was advancing at too quick a pace for researchers to continue safely.
The question to ask is not whether this claim is right, but why it was said now. That is because, if this is right, Anthropic must have known this before the essay was written and the accompanying announcement was made, and I cannot believe that a comment from one whistleblower is what has tipped the balance. So, what has?
Of course, I can only speculate, but let me offer three suggestions as to why I think this might have happened now.
Fear
The first, and most obvious, is that those running these AI organisations are genuinely frightened of what they have created. They might really fear the possibility that they have let the genie out of the bottle. I cannot dismiss this possibility, but I do not think that explains the timing by itself.
Finance
The second, which I think is considerably more likely, is that likes of Dario Amodei are actually terrified of something else, and that is the financial risk of failure that they are facing. This might arise for three reasons.
Firstly ,they have asked for vast amounts of funding, and there is no one who knows whether a return can be generated on that money. There is no proof as yet that the AI business model can make a return, let alone on the money being thrown in its direction.
Secondly, the complex financial arrangements being put in place for these purposes might themselves unravel. Many questions have been asked by reputable sources about the complex cross-financial arrangements that have been structured between the AI companies, data centres and existing software enterprises. Whether those will work is unknown. They might be called novel. They might also be unworkable. Some assumptions, including loans securitised on computer chips, look questionable in terms of their value basis, and many have said so.
Thirdly, under this financial heading, it is just possible that Anthropic cannot work out how to usefully spend the money it has got. Do not underestimate the possibility that they just do not know. It has been easy to raise; it may be harder to use. They may want time to think about that. Any excuse will do in that case.
Funk
The third point is something that few would say, which I am happy to do. I suspect that Dario Amodei is in a blind funk, and he may not be alone. In other words, he literally has no idea what to do. He founded Anthropic, but nothing prepared him for what it has become. He is supposedly a guru, but he is also a human being. Put the two together, and at the moment, the human being is showing.
I can say from my own experience that facing tough decisions when leading an organisation is a lonely task, sometimes frightening, and certainly very difficult. That is especially true when you know the buck stops with you, and I've been in that position. I am not pretending that I ran an organisation like Anthropic. I am saying I have some understanding of the loneliness that the responsibility to yourself, to many others, to your organisation, and to those outside who are dependent upon it brings.
It is possible at this moment that Dario Amodei is indicating something that other leaders inside AI may not feel. More likely, however, others amongst those who are supposedly gurus in this area may well be in the same place.
They are out of their depth.
They are untrained for this.
Most especially, they do not know whose advice to rely upon on this when so many will be biased because their own fortunes will be at stake, and they do not want their leadership to supposedly prejudice their personal situations, when that possibility clearly exists. Objectivity, therefore, might well have departed the scene, and that will make the loneliness of chief executives even harder to bear.
I am not saying that these people are not well rewarded for what they do. They are, totally disproportionately. But it is important to remember that they remain human beings despite all of that, and fear to the point of terror can afflict them. I think this may be what we are seeing.
Does any of this make things feel better? No, not at all, is my honest answer, bar one thing, and that is that when you feel like a headless chicken, taking time to think usually helps. The call for a halt does, then, make sense. The reason given for it might, potentially, be spurious, but heeding the request may still be appropriate.
But one word of warning. Absurd spending on AI has maintained the impression of a functioning economy in the USA, and maybe beyond. Slow that spending, and the risk of the downturn I have been talking about grows considerably. That is inevitable. This cloud may not have a silver lining.
Thanks for reading this post.
You can share this post on social media of your choice by clicking these icons:
There are links to this blog's glossary in the above post that explain technical terms used in it. Follow them for more explanations.
You can subscribe to this blog's daily email here.
And if you would like to support this blog you can, here:
Buy me a coffee!
There’s an idea interesting documentary about how AI can go badly wrong called Terminator 2: Judgement Day. Worth a look.
🙂
Now that is funny, for sure.
Took my 15 y old to see it in the cinema last week. It’s back in theatres for 35th anniversary. Made a huge impression on me when I was a teen and it came out
They’re getting extremely desperate for money. The only way to grow and improve upon their technology involves mining obscene amounts of rare earth minerals, and just running their models takes nearly petabytes of VRAM. These “godlike creatures” they’re trying to scare everyone about have their own physical constraints that make them not a danger to humanity at all.
Your post makes a key point about human development and the State’s role in it as well as the market’s role. ‘Market-led’ is not always best – it should be a close partnership. It was state funding that started this whole internet thing off anyway but you would know that because of the huge self-aggrandizing going on in silicon valley.
I think that one of the enabling factors of AI is simply cost cutting – less people = less wage bills, pension contributions and other costs. Penny pinching and greed.
It looks to me as though AI developers are close to or arrived at that point of creating something that only they really understand. What they have created is something as Jaron Lanier has noted is rather sneaky, something that we have also poured huge sums of human pedagogy into and whose raison’d’etre is based on EXTRACTING stuff from people. To us it looks helpful, but what are its parameters of ‘efficiency’ – or rather whose parameters have they been kitted out with? What biases has it been given? Whose biases?
I think that now is the time that States everywhere need to step in and see what has actually been going on. Now is the time for cooperation. Power like this needs to be shared and managed. This is a key moment as even States seem unprepared for the consequences.
Two things I would note. Token prices for AI usage are becoming so expensive that companies are beginning to hire back people they’ve released as it’s cheaper for them to pay for people instead of AI function.
The other is that, in a world where CEO’s actually managed to cut their staff numbers significantly and the dream of low employees they have is realised, who will have money to but their products?
Unless there’s a sudden impulse towards UBI, very few humans will be left who have the funds to purchase anything other than the barest of necessities. The billionaire class don’t spend their money on the things that keep most businesses profitable, so I’m curious what they think they’ll do when they have no paying customers left
They really do not understand macro.
I think the documentary your looking for is Colossus The Forbin Project!
To further explore the financial argument, the biggest danger to the big seven companies is that LLMs become a commodity, with competition from hundreds or thousands of models. All the trillions of dollars of investment is on the assumption that everybody needs AI and we have to get it from them.
If we can all run local AI models for free or at low cost then there is no way they will be able to make the returns they need. These companies are calling for regulation to try and stop new competitors emerging.
Yes Richard, this panic is driven largely by the three F’s. Not the Fear that Anthropic’s Mythos and similar agents are going to end civilisation as we know it but that this industry is never ever going to be a Financially viable business which is driving the CEOs into a deep Funk. Many of the hyperscale data centres will never be built, the majority of NVIDIA GPUs with an effective lifespan of three years are sitting unused, token prices are being hiked, prepayment is now required for API usage and now, today, we have Altlman delaying OpenAI’s IPO and other CEOs calling for development to be slowed down. The ultimate aim here is to get the US government to bail them out as with the banks in 2008. Ed Zitron has been calling out this industry as unsustainable for years: he is correct. I can feel a re-reading of Jock Young’s excellent 1972 “Folk Devils and Moral Panics” coming on.
The only way I can see this technology not having a future of astronomical size is if it starts to be rejected by the public for some reason.
It seems obvious to me that, if embraced, the intention will be to embed AI into pretty much everything technological thereby making it the entire human/tech interface used globally by everyone.
Military applications are always the front runner so I would expect to see blanket coverage in this area including within robotic battlefield units.
As it filters more and more into the public domain we should also expect it to see it coming into robotic home assistants.
Of course the argument for attempting to embed this technology into the brain will always be chomping at the bit in the background and the public will eventually be all for having it imposed on dangerous prisoners to prevent them from harming anyone else. Others will come later.
This AI flood is not so much a new technology but rather a transformation of the entire way in which human civilisations think and behave with accelerated progress into full transhumanism.
Should we be afraid of it. I don’t know but we should certainly be in no doubt about the path it has opened and must make a decision on whether this is the future we want to see realised.
I am not sure the public will get a say in this.
The public will rarely use the versions that could go out of control – the AI agents.
I know the principle. I have learned how they are written. Do I need them? Not that IU can see, and I am tech heavy and a geek.
I read quite a lot of your posts Richard, and although I’m not always in full agreement I recognise that you know a lot more than I do. Sometimes you come across as being against almost everything. But this post reminds me why I keep reading you: walking your own talk and showing empathy with people as people, even if they are engaged in things that I (and you, I suppose) think are making the world worse.
I am profoundly for people and for saving our planet.
A lot of people do not seem to be so.
There are plenty of warnings in science fiction. I mention some of them, and this article, here: https://suenethercott.substack.com/p/the-end-of-the-world-as-we-know-it
I recently came across the Ed Zitron AI critiques in the Techreport podcasts. He doesn’t hold back.
He says the AI “boom” is a total hype fuelled by a circular market benefitting no one but Nvidia.
The business models are unsustainable and will probably crash in 2027 before OpenAI and Anthropic can go public.
On top of that, the AI coding is very “buggy “ and getting more expensive as they try to move to a token system.
I have no idea of Ed’s political stance but his sharp and sometimes funny critiques of AI and the “tech genius brigade “ are worth a watch.
I’ve also seen the Ed Zitron stuff, and it makes me think that the suggestion of a slow down might be a way to get public money to bail them out – “Hey, we’ve slowed down development to save humanity, now by being moral we’re going bust, so we need a big chunk of public money to compensate us for our virtue !”
You, maybe wholly appropriate, cynic.
I think the real fear these people have is fear of a coming Butlerian Jihad. If you’re up on your Dune lore, you’ll remember that not only did the Butlerian Jihad destroy all the thinking machines, but also all the people who created and used them to control others.
I had my telephone updated by Virgin to digital ( no choice)..but it would not allow anyone to phone me. Contacted Virgin..engineer coming..ok…so in the meantime.. just out of interest I gave the details of the problem to an AI bot. I was given a very informative list of answers to the problem. Wrote it down as bot suggested and gave it to the engineer.Engineer looked at me as tho I was nuts. Do you understand the problem now… I said full of my own brilliance… Nope this is all garbage says engineer…your phone is just too old fashioned missus to use digital service..need a new one…problem solved.So much for AI bots.