By Katrina Manson and Emily Chang
Anthropic PBC’s boss said he doesn’t know what role his artificial intelligence model played in a missile strike that killed an estimated 120 children at an elementary school in Iran, reflecting a broader knowledge gap for AI executives who are increasingly selling advanced AI tools to the US military.
But he said the use case in this instance didn’t violate the company’s policies, arguing military decision makers make terrible mistakes even at the best of times.
“Look, we don’t have access to, we don’t know exactly how these models were used,” Dario Amodei, Anthropic’s CEO and cofounder told Bloomberg’s The Circuit with Emily Chang in an interview, when asked whether his company’s AI tool Claude played a role in the February 28 strike — the first day of US operations in Iran — on the school in Minab. He described the strike against the school as “a really terrible thing to happen.”
“The principle that we have established, and I think the principle that was obeyed here, is a human makes the final decision,” Amodei said. “I don’t know what role Claude or any other AI had, but if this isn’t an illustration why that principle is so important, I don’t know what is.”
The Pentagon, which hasn’t publicly claimed responsibility for the strike, is investigating the incident.
A long-running debate about the role of AI at war pits those who hope AI will reduce mistakes in war, save lives and deliver victories against campaigners and experts who worry that speeding through targets will make war worse and lead to greater civilian harm. There is also debate about the extent to which technology companies should be privy to how their AI tools are used and whether they should be held responsible for outputs that result in mistakes.
Hamza Chaudhry, at the Future of Life Institute, a group that emphasizes the risks of military AI, warns that AI targeting processes could ultimately speed up so fast that nominal human decision-making amounts to little more than a “rubber stamp.” As a result, he said, the expanded scale of combat could result in the taking of many more lives.
Amodei’s comments bolster previous reporting from Bloomberg News that AI frontier companies have little oversight of the powerful, sometimes unreliable and potentially deadly tools they are sending into combat. At the same time, the Pentagon is seeking to accelerate its AI adoption on the battlefield.
The Anthropic CEO triggered a row with the Trump administration earlier this year by drawing a line at using his company’s AI tools in fully autonomous weapons and mass domestic surveillance. As a result, the Pentagon made the unprecedented decision to designate the US company as a supply-chain risk, triggering a lawsuit from privately held Anthropic that is ongoing.
Nevertheless, Amodei’s AI is playing a role in the US war with Iran.
US Central Command is using an AI-assisted platform named Maven Smart System that uses Claude and other AI tools to help generate so-called points of interest, help its personnel make decisions and speed up processes for its military operations against Iran. The command has emphasized the unusually high quantity of targets that US operators have hit in a short period of time: striking 1,000 targets in the first 24 hours of operations and 13,000 targets by April 6, little more than a month after operations began.
Asked whether he was comfortable with Claude’s role in hitting more targets and helping to kill people more quickly, Amodei said he supported the US having the ability to be more effective militarily, arguing that the ability to be “stronger” deters rather than causes wars.
“You basically have to leave policy in the hands of the military decision makers,” he said, noting that a human makes the final decision.
Amodei argued that developing AI warfare could help avoid World War III and help defend Taiwan from invasion from China, so long as the technology wasn’t used without limits or in a way that undermined democratic values.
“When I see Russia invading Ukraine, when I see the risk of China invading Taiwan, it worries me that we have a kind of resurgent authoritarian bloc, that they’re very aggressive and that we need to defend ourselves,” he said.
But even some supporters of AI warfare are increasingly wary of the risks involved.
“There’s a lot of governance pieces that, in my opinion, are missing,” Jack Shanahan, retired Air Force lieutenant general and former director of Project Maven, the Pentagon effort that birthed Maven Smart System, told a Stanford University workshop devoted to the future of decision-making last week. He warned that integrating Claude into Maven Smart System could lead to unexpected impacts and dilute the role of human judgment.
While one advantage of AI is to give humans more time to make decisions, Shanahan said he worried that “Type A” personalities in the military may use that extra time to make more decisions rather than better decisions.
“If you make more decisions rather than the right decisions, you may have a very flawed decision-making process,” he said. “You may have a thousand targets, but are they the right targets?”
