A conference at the United Nations in Geneva opened on Monday to address the regulation of lethal autonomous weapons powered by artificial intelligence. Officials are debating how to curb these systems, often termed killer robots, as technology outpaces legal frameworks.
Lethal Autonomous Weapons Systems can identify, select, and engage a target without human intervention, according to Jean-Marc Rickli, head of global and emerging risks at the Geneva Center for Security Policy.
Simpler autonomous weapons have existed for decades. An anti-personnel mine is autonomous because it explodes when someone steps on it, explained Laure de Roucy-Rochegonde of the French Institute of International Relations.
The proliferation of artificial intelligence-enhanced drones changes the equation. These weapons can select targets that were not predetermined, adapting to their environment without help, Rickli noted. Regulators face a central challenge in determining the level of human control needed.
Governments adopt differing policies on oversight. The United States refers to appropriate levels of human judgment in use. France insists a human must remain in the decision-making loop, banning fully lethal autonomous systems while permitting partially autonomous ones.
The human role is ensured through appropriate control by a military commander, a concept that varies depending on system criticality and the operational context, said Rear Admiral Vincent Sebastien, deputy director of the French defense ministry agency for defense artificial intelligence.
UN Secretary General Antonio Guterres and International Committee of the Red Cross president Mirjana Spoljaric warned in August that the world is close to crossing a moral red line involving machines targeting humans. They stated that a race toward greater autonomy in weapon systems is underway.
Analysts caution that the Geneva discussions have a narrow scope. The agenda misses broader issues surrounding artificial intelligence on the battlefield, particularly target generation. Rickli pointed to recent conflicts where high volumes of targets were engaged daily inside Iran.
Dario Amodei, head of the artificial intelligence firm Anthropic, warned this year that frontier systems are not reliable enough to power fully autonomous weapons. De Roucy-Rochegonde questioned what kind of human supervision is required to ensure an operator exercises genuine judgment rather than rubber-stamping machine recommendations.
Major powers view the technology as a strategic advantage. Swarms of autonomous drones aim to neutralize adversaries through saturation, and the Chinese defense ministry described them in January as the new face of warfare. Gerald Mako of Cambridge University noted that the first state to normalize fully autonomous lethal strikes will reset expectations for everyone else.
