Shaping the future: Autonomous Weapons Systems

 National Centre for Peace and Conflict Studies, University of Otago

It is a pleasure to be here in Dunedin, and to have this chance to talk with you about key issues in my disarmament and arms control portfolio. It was one of my objectives in taking on this role that the Government should increase its engagement on disarmament issues. These are issues with a long whakapapa in our country, but also significant implications for the future. It is important, then, that you see yourselves in our work too – New Zealand’s credibility in this area relies on it

Can I please also take this opportunity to acknowledge Professor Richard Jackson and the work of the National Centre for Peace and Conflict Studies for your valuable contributions to the discourse on peace and security, the work you do makes a difference and it serves us well. 

Today, I will share with you New Zealand’s priorities for disarmament and non-proliferation. These have been set against a global security environment in which we see the re-emergence of great power rivalry, challenges to international law and the global rules-based order, and rapid technological change that is outpacing any potential Government response. These global tensions have compounded the divisions and inefficiencies of key disarmament fora just when they are needed the most.

At the same time, disarmament and non-proliferation must compete for necessary resources and political attention when other important global challenges, such as the COVID-19 pandemic and climate change, are leaving a lasting impact on communities around the world.

This requires New Zealand to be even more thoughtful about where we put our efforts. The disarmament strategy I have led recognises these realities, and provides a framework to keep us focused, motivated, ambitious, and flexible. This approach will hopefully position us to respond to new and emerging challenges and opportunities as they arise.

But I want to be clear that neither these challenges, nor the dysfunction in the international environment, lessens the critical importance of work on disarmament and arms control. While New Zealand invests heavily in diplomacy and collective action, armed conflict is always a possibility when peaceful efforts fail to resolve civil conflicts, geo-political tensions, or disputes over land, water and other resources. And so long as this dynamic remains, human security depends upon a robust set of rules and norms regulating the weapons used in conflict. In prohibiting, stigmatising, and disincentivising unacceptable methods of warfare, we can play a small role in mitigating the effects of conflict and keeping a small window open for negotiation and, ultimately, peace.

New Zealand’s Disarmament Strategy in 2021

At the heart of our work, then, remains a steadfast view that progress on disarmament is not conditional on an improved international situation, but can contribute to its achievement. It also reflects the reality that, while there are many factors beyond our control, we can make a difference.

This work is an embodiment of New Zealanders’ belief that our interests – when it comes to peace and conflict – are best served by pursuing collective human security, and not a narrow understanding of national security. This enhances the safety and security of civilians, demonstrates our independent foreign policy, and reinforces our global reputation as principled, responsible and reliable. Our advocacy and determination on disarmament has strengthened international humanitarian law and ensured a continued focus on the protection of civilians.

New Zealand’s long-standing positions on disarmament are an important part of our foreign policy, and we can trace their origins right back to the contribution made by former Prime Minister Peter Fraser to the establishment of the United Nations after the Second World War.

Following that, and in light of the unfolding horror of nuclear testing in the Pacific, New Zealanders took to the streets to reject the right of states to test these weapons – both due to their very nature as an inhumane tool of war, but also due to the devastation the tests themselves were inflicting on communities and ecosystems across the Pacific, a region without any stake in nuclear war.  

These events triggered a powerful protest movement in our country, and the Fourth Labour government signed the Treaty of Rarotonga declaring the South Pacific a zone free of nuclear weapons, and passed the Nuclear Free Zone Act, formalising our nuclear free status and banning nuclear powered ships from entering New Zealand waters. This policy quickly became a cornerstone of our independent foreign policy, and eventually attracted significant cross-party support in Parliament.  

New Zealand’s steadfast support of international law, and our determination to see inhumane weapons banned continued into the 1990s and 2000s with the international movement to ban anti-personnel landmines and cluster munitions, in which we played a central role. At the centre of these movements was a belief that each of these weapons breached key principles of International Humanitarian Law, and we wished to see them outlawed for good. We continue to call on all countries to sign and ratify both conventions.

Alongside these core disarmament developments, New Zealand has for several decades advanced a progressive foreign policy agenda that pursues the common good, and advances our collective human security. Take, for example, the significant role we played in mediating peace during the civil war in Bougainville; the role we played as Chair of Security Council deliberations on the then unfolding genocide in Rwanda; or our firm stand against the Bush-Blair invasion of Iraq. This willingness to speak up and work for the global common good, even at times against the headwinds of great power politics, has become part of who we are.

It is no surprise, then, that our disarmament work is built on these values, including those outlined by our Foreign Minister, the Honourable Nanaia Mahuta during her Waitangi Day speech. 

Through our efforts on these issues we demonstrate our commitment as kaitiaki (or stewards of intergenerational well-being) in the interests of our global whanaunga (connectedness or shared sense of humanity). We show manaaki – kindness or the reciprocity of goodwill – both in our pursuit of outcomes that improve the lives of people around the world and in our approach to difficult disarmament negotiations. And it is through mahi tahi and kotahitanga – collective benefits and shared aspiration – that we strive to work with all partners both domestically and internationally to achieve our disarmament objectives.

So what are these objectives?

The strategy is grouped into three major themes: nuclear disarmament and non-proliferation; strengthening international humanitarian law; and shaping the future.

Our nuclear-related objectives reflect that eliminating nuclear weapons – due to their immense destructive power and the existential threat they pose – is the most urgent item on the international disarmament agenda. While nuclear disarmament has received a much-needed boost from the entry into force of the Treaty on the Prohibition of Nuclear Weapons earlier this year, prospects overall are not promising. Nuclear arsenals are being modernised and expanded, and there continue to be serious challenges to the nuclear non-proliferation regime -It is often unacknowledged, but the risks of a nuclear detonation are continuing to grow. Meanwhile, the effects of nuclear testing continue to be felt, generations later, with devastating impacts including in the Pacific.

In this area, New Zealand will pursue three major disarmament and non-proliferation objectives over the coming year, we want to:

• Contribute to a positive outcome to the Review Conference at the Nuclear Non-Proliferation Treaty, including a meaningful result on nuclear disarmament;
• Increase ratifications and signatures of the Treaty on the Prohibition of Nuclear Weapons and achieve a positive outcome at its 1st Meeting of States Parties; and
• Achieve progress on nuclear legacy issues in the Pacific, supporting the work of the Pacific Island Forum Secretariat, building on the 1st meeting of parties to the Treaty of Rarotonga.

Our second work stream focuses on international humanitarian law. Reflecting the humanitarian considerations that are at the heart of our approach to disarmament, New Zealand places value on upholding – and where necessary, strengthening – IHL. This is the body of international law that restricts and regulates the means and methods of warfare in order to limit the effects of armed conflict and protect civilians. And it is a body of law that is under threat.

Recent conflicts provide an overwhelming body of evidence IHL is being blatantly violated. Ratification of key treaties has slowed and implementation is patchy. And as I will touch on shortly, states have been slow to respond to technological developments in a way that ensures IHL remains responsive to new challenges.

New Zealand will focus on two objectives in this space:

• Adoption and promotion of an ambitious political declaration on explosive weapons in populated areas; and
• Increase uptake of key disarmament treaties (including the Arms Trade Treaty and the Convention on Cluster Munitions) in the Pacific and beyond.

The third major strand of New Zealand’s disarmament strategy is dedicated to shaping the future, with a particular focus on two issues: outer space and autonomous weapons systems.

As states move to exploit the opportunities space presents, we see growing evidence of irresponsible behaviour threatening to place space systems at risk. Greater competition and more activity threatens the sustainable and peaceful use of outer space – the weaponisation of which could have severe, and far-reaching consequences on Earth.

Back closer to Earth, we see growing concern around the legal, political and ethical implications of autonomous weapons systems, where a decision to kill could be made without human control or involvement. The development of autonomous technologies, including those linked to artificial intelligence, is rapidly outpacing global policy and regulation.

With these challenges in mind, New Zealand will focus on two key objectives:

• Develop a national policy and engaging actively in multilateral processes on responsible behaviours in space and related issues, including on space weaponisation; and
• Develop a national policy on Autonomous Weapons Systems that is fit for purpose and future-proof, and which forms the basis for our international engagement on this issue.

It is this latter topic that I’d like to focus on for the rest of this session, both to update you on the policy development process and to invite your views on the work ahead.

Autonomous Weapons Systems

Artificial intelligence has potential to improve humans’ lives, but also to disrupt our societies for the worse, and even threaten our very existence.

Human society has benefited massively from advances in computing technology over recent decade, which have become integral to our daily lives and translated to real improvements in wellbeing.

But machine learning technology, robotics and artificial intelligence are also being rapidly developed and sought out by militaries around the world eager to remain on the cutting edge, amidst increased rivalry and uncertainty. This presents enormous ethical concerns. At its most autonomous, this technology dehumanises people and introduces biases that could further bake-in systematic discrimination and the persecution of minorities. It endangers civilians by eroding their protections at the heart of international humanitarian law, and it creates real risks to international peace and security, including by potentially kicking off a new arms race.

The line between science fiction and industrial fact is increasingly becoming blurred. Imagine a swarm of armed drones being unleashed on a packed sports stadium by a terrorist cell; or an autonomous drone patrolling an area of desert, selecting a target using recognition software trained to identify a uniform, weapon, gesture or face, then taking the decision to stun, maim – or at the most extreme of the spectrum – kill the target. In theory, swarms could include tens of thousands of drones, creating what some scientists have described as a weapon akin to a low-scale nuclear device.  Or consider the prospect of autonomous tanks and ships, unmanned and making targeting and kill decisions alone.  With militaries less constrained by the political, moral, and practical constraints that come with deploying their own citizens in the battlefield, the potential for a continuous global battlefield emerges.

This all raises a number of practical questions. Would it be possible to programme a machine to make decisions consistent with key principles of international humanitarian law? How does a robot discriminate between a combatant and a civilian, or decide whether the action it is taking is proportionate?  This by its very nature is a subjective decision. Do we want to remove the subtle and nuanced reasoning essential to these decisions, thereby removing the humanity that underpins them? 

A lack of human responsibility over these systems also poses challenges for accountability. Who, for example, would be responsible if an autonomous system decided to bomb a hospital or a school? And what happens when these systems malfunction?

Many of you will remember the story of the Soviet officer on duty the morning of 26 September 1983, when the early-warning system he was manning wrongly detected an incoming nuclear strike. Military protocol dictated he inform his superiors, who would then – as per military protocol – launch their own counterstrike. According to the automated system, there was a very high degree of confidence a launch was incoming. But the duty officer applied his humanity to the situation. Something didn’t feel right, and in doing so – in failing to call the ‘strike’ in, he almost certainly prevented setting the world off on a path to nuclear annihilation. And this is only one example.

To many of us the prospect of delegating the decision to kill to a machine will simply be morally abhorrent. Moreover, these systems are often very opaque, being by their very nature black box systems. Machine-learning is data driven, through the consumption of screeds of information to make more-and-more accurate predictions. Moreover, militaries keen to protect their edge on their battlefield are unlikely to disclose what will be considered highly sensitive information, which raises further issues, including how victims and their families could ever receive justice in the event of a violation of Hillarie civilians really safer when the owner of an autonomous weapon, let alone the general public, does not know exactly where or when, or what, they will destroy?

Tech workers - including those at Google, Microsoft, and Amazon - have been sounding the alarm since 2015 that their work, research and product development are being adapted for use in warfare. Some companies have gone further, with leaders from Tesla, Google and Apple all calling for a ban on fully autonomous weapons.  Robotics companies, too, such as Clearpath Robotics have been actively working with the international campaign to prohibit autonomous weapons systems.

And yet, there are already a number of weapons systems in operation today that have significant autonomous characteristics. While these fall short of what would be considered fully autonomous weapons, it does show that modern warfare is already embracing this new technology.

As you can see, the legal and ethical challenges posed by autonomous weapons systems – and by their inherent unpredictability – are profound. But they also pose a significant policy challenge given their intersection with emerging technology, foreign policy and military capability.

So what are we doing?

Because of the importance of this issue, and the speed at which autonomous technologies are being developed, I have directed the Ministry of Foreign Affairs and Trade to accelerate the development of a comprehensive New Zealand policy on AWS, and to prepare for a more active role internationally.  Accordingly, MFAT has launched a domestic multi-stakeholder consultation process to support this process. 

We kicked off this work in April, with a workshop which brought together academics, activists, officials, Defence personnel and entrepreneurs. This was extremely useful in demonstrating the breadth of the challenges posed by autonomous weapons, and a widely held view that New Zealand should step-up its efforts at home and abroad. The mahi continues, and engagement including via forums like this, is an important component of it.

Since then, the International Committee of the Red Cross has released its policy position on AWS, which provides a clear account of what’s at stake and why urgent progress is required to adopt new legally-binding rules to address the challenges posed by this issue.

As I hope I’ve been able to outline today, the work we are doing cuts to the heart of the future of warfare on how best to ensure the safety and security of New Zealand and our partners in a world where autonomous technology is increasingly ubiquitous; and on how to uphold the global rules based order and protect the disarmament and humanitarian work we’ve contributed to over decades.

I have committed New Zealand to building an alliance of countries working towards an international and legally-binding instrument prohibiting and regulating unacceptable autonomous weapons systems.

This of course relies on other nations joining us in this effort. As Minister I have and will continue to be active in taking this up with my ministerial counterparts overseas, because ultimately if we want to make progress on a legally binding instrument it will take a group of countries, alongside civil society, to lead the way.

At this point it is important to recognise the vital role of civil society, as always, leading the way in confronting these frontier issues – in this case it is the International Campaign to Ban Killer Robots headed up by New Zealander Mary Wareham.

New Zealand – like many of our partners – is currently in the throes of developing a detailed policy position, and it’s important that we do this, to determine the exact parameters of this work, and its broader impacts. But it will come as no surprise to anybody that New Zealand remains deeply supportive of ensuring international humanitarian law remains up to date and relevant, and that when ICRC the guardians of IHL say we need a new international instrument to achieve this, we support them.

We will of course continue to harness the knowledge and experience of our local academics, activists, and officials, which forms the basis of our engagement with overseas partners. And we are working closely with civil society and the International Committee of the Red Cross to share notes, and tap into their unique expertise. This of course is necessary in such a technical field; but that complexity should not scare us off. We have, of course, scaled similarly large tasks in the not-too-distant past. I look forward to also engaging further with some of you in this room as we advance this mahi further. Thank you.

Nō reira, tēnā koutou, tēnā koutou, tēnā koutou katoa.

Please note: Speech as delivered may differ slightly.