全球智库信息集成服务系统

Insights from a wargame involving AI and autonomous systems

• Manned systems may be better for deterrence than unmanned ones.
• Replacing manned systems with unmanned ones may not be seen as a reduced security commitment.
• Players put their systems on different autonomous settings to signal resolve and commitment during the conflict.
• The speed of autonomous systems did lead to inadvertent escalation in the wargame.

Implications for deterrence

• Autonomous and unmanned systems could affect extended deterrence and our ability to assure our allies of U.S. commitment.
• Widespread AI and autonomous systems could lead to inadvertent escalation and crisis instability.
• Different mixes of human and artificial agents could affect the escalatory dynamics between two sides.
• Machines will likely be worse at understanding the human signaling involved deterrence, especially deescalation.
• Whereas traditional deterrence has largely been about humans attempting to understand other humans, deterrence in this new age involves understanding along a number of additional pathways.
• Past cases of inadvertent engagement of friendly or civilian targets by autonomous systems may offer insights about the technical accidents or failures involving more-advanced systems.

The greater use of artificial intelligence (AI) and autonomous systems by the militaries of the world has the potential to affect deterrence strategies and escalation dynamics in crises and conflicts. Up until now, deterrence has involved humans trying to dissuade other humans from taking particular courses of action. What happens when the thinking and decision processes involved are no longer purely human? How might dynamics change when decisions and actions can be taken at machine speeds? How might AI and autonomy affect the ways that countries have developed to signal one another about the potential use of force? What are potential areas for miscalculation and unintended consequences, and unwanted escalation in particular?

This exploratory report provides an initial examination of how AI and autonomous systems could affect deterrence and escalation in conventional crises and conflicts. Findings suggest that the machine decisionmaking can result in inadvertent escalation or altered deterrence dynamics, due to the speed of machine decisionmaking, the ways in which it differs from human understanding, the willingness of many countries to use autonomous systems, our relative inexperience with them, and continued developments of these capabilities. Current planning and development efforts have not kept pace with how to handle the potentially destabilizing or escalatory issues associated with these new technologies, and it is essential that planners and decisionmakers begin to think about these issues before fielded systems are engaged in conflict.

• Chapter One

  Introduction

• Chapter Two

  Key Deterrence Concepts

• Chapter Three

  Artificial Intelligence and Autonomous Systems

• Chapter Four

  Potential Futures in a World of Proliferated Artificial Intelligence and Autonomous Systems

• Chapter Five

  A Wargame of Artificial Intelligence and Autonomous Systems

• Chapter Six

  Wargame Insights and Debates

• Chapter Seven

  Implications for Deterrence

• Chapter Eight

  Implications for Decisionmaking

• Chapter Nine

  Conclusion and Areas for Further Research

• Appendix A

  General Morphological Analysis