全球智库信息集成服务系统

• The researchers apply the cross-industry standard process for data-mining to the AI application explored in this analysis. This task requires two primary data inputs: (1) a sufficiently large sample of officer performance narratives and (2) outcome labels that provide information about which narratives indicate the best job performance.
• The researchers find that there is a business need for the AI performance-scoring system. The system could be used as a tool to facilitate policy analysis, assist development teams, enable professional development, or aid in competitive selection decisions.
• Extracting and digitizing large amounts of officer evaluation data from the existing archive is feasible, but, because the process of digitizing the text from older documents requires extensive tuning and computation time, the process could present a potential challenge to future efforts by Air Force practitioners.
• Initial model results are promising: Standard machine-learning algorithms accurately predicted an officer's performance quality by identifying known signals in the performance narrative text without explicit programming.
• Implementation concerns regarding privacy, fairness, explainability, and other unintended consequences are greatest if the system were used to make decisions that would alter officers' careers. These considerations could be less of a barrier to implementing the system for other purposes.

Both private and public organizations are increasingly taking advantage of improvements in computing power, data availability, and analytic capabilities to improve business processes. These trends have prompted U.S. Department of Defense policymakers to become more interested in whether adopting data-enabled methods would facilitate more-effective management of department personnel. In this report, RAND researchers explore one such application that would enable the U.S. Air Force to leverage existing data for improved human resource management (HRM) policies and practices. Specifically, the researchers develop a performance-scoring system that uses artificial intelligence (AI) and machine learning, which would enable the expanded use of performance narratives in HRM processes. The main purpose of this report is to serve as a worked example (i.e., a step-by-step solution to a problem) for Air Force policymakers as they consider how to approach the potential ways in which AI can improve HRM processes.

• Chapter One

  Introduction

• Chapter Two

  The Business Need for Performance Metrics in Human Resource Management Decisions

• Chapter Three

  Understanding Officer Evaluation System Data

• Chapter Four

  Processing Performance Narratives to Create Analytic Data

• Chapter Five

  Modeling the Relationship Between Performance Narratives and Promotion

• Chapter Six

  Evaluating Other Implementation Considerations

• Chapter Seven

  Conclusion and Policy Implications