全球智库信息集成服务系统

The Intelligence Community Is Strongly Emphasizing the Use of Structured Analytic Techniques to Promote Rigorous Analysis But Has Made Little Effort to Assess Whether They Are Measurably Improving the Quality of Analysis

• One primarily qualitative method to evaluate these techniques would be periodic in-depth reviews of intelligence production on a variety of topics to ascertain how frequently structured techniques are used and which ones are most effective.
• In addition, quantitative research could examine the extent to which the use or nonuse of structured techniques correlates with the quality of analytic output as measured by product evaluation staffs, while controlled experiments could test these methods' contribution to intelligence analysis.

RAND's Preliminary Review of a Limited Sample of Intelligence Publications Finds That the Minority of Those Employing Structured Techniques Addressed a Broader Range of Potential Outcomes and Implications Than Did Other Analyses

• In some cases, however, key assumptions and the logic behind specific techniques were not transparent.
• More comprehensive reviews, along with interviews of analysts, managers, and consumers, could determine how effectively agencies are employing structured techniques and provide lessons learned on best practices and pitfalls of their use in intelligence analysis.

Structured analytic techniques (SATs) are a key part of the rigorous analytic tradecraft the Intelligence Community (IC) has pursued in recent years, but so far these techniques have received little systematic evaluation. This report argues that the assessment of SATs is essential, albeit difficult; suggests specific questions that should be part of that assessment; and proposes several methods for ascertaining the practical value of SATs. The report also offers the results of a pilot study that explored conducting such an evaluation via structured interviews with analytic practitioners and a qualitative assessment of a body of IC products to examine the incidence and utility of SATs. These preliminary efforts do not offer definitive conclusions about the value of SATs but illustrate how the IC might evaluate them more systematically.

• Skip to page content

Objective Analysis. Effective Solutions.