全球智库信息集成服务系统

The Formative and Summative Evaluations Yielded Similar Findings

• The main finding from the formative evaluation is that the development of SimCoach had a greater focus on user experiences than on the outcome of interest for our study — the efficacy of SimCoach in improving users' intentions to seek help.
• The main finding from the summative evaluation is that participants who used SimCoach did not report greater help-seeking intentions than participants receiving no intervention.

This report describes RAND Corporation researchers' assessment of SimCoach, a computer program featuring a virtual human that speaks and gestures in a video game–like interface, designed to encourage service members, especially those with signs or symptoms of posttraumatic stress disorder (PTSD) or depression, to seek help to improve their psychological health. The assessment included a formative component assessing SimCoach's design, development, and implementation approaches and a summative component assessing outcomes among participants in a user experience survey and a randomized controlled trial (RCT). Results of the formative evaluation identified both strengths and opportunities for improvement. For example, although SimCoach development processes were well-aligned with best practices for software engineering, SimCoach content development and evaluation processes could have been more tightly coupled to best practices in psychological health. The summative evaluation RCT did not show any SimCoach-related benefit in intent to seek help compared with that of control users not exposed to any intervention. However, secondary outcomes indicated that SimCoach users had satisfying experiences without distress. If SimCoach development is continued, greater attention to clinical processes and outcomes is needed so that the program can have its intended impact on help-seeking for PTSD and depression.

• Chapter One

  Introduction

• Chapter Two

  Formative Evaluation

• Chapter Three

  Summative Evaluation

• Chapter Four

  Summary and Recommendations

• Appendix A

  Revisions to SimCoach Beta Recommendations

• Appendix B

  Additional Distress Signal Phrases

• Appendix C

  SimCoach Beta Participant Comments

• Appendix D

  SimCoach Proposal

• Appendix E

  SimCoach Onsite Evaluation: Preliminary Results