全球智库信息集成服务系统

It Is Possible to Find Assessments of Many Hard-to-Measure Competencies, but There Will Be Challenges in Gathering Relevant Information on Quality and Use

• A large number of relevant assessments could be identified and cataloged, but some information about the assessments was hard to gather.
• There are logical steps to follow to collect assessments and relevant information and prepare them for the repository, starting with clearly identifying the constructs of interest, including developing search criteria and conducting searches, and concluding with collecting and organizing the information about each identified measure.

To Be Effective, an Online Repository Should Include Five Key Functions: Search, Display, Evaluate, Use, and Connect

• Interviews with potential repository users confirmed the potential value of such a repository and highlighted the importance of these five functions to make the repository usable.
• A repository can serve as an important first step in helping users find measures and evaluate their suitability for decisionmaking.

A Number of Actions Will Need to Occur to Develop an Actual Repository

• This study might be considered a pilot test for building a repository.
• The steps needed to develop a repository include defining the goals and purposes of the repository, searching for measures of constructs, and organizing the information and materials collected.

The William and Flora Hewlett Foundation engaged RAND to conduct research related to the conceptualization and measurement of skills for deeper learning (e.g., critical thinking). This report explores the feasibility of and challenges associated with building a repository of assessments of hard-to-measure competencies, such as those associated with deeper learning.

This feasibility study focused on two aspects of building and maintaining a repository of measures. First, the authors examined the procedures needed to collect, review, document, and catalog assessments for a computerized database. As part of this effort, the authors built a small database of assessments of hard-to-measure competencies, focusing on measures applicable to K–12 students in typical school settings in the United States. Second, the authors examined web-based archives to identify the best functional modules and user-interface features to incorporate into a repository of measures. The study emphasized the collection of assessment information and materials more than the design of a website. Overall, this feasibility study was exploratory in nature. The results are encouraging, but the authors identified some challenges to be overcome in assembling the information needed for a repository.

• Chapter One

  Introduction

• Chapter Two

  Collection of Assessments for the Repository

• Chapter Three

  Design of the Repository User Interface

• Chapter Four

  Findings and Recommendations

• Appendix A

  The Database of Measures Developed in This Feasibility Study

• Appendix B

  Measure Profile for California Critical Thinking Disposition Inventory

• Appendix C

  Descriptions of Example Websites Reviewed