全球智库信息集成服务系统

Long-term projects need the flexibility to accommodate change

• During this project, there were changes in sites' data collection procedures, data management systems, and research emphasis.

Success might depend on establishing good relationships with counterparts in district offices

• Concerns about accurate data handling are essential to a successful data-based analytic project, but it might be equally important to invest in building positive working relationships with client data administrators and information technology personnel. Many times, progress occurred because of the strong professional connections developed with counterparts in the sites.

Redefining and rescaling existing measures required creative thinking to define comparable metrics across sites

• Scaling and categorizing questions surfaced repeatedly in different contexts. Different kinds of solutions worked in different contexts, and a successful data-based analytic project will benefit from creative thinking about ways to rescale or reclassify based on existing measures.

Collaboration with district staff was critical in the deidentification of student data and the substitution of new unique identifiers

• Confidentiality requirements necessitate data use agreements and data safeguarding procedures that will add time and complexity to the project.

The Intensive Partnerships for Effective Teaching initiative, which was funded by the Bill & Melinda Gates Foundation, was a multiyear effort to improve student outcomes—particularly high school graduation and college attendance among low-income minority students—by increasing student access to effective teaching. The RAND Corporation worked with the foundation to collect and warehouse data from participating sites and to produce annual data dashboards that presented quantitative information about key indicators of the progress of the reforms. During the course of this project, the RAND data team conducted four key activities: (1) defining the metrics that would be used to monitor and assess annual progress and that would appear in the dashboard, (2) collecting the data from the sites to compute the metrics, (3) managing and standardizing the data, and (4) creating the dashboard and reporting the metrics to the sites and the foundation.

This report discusses the challenges in defining metrics and collecting data. It also describes how the RAND data team addressed those challenges. Specifically, the authors examine challenges and recommendations in four areas: (1) issues related to defining metrics used to track system performance; (2) issues related to data collection; (3) issues related to managing and standardizing data across sites; and (4) issues related to data confidentiality, data sensitivity, and partnerships. The authors also draw overarching lessons related to the systematic use of education data for periodic program monitoring.

• Skip to page content

Objective Analysis. Effective Solutions.