全球智库信息集成服务系统

Urban school districts regularly find themselves at the center of education reform efforts because of large gaps in academic performance between students based on income, race, and ethnicity. Since 2002, the National Center for Education Statistics has tracked the National Assessment of Educational Progress (NAEP) scores of select urban school districts through the Trial Urban District Assessment (TUDA).

But interpretation of these scores is complicated by differences between student populations, both across districts and over time. We use student-level data to tease out how much of this progress has been driven by changes in student demographics and compare how districts serve students with similar backgrounds and needs.

Using Student Demographic Characteristics to Adjust TUDA Scores

Raw TUDA scores do not reflect the fact that urban school districts may serve very different student populations. For example, Houston serves more English language learners than Atlanta, and Hillsborough County, Florida, reports a smaller share of students receiving free and reduced-price lunch (FRPL) than Cleveland. A comparison of raw scores across these districts would not account for student differences. 

To account for demographic differences between districts, we adjust TUDA scores based on students’ gender, race or ethnicity, frequency of English spoken at home, family structure, age when the test was taken, and parents’ educational attainment. We also adjust for FRPL status, English language learner status, eligibility for special education, and the accommodations received on the assessment, but we note that these factors are somewhat related to district and state policy.

FRPL status typically serves as a proxy measure for student poverty, but this measure has become less reliable with the introduction of the Community Eligibility Provision, which provides universal free lunch to high-poverty schools. States vary in how they report FRPL in these schools, which leads to a potential overcount or undercount of low-income students. To counter this, we build an imputed FRPL variable for students enrolled in provision schools.

Estimating Demographically Adjusted 2017 TUDA Scores

We adjust districts’ TUDA scores to compare how students scored relative to their demographically similar peers. This can identify districts that are serving their students well relative to the performance of students with similar backgrounds and needs.

Because TUDA districts are selected, in part, because they serve higher shares of Black and Hispanic students and students from low-income backgrounds, districts’ adjusted scores tend to be higher than their raw scores. Our adjustment reduces the spread of district TUDA scores—the difference between the lowest-scoring and the highest-scoring districts—by about 29 percent.

But even after controlling for demographic characteristics, there are still substantial differences in academic performance. Averaged across the four tests, the difference between the highest adjusted score and the lowest is nearly one standard deviation. School districts that performed at the top of our TUDA adjusted scores—Boston, Miami, and Austin—are in states that historically perform among the top in our adjusted NAEP scores as well.

Understanding Changes in District Performance since 2005

We have shown that NAEP scores for students in large cities have increased faster than the national average. This improvement could be attributable to improvements in instruction or curriculum or other district changes, but it could be because of changing student demographics. If a district enrolls a higher share of students from more advantaged backgrounds over time, TUDA score increases could be driven by these demographic changes, rather than by policy changes or other factors.

To understand the potential effects of demographic changes, we look at the 11 TUDA districts that were a part of both the 2005 and 2017 samples. We estimate the relationship between students’ TUDA scores and their demographic characteristics in 2005 and predict the district’s score in 2017 based on changes in these characteristics.

Our analysis indicates that some TUDA districts have made large gains that cannot be fully explained by changes in the student population. These gains were particularly large in San Diego, Chicago, Los Angeles, and the District of Columbia Public Schools.

Although it is tempting to attribute these changes to school district policies and practices, we cannot exclude unobservable factors that might contribute to student achievement. Changes to local safety net programs or differences in student exposure to crime or pollution could all affect students’ TUDA performance. Other student-level characteristics, such as families’ perceptions of academic achievement and students’ personal motivations, could also contribute to these di