全球智库信息集成服务系统

Federal school improvement policies, by design, have historically targeted the schools struggling the most. Under the No Child Left Behind Act (NCLB), states were required to identify and support chools that failed to meet benchmarks, which led to overclassification and thinly spread resources. Beginning in 2012, however, under the Obama administration’s waivers from NCLB, states instead have identified schools that performed in the lowest 5 to 15 percent statewide—a shifted focus to target resources that was driven in part by limited state capacity. The latest iteration of federal law maintains the requirement that states identify the lowest-performing schools, but now, districts will be responsible for intervening where schools need the most support. As a result, the Every Student Succeeds Act (ESSA) largely shifts the school support role from state to district hands.

Importantly, states will continue to play a role in supporting schools that they have identified for improvement under ESSA. Yet, given districts’ local locus of control and prime positioning as change agents, they have greater capacity to create systems that support all schools.¹ This broadening of district support, however, does not have to come at the expense of schools that are struggling to make gains. Rather, districts can create, implement, and sustain systems that help all schools—not just those at the bottom—to continuously improve.

Denver Public Schools (DPS) provides a window into this work. As the largest school district in Colorado, DPS represents 10 percent of the state’s K-12 student population.² The district uses a school rating system called the School Performance Framework (SPF) to inform its Tiered Support Framework (TSF), which categorizes all district-run schools into three levels of support: universal, strategic, and intensive.³ The universal tier includes foundational supports available to all schools, whereas school-specific improvement plans inform supports for schools in the strategic tier. Schools in the intensive tier receive district support for accelerated improvement and school redesign.

Both the district’s school rating system and the TSF are unique to Denver, which leverages its district capacity to provide supports for all schools while ensuring that higher-need schools receive additional and differentiated supports and resources.⁴ Altogether, the district identifies, plans, supports, and monitors school performance in order to help all schools improve.

By analyzing the academic growth of DPS schools over time, one can better understand how the district’s framework supports school performance. Comparing the growth of lower- and higher-performing DPS schools with the rest of Colorado and the growth of DPS schools receiving more rigorous supports with that of peer schools across other districts yields particularly promising results. Not only did Denver schools show greater growth than the rest of the state in recent years, but analyses suggest that DPS schools in need of additional tiered supports made greater gains than similar schools in Colorado. Specifically:

• In recent years, all DPS schools—including both higher- and lower-performing schools—showed greater growth than the rest of the state, particularly in math.
• When controlling for school demographics, students in DPS schools receiving more rigorous supports in the strategic or intensive tiers showed comparable gains in ELA and math relative to students in higher-performing DPS schools.
• DPS elementary schools receiving strategic or intensive tier supports showed greater schoolwide gains—in addition to greater gains for Hispanic students and students eligible for free or reduced-price lunch—as compared with similar schools in the state.
• When compared with peer schools, however, DPS elementary schools receiving strategic or intensive tier supports had larger achievement gaps between students who were and were not eligible for free or reduced-price lunch.

Importantly, DPS’ Tiered Support Framework is not the extent of the district’s school improvement efforts. The district is using a variety of strategies to increase school quality and close opportunity gaps, including investing in early learning opportunities, expanding high-quality school options throughout the district, and developing strong pipelines for teacher leadership.⁵ The district’s gains therefore cannot be attributed to its tiered supports alone, but an analysis of the framework and school progress can provide a window into DPS’ efforts and outcomes.

Accordingly, this report will explore how Denver’s school rating and improvement systems support school performance across the spectrum. Although academic achievement is not the only barometer of school performance, the analysis in this report will use test scores and student growth as outcome measures, given the significant role these metrics play in Denver’s school rating system and the availability and simplicity of data.⁶ The results of this analysis suggest that other districts can learn important lessons from Denver’s approach to continuous improvement for all schools.

Denver Public Schools has an accountability and support system that is unique to the district and aligns with the DPS vision of school quality. The district annually classifies all schools into five performance categories, which it uses to help determine which of the three tiers of supports each district-run school receives. DPS supports alternative schools—schools that serve large percentages of high-risk students or students with special needs—through a unique accountability, support, and funding structure and uses a separate tiering process for charter schools.⁷ As a result, alternative schools, charter schools, and their tiering processes are excluded from the analysis.

In addition to a district rating and a support tier, Denver schools receive a classification under Colorado’s statewide accountability system, which assigns schools to one of four school performance framework plans: “performance,” “improvement,” “priority improvement,” or “turnaround.” Denver goes through an annual review process with the state in order to align the state and district school classifications.⁸

In the 2014-15 school year, Colorado administered new assessments, the Colorado Measures of Academic Success (CMAS). As a result, that year, both the state and Denver did not classify schools using their respective rating systems; school ratings data at the state and district level are available for the 2015-16 and 2016-17 school years.⁹ In compliance with ESSA accountability requirements, Colorado has also identified schools for comprehensive support and improvement or targeted support and improvement.¹⁰

Identification process

Denver Public Schools annually classifies school performance using five performance ratings: “distinguished,” “meets expectations,” “accredited on watch,” “accredited on priority watch,” and “accredited on probation.”¹¹ These school performance determinations are based on a combination of measures, including:

• Academic achievement and growth from student assessments, including state assessments in English language arts (ELA), math and science; college and career readiness assessments such as the PSAT and SAT; early literacy assessments; and English language proficiency assessments
• Student satisfaction survey results; parent satisfaction and engagement survey results; and student attendance rates
• Postsecondary readiness status and growth, which include PSAT and SAT results; graduation and dropout rates; remediation rates; and results from Advanced Placement, International Baccalaureate, and concurrent enrollment courses
• Academic gaps between historically underserved students and their peers¹²

In 2017, the most recent year for which DPS school ratings data are available, the district classified half of schools as “meets expectations”—up from about 40 percent the previous year. In both 2016 and 2017, DPS classified about one-third of schools as “accredited on watch,” and between 2016 and 2017, it significantly reduced the number of schools that were “accredited on probation.” (see Figure 1 below) Notably, in response to community concerns that the district inflated scores by overstating gains on early literacy assessments, the district has proposed changes to how it calculates school ratings, which tend to shift at least slightly on an annual basis.¹³

Denver uses these school ratings and other key data to categorize district-run schools into the three support tiers that make up the district’s Tiered Support Framework. According to the district, the universal tier focuses on continuous improvement to move schools from “good” to “great.” The goal of the strategic tier is to reverse the trajectory of higher-risk schools through preventative supports and resources. Lastly, the intensive tier provides the highest level of support for schools, using school turnaround or other improvement and intervention strategies.¹⁴

Overall, TSF identification involves three steps. First, the district reviews school ratings and indicators that correlate with future performance, such as voluntary teacher turnover, annual climate survey teacher feedback, and parent and student satisfaction. Second, where needed, the district performs a qualitative assessment of school culture and instructional structures. Third, based on data gleaned from the first two steps, district leadership makes recommendations on final tier placement.¹⁵

This identification process occurs each year following the release of the district SPF, typically by the end of September or mid-October.¹⁶ For example, in fall of 2016, using the 2016 school ratings and other indicators described above, the district identified the TSF tier for each school, which guided improvement planning and supports for the remainder of the 2016-17 school year as well as for the 2017-18 school year.¹⁷ During the 2016-17 school year, schools identified in the strategic or intensive tier conducted a needs analysis informed by a third-party school quality review; engaged in immediate- and long-term improvement planning; and implemented “quick-win” improvement strategies, such as targeted tutoring programs, teacher training, or social-emotional supports. In the 2017-18 school year, schools implemented their full improvement plans, which aligned time, people, and money.¹⁸

Based on 2016 school ratings, more than two-thirds of schools were identified in the universal tier for the 2017-18 school year; these schools were primarily classified as “distinguished,” “meets expectations,” or “accredited on watch.” Another third of DPS schools, from four of the five school rating categories, were split nearly evenly across the strategic and intensive tiers. Notably, because the district supports schools in the intensive tier for a minimum of five years, some schools that the district classified as “meets expectations” in 2016 continued to receive support in the intensive tier during the 2017-18 school year. (see Figure 2 below)¹⁹

Tiered supports

The universal tier outlines the supports that are available to all schools, including coaching on data-driven instruction, professional development, and community engagement programs such as the Parent Teacher Home Visit program. Strategic supports are informed by school-specific improvement plans, which can include increased time for teacher planning, school leadership team support, and wraparound services provided by external experts. Schools in the intensive tier, whose supports are driven by school-specific needs and designs, receive district support for school redesign, from redesigning critical elements of the school to redesigning the entire school. Whole school redesigns typically include replacing school leaders and staff or opening a new school in partnership with the community.²⁰

In addition to planning and implementation support, schools in the strategic and intensive tiers may receive targeted leadership development and resources. For example, a cohort of principals and school leadership teams from schools in both tiers receives one or two years of professional development through the University of Denver, the University of Virginia, or the Relay Graduate School of Education. Both tiers of schools also receive additional funding to use at their discretion, guided by their school improvement or redesign plans.²¹

Progress monitoring is also foundational to the district’s school support and accountability model. All schools, regardless of tier, have a regular review of their improvement plans with instructional superintendents, who supervise principals and provide accountability as well as school support; this occurs approximately every four to six weeks, alongside typical weekly visits. Schools in the intensive tier have more frequent school visits, and after three years of implementation and monitoring, the district evaluates if the highest-priority schools—those that implemented a whole-school redesign model—are on course to exit that tier after year five.²² Ultimately, schools remain in the intensive tier for a minimum of five years so that the district can provide ongoing support for schools to make and sustain gains; however, schools may receive longer support or intervention if improvement goals are not met.

The goal of this analysis is to examine how the performance of Denver Public Schools, all of which receive tiered district supports, compares over time with that of the state and peer schools across other districts. Given Colorado’s transition to new state assessments in the 2014-15 school year, data are restricted to the three most recent years of available state assessment results, from the 2014-15, 2015-16, and 2016-17 school years.²³

Approximately 1,400 Colorado schools—about 120 of which are in Denver—are included in the analysis, excluding charter schools and alternative schools.²⁴ On average, excluding Denver, Colorado schools represented 45 percent free or reduced-price lunch-eligible students, 31 percent Hispanic students, 59 percent white students, and 3 percent black students. Meanwhile, Denver schools represented 69 percent students eligible for free or reduced-price lunch, 56 percent Hispanic students, 25 percent white students, and 12 percent black students.

As part of the CMAS assessments, students in third grade through ninth grade take the Partnership for the Assessment of Readiness for College and Careers (PARCC) exam in ELA and math; they also take CMAS exams once in social studies during elementary and middle school and once in science during elementary, middle, and high school.²⁵ This analysis is restricted to school-level ELA and math scale scores—which measure student performance at a point in time—as well as ELA and math growth scores, which measure student improvement over time. Since growth scores capture academic progress, they are important for determining school effectiveness, particularly for districts that serve more disadvantaged students and have lower scale scores.

Scale scores

On the CMAS, students receive scale scores between 650 and 850, which are reported at the school level as mean scale scores, or the average performance of students in the school.²⁶ Compared with raw scores, or the total number of points students earn on the test, scale scores adjust for slight differences in test form difficulty and enable comparisons across years.²⁷ In the 2014-15, 2015-16, and 2016-17 school years, the state consistently outperformed DPS schools in ELA and math. (see Figure 3 below)

On average, these differences translate to Colorado schools, excluding DPS, outperforming DPS across all three school years by 4.5 points in math and 4.8 points in ELA.²⁸ However, controlling for measures such as the race and ethnicity of students and poverty—using the percent of students eligible for free or reduced-price lunch as a proxy—Denver schools were associated with greater gains of 8.4 points in math and 8.5 points in ELA, relative to other Colorado schools.²⁹

Growth scores

Colorado uses student growth percentiles in order to calculate how much growth a student made relative to his or her academic peers, which is determined by previous test score history. These scores range from 1 to 99. For example, a student with a student growth percentile of 75 improved as much or more than 75 percent of his or her peers. Schools report their median growth percentile, which is the median student growth percentile of all the students in the school.³⁰

As Colorado administered the CMAS for the first time in the 2014-15 school year, growth data are available for the 2015-16 and 2016-17 school years. In both of those school years, all DPS schools showed greater growth than the rest of the state in ELA and math. (see Figure 4 below)

On average, these differences in growth translate to Denver schools outgrowing other Colorado schools by 2.2 percentiles in math and 6.3 percentiles in ELA.³¹ After similarly controlling for school-level measures of poverty, the race and ethnicity of students, and other measures, Denver schools were associated with greater gains of 5.6 percentiles in math and 7.4 percentiles in ELA, compared with the rest of the state.³²

Strategic and intensive schools

To better understand how Denver supports school progress, schools can be analyzed through the lens of the district’s Tiered Support Framework, which organizes resources by the three tiers of support based on school need. DPS created the TSF in the 2012-13 school year in order to support both the district’s highest-needs schools and those in need of preventative support, in addition to schools that were on track for success. In the 2014-15 school year, DPS amended its policy so that schools identified for intensive support would remain in that tier for five years, regardless of improvements in performance. This policy was applied for the first time during TSF identification in the fall of 2015.

As examined above, in the 2015-16 and 2016-17 school years, DPS schools were associated with greater growth than the rest of the state. In order to understand which schools are driving this growth, one can unpack DPS’ academic outcomes by the district’s lower- and higher-performing schools through the lens of the TSF. Accordingly, this analysis will make a distinction between DPS schools that have and have not been classified in either of the more rigorous strategic or intensive support tiers over time.

Out of the 122 DPS schools included in the analysis, 52 were in the strategic or intensive tier at least once between the 2012-13 and 2016-17 school years. These schools will be referred to as the “strategic and intensive schools.” During this same period, the remaining 70 DPS schools were exclusively in the universal tier and will thus be referred to as the “universal schools.” Charter schools and alternative education campuses are excluded from the analysis because they have separate district support systems.

On average, the strategic and intensive schools included in the analysis represented 87 percent free or reduced-price lunch-eligible students, 71 percent Hispanic students, 15 percent black students, and 9 percent white students. Compared with universal schools, the strategic and intensive schools had significantly higher rates of poverty, greater shares of Hispanic and black students, and fewer white students.

Like DPS schools overall, universal schools showed significantly greater growth, on average, than the rest of the state: 5.0 percentiles in math and 9.3 percentiles in ELA.³³ Controlling for school-level measures of poverty and the race and ethnicity of students, among other measures, universal schools were associated with greater gains of 6.6 percentiles in math and 9.6 percentiles in ELA comparatively.³⁴ The strategic and intensive schools grew slightly less than the state in math and slightly more than the state in ELA.³⁵ However, similarly controlling for school-level demographics, strategic and intensive schools were associated with greater growth than the rest of the state: 3.4 percentiles in math and 2.8 percentiles in ELA, though the difference in ELA is not statistically significant.³⁶ (see Figure 5 below)

Within the district, universal schools had significantly higher scale scores and growth scores in both ELA and math than the strategic and intensive schools. However, after controlling for school-level measures of poverty and the race and ethnicity of students in the two groups of Denver schools, these gaps narrowed significantly. In particular, strategic and intensive schools were associated with as much growth as DPS schools in the universal tier that needed less rigorous supports, as differences were not statistically significant. (see Table 1 below)