Overview

Dashboard Alternative Status Schools (DASS) face a common challenge, lacking ways to evaluate their student progress against similar comparators. How should students have performed? Is performance higher or lower, compared to DASS schools with similar programs? Comparative data is helpful for internal understanding and for external accountability purposes. Many DASS schools want more robust ways to set their school performance in context.

To address this challenge, a group of DASS charter school networks is collaborating on a pilot project to compare student growth on benchmark assessments. CSDC is facilitating this work through a Professional Learning Network (PLN) supported by the California Consortium for Educational Excellence (CCEE). Broadly, these CCEE-funded PLNs aim to strengthen the use of the California School Dashboard and Local Control Accountability Plans as tools for continuous improvement. The DASS members of the PLN were unified in their strong feeling that the most important “improvement project” to their school would be development of comparative data.

Seeking Partners: An Invitation to Work with Us

The PLN is seeking DASS schools, charter and non-charter, to share benchmark test data this year. Over 1,000 DASS schools serve more than 160,000 students in California annually, so resulting data sets could be very useful. Data sharing will begin in winter 2019. Benefits to participating schools include the following:

  • Receive reports comparing their students’ performance to that of other schools in the set
  • Get access to anonymized data sets to perform disaggregated data analysis
  • Connect with DASS colleagues on data use
  • Help advance our collective understanding of what DASS student progress rates
  • Help establish acceptance of benchmark assessment data as a basis for accountability

The PLN also invites input and engagement from interested parties. Please contact Susanne Coie at scoie@chartercenter.org with questions and comments, or join an email listserv for periodic updates.

Privacy

Data privacy is of paramount importance. To ensure privacy, the University of California at Los Angeles will store anonymized data on a secure server. Schools will not have access to other school’s data except in anonymized form.

The Product

While many types of academic and non-academic data help demonstrate the impact of the program on students, the PLN chose benchmark assessment data as the top priority for cross-school comparison. Our initial aim is to create a large database of benchmark assessment data, with separate comparative sets based on the following:

  • School type: Results from different models - classroom-based, non-classroom based/online, juvenile court, special education focus - compared separately 
  • Assessment type: Results from MAP, Ren Star or CASAS compared separately

Each comparative data set will ideally be disaggregated by the following:

  • Student subgroup
  • Performance bands
  • Length of enrollment
  • Grade level bands

To achieve a more robust metric, some of the following steps will be used in the data analysis process:

  • Change/time: Schools test at different intervals, and individual students’ test-taking patterns may vary. To achieve a useful growth metric, the analysis will look at growth over time, taking into account the number of days between tests. 
  • Exclusion of outliers: Data will be cleaned to exclude nonsensical outliers, for example, a student who appears to grow six grade levels between assessments.
  • Individual student growth, not cohort growth: Cohort change does not work as well for small schools or for schools with high student turnover, since the baseline is different. This model is based on individual student growth.

Who?

The Charter Schools Development Center facilitates the PLN, whose DASS members include Learn4Life, Learning Works, Opportunities for Learning/Options for Youth, John Muir Charter School, and New Opportunities/Family First Charter Schools. These DASS school networks enroll over 35,000 students annually. All DASS schools are encouraged to participate in this pilot.

Concerned that your results may not so look so hot? Please consider sharing your data anyway, for two important reasons. First, it is important that the data set include schools performing at all levels, so the data doesn’t skew high or low. If only higher performing schools participate, the results could look artificially high.

Second, your results may be closer to the norm than you think. DASS schools serve a population with special needs, and it takes time to have an impact. Is there a DASS administrator who doesn’t wish for a greater impact, sooner? Ideally, this work will help surface bright spots and support continuous improvement work.

Alternative Metrics

Don’t use standardized tests to show growth? Let us know what metrics matter most to you. We are interested in supporting collaboration around other metrics as well. Contact us and share your needs and interests in data collaboration.

Upload Your Data Now

Ready to get started? Please join our email listserv and provide the requested uploading information through the link or contact us directly. UCLA will prepare a folder for uploading, and we will email you an Excel spreadsheet for formatting your data. Participating schools will receive comparator data analytics with other schools’ names anonymized. Participating schools will also be invited to give input and help us make the data reporting as useful as possible.

To receive email updates, join our email listserv.