District Leaders Measure Learning Loss: No Assessment Scores Necessary

March 26, 2021

“We know that there’s learning loss. Everyone is going to have learning loss,” says Chief Academic Officer Patty Cooper. “But we don’t want to make excuses.” Cooper is one of many K-12 school district leaders in the U.S. striving to make sense of what has gone on during the pandemic. In the course of a few weeks in early 2020, schools across the nation pivoted to remote learning, and instruction came from the U.S. Department of Education that state assessments would be waived. “Students need to be focused on staying healthy and continuing to learn. Teachers need to be able to focus on remote learning and other adaptations,” former Education Secretary Betsy DeVos said in a March 2020 statement. “Neither students nor teachers need to be focused on high-stakes tests during this difficult time.” One year later, K12 data experts who had previously relied on state assessments to measure students’ growth are figuring out how to identify and measure learning loss in the absence of state test results.

Identify and Measure Learning Loss: Webinar on Demand

Sometimes called achievement gaps, the term learning loss refers to a reversal in academic progress after an extended instructional break or disruption. This event is nothing new to district data leaders. Data teams made up of individuals representing departments across the district, including curriculum and instruction, assessment and accountability, principals, interventionists, and others, gather multiple times a year to analyze scores and make recommendations for instructional decisions in response to disruptions such as summer breaks. For a year with increased interruptions in instruction time and no assessment scores, identifying and measuring learning loss is tricky.

Data Scientist Dr. Joy Smithson anticipated these hurdles for school districts. Working with K12 data analytics and communications provider SchoolStatus, Dr. Smithson assists districts with student data analysis: Which student populations need help? Who has demonstrated growth and who hasn’t? How can educators take action quickly?

“To measure loss, you need two ingredients,” explains Dr. Smithson. “A baseline value and a current value.” In a normal year, the baseline value is the prior year’s state assessment, and the ‘current’ or recent value for comparison is the year-you’re-in state assessment (often taken in spring of the academic year). Educators can determine which students need more attention in what subject areas, based on the difference.

Screen Shot 2021-03-26 at 12.11.11 PM

Dr. Smithson presents two options for identifying and measuring learning loss in unusual circumstances this year. “Recent research suggests that it’s okay to use the 2019 data as the baseline or prior value,” Dr. Smithson explains. “Though this data point is a bit older than we would like, we know from a few different studies that the 2019 value serves as a valid baseline value.” Because of the age of the 2019 state assessment data, however, Dr. Smithson encourages a second option using benchmark data. “I recommend using benchmarks for two reasons,” says Dr. Smithson “First, they are intended to align with your state assessment, so it should be a comparable measure. And second, for those of you with oversight for lower grade levels, 3rd grade, 4th grade, you may not have 2019 data on a student unless they failed, and benchmark data is certainly preferable to no data.”

Once the method of measurement is selected, district leaders can begin to analyze the data to determine where to target interventions.

“Looking at trends and comparing groups, that’s where you are going to see that all things aren’t equal,” says Dr. Smithson. “A common theme in the literature on learning loss and the experience of COVID-19 is that demographic groups are impacted differently. Being able to compare groups and students in different risk categories will be helpful for addressing these inequities.”

“We’ve always had a pretty good idea that there are certain demographic groups that underperformed on state assessments and in the classroom,” says Director of Curriculum and Instruction Dr. Robert White. Like many district data experts, White had his eye on student growth and learning loss before the pandemic hit. “We never really got a quantifiable idea of who was most vulnerable until last year when we didn’t take the math assessment.”  During the summer White and his team had time to examine the data in new ways. “We didn’t have as much data to go through,” he explains. “So we started looking at historical data of how our demographic subgroups had performed.” 

Both Dr. White and Mrs. Cooper use SchoolStatus for data analysis and have benefitted from the direct assistance of Dr. Smithson. Cooper explains that the way they measured student achievement meant that they were well-positioned to analyze COVID-19 related learning loss. They had already been tracking student growth with their customized ABA (Attendance, Behavior, Academics) report. “It’s something that we’ve used for years,” explains Cooper. “It wasn’t necessarily targeted toward COVID. Although it’s definitely on the radar now.” 

Mrs. Cooper’s ABA Report is one of many custom reports she uses to dive deep into student data. All student data from systems such as benchmarks, assessments, LMS, and the district SIS are integrated into SchoolStatus. “My principals and interventionists would have to spend a lot of time collecting all of that information and inputting it into an Excel sheet,” she explains. “So with SchoolStatus digitizing that for us, it helps us spend our time analyzing and taking action on what the data is telling us as opposed to taking so much time collecting the data.”

Screen Shot 2021-03-25 at 3.27.49 PM

With student data at their fingertips, Mrs. Cooper’s team gets down to creating Individualized Education Plans and communicating with parents right away. Having all the data in the same place allows Cooper’s team to pick up on issues they may have otherwise missed. “Everything is talking to SchoolStatus,” she says. That includes data regarding communication between school and home, a component most data analytics tools don’t consider. SchoolStatus embeds communication tools that automatically record and log all parent contact–meaning the frequency and content of parent communication is now a productive piece of the puzzle.

“We just had a child that’s never been on the radar for ABA, and all of a sudden, this student shows up,” Cooper explains. “So we go all the way into the communication piece to say, okay, what has the communication with the parents been? And is it the appropriate communication with the parents?”

District leaders have demonstrated an impressive ability to keep students on track with a focus on acceleration, not remediation. To do that, parents have to be involved and engaged. “Parents are a huge part of it,” says Dr. White. “Parent engagement becomes magnified in the situation we find ourselves in, or maybe trying to get ourselves out of right now.”

With a significant focus today on finding ways to remedy the loss of instructional time that has occurred during the pandemic, educators are looking to all forms of data. The Federal government recognizes that it will take significant resources for districts to address and make up lost ground. To succeed with the goal of closing achievement gaps, schools need more efficient ways to analyze their student data and identify which subgroups and which students need the most support. “The question in our classrooms is not ‘does everybody understand’,” says Dr. White. “But making it a point to make sure that these groups that historically do not perform as well are getting our message, our instruction. Otherwise, we’re just giving this lip service, which would be a tragedy.”

__________

8c0cfae5-1294-4c0b-bc98-4672427b2982

Dr. Joy Smithson
Data Scientist @SchoolStatus


7f460da9-b444-47e2-8981-f81ee7d0bca6

Mrs. Patty Cooper
Chief Academic Officer @Pass Christian School District


00e10c29-10b4-4020-89dc-65d87e5ed428

Dr. Robert White
Director of Curriculum & Instruction @Columbia School District



SchoolStatus for Learning Loss

SchoolStatus is the district-wide communication tool that integrates key student data in order to increase communication among educators, district administrators, and student families. The company’s solution aggregates individual student data, such as state assessments, attendance, and grades in an easy-to-visualize format and offers the option to communicate with student families via call, text, or email. Through SchoolStatus, millions of communications have occurred on the classroom, campus, and district level. For more information about SchoolStatus, visit http://www.schoolstatus.com.

Recent Posts

   

Subscribe to Email Updates