Paul W. Bennett: Why education policy is failing us in COVID-19 Times

Twenty years ago, former Ontario Institute for Studies in Education dean and Ontario education deputy minister Dr. Bernard Shapiro identified a fundamental problem afflicting the education sector: “All policy decisions are made by leaping over the data.”

Under normal circumstances, the tendency to generate education policy on the basis of popular school change theories or in the absence of evidence-based research may not be of such great importance. The ongoing COVID-19 crisis is not one of those times.

After two school years of pandemic disruptions, the critical disconnect is becoming more visible to parents, students, and the informed public. Navigating our way out of the crisis will require sound, consistent, and effective responses from provincial and district education authorities. That, in turn, would be greatly enhanced by quality evidence-based research assessing the impact of pandemic school shutdowns and disruptions. For that to happen, it would have to exist.

Canada’s provincial school systems collect student data, but rarely apply it in addressing student learning challenges or unmet needs. Far too much education policy is driven by school change theories, often faddish, reactive and off-the-mark, and provides solutions to problems that don’t really exist. A great deal of what passes for education policy debate is also narrowly circumscribed by subterranean ideological alignments that turn issues like student testing into proxies for recurrent battles between the Left and the Right. That also explains the undercurrent of resistance to system-wide student assessment, even now in a time of crisis.

Back in June of 2011, Dirk Van Damme, Head of the OECD’s Innovation and Measuring Progress Division, stunned a Toronto gathering of prominent international educators at OISE with a rather harsh assessment of the state of education research. “It’s mostly of low quality,” he said, “and we need to be more hygienic when using the word research.”

While Van Damme recognized that education research was improving, he claimed that much of the “research” lacked credibility because researchers began from “fixed ideological positions” and limited their work to small scale projects with limited broader applicability. He warned then, a decade ago, that we were not “preparing students for 21^st century challenges.”

Not much has changed in the intervening years, judging from our preparedness for and response to the COVID-19 crisis. The most recent national study, “Children and Schools During COVID-19 and Beyond,” produced in August 2021 for the Royal Society of Canada, conveyed a sense of urgency to combat this “generational catastrophe” and attempted, not altogether successfully, to unearth evidence of how the massive disruption and school closures have impacted the learning of 5.7 million Canadian students in the “pandemic generation.”

Studying the RSC policy briefing report does give you a pretty good sense of the current shape and quality of education-based research. Social and emotional well-being and children’s mental health are the clear priorities of the vast majority of researchers who are mostly trained in child psychology and educational sociology. It’s little wonder then that the report focuses on social and emotional impacts and, to a large extent, on “notable threats to children’s well-being, educational success, and healthy development”, in that order. It’s also clear that the COVID-19 crisis may well have reinforced the commitment of researchers to transform schools into “therapeutic institutions” for all children.

One of the nine RSC essay chapters, “Estimates of Student Learning During COVID-19 School Disruptions,” made a valiant effort to fill the vacuum. University of Toronto researcher Scott Davies and University of Waterloo professor Janice Aurini confront the problem squarely: “School disruptions over 2020 and 2021 have likely had a significant impact on children’s learning.”

What we do know is worrisome. “Canada lacks high-quality and large scale data that can be used to directly measure any impacts of those disruptions on student achievement,” Davies and Aurini confirm. “Compounding this problem, provinces like Ontario cancelled their planned standardized testing in 2020 and 2021, precluding the possibility of comparing achievement shortly before and after the school closures. Available studies of achievement are limited to single school boards or handfuls of schools, or parent and teacher surveys that can only capture their perceptions of student learning.”

With few exceptions, Canadian researchers have also ignored sound research on “the summer slide” which formed the basis for early estimates of school shutdown setbacks.

Forced to rely upon international studies and research data models, Davies and Aurini claim that the spring of 2020 disruptions alone resulted in “enduring three month learning shortfalls and gaps growing between the quartiles up to 1.5 years.” They conclude that, “Most Canadian students struggled, as did students elsewhere in the world, gaining little ground and soon disengaging from schooling partially or fully.” While students resumed more normal patterns of learning during the interrupted 2020-21school year, the problem was compounded when students “reached a threshold of ‘pandemic fatigue’ and grew tired of online learning.”

The long-term impacts of learning shortfalls are now surfacing, again documented first by international research (UNESCO 2021 and UNICEF 2021). The only significant Canadian research, so far, focuses on social and emotional impacts, often to the exclusion of academic achievement. Poor mental health obviously adversely affects student achievement, but—as Davies and Aurini, point out —the reverse is true because “students who feel they are not achieving will have their well-being compromised”.

False binaries bedevil Canadian education research and are much in evidence throughout the RSC report. The idea that prioritizing academic achievement is at odds with student well-being is not defensible. That faulty assumption was nicely laid to rest in 2020 by University of Cambridge researcher Tania Clarke in a research article exposing “the dangerous discourse of trade-offs in education.” Academic achievement and student well-being are, more often than not, reciprocal and mutually reinforcing. Simply put, doing better academically improves your outlook and sense of well-being.

What’s standing in the way of securing better achievement data? Some of Canada’s most prominent faculty of education researchers are openly hostile to provincial every-student assessment, one of the primary means of generating student data and assessing the extent of learning loss affecting COVID-19 era students. That research gap is exemplified, once again, in the RSC report. Two of the chapters, in particular, produced by University of Ottawa professors Andy Hargreaves and Joel Westheimer, attempt to steer public education away from measuring learning and setting clear expectations. Like much of the current school change literature, those essays privilege student-well being over academic achievement, denigrate the term “learning loss,” and seek to limit or eliminate completely system-wide student assessment.

Actual data from parent surveys, school district reports, and quantitative studies suggest a major disconnect between educational experts, parents, and classroom teachers. Surveys conducted by the Alberta Teachers Association (ATA 2020) and Canadian Teachers Federation (CTF 2020) demonstrate the depth of parent and teacher concerns over erosions of children’s skills and mental health. After the first phase of COVID-19 shutdowns, parents in the Hamilton-Wentworth Board also expressed a strong desire for more teacher-led synchronous learning activities during regular school hours. The vast majority of parents, when given the choice, still opt for in-person schooling, with the possible exception of those who live in multigenerational households. Summer learning loss recovery programs have, according to Davies and Aurini, proven popular with parents who choose them for their children.

The identified learning shortfalls will not go away. Davies and Aurini caution us not to brush the problem aside because the COVID-19 school disruptions may well “trigger a series of negative consequences” in the coming years. Taken together with the solid evidence of adverse mental health impacts, the situation cries out for high quality and timely data that can guide educational policy while also speaking to the legitimate concerns of parents, teachers, and the public.

Yet, Davies and Aurini could not identify a Canadian jurisdiction anywhere that routinely uses seasonal learning designs, ideally first in September and then in June, to generate such data.

Student deficits in foundational literacy and numeracy are emerging as post-pandemic response priorities. All Ontario provincial student assessments conducted by the Education Quality and Accountability Ontario agency have been restored in 2021-22, after two years without them, a step in the right direction. In other provinces, restoring the standardized tests would be best achieved with the establishment of comparable independent student assessment agencies reporting to the public and mandated to oversee all Grade 3 and Grade 6 assessments and to support the ongoing creation of seasonal learning data.

Everyone concerned about the current state of K-12 education in the wake of COVID-19 should be demanding better from the system’s gatekeepers. It’s not too late to resuscitate every-student provincial assessment, tackle the learning shortfalls, and begin preparing for the consequences of future school disruptions.