ACCELERATING STUDENT ACADEMIC RECOVERY
The EdResearch for Action Overview Series summarizes the research on key topics to provide K-12 education decision makers and advocates with an evidence base to ground discussions about how to best serve students. Authors – leading experts from across the field of education research – are charged with highlighting key findings from research that provide concrete, strategic insight on persistent challenges sourced from district and state leaders.
Emily Morton | CALDER at the American Institutes of Research
Ayesha Hashim | Northwest Evaluation Association (NWEA)
September 2023 | Brief No. 26
What strategies are most effective for intensive academic intervention?
- High-impact tutoring is widely considered to be one of the most effective ways to help students quickly make up lost academic ground.
- Intensive, short-term interventions during summer vacations and other school breaks have boosted achievement in some schools and districts.
- Double-dose math classes have been shown to be effective in helping students increase their math proficiency.
Additional Practices to Consider
- Academics-focused afterschool programs, staffed by certified teachers with high-quality curriculums and sufficient dosage, can significantly raise student test scores.
- Computer-assisted learning that supplements core instruction is a promising strategy for boosting student achievement, particularly in math.
- Increasing instructional time by extending the school day or year can lead to gains in student achievement, but the effects are mixed and depend on how the time is used.
Practices to Avoid
- Grade retention, while popular among many policy makers, can have negative consequences if not executed equitably and paired with substantial additional support for students.
- Interventions that replace, rather than supplement, core instruction time can negatively impact student achievement.
- Teaching more advanced content without providing additional support is unlikely to be successful.
Effective Implementation Considerations
- Academic interventions that provide scaffolding for grade-level content lead to larger gains in student achievement compared to those that solely reteach content from previous years.
- Interventions often have larger positive effects when they attend to students’ social and emotional needs alongside academic learning.
- Clear and persuasive communication with families is key to investment in the program regardless of model design.
Breaking Down the Issue
Three years after the onset of the pandemic, there is little evidence of academic recovery in the U.S. The latest data reveal a sobering reality: In spite of many school districts’ efforts to accelerate learning, students remain far behind pre-pandemic levels of achievement. Even more troubling, many students’ did not accelerate their progress at all last year; on average, their rate of learning decelerated, putting them even further behind. Between fall 2022 and spring 2023, according to a recent study, students in grades 4-8 made significantly slower progress in reading and math than was typical in pre-pandemic years (and progress was especially slow for students attending high-poverty schools and students of color). At this rate, the average student would need, on average, an extra 4.1 months of reading instruction and 4.5 months of math instruction to match what they had learned in a typical school year prior to the pandemic.
The 2022 NAEP math and reading results were similarly troubling: Fewer students achieved Proficient or Advanced levels than in 2019, and greater numbers of students scored at the Below Basic level. In other words, the findings suggest that fewer than one-third of the nation’s 4th and 8th graders are currently performing at or above grade-level.
|PERCENT OF STUDENTS SCORING BELOW NAEP BASIC
Source: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2023.
School districts that serve high percentages of low-income and minority students and that were remote or hybrid for longer periods experienced the largest initial negative impacts on test scores, and they remain the farthest behind pre-pandemic achievement levels. For example, just to return to pre-pandemic levels of inequality in achievement, students at high-poverty schools are estimated to need an additional month of schooling relative to students at low-poverty schools.
Across districts, middle school students also tend to have a longer road to recovery than elementary students. Whereas the average student in grades 3 through 5 is estimated to need about 2 to 3 months of additional schooling to catch up in reading and math, the average middle school student needs 5 to 6 months. Estimated recovery timelines for math are similar to or slightly longer than those for reading in most grades.
Schools have always looked for ways to accelerate the progress of students achieving below grade-level. However, researchers have found that some of the most common approaches to helping those students catch up to their peers (such as grade retention, accelerating entry into advanced courses, and providing optional, on-demand virtual resources) tend to be ineffective. At times, such interventions even exacerbate inequalities, having negative long-term effects that outweigh any benefits students’ may experience in the short term.
In that case, what can educators do to identify and support students who’ve struggled academically since the onset of the pandemic? Fortunately, certain interventions have proven to be effective at helping students catch up and keep making progress, while allowing them to participate in core classes with their peers.
Which interventions have strong evidence of effectiveness at accelerating student learning in reading and math?
Multiple studies provide strong causal evidence for the effectiveness of the following practices. This comprehensive research not only demonstrates program impact but also allows for a better understanding of the factors contributing to their success.
High-impact tutoring is widely considered to be one of the most effective ways to help students quickly make up lost academic ground.
- A review of almost 200 rigorous studies found that high-impact tutoring (defined as tutoring delivered 2-3 times per week for at least 30 minutes per session with four or fewer students in a group) was one of the few school-based interventions that consistently demonstrated large positive effects on both math and reading achievement.
- Recent estimates of impacts of effective K-12 high-impact tutoring programs in math and reading range from 0.12 (online tutoring with groups of 3-4 students for 3 hours per week) to 0.40 (small-group in-person tutoring for 50 minutes every school day) standard deviations (or SD; a measure of the strength and statistical significance of an intervention’s effects). To put these figures in perspective, the average 5th grade student gains about 0.40 SD in one school year. This suggests that supplementing regular instruction with high-dosage tutoring can help low-achieving students make considerable progress toward catching up to their peers.
- Effective programs tend to include the following design principles: They operate during the school day without displacing classroom time, are offered three or more times per week, employ skilled tutors, are conducted in small groups (up to four students), and align with the classroom curriculum. Establishing contracts that incentivize providers to adhere to these principles is a promising strategy.
- In addition to adhering to these design principles, school and district leaders must also meet challenges related to staffing, scheduling, and student engagement, or even a well-designed program will be unlikely to succeed.
- Optional, on-demand tutoring programs are common but unlikely to be used by targeted students as intended. Recent evidence on a California charter district’s use of an on-demand, virtual math tutoring program found that, without nudges to participate, only 19% of middle and high school students ever accessed the platform. Struggling students were even less likely to log on, raising concerns that opt-in resources may exacerbate inequalities rather than reduce them.
- Saga Education’s tutoring model is one example of an effective program. In one implementation, pairs of students in 9th and 10th grade received 50 minutes of math tutoring every school day from a paraprofessional tutor at a cost of $3,200-$4,800 per student per year. The content of the sessions was split between material from previous grades and grade-level material aligned with regular classroom instruction. One year, sessions replaced a second hour (or “double dose”) of Algebra and increased students’ scores by 0.18 SD. Another year, sessions replaced an elective course such as art or physical education and increased test scores by 0.40 SD.
- While some effective tutoring programs can be expensive, their large average effects make them highly cost effective relative to many other educational interventions.
Schools and districts have boosted achievement by offering intensive, short-term interventions during summer vacations and other school breaks.
- Reviews of effects of summer learning programs across multiple studies find significant positive impacts on math scores (+0.10 SD), but reading outcomes vary. Some studies have found no impact and others have found gains more consistent with those in math (+0.09 SD).
- Summer programs are highly variable in terms of their design and effectiveness, and many have had no impact on achievement. Effective programs tend to include the following design principles: student attendance requirements (at least 20 days), highly effective teachers, lesson plans that align with school-year standards, and enrichment activities. A specific math focus and spending more time per day on math instruction were also associated with larger gains in math. Reading programs were associated with better outcomes when they had 4-8 hours of instruction per day, 70 to 175 hours of total instruction, and class sizes smaller than 13.
- Short-term learning programs during school breaks (often called “vacation” or “acceleration” academies) can also effectively increase student achievement. For example, in Lawrence, MA, week-long acceleration academies for struggling students in grades K-8 led to 0.12 and 0.11 SD, in math and ELA respectively. The programs provided small-group instruction (10-12 students per group) taught by selected teachers during two separate week-long vacation breaks. Students received about 25 hours of extra instruction per week at a cost of $800 per student (which included incentives for students, such as $40 gift cards for those with perfect attendance).
- A very similar math vacation academy that served a representative population of 6th and 7th grade students in struggling schools in Springfield, MA found a weaker but positive impact (0.07 SD) on math achievement and no impact on ELA achievement.
Double-dose math classes have been shown to be effective in helping students increase their math proficiency.
- Double-dose courses can be offered as an additional, separate class period or as a single extended period. They typically replace an elective and serve all students or target a subset of low-performing students.
- In Chicago, 9th grade students with low math test scores were enrolled in an additional algebra support class (which was typically taught by their usual teacher, who received updated curriculum and additional professional development). Participating in the double-dose period increased students’ spring algebra test scores by 0.20 SD.
- Double-dose courses in English, reading, and vocabulary have not generally yielded positive effects on test scores over and above regular instruction. Impacts of a 2004 Florida double-dose policy targeted at low-performing students were initially promising — the ELA double dose resulted in a 0.11 SD increase in students’ reading scores the first year — but the effect faded out over the next two years.
Additional Practices to Consider
Which interventions have more mixed evidence of success?
The following practices have fewer causal studies demonstrating their effectiveness, and less evidence is available on how to effectively design these programs to impact student achievement.
Academics-focused afterschool programs, staffed by certified teachers with high-quality curriculums and sufficient dosage, can significantly raise student test scores.
- A 2018 review of research on out-of-school time programs finds that programs with certified teachers, high quality curriculums, and sufficient dosage can have significant positive impacts on student achievement.
- For example, an afterschool reading program for 4th-6th graders for four hours per week over 23 weeks led to notable gains in vocabulary (0.23 SD) and comprehension (0.32 SD).
- Another study focused on math instruction for 2nd-5th graders involving certified teachers and groups of around 8 students meeting 3-4 times a week for a total of 180 minutes per week resulted in positive effects on test scores after one year (0.09 SD). However, the study also examined a similar reading program but found no impacts on achievement. There is some evidence that this was because the reading program curriculum was implemented with less fidelity and consistency across instructors.
- Multipurpose and specialty afterschool programs that focus on homework help, recreation activities, and other enrichment activities (e.g., arts, sports, technology) can have other positive impacts on students, but are not linked to increased student achievement.
Computer-assisted learning that supplements core instruction is a promising strategy for boosting student achievement, particularly in math.
- Computer-assisted learning (CAL) programs provide “personalized,” adaptive content to address student’s learning needs. CAL programs can range from light-touch interventions that provide practice opportunities, to courses with entirely new curricula. The level of support students receive while accessing the programs can also vary, from actively engaging with a teacher about the program content to working independently.
- A synthesis of 29 experimental studies found that most K-12 CAL programs showed positive effects, particularly in math. However, there is limited evidence of the effectiveness of CAL in other subjects.
- A review of studies on the CAL program ALEKS found large, positive impacts on math when the program was used to supplement traditional instruction (+0.43 SD).
- The effectiveness of CAL depends largely on how it is implemented, and less is known about the specific features of CAL programs that make them more or less effective. A recent study in Russia finds no additional benefit of spending 40-50 minutes per week on the software relative to spending 20-25 minutes per week on the software.
Increasing instructional time by extending the school day or year can lead to gains in student achievement, but the effects are mixed and depend on how the time is used.
- A 2022 review of experimental research shows that increases in the number of days with academic classes in a school year can improve student achievement. The programs studied range from 10 additional days to six weeks, with the common theme that more days are correlated with higher test scores.
- However, research on lengthening the school day is more mixed. The estimated impact of adding up to two hours range from no effect to an increase of 0.25 SD in ELA test scores. Smaller increases in the length of the day are generally associated with smaller impacts.
- Researchers hypothesize that extending the school year may be more effective than lengthening the school day because it requires fewer organizational changes. Extending the day often requires schedule adjustments and new instructional methods.
- Substantial learning time could be recovered within current school days by minimizing student absences, suspensions, and tardiness, as well as teacher absences and other outside interruptions (e.g., intercom announcements).
Practices to Avoid
Which practices have insufficient evidence of effectiveness?
Grade retention, while popular among many policy makers, can have negative consequences if not executed equitably and paired with substantial additional support for students.
- Proponents of grade retention argue that providing struggling students with an additional year of instruction could have a larger positive impact on student achievement than other interventions if the additional year were to result in a typical year’s learning gains. However, opponents cite mixed evidence on academic gains from retention and negative effects on students’ self-esteem, motivation, and equity concerns.
- Studies of elementary school retention policies in Florida, Indiana, Mississippi, Chicago, Louisiana, and New York City generally find short-term academic benefits, but some find these positive effects fade out in middle or high school. Alternatively, research on middle school retention consistently finds negative long-term impacts, including lower rates of course completion and high school graduation.
- Grade retention is costly, with estimates of the expense ranging from about $4,300 to $10,700 per student. However, the cost of retaining a student for one year is less than enrolling a new student, as these costs are less than average annual per-pupil spending.
- Effective retention policies require intensive instructional support, as retention alone doesn’t guarantee academic improvement and can have adverse consequences.
Interventions that replace, rather than supplement, core instruction time can negatively impact student achievement.
- A 2021 review of 40 studies upholds the well-established finding that separating struggling students for all core instruction has a small to medium negative impact their achievement relative to participating in core instruction with their higher-performing peers.
- A study of 146 elementary schools delivering reading interventions through the Response to Intervention (RTI) framework had no significant impact or a negative impact (0.17 SD) on achievement across grades 1-3. The authors speculate that the null and negative effects can be attributed — at least in part — to pulling students out of core instruction for the intervention, which was not consistent with the design of the program but happened for 40% of participating students.
Teaching more advanced content without providing additional support is unlikely to be successful.
- “Algebra-for-all” programs — programs that accelerate course taking so students take algebra in 8th grade regardless of readiness — have been found to negatively impact students’ grades. Studies of “algebra-for-all” in North Carolina and California found particularly negative effects on achievement for the low-performing students that the policy intended to help.
- Similarly, a Chicago Public Schools policy that required low-performing ninth grade students to enroll in college-prep algebra and English I courses rather than remedial classes resulted in increased failure rates and lower grades, and did not impact students’ test scores or graduation rate.
- Research syntheses caution that accelerated learning programs such as grade-skipping or taking advanced courses only benefit students who are screened and selected based on academic readiness, social and emotional maturity, and motivation for accelerated learning.
- While student readiness for advanced coursework is important, school-level variation in the provision and allocation of advanced courses can reproduce racial and socioeconomic disparities in advanced course enrollment. Decisions on student readiness for accelerated learning may therefore exacerbate inequities in the distribution of educational opportunities without race- and equity-conscious action from local decision-makers.
Effective Implementation Considerations
How can schools create the conditions for academic interventions to succeed?
Academic interventions that provide scaffolding for grade-level content lead to larger gains in student achievement compared to those that solely reteach content from previous years.
- Designing interventions to provide “just-in-time” supports to access grade-level material can make them more effective. A study of a Pittsburgh Public Schools afterschool tutoring program found that addressing specific gaps in students’ previous learning led to larger reading gains (+0.17 SD) than focusing only on grade-level or remedial content.
- A 2021 study of the CAL program Zearn also found positive effects of providing scaffolding for grade-level content. Students who had struggled with 3rd grade math were able to succeed in 4th grade math lessons while being retaught 3rd grade concepts and skills. By comparison, students who accessed only 3rd grade math lessons learned less and fell further behind.
- Aligning interventions with classroom instruction may also amplify their efficacy. A 2008 evaluation of tutoring on math problem-solving was found to be more effective when it was closely aligned with classroom content and pedagogical approaches.
Interventions often have larger positive effects when they attend to students’ social and emotional needs alongside academic learning.
- Focusing on developmental relationships as an “active ingredient” in academic programming, rather than focusing only on incentives and accountability, can have larger effects on student learning, especially for at-risk children. These relationships involve emotional attachment, talking and listening to each other, having meaningful interactions, and making sure everyone feels respected.
- Recent experimental studies suggest that assigning low-income youth to mentors who serve as role models (and who provide frequent social reinforcement and feedback) can lead to significant and persistent gains in social, behavioral, and academic outcomes.
- Tutoring programs with the largest impacts have sustained tutor-student relationships focused on clear academic and social emotional goals.
Clear and persuasive communication with families is key to investment in the program regardless of model design.
- Communicating the value of academic interventions to families, including the magnitude of unmet student needs and the ways in which interventions address specific challenges, can improve student uptake and attendance in such programs.
- When eligible students do not register or attend programs, researchers have cautioned that families need transparent and timely information on available services such as student-teacher ratios, tutor qualifications, academic content that will be covered, and program logistics, cost, and registration.
This EdResearch for Action Project brief is a collaboration among: