Researchers reviewed evaluations of summer programs in the United States serving students in the summer before kindergarten through the summer before 12th grade.

The report examines the following three research questions:

• What summer programs serving K–12 students in the U.S. education system have been recently evaluated?
• What summer programs have yielded evidence that meets ESSA Tier I–III standards?
• What are the characteristics of summer programs that meet ESSA Tier I–III standards?

Eligibility Criterion

Studies had to have a treatment and a comparison group. They also needed to measure an outcome related to such areas as academic achievement, academic attainment, or social and emotional competencies.

One eligibility criterion required that at least half of the program services be offered during the summer. Researchers also excluded evaluations of programs in which the summer activity represented less than half of the services provided to students.

The review used two primary eligibility criteria related to research designs. The first one stated that analyses must compare the outcomes of two distinct groups of participants. The second was that the comparison condition should  involve no summer programming at all or programs that were not expected to influence the outcomes targeted by the program being studied. 

Assessing the Four Evidence Tiers

ESSA requires that “activities, strategies, and interventions” funded under the legislation be supported by evidence of their effectiveness. The law specifies four tiers of evidence. But tier IV does not require having a published evaluation of the specific intervention. Because the report focuses on study findings as the unit of analysis, researchers could not consider this tier as part of their review.

To support states’ responsibility to apply ESSA evidence tiers to activities funded under the law, the USDOE in 2016 issued nonbinding, non-regulatory guidance for using the evidence tiers in practice. In this report, researchers provide information on whether a program has evidence that meets the first five criteria (rigor, positive result, broad application, absence of harm, and overall effectiveness).

Researchers conducted a comprehensive search of the major electronic databases of indexed scientific literature and relevant websites on summer program research to identify evaluation reports. The documents corresponding to the 83 studies that passed all eligibility criteria were then subject to in-depth reviews per ESSA evidence criteria. A team of three researchers carried out these in-depth reviews.

Another team of three researchers extracted the following available data from 46 studies of 43 programs assigned to Tiers I or III:

• Intervention features such as main components of the program, duration, staffing, participant-staff ratio for instructional periods
• Study participants and location including locale, total number of participants and sites, and race and ethnicity of participants, among other factors.