After-School Programs for High School Students
An Evaluation of After School Matters
- Author(s)
- Barton J. Hirsch, Larry V. Hedges, Julie Ann Stawicki, and Megan A. Mekinda
- Publisher(s)
- Northwestern University
Summary
How we did this
The research involved a randomized controlled trial using a mixed methods assessment approach. Youth who applied for a specific apprenticeship were randomly assigned to either the intervention (apprenticeship) or control group. Both intervention and control youth were assessed prior to the onset of the apprenticeship (pretest) and after the end of the spring semester, when the apprenticeship ended (posttest). Background, implementation, and outcome measures were assessed via multiple sources and methods. The researchers studied a total of 13 apprenticeships and their respective controls yielding a total of 535 participating youth.
This rigorous evaluation investigates whether participation in Chicago’s After School Matters (ASM) apprentice-like program results in gains in positive youth development, marketable job skills, and academic performance, as well as reductions in problem behavior. It compares results for teens in ASM to results for similar students.
The study finds statistically significant benefits for After School Matters students on some measures of youth development and reduced problem behaviors, but no differences in job skills or school performance.
The authors discuss two contrasting interpretations of the data. A positive perspective emphasizes that ASM was able to obtain significant positive results despite several factors that worked against doing so—for example, lack of substantial extra support for implementation. The other view of the findings emphasizes that few significant effects were found, effect sizes were generally small, and that testing a more representative sample of ASM instructors may eliminate the few positive impacts that were found.
To best enable youth to achieve key ASM outcome objectives, core program design features provide a promising foundation. But important elements of what happens in the apprenticeships need improvement.
Key Takeaways
- After School Matters had a significant impact in areas that are important to adolescent development and to policy. There were statistically significant benefits in youth development and reduced problem behaviors.
- The researchers did not find any significant differences between the groups for marketable job skills or academic outcomes.
- Quantitative and qualitative findings indicate that ASM programs need strengthening. Youth need to be more fully engaged, and greater effort needs to be put into building high-quality products or performances.
What We Don't Know
At the time of this study, very few randomized evaluation studies had been done on afterschool programs. To the best of the authors' knowledge, only one such study had been done with an exclusively high school sample. High-quality evaluation studies can provide information that is crucial to program development for strengthening the ability of programs to produce strong effects. In turn, redeveloped programs need to be subjected to further evaluation.
According to the researchers, it is critical that potential effects be more clearly conceptualized and new measurement instruments developed. They did this for marketable job skills, but much work remains to be done with respect to positive youth development.
