Connecting the Dots

Data Use in Afterschool Systems

People and processes are as important as technology, according to early findings from a study of the use of data in afterschool systems.

April 2016

Download Report

A boy and girl play with chips during an afterschool program.

Table of Contents

Overview

Research Approach

Ideas In Practice

Author(s)
Julie Spielberger, Jennifer Axelrod, Denali Dasgupta, Christine Cerven, Angeline Spain, Amelia Kohm, and Nicholas Mader

Publisher(s)
Chapin Hall at the University of Chicago

Page Count 88 pages

Summary

How we did this

The researchers conducted in-person site visits with each of the nine cities participating in Wallace's Next Generation Initiative during the winter and spring of 2014. They interviewed key stakeholders and observed selected meetings related to the development of data systems and use of data. Across the cities, 104 interviews were conducted (an average of 11–12 per city) and 13 meetings were observed. Stakeholders represented a range of organizations and roles.

Afterschool programs can help keep children safe and foster skills they need to succeed in school and life. But too often afterschool opportunities are simply not available for children from low-income communities. Many cities are creating afterschool systems to help ensure that high-quality programs are available to more children, particularly those often denied access. This interim report finds that developing and maintaining effective data systems can help such widespread efforts.

Citywide afterschool systems

The report presents early findings from a study of how afterschool systems build their capacity to understand and improve their practices through their data systems. It examines afterschool data systems in nine cities that are part of The Wallace Foundation’s Next Generation Afterschool System-Building initiative, a multiyear effort to strengthen systems that support access to and participation in high-quality afterschool programs for low-income youth.

The cities are:

Baltimore
Denver
Fort Worth, TX
Grand Rapids, MI
Jacksonville, FL
Louisville, KY
Nashville, TN
Philadelphia
Saint Paul, MN

All of these cities were in various stages of collecting and using data already to inform their afterschool system building, improve program quality, and make decisions about the allocation of resources. They viewed the development of the capacity for data use as an important part of their system building, but they were still developing their capacity.

Approaches to data

Prior research on data use in afterschool systems has focused more on the implementation of technology than on what it takes to develop and sustain effective data use. This study finds that the factors that either enable or hamper the use of data in afterschool systems—such as norms and routines, partner relationships, leadership and coordination, and technical knowledge—have as much to do with the people and process components of the systems as with the technology.

Strategies that appear to contribute to success include:

Starting small. A number of cities intentionally started with a limited set of goals for data collection and use, and/or a limited set of providers piloting a new data system, with plans to scale up gradually.
Ongoing training. Stakeholders learned that high staff turnover required ongoing introductory trainings to help new hires use management information systems and data. Providing coaching and developing manuals also helped to mitigate the effects of turnover and to further the development of more experienced and engaged staff.
Outside help. Systems varied in how they used the expertise of outside research partners. Some cities identified a research partner who participated in all phases of the development of their data systems. Others used the relationship primarily to help analyze and report data collected by providers. Still others did not engage external research partner but identified internal staff to support the system. In any of these scenarios, dedicated staffers with skills in data analytics were key.

Key Takeaways

Developing the capacity to use data depends as much on people and processes as it does on technology.
Citywide systems use data for many purposes, including improving program quality, managing resources, identifying students who might need additional support, etc.
Afterschool data systems are continually evolving based on new technology, knowledge, priorities, which can change their management strategy.
In selecting indicators and outcome measures for their data systems, city afterschool leaders wanted them to be useful and actionable for multiple stakeholders, including providers, school staff, and city leaders.
Building the capacity to use data is not an easy or a straightforward process and requires time, patience, persistence, and flexibility.

Collecting and having the ability to analyze data citywide will allow us as a city, as a group of stakeholders, to make informed decisions, set priorities, solve problems and reach a consensus on a citywide strategic plan for out-of-school time. Data drives everything.