Sep 07

Evaluating afterschool: What my toddler taught me about evaluation

By Allison Riley, PhD, MSW, Senior Vice President, Programming and Evaluation at Girls on the Run International. Girls on the Run is a physical activity-based positive youth development program that inspires girls to be joyful, healthy, and confident using a fun, experience-based curriculum that creatively integrates running.

The Afterschool Alliance is pleased to present the seventh installment of our “Evaluating afterschool” blog series, which answers some of the common questions asked about program evaluation and highlights program evaluation best practices. Be sure to take a look at the firstsecondthirdfourthfifth, and sixth posts of the series.

My two-year-old daughter and I like to take walks together when I get home from work. Whether we are headed to see the neighbor’s chickens or visit a friend, we always have some goal in mind when we walk out of the door, though my toddler typically doesn’t take the most direct path. Even if I try to rush her along so we can more quickly reach our destination, she is sure to pause when a good learning opportunity comes her way. When I follow my daughter’s lead, our walks are purposeful yet flexible, and I always learn more, too.

As it turns out, my daughter’s approach to a walk translates well to my workday world. As someone who’s spent my career evaluating youth programming, I have learned the importance of having a clear purpose and goals for a project while being flexible and responsive to information gathered during the evaluation process. Let’s look at a recent Girls on the Run study as an example.

Choosing a path

In 2013, Girls on the Run International embarked on a journey to identify evaluation measures that align with our curriculum in preparation for a rigorous external evaluation study of our program’s effectiveness.

When our preliminary study revealed inconsistencies in how the program was being delivered across our program sites nationwide, we had two choices:

  1. Push forward as planned toward our original goal of conducting the external study, or
  2. Alter our path to be responsive to the information uncovered during the preliminary data collection

While it was tempting to proceed as planned, we understood that without addressing these delivery inconsistencies we would not be able to reach our ultimate program impact goal. While it wasn’t in the original plan, we shifted our resources into a National Coach Training initiative designed to equip our network of more than 50,000 volunteers with the tools and resources they needed to provide a supportive climate and implement our curriculum. After developing and launching this strategy we were able to redirect focus back to the external study.

Making it happen

Here are a few strategies to consider to ensure your evaluations have a clear goal while being flexible and responsive to information you uncover along the way.

  1. Build a shared vision with key stakeholders from the beginning. Key stakeholders and program evaluators must have a shared vision of the goals of any evaluation activities and must be ready to be flexible and responsive. At Girls on the Run, being responsive meant extending our evaluation timeline and shifting resources to address the needs revealed during the first phase of the evaluation. Without support from key stakeholders including the board of directors, our academic partner, and our funder, it would have been very challenging to alter our course.
  2. Be serious about learning along the way. As funders and program leaders have begun to rely more heavily on data and measurement, the emphasis on program evaluation has increased. While this growing focus is exciting, it can also lead to a desire to put all of our energy into showcasing impact. With this single-minded focus, program improvement can often take the back seat. Like my daughter on our walks, it is important to place a high value on learning along the way to your destination.
  3. Build in continuous feedback loops. It is tempting to make a plan, launch the evaluation, and then part ways with stakeholders until the report is in hand. In order to keep everyone accountable to the original purpose and goals (and identify any needed adjustments) schedule checkpoints throughout the evaluation project.
  4. If something’s not working, it’s okay to change course. This could mean changing your survey questions or response scales because they aren’t working, or looking at approaches to ensure consistent program delivery before proceeding with the next steps of an external study. Know that you will get to your ultimate destination.
  5. Make the hard decision. It can take courage to step back, slow down, or forge a different path. But the hardest decisions are often the ones that have the greatest payoff when things are all said and done.

For us, being flexible and responsive was not the easy choice, but in the end, we were able to maximize program impact through a focus on program quality and measure that impact with a rigorous, independent study. I suppose my daughter had the right approach all along – purposeful, yet flexible. We will make a great team on the next Take Your Daughter to Work Day.