Dr. Rebekka Lee is a Research Scientist at the Harvard T.H. Chan School of Public Health. She has spent over a decade working at the Prevention Research Center on Nutrition and Physical Activity, where she currently applies implementation science and mixed methods to evaluation & research projects with partners at the Boston Public Health Commission, Massachusetts Department of Public Health, and YMCA. She was the principal investigator of an NCI-funded grant to study training implementation strategies for scale up of the Out of School Nutrition and Physical Activity (OSNAP) afterschool nutrition and physical activity intervention. She also supports the Community Engagement Program at Harvard Catalyst: The Harvard Clinical and Translational Science Center. Dr. Lee uses the Resarch-Tested Intervention Programs (RTIPs) resource to teach about evidence-based interventions in the Leaders in Health community training program that she co-directs and in graduate courses on program planning, evaluation, and implementation science.
Questions & Answers
One of the great things about the Out of School Nutrition and Physical Activity (OSNAP) intervention is that it is designed for afterschool staff to take the lead on how best to implement healthy changes at their sites. We advise that the time allotment and content of learning community sessions is not changed and sites stay focused on the 10 healthy OSNAP goals, but how the knowledge and skills developed are translated into physical activity and snack changes can be decided by the afterschool team. One area we’ve seen successful adaption has been the type of technical assistance provided between learning community sessions – this was originally designed as a phone call with email follow up, but partners have also used text reminders and site visits to keep sites on track. Our local health department partners have also added several slides to the beginning of the first session to connect OSNAP to local data and needs. We have seen successful adaptation to programs that serve younger preschool age children and to the summer camp setting. Any youth program that serves snacks or meals can likely benefit from OSNAP!
Some facilitators to implementation are on-site kitchens for preparing and serving snacks and meals, school support among school-based programs run by outside providers like the YMCA or Boys and Girls Club, and an experienced site director. On the flip side, staff turnover can be a particularly large challenge, given the largely part-time, low wage afterschool workforce.
Yes, we suggest that practitioners use the Out of School Nutrition and Physical Activity Observational Practice Assessment Tool (OSNAP-OPAT) to evaluate the impact of the intervention on healthy practice changes. This is a validated observational tool that site directors complete for 5 days in the fall and the spring for just about 10 minutes each day. We’ve also integrated the tool into the OSNAP learning community sessions to help practitioners focus on the goals with greatest areas for improvement early in the year and identify successes to share with leadership and partners at the end of the year.
We also have evaluation surveys and fidelity checklists available on our website for each learning community session to assess participant experiences and determine whether the intervention is being delivered as intended: http://osnap.org/resources/osnap-materials/osnap-learning-collaboratives/
We are working on a study to assess the effectiveness and cost of two training models – an in person training-the-trainer and online approach – for scaling up the Out of School Nutrition and Physical Activity (OSNAP) intervention. Collecting data on implementation cost alongside effectiveness allows us to address questions about the financial investment required as interventions are taken to scale. The primary objective is to determine the impact of each implementation strategy on overall healthy changes in afterschool practices. The secondary objectives focus on nine specific healthy practices (e.g. offerings of physical activity, water, fruits and vegetables, and sugary drinks) and assess implementation outcomes such as cost, reach, adoption, and fidelity.