Evaluating Reach Up implementation, impact, and scaling up processes
Reach Up is a very popular program among ECD researchers and program designers. And for good reasons. The pilot of this Jamaican program, which is based on early childhood supplementation and stimulation, was conducted between 1986-1989 and included rigorous evaluations that showed that 22 years later the intervention continued to have benefits for cognitive development, school enrolment and performance, mental health, and violent behavior. These long-term effects had a major international impact, leading to the development of similar programs in 14 countries around the world. But now the focus is back on Jamaica: The Health Ministry plans to implement Reach Up as a nationwide home visits program. This provides a unique opportunity to evaluate and document the sustainable expansion and integration of a successful pilot at the national level. The evidence that will be gathered will be highly relevant for the region and the world at large.
Problematic
Knowing that we have successful interventions is of course exciting but embedding them into existing services is no easy task. It is vital to articulate sectors to ensure sustainable services for all children, but this involves a number of complexities yet to be uncovered. For example, articulation initiatives as well as evidence on their effectiveness are still needed. In this sense, it is imperative to generate knowledge oriented to scaling up successful programs to a national level. This scaling up should provide good quality results and navigate the challenges posed by ability limitations in services such as design, program intensity, personnel training, quality sustainability, availability of human resources, and other assets needed for its persistence over time.
Solution
This project will take full advantage of the valuable opportunity offered by the Health Ministry to conduct a detailed evaluation. This evaluation will help us understand the factors behind the degree of success and effectiveness of interventions and to identify any weaknesses that may emerge as the program merges with existing services. The Reach Up home visits intervention will be managed in its totality by the Health Ministry, whereas evaluation of the processes, impact and costs will be handled by the West Indies University together with the IDB through the Early Childhood Development Innovation Fund. This evaluation will help understand what makes an intervention successful and identify any weaknesses that may come up as the program continues to scale up.
Additionally, an experimental intervention will be conducted where parents will be reminded of the importance of good parental practices via text messages that will be sent in the periods of time between home visits. As the program was implemented during the COVID-19 pandemic, the text message strategy was adapted to provide families with simple activities that the caregiver and the family can perform to stimulate children’s development during the temporary closure of institutional services. In addition, a manual was developed to help parents with their childrearing tasks. This manual can be easily adapted for use in different contexts within the region.
Evaluation and Impact
Given the magnitude and relevance of its scaling up to national level, the project has three types of evaluation: i) Implementation processes evaluation. It measures the effectiveness and use of supervision instruments and monitoring of visits, program strengths, facilitators, and challenges in areas such as training, organizing visits, program strengths, health providers commitment, supervision, and management. ii) Assessment of impact on children development and family environment. The program’s impact will be evaluated experimentally with children chosen on a random basis to either receive the program now or be included in a waiting list. iii) Cost evaluation. All required procurement and implementation costs will be revised, including the time devoted by staff to deliver the program, new investments required, and costs of training, material, and additional personnel.