Zambia was ranked last in the 2011 Southern and East Africa Consortium for Monitoring Education Quality (SACMEQ), and falls well below the SACMEQ average in both numeracy and literacy.1 A 2014 national assessment found that 65% of Zambian Grade 2 learners were unable to read a single word in their local language.2
Faced with these low levels of learning, the Ministry built an intervention based on an idea proven to be effective by several randomised evaluations. Instead of copying an approach from abroad, the Ministry worked to understand the reasons why an approach worked elsewhere, then mapped these reasons to its own context and made decisions about how to scale based on its unique opportunities and constraints. The partners and the Ministry built in systems of monitoring, data analysis and continual review meetings to ensure the programme improved over time.
In 2015, J-PAL Africa staff began speaking with the Ministry of General Education to hear their concerns and priorities. The team learned that remedial education was a top priority.
The Ministry had rolled out a new curriculum to address low learning levels, but remained concerned about learners who were already below grade level. The Ministry had already sent out a directive telling schools to focus on remedial learning, but had left it to schools to identify effective ways of delivering this.
Given this focus, J-PAL offered to share the details of the research on Teaching at the Right Level (TaRL). The Ministry responded with alacrity, using funds from the Global Partnership for Education with additional support from UNICEF and J-PAL’s Government Partnership Initiative (GPI) to launch an 80-school pilot of the intervention. The Ministry invited other cooperating partners, including UNICEF, DfID, the British Council, and Innovations for Poverty Action Zambia, to a Catch Up working group, which hatched the plans for the pilot. The Ministry also partnered with VVOB – education for development for additional mentoring and monitoring support during the pilot.
The first step was to train key officials in the policy lessons from the randomised evaluations. While this intervention has been robustly proven to work across contexts, not all models lead to improvements in learning outcomes. For example, volunteer-led models have been shown to lead to the quickest learning gains but are hard to scale and sustain. Teacher-led models show a smaller impact per child on learning outcomes, but they are more scalable and sustainable. Research shows that teachers deliver the best outcomes when supported by frequent visits from mentors.
The J-PAL team supported the Ministry in learning this nuance by presenting on the evidence, creating an implementation booklet about the evaluated programmes, taking two key Ministry officials on a learning journey to see TaRL in India, and introducing the Ministry to Pratham, the Indian NGO that developed the Teaching at the Right Level approach. In addition, Pratham created a team of four staff members to support Zambia. This team has repeatedly visited Zambia to meet with and train Ministry officials and continually provides remote support.
Timeline of Zambia Catch-up Programme
Ministry creates Catch-Up working group
Full-time J-PAL staff hired in country
India learning journey for Ministry officials
Pratham/J-PAL lead training for 4 pilot districts
USAID commit funding for scale
Pilot review meeting and decision to scale up to 1800 schools
Training for 12 districts
Training for teachers in 470 schools
The pilot confirmed that the majority of Grade 3 to 5 learners in Zambia are lacking basic reading and mathematics skills, with more than half of children in Grades 3 to 5 in Catch Up pilot schools at baseline unable to read words.
The process monitoring results found that the programme was well implemented. Monitoring largely occurred as planned, and teachers stuck to the key principles of TaRL. Furthermore, they continued to implement the programme over time.
Zambia’s baseline scores were roughly comparable to the baseline in Haryana, India, where a TaRL intervention was proven to be effective in a randomised evaluation.
Learning outcomes improved significantly during the pilot period. According to the government data, the share of children who could not even read a letter fell by 26 percentage points from 33% to 8% during the pilot period, and the share of children reading with basic proficiency (a simple paragraph or a story) grew by 18 percentage points from 34% to 52%. In arithmetic, the share of students in the beginner group (who could not even complete two- digit addition sums) fell by 16 percentage points from 44% to 28% and the share of students with basic proficiency (able to complete two digits subtraction) rose by 18 percentage points from 32% to 50%.
Baseline and endline learning outcomes
The Ministry of General Education is eager to scale the programme to more children in Zambia. The Ministry committed to scaling Catch Up to 1,800 schools by 2020, and is poised to roll out the programme to 470 schools in 2018.
Although the pilot has concluded, the team looks at this next phase of scale as another opportunity to learn and improve. Already, the team made a number of improvements since the pilot, including:
- Development of master trainers (selected from province, districts and zones) who were involved in the pilot to lead the scale up.
- A more intensive training for new master trainers and the creation of training programmes, videos and step-by-step guides for facilitators on how to train mentors and teachers. Pratham staff came back to Zambia to support the 10-day training of government master trainers and continues to support these officials with occasional visits.
- Improvements made to the teacher guides, lesson procedures, and activities in the programme following analysis of pilot learning outcome data and focus group discussions with Ministry and VVOB staff from the provincial level, district level and school level.
- VVOB support in the scale-up is provided to the zone, district, and provincial leaders of the programme rather than directly to the teachers. This system helps promote government ownership of the programme.
- Further strengthening the monitoring and feedback system, with more data collection and analysis conducted at the school level by senior teachers. This encourages data-based decision-making to take place at school, and makes aggregation and analysis easier at the zone and district level.
1. Southern and Eastern Africa Consortium for Monitoring Educational Quality (SACMEQ).“What are the levels and trends in grade repetition?” SACMEQ Policy Issue Series available from www.sacmeq.org, 2010.
2. USAID, UKAID, and the Zambian Government, “Proposing Benchmarks and Targets for Early Grade Reading and Mathematics in Zambia.”