Evaluation with impact – creativity and rigour
I recently conducted a short evaluation for a project which started 20 years ago.
In 1995 the nurse Hazel Stuteley came across a small run down estate in Cornwall. Public agencies had almost given up this place, the police saw it almost as a no go area, children were expected to fail at school, unemployment was high. Hazel mobilised the community and forged cooperation, using existing resources to improve people’s lives. Years later, in November 2015 she decided to have this project evaluated to demonstrate the impact of her and the community’s effort.
Helping people to improve their lives was at the forefront of Hazel’s mind, not collecting data. But with the help of a newspaper article, some existing data points, and the rest of the country as the counterfactual I was able to construct sufficient evidence which after a 70% optimism bias had been applied still showed a significant net benefit of her and the resident’s work.
My methods consisted to setting out with the data available, rates of post-natal depression, childhood asthma, low school performance, high crime rates, bad housing stock and unemployment the situation before the intervention. Looking at the estate again after the intervention as and when data had become available and comparing this to the changes across the country using a difference in difference approach. Some of the data relied on the memory of the protagonists. To ensure that there was no positive bias in the results, I applied a 70% “optimism bias”. Despite this big reduction, the results still showed that the Beacon Estate had improved compared to the rest of the country, improving the lives and chances of the people who live on it.
Hazel is now using this report to support and encourage others who face similar situations to show that it is possible for people to change their lives. “Even an economist said so!”
Ulrike Hotopp, Director LIVE Economics ltd, Chief Economist at Simetrica at the time of this evaluation