Measuring Impact

 

Rigorous measurement and evaluation is an integral part of Living Goods’ DNA.  We use a thorough system for setting targets on and tracking operating metrics linked to our key impact areas.

 

 

Tracking Metrics on Health and Wealth

 

To set targets and strategies to reach our goal of reducing child mortality by at least 15% we built an extensive logical framework that lays out several measurable outcomes and the underlying behavior changes required to affect them. Those metrics include treatments by disease and age, pregnancies supported, post natal follow ups, facility deliveries, as well as treatment and referral follow ups. Living Goods tracks the performance of those key health metrics, sales, and profitability across all branches. We provide all our stakeholders with quarterly updates on each of the following key indicators:

 

  • Pregnancies supported per agent per month
  • % of deliveries in a facility with a skilled attendant
  • % of newborns visited in the first 48 hours by an agent
  • # of under-5 malaria treatments per agent
  • # of under-5 diarrhea treatments per agent
  • % of malaria and diarrhea treatments followed up
  • % of high-impact items in stock
  • Sales per agent
  • Sales per branch
  • Branch profitability

 

 

We’re currently expanding our key operating metrics to include economic impact on agents and customers. We will aggregate customer and agent level economic impacts including agent earnings and incentives, money saved from using efficient household technologies like clean cook stoves as well as the transport savings from home delivery and cost savings from lower priced products.

 

 

Using technology to collect and analyze data

 

Our extensive mobile platform streamlines measurement, optimizes quality control and monitoring, and deepens impact. All Living Goods agents report on key health activities via mobile, speeding the collection of data and allowing real time access to impact data. Living Goods is also using randomized follow up calls to improve quality control and monitoring. In addition, by mid-2013, Living Goods will complete a comprehensive data warehouse to aggregate sales by agent, health metrics, mobile data and agent and customer profiles. Taken together the mobile and MIS systems will enable Living Goods to analyze and manage performance to deepen impact and sustainability of the model.

 

 

Randomized Control Trial

 

Living Goods is implementing a best-in-class, large-scale, randomized control trial (RCT) to assess the impact of our operation. Randomized evaluations are considered the “gold standard” in evaluation design. RCTs are most commonly used by medical researchers in clinical trials and are increasingly used in social science to measure the impact of a program or intervention. A randomized evaluation determines the extent of impact of an intervention by comparing the outcomes of communities or individuals who received the program with those who did not. Living Goods study is being administered by a leading independent researcher – Professor Jakob Svensson, an affiliate of the Abdul Latif Jameel Poverty Action Lab (J-PAL), a member of the Swedish Academy of Sciences, and professor of economics at the International Institute for Economic Studies at Stockholm University. Baselines for the RCT evaluation were taken in 2010. This study will allow us to measure our primary objective of reducing under-5 child mortality and morbidity. J-PAL is also fielding a behavior tracking survey designed to provide directional input on our progress toward our ultimate objective of under-5 child mortality. The behavior tracking surveys also use a randomized control design but focus on a smaller subset of indicators that Living Goods can track more frequently.