Using the Virtual Design Lab to rapidly test health interventions

The Community Health Innovation Network (CHIN) is a three-year initiative implemented by Living Goods and Medic Mobile. CHIN is aimed at promoting creative solutions and designing and testing a pipeline of innovative ideas that integrate community health care with health facilities, while extending high-quality diagnostic technologies to effectively reach clients at the community level.

In this issue of the Newsletter, we’ll be talking about the Virtual Lab (vLab) (1): a collaborative team that is addressing the problem of how to serve our clients while working remotely.

We’ll also cover how the vLab is helping us respond to COVID, and other health conditions (2): some of Our Findings (3) from our work, and how we are using a rapid sprint approach (4) to quickly design, deploy and test a series of mobile health interventions. These solutions have the shared aim of allowing clients to quickly and easily self-report symptoms and receive treatment for COVID, ante and post-natal care and many other conditions.

The vLab

Standing for ‘Virtual Design Lab’, the vLab is a small team comprised of Living Goods, Medic Mobile, Accenture Development Partnerships, and the Busara Center for Behavioural Economics.

This partnership is a key part of Medic’s Community Health Toolkit (CHT),

The vLab’s purpose is to ensure that the CHIN benefit from community engagement and ensure a human-centred approach in all health interventions, despite the travel restrictions necessitated by the COVID-19 pandemic.

The vLab uses a sprint approach and virtual collaboration tools to remotely design, prototype and test innovations in communities. Methods used include A/B Testing different SMS workflows to gain insight into the most effective designs, using Behavioural Economics to nudge clients into health-seeking behaviours, and running (remote) outreach interviews and focus groups to gain qualitative feedback.

Using the vLab to respond to COVID and maternal, newborn and child health (MNCH)

A part of our COVID response has been an innovation we are testing through the vLab called CIHA (the Client-Initiated Health Assessment). We have been testing CIHA in peri-urban and rural regions across Kenya.

CIHA is a self-triage platform that allows users to conduct a self-assessment by indicating their symptoms on a simple SMS workflows, without waiting for a CHV to visit. The platform then analysis their response and predicts the likelihood of a certain illness. It aims to empower caregivers and household members allowing them to screen themselves for certain illness and initiate access to healthcare.

Within CIHA, we have developed self-screening workflows for COVID-19, ICCM (Malaria, Pneumonia, Diarrhoea), Antenatal Care (ANC) and Postnatal Care (PNC). Once a user has responded to the SMS prompts, depending on the diagnosis, a task for a home visit is triggered, or they are given the option to reach out to their link CHV via free SMS.

In total, 1194 households were sensitized on CIHA through system prompts, 119 users have responded to these messages and 23 have received follow ups as a result (CHVs only refer and follow-up suspected cases).

Our Findings

Through an intensive series of design and test sprints, we have uncovered the following pieces of insight:

  • The CIHA workflow showed a positive response rate of 65%(excluding opt-outs and spam) with a workflow completion rate of 70%.
  • Response rate ranges for COVID workflows range from 6% – 13% and completion rate ranges from 3%- 11%. From our qualitative testing, this is most likely due to mistrust of COVID-related messages. Clients frequently view these as being part of a government contact-tracing scheme and are suspicious of responding as they fear having to forcibly quarantine.
  • Wednesdays and Fridays are the most optimal days to engage with clients:

  • Afternoons and evenings are the most optimal times to engage with clients:
  • Client education levels affect the amount of time taken to finish workflows: clients with a post-secondary school level education completed the workflows more than twice as fast (7 minutes) as those without (15 minutes).
  • We have been able to demonstrate improvements to uptake by the following behavioural nudges:
    • Day & Time of Engagement – data showed that clients are more responsive and willing to complete an assessment in the afternoon and towards the end of the week
    • Message Language – better response rate on messages written in Swahili but generally there was no significant difference in the completion rate between the two languages
    • Message Structure – A clear call to action in a message gives better response and completion rate
    • Behaviour change communication by distributing a sticker with health education and a helpline.
    • Incentivising by awarding letters of recognition to CHVs

Overall, although due to a complex social landscape, COVID SMS interventions are difficult to drive high levels of workflow completion. Additionally, as we are not integrated with the Kenyan Government’s own test system, we are unable to provide COVID-19 tests based on CIHA diagnoses: a key value proposition for such a service.

However, responses to ICCM, ANC and PNC workflows have been far more promising, with 77.5% of initiated self-assessments being completed.

Finally, the vLab is proving that a working self-assessment flow can be configured and deployed successfully and quickly. The user journey is functioning, with clients able to fill in their symptoms and receive follow ups from CHVs.

Thanks to Beatrice Wasunna, Maria Ma, Franciscah Nzanga, Isaac Mwangi, Stephen Odindo, Pooja Gupta, Tabitha Mberi, Maryanne Mureithi and Nick Valenzia, with leadership support from Isaac Holeman (Medic Mobile) and Andrew Karlyn (Living Goods).

Rapid Sprints: Our Process

The Lab is not a physical location or permanent team, but a framework of design processes and principles that will enable a lean, agile team to use Living Goods’ existing structures to rapidly prototype, test and iterate early-stage innovations.

The high level vLab process is as follows:

The work of the vLab has given us a strong platform to run a series of tests using behavioural nudges:

  • ‘Messenger Effect’: Having messages appear as ‘from your Local CHV’ to build trust with clients
  • ‘Gain vs. Loss Framing’: Messages emphasise benefits (‘this is important for having good health’) or costs ‘you could risk ill-health’
  • ‘Identified vs Statistical Lives’: Messages identify personas similar to the client ‘Grace recently took this test’ or notify that others in the community are participating ’32 other people in your town took this test’

Learnings for Other organisations

Other implementors working to deliver health information and assessments via SMS should be aware of the following:

  1. COVID Interventions are complex

The level of misinformation around COVID has created a difficult landscape to remotely engage with clients about their health. Clients frequently either didn’t believe that COVID existed, believed it was a Western/urban problem, or mistakenly feared losing their livelihoods should they be forced to quarantine. This contributed to low engagement numbers when COVID was named in an SMS.

  1. Trust comes first

Clients will not respond to messages if they are not evidently from a trusted source, such as ‘your Local CHV’. This is more crucial than ever, due to the mistrust and stigma surrounding COVID.

  1. Beware message fatigue

Clients have reported receiving multiple messages (up to 10) messages in a day from various providers. Health messages can be drowned out amidst this noise, so implementors think carefully about how to design their message language in order to give them the best chance of being noticed.

  1. Time is of the essence

Implementors should think carefully about the time of day, and day of the week to send messages. Late afternoons and evenings on Wednesdays and Fridays were optimal for our client population.

  1. Phones and SIMs are frequently shared between families.

Careful analysis needs to be run on who to target with health messaging and when: the father as the traditional head of the family, or the mother as the head of the household.

  1. Network Connectivity issues

Network connectivity and electricity access issues must be considered for the successful deployment of a similar intervention

What’s next for the vLab?

Our work with the vLab is in full swing through March 2021. Look out for further updates on our work in the New Year!

Back to Top

We use cookies to improve user experience and analyze website traffic. By using our site, you agree to our use of cookies. You can change your cookie settings at any time through your Interent browser. Please read our Privacy Policy for full details.