Evaluations of Digital and Data-Driven Solutions in Healthcare

If you like all things healthtech, pizza and Pacman networking (you might not know what this one is yet, but I’m pretty sure you’ll like it!) then I’m sure you loved OHT Scotland’s third event: Evaluations of Digital and Data-Driven Solutions in Healthcare. But if you didn’t make it, fear not! We have a summary of the highlights here. by Elsie Horne

Firstly, all of us at OHT Scotland would like to say a massive thanks to Microsoft Edinburgh for opening their office to 50 healthtech enthusiasts for the evening, and especially to Andrew Ord, Peter Jamieson and Faith La Grange for hosting the event.

Faith, Director of Local and Regional Government at Microsoft, kicked off the evening by highlighting an exciting collaboration between Microsoft, Kensci, Storm ID and NHS Greater Glasgow and Clyde. These organisations are working together to harness digital innovation to transform the care pathway for people with chronic obstructive pulmonary disease (COPD), a condition which affects more than 1.2 million people in the UK. You can find out more about this collaboration on this Microsoft Industry blog post.

This was followed by 15-minute presentations from three speakers. First up was OHT Scotland’s very own Kathrin Cresswell, Chancellor’s Fellow and Director of Innovation at the University of Edinburgh’s Usher Institute of Population Health Sciences and Informatics. Kathrin’s presentation summarised previous approaches to large-scale, digitally enabled change in the NHS, and focused on the independent evaluation of Global Digital Exemplar programme currently being carried out at the University of Edinburgh. Kathrin highlighted the fact that we still don’t really know what digital excellence is for the NHS, let alone how best to get there. This challenge stresses the need for rigorous evaluation of health innovation systems, to make sure the assumptions we have made about the path to digital excellence hold true in practice. It can take many years to realise hard clinical outcomes, and Kathrin emphasised the need for regular qualitative evaluation along the way. She described qualitative evaluation as a means of avoiding a car crash before it happens!\

Next up was Michelle Brogan, Service Development Manager at the Digital Health and Care Institute, who spoke about her experiences as national lead for the Home and Mobile Health Monitoring (HMHM) portfolio. Michelle discussed some of the aspects which make the evaluation of health tech implementation so challenging: unstable environments; systems with many moving parts; health tech as an enabler rather than intervention; the time delay in realising hard clinical outcomes and financial benefits. She referenced the important work which aims to address these challenges, in particular, the nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) framework being developed by Trish Greenhalgh’s group. To find out more about the evaluation of the HMHM portfolio, check out this summary report of the Technology Enabled Care Programme.

Our final speaker, Alistair Connell, joined us live from San Francisco to speak about DeepMind Health’s Streams App, which is used to help clinicians better identify and treat acute kidney injury (AKI). Alistair worked as a nephrologist in London, so has the first-hand experience of the inefficiencies in the current AKI care pathways. He is now undertaking a PhD with DeepMind to examine the effects of this digitally-enabled care pathway for patients with AKI, facilitated by the Streams App. To find out more about the evaluation of the Streams App, check out the published protocol which provides a transparent framework for the evaluation. Alistair left us with a little teaser: there are currently three manuscripts in submission reporting the results of this study, so watch out for publications in the coming months!

While Kathrin, Michelle and Alistair gave accounts of evaluating three very different solutions from three very different angles, there were recurring themes throughout all of their presentations. Three themes that stood out for me were:

Incorporate qualitative evaluation from the start! This can help to address problems early.Do your groundwork! This is generally poorly done, but it’s important to know what baseline situation is before introducing a solution.Cost-benefit analysis is difficult but essential for a solution to scale.

These themes were reiterated in the Q&A sessions when Debbie Wake spoke about their mixed methods approach to evaluation, and some of the challenges they are facing in expanding My Diabetes My Way to NHS England.

Finally, what on earth is Pacman Networking?! Just make sure that when you’re huddled in a circle deep in conversation about health-tech and stuffing your face with pizza, that you leave a space in your circle for a new member to join (space in circle = Pacman’s mouth).* Thanks to Catherine Stables for suggesting this one – let’s hope it becomes a OHT tradition!

* Optional extension: shuffle round the room in your circle eating dots and avoiding ghosts.