In this guest blog, two NHS patients, and the AI Ethics and Governance Lead at the AI Centre for Value Based Healthcare (AIC) share what makes a successful patient collaboration when developing healthcare AI. The AIC is a consortium that develops and deploys healthcare AI, and proposals to do this are overseen by the Data Allocation Committee, to which the co-authors belong. The committee is comprised of patients, clinicians, data protection experts, and scientists, and they have been reviewing healthcare AI proposals for the last four years.
--
When it comes to healthcare AI development, collaborating with patients is an obvious, but often neglected, choice. Patients know the setting the AI will enter, so they know what would or would not improve the situation. It is also their health that pays the greatest cost for unreliable outputs.
We - Robin (AI Ethics and Governance Lead), Stephen and Clive (NHS patients) - have worked together for the past 4 years. We have reviewed NHS AI infrastructure plans, dug into the meaning of contracts, created guidance for AI developers, and successfully reviewed more AI development projects than we can count. Here are some of the important factors we have discovered for genuine and productive collaboration:
Power and expectation:
Let’s start by acknowledging the elephant in the room: talking at a patient does not mean you are talking with them. This is the threshold for collaboration. When Stephen reviews the quality of the public involvement in an AI project proposal, he will often say:
“You can tell if the involvement has been meaningful because something has changed after the conversation.
All too often a critical dialogue between patients and researchers is avoided because it is feared it will not be of real value. So, consultation skims over detail and prompts brushstroke responses with little relevance to a specific project. The more specific the consultation, the more likely it is that you will find changes are made where patient recommendations have helped fine tune”.
Once you’ve breached the collaboration threshold, there are different degrees of power. To test where the power is, ask who has final say on a decision. Does this level of power match how the collaboration is described? If you describe giving patients complete power, but in fact can override any decision you dislike, what does that say about the level of respect the project holds for the patients? Honest discussion about limitations of power and involvement is an important part of working with expectations for all parties when patient involvement is a real desire. It helps to develop trust which is a vital ingredient.
Sharing power must be thought of from the start of the project so it can be built into the funding plan. If you are unable to pay for their time, travel, and childcare, then you usually cut off a significant segment of the population.
Safe Spaces:
Giving patients power often means putting them in a room with other powerful people, who speak jargon and are assured they belong there. What do you imagine that feels like for some patients, especially ones who society has repeatedly told they don’t belong?
Uncomfortable, perhaps? Feeling like it is safer to stay quiet, maybe? Is it productive -or kind- to leave them feeling that way?
This is why we advocate for safe spaces. We have “debriefs” where patient members and Robin meet to discuss meetings in private. We go through unclear topics, and speak honestly and candidly to get to the root of a question. Robin will then take away any actions that emerge and, if preferred, share any points anonymously in the subsequent executive meeting.
We have also found just how much the chairing and planning of the executive meetings from a perspective of encouraging involvement, is a significant factor in successful collaboration. This is an active process which reinforces the ability of patients to challenge and question professionals in what can otherwise be an intimidating or excluding environment.
Transparency:
Imagine a developer has found a group of patients that want to collaborate on their AI development, and they open with a simple question:
How do you feel about Blerg-bot-24 deciding when your next GP appointment will be?
To answer that you probably need to know what Blerg-bot-24 is. How does it work? Is it overseen by GP staff? Why is this option being looked at?
A question that can seem simple to a developer can demand a lot more answers than they expected. This is where the power is tested. Say the developer stated the project will be led by the patient, but the patient, after learning how blerg-bot-24 works, thinks it will be a terrible idea. What do they do? They can build a new path to something their patients want in their care, strengthening the products value and their relationship with the patients, or they can ignore the patients insights, revealing the project was never a collaboration, but an attempt at getting patient backing. A large part of navigating collaborations is about engaging relationships with respect and ensuring expectations by all parties are accounted for.
Meaning and respect:
When I (Robin) got to know Clive he said something that has always stuck with me:
“Developers don’t realize that when they ask to learn from your cancer diagnosis, they are taking a piece of you. Every time”.
Imagine you went through something life-changing, like cancer, what would that be like? It is not just treatment. It is facing your mortality. It is isolation. It is exhaustion. It is wondering if you will walk your daughter down the aisle.
When a developer digs up that experience to understand how their product should work, what do they owe the patient? At the very least, they should know what they are asking of them - the exposure of a meaningful and potentially painful experience - and treating it with care. They should not, as all too often happens, go cold as soon as they have what they need.
The dialogue between professionals and patients must be seen as something ongoing where the lived experience of patients is given value throughout the research process. This is a recognition of the time and dedication which patients display by their willingness to engage and participate.
Further, some experiences will highlight unique sides of the illness the Developer is looking at. A clinician (as a patient) will probably have a different experience to a priest (as a patient). A western European will likely have a different experience from an eastern European.
We often talk about this difference in experience in terms of diversity, and there is growing recognition about how this can adversely impact on populations who may be traditionally missed out because of their race, and gender. Diversity is much greater than this and can impact the same people in complex ways which include gender, wealth and social status, disability, sexual orientation, racial origins, literacy, native spoken language to name but some.
It’s vital that we strive to capture the views of those who form these groups who have least influence, no matter how difficult this may prove to be. To do this we have to find ways of breaking down language barriers, addressing issues around how and when consultation with groups happen, and properly recognising that ignored or unreached groups have just as vital insights into research which will help to fine tune development and roll-out of services. The research community must be actively willing to accommodate and potentially change the way they work and see themselves in their relationship with public and patients.
Conclusion:
Respecting the inherent value in a patients lived experience can bring so much more than a nod of approval. It can change everyone involved. However, it is not easy. It takes planning, honesty, and sharing power. It can mean wandering into the unknown.
However, if you want patients to be brave enough to step into a room full of strangers, share an important experience and divulge their ideas, then you need to be brave enough to listen.