How would you feel if the decision about how many courses of chemotherapy were to be allocated to your cancer treatment was made by an Artificial Intelligence (AI) agent?
Not just a number: A new survey was conducted by Macquarie philosophers to understand if people feel dehumanised when it comes to integrating AI into healthcare.
Many clinical settings already use different applications of AI to analyse test results, speed up diagnoses and even guide treatment decisions – but not everyone is comfortable with outsourcing decisions about their healthcare to a machine.
“We need to think more deeply about the impact on people and about the way they feel about AI making various decisions about their health,” says Associate Professor Paul Formosa, a philosophy and ethics scholar at Macquarie University.
If having AI involved in their healthcare makes people feel dehumanised, patients may struggle to accept its decisions or recommendations, no matter how accurate, or efficient, Formosa says.
“Image recognition is something AI is very good at – and there’s examples where AI trained on hundreds and thousands of images of eye retinas, for example, can perform as well or even better than humans in detecting certain diseases,” he says.
However just because AI is effective at some functions – doesn’t mean it is warranted in all circumstances, he says.
Preferring humans
Formosa and fellow researchers recently surveyed more than 470 people about different healthcare scenarios involving AI or human decision-makers, asking them whether they felt they were treated in a dignified and respectful way.
Respondents showed some clear preferences about how and where they would like to see AI used in health care, he says.
People are concerned that AI, compared to humans, can’t account for their uniqueness, and that some things can’t just be reduced to a number.
“People have a general preference for ‘assistive AI’, where the AI is part of the decision-making process, rather than autonomous, where it’s making decisions without a clinician,” Formosa says.
They also preferred to have a human decision-maker where a diagnosis was made, he says.
“People are concerned that AI, compared to humans, can’t account for their uniqueness, and that some things can’t just be reduced to a number.”
However, there were signs that people were less concerned when AI was involved in decisions about resource allocation – for example, in getting an appointment with a specialist, because it was seen as fair or impartial – provided that the outcome was positive – in other words, that they were able to obtain an appointment.
Reducing Stigma
Formosa says that the results summarise the majority of preferences – but the area is complex and there are some patients, and some situations, where AI may be preferred.
Acceptable: The survey found people were less concerned about AI involvement in healthcare when performing tasks such as making an appointment.
For example, research shows that many people have negative reactions to seeking medical treatment for things that they feel they will be judged on – such as illnesses caused by smoking, or mental illnesses.
“People may be more comfortable interacting with an AI for these conditions,” he says, adding that this is an area for future research.
Perceptions of being ‘dehumanized’ can also depend on how the AI is integrated into the healthcare process, he says.
“If you only interact with an AI – it takes your symptoms, delivers diagnosis and treatment decisions – that could be quite dehumanising; but if you give your symptoms to a human doctor, who then sends off the data to the AI, receives the results and diagnosis, and then the human doctor presents the results to you personally – then the perception can be quite different,” he says.
Skill shortage
Formosa says that before integrating AI into healthcare, we need to step back and consider the more general question: should we offload ethical decisions to AI and machines?
“We need to drill down and work out what scenarios are fitting for AI, and where is it not appropriate.”
Offloading certain roles to AI can have broader impacts, he says – including a decision about what skills we prioritise for humans to retain.
“In diagnostic cases, for example, if doctors give over certain tasks to AI then they may lose those skills, so that also means we are making decisions about what skills really matter. And over time, this could impact the safety of these technologies, too.”
is an Associate Professor in the Department of Philosophy and a member of the
