AI governance in clinical practice

Why is AI governance needed?

As with any technology or process you introduce into your clinical practice workflow, it’s important to have an officially documented and internally approved policy for the use of an AI. Such a policy generally covers the adoption, implementation, use, monitoring, and review of AI technologies. It aims to improve the quality of care and mitigate the occurrence of medico-legal incidents in a medical practice.

Case study

• With the consent of your patients, you start using an appropriate AI scribe tool within consultations
• You use the tool to transcribe consultations and generate clinical notes, which you review before transferring into the medical record and deleting the transcribe
• When reviewing a patient’s notes, you notice a “suggested investigation” that you did not request
• After investigating the scribe tool, you find that the AI scribe tool is set to auto-update, which has introduced new features, enabled by default, including “suggested investigations” that auto-populate into your transcription notes

Result

• You contact MDA’s medico-legal advice team, who highlight the potential risks of AI tools creating clinical judgements and potential ambiguous authorship of medical records
• MDA advises you to audit all your clinical notes, from the date of update up until the present
• When reviewing a patient’s notes, you notice a “suggested investigation” that you did not request

 

What is an AI governance policy?

An AI governance policy can include information and guidance for your organisation that:

• Helps doctors and support staff understand the capabilities and limitations of the technology through processes of data gathering and analysis
• Sets timeframes for reviewing procedure and mechanisms for implementing changes
• Identifies organisational risks and details plans for reporting and addressing potential incidents, including internal accountability and external support for crisis resolution

Case scenario: how MDA can support your practice

• MDA also assists by providing you with MDA’s guide on AI Governance Policy in Clinical Practice
• This guide provides a structured framework to support the legal, ethical, safe, secure, and effective use of artificial intelligence (AI) technologies within clinical practice and healthcare organisations
• It applies to both healthcare-specific AI technologies and general-purpose AI tools that may be used in connection with clinical, administrative, educational, research, or operational activities
• This framework helps you identify areas for improvement, including implementation controls, monitoring and incident management and periodic review of use of AI tools

What is silent scope creep in AI technology?

Silent scope creep happens when an AI tool begins introducing features outside its original scope, without explicitly flagging this change in the system.

It's important to remember that AI scribes are a class of non-regulated technology (unlike AI diagnostics, which need to be TGA approved). This means that developers can alter a program after you purchase it, introducing or remove features, which then update automatically.

This is a feature (or pitfall) of all modern consumer software, which are increasingly marketed on a subscription basis, and highlight the need for continuous review of all technology that is introduced to clinical practice.

If you have any questions about using AI in your practice, you can contact MDA's medico-legal experts on 1800 011 255 or email advice@mdanational.com.au.

For more information on clinical governance standards in Australian healthcare, visit https://www.safetyandquality.gov.au/national-standards/nsqhs-standards/clinical-governance-standard