AI in healthcare is here to stay. How do we make sure it’s safe?

Hiqa has launched Ireland’s first national guidance on the use of AI in health and social care, which aims to help services and staff to implement AI tools in a way that ensures accountability, human judgment and person-centredness, writes Rachel Flynn, its director of health information

AI can reduce the administrative burden and increase efficiencies in the healthcare system by improving workflows and automating administrative tasks, potentially leading to shorter waiting time for appointments or fewer cancellations.

If you have been to the GP recently, you may have noticed a sign reading 'Ambient AI is in use'. A GP or nurse using AI to help with notetaking and decision support so they can focus more fully on the patient in front of them, while not entirely without risk, is generally understood and accepted, particularly at a time where there is a shortage of healthcare professionals and patients may have complex needs.

But what about situations where AI is given an enhanced role, such as reading scans and making diagnoses in healthcare, or monitoring residents in social care settings to assess and forecast potential risks to their wellbeing?

While AI is a promising tool to help address some of the challenges in our health and social care system — an ageing population, increasing rates of chronic and complex disease, a shortage of health and social care workers — the idea that an AI tool would become more involved in decisions that affect your care is still an uncomfortable thought for many.

It’s against this backdrop that Hiqa has launched Ireland’s first national guidance to inform the use of AI in health and social care, which aims to help services and staff to implement AI tools in a way that ensures accountability, human judgement and person-centredness.

There are, after all, many benefits to using AI tools in these settings — AI can support more personalised care, identify important patterns, and provide humans with essential information and analysis to aid quick decision-making. 

It can also reduce the administrative burden and increase efficiencies in the healthcare system by improving workflows and automating administrative tasks, potentially leading to shorter waiting time for appointments or fewer cancellations.

It is also undoubtedly here to stay — the Department of Health recently published the first national strategy dedicated to the safe, responsible, and effective application of artificial intelligence in health and social care, which focuses on the deployment of AI over the next five years. 

Rachel Flynn: 'The adoption of AI tools in health and social care is a significant transition, but it’s one that can be made more positive when it is guided by best practice, underpinned by transparency and accountability, and centred around the needs and voice of the patient.'

Also part of this system-wide approach to AI in health and social care is the HSE’s AI Implementation Framework, which provides a toolkit for implementing AI projects across the HSE.

However, the use of AI in health and social care is not without its risks. The EU Artificial Intelligence Act, which provides a comprehensive legal framework for AI, categorises AI tools by their level of risk, which for the type of tools typically used in health and social care settings, is classified as high. 

In Ireland, the EU AI Act will be implemented through the Regulation of Artificial Intelligence Bill 2026, which recently passed through the Seanad and will establish Ireland's AI regulatory structures and create a new AI office.

The potential perils range from the use of data without a person’s consent, to staff uncritically accepting AI outputs without human oversight, the introduction of bias, or even deskilling of the workforce due to overreliance on these exact tools. In this context, it’s important that we use them in the right way.

Hiqa has the unique role of promoting safety in health and social care, and of developing tools to help services and staff implement national standards in these settings. Our AI guidance builds on national standards and is informed by our existing work in this area.

To help services and staff navigate the new opportunities and challenges that AI brings, we have structured our AI guidance into four key sections, based on our key principles for person-centred care and support: accountability, a human-rights based approach, safety and wellbeing, and responsiveness.

In the case of principle two, a human-rights based approach, we highlight the importance of maintaining human connection, and of ensuring people can make informed decisions based on their own needs and preferences about the use of AI in their care. 

In most cases, this means staff should take the time to explain to patients and residents exactly which AI tool is being used, why it’s helpful and what personal data it will require. It also means being open to questions and exploring alternative options as needed.

Over the last year, we’ve spoken with service providers, staff, members of the public, policy-makers and researchers, whose feedback has been invaluable in making this guidance practical and relatable for people working and participating in health and social care settings. 

While we hope it will educate and support staff to feel confident in their ability to use AI safely and responsibly, we also hope this guidance will help members of the public to feel more informed about what to expect from the use of AI tools in their care as we navigate further into the digital age.

The adoption of AI tools in health and social care is a significant transition, but it’s one that can be made more positive when it is guided by best practice, underpinned by transparency and accountability, and centred around the needs and voice of the patient.

• The guidance is now available at hiqa.ie. Rachel Flynn is Hiqa’s director of health information and standards