Civil liberties groups caution against public bodies using AI

The Oireachtas committee on artificial intelligence will hear from the ICCL an IHREC on Tuesday, October 21. Stock picture

The use of artificial intelligence (AI) across the public service in Ireland carries “high risks”, civil liberties bodies are set to tell the Oireachtas on Tuesday afternoon.

In submissions to the Oireachtas committee on artificial intelligence on Tuesday, the Irish Human Rights and Equality Commission (IHREC) and Irish Council for Civil Liberties (ICCL) are set to warn that the State is lagging behind in terms of living up to its responsibility to use AI responsibly.

IHREC chief commissioner Liam Herrick is to tell the committee — which has been meeting with representative groups from across the social spectrum to discuss the now-prevalent technology — that using AI is particularly risky when it comes to making “decisions about essential entitlements and supports”. Mr Herrick is set to say: 

We have seen examples from other jurisdictions, including the Netherlands, where flawed AI systems led to serious and systemic rights violations in welfare provision. 

That statement refers to a multi-year AI algorithm project put in place by the municipal authority of Amsterdam, which sought to use AI in the fairest manner possible to evaluate and identify welfare applications for review which were potentially problematic.

Irish Council for Civil Liberties executive director Liam Herrick. File picture: Robbie Reynolds

Once the algorithm was deployed, it quickly became apparent that it was more likely to wrongfully flag the parents of children for investigation, broadly understood to have resulted from the algorithm having been trained on previous welfare case files, which themselves would have been dependent upon the sometimes-flawed decision-making of human inspectors.

Mr Herrick is expected to tell the committee: 

The risks associated with AI are not evenly distributed, and discriminatory outcomes are being documented across a range of protected characteristics, including gender, disability, race, family status, and age. 

“It is essential that the roles, responsibilities, and powers of each designated body are clearly defined and supported, to ensure accountability and enforcement.”

Separately, the committee will hear that the State has a responsibility to properly inform the public in terms of how and to what extent it is using AI as part of its working processes.

Kris Shrishak, senior fellow with the ICCL and an expert in the field of AI and its impact on fundamental civil rights, is set to tell the committee that the council has uncovered issues in terms of how the State has procured AI solutions in the past.

With regard to a chatbot procured by the Department of Justice, he is to say the department refused to name the vendor due to it being “commercially sensitive”.

In terms of due diligence prior to the acquisition of three separate chatbots, the same department “did not run a tender process, performed no risk assessment, no bias tests, and no environmental impact assessment”, Mr Shrishak is due to say, information the council gleaned via a series of freedom of information requests.

“The Department of Public Expenditure ... should provide clear guidance on the procurement of AI systems and services,” he is due to say, adding that a publicly accessible central register of all the algorithms used by public bodies should be made available by the Government.

The Government last month announced the formation of a National AI Office, which is set to go live next year, a move which the Department of Enterprise declared would see Ireland ‘lead the way in EU AI regulation’.

However, Mr Shrishak will suggest that there is a “huge dearth” of AI expertise among the nine fundamental rights bodies — including Coimisiún na Mean and the Data Protection Commision — which will represent the new office.