AI risks over-reliance on robotic decision-making

The five biggest threats posed by artificial intelligence are:

• Inaccurate outputs,
• Legal and reputational damage,
• Data-protection breaches,
• Cybercrime, and
• Over-reliance on automated decision-making.

That’s according to a global survey of business leaders.

James Wall, executive director of cyber technology at Gallagher Specialty, presented the findings of the insurer’s 2026 AI Adoption and Risk Benchmarking survey at the ALG Corporate Crime and Regulation Summit (14 October).

The survey found that AI adoption was accelerating faster than organisations’ ability to put appropriate governance frameworks in place.

Weaknesses

Other key findings included:

• 63% of businesses surveyed were using AI in 2026 – up from 45% in 2025,
• 86% reported a positive impact on workforce productivity, while 82% said AI was already having a positive impact on their business,
• Weaknesses remained in risk management, accountability, staff training, and human oversight,
• More than half of respondents reported significant skills gaps in implementing AI, and
• Businesses took an average of 28 months to achieve a full return on their AI investment.

In his introduction to the Croke Park event, Kenan Furlong, partner in A&L Goodbody, said AI was the summit's main theme because it was the most requested.

Expanded use of technology had brought great opportunity, Furlong said, but also great risk, such as the potential loss of privilege if confidential material is entered into non-confidential AI models and how firms developing their own tools could incur additional obligations under the EU AI Act.

He flagged AMLA’s upcoming guidelines as “the gold standards that regulators and law enforcement will expect you to comply with in your AML applications”.

Growing scrutiny

Furlong also pointed to increased regulatory enforcement, and growing scrutiny of individual conduct, including the possibility of social-media activity affecting suitability for senior roles.

James Wall joined ALG lawyers, Stephen D’Ardis (partner), Laura Mulleady (partner), Chris Bollard, and Catherine Moloney on a panel that used a fictional AI failure to map the steps required in such an event.

The scenario involved the use of an AI chatbot to categorise complaints in a financial-services company.

Six months after launch, a Central Bank thematic review has discovered that, because of a linguistic ambiguity, every complaint has been categorised as low priority.

The panel’s central message was that firms needed to be able to respond decisively before they had all the answers and that AI governance must be treated as an integral part of business risk management.

The priority should be to establish what has happened and prevent further harm.

A rapid assessment should establish how the system operated, how it malfunctioned, and the extent of the damage.

The immediate response should include:

• Containment: consider taking the tool out of service while the cause of the malfunction is investigated,
• Continuity: how will the function be replaced without introducing new risks,
• Legal exposure: establish the scale and materiality of the problem and assess potential legal liabilities,
• Communication: inform affected customers promptly and engage with the Central Bank, demonstrating that decisive steps are being taken,
• Escalation: keep the board informed as technical information becomes available, following agreed escalation procedures.

Incident response

Action should not be delayed until the full picture emerges, and a well-rehearsed AI incident response plan should set out communications arrangements, escalation triggers, and who has authority to make key decisions.

From an insurance perspective, James Wall said that two early calls should be made:

• External regulatory counsel, to advise on regulatory requirements, reporting obligations, and remediation,
• Specialist technology forensic investigators, to establish why the system failed, how the malfunction occurred, and what needed to change.

The insurance broker should also be notified promptly, in line with the relevant policy and incident-response arrangements.

The panel said that regulators would want more than a general explanation of what went wrong.

They would want to understand the prompts that led to the malfunction and, crucially, what had been done to prevent a recurrence.

An AI incident should not be treated as a single-regime legal problem.

The company’s role under the EU AI Act should first be established.

A firm using a licensed or procured system will generally need to consider its obligations as a deployer, while developing a system itself or placing it on the market under its own name could also bring it within the definition of a provider.

That assessment should be revisited as further facts emerge.

The wider regulatory framework applying to a financial-services business must also be considered – including conduct, governance and oversight requirements.

If relevant, the impact on consumers must be central to the response.

Firms need to understand not just which AI systems they use, but what those systems are being used to do.

The complaint-triage system in the scenario was not considered a prohibited AI practice under article 5 of the AI Act, nor was complaint triage itself considered to fall within the act’s high-risk categories.

However, the position could change if the system were repurposed, for example, to assess insurance eligibility.