Regulator reviews AI usage to strengthen the rules

By Resolve Editor Kate Tilley

The Australian Securities and Investments Commission (ASIC) is reviewing use of artificial intelligence (AI) in the sectors it regulates, including insurance, because it acknowledges the current regulatory environment is inadequate.

The Federal Government’s interim report on AI regulation has said: “Existing laws likely do not adequately prevent AI-facilitated harms before they occur, and more work is needed to ensure there is an adequate response to harms after they occur.”

ASIC chair Joe Longo, speaking at the UTS Human Technology Institute’s Shaping Our Future Symposium in January, said ASIC’s review would help the regulator understand the gap between current governance and regulation and the ideal.

But he stressed AI was not an unregulated “wild west”. “The responsibility towards good governance is not changed just because the technology is new. There’s plenty of scope right now for making the best use of our existing regulatory toolkit,” Mr Longo said.

ASIC was “willing to test the regulatory parameters where they’re unclear or where corporations seek to exploit perceived gaps. That means probing the oversight, risk management, and governance arrangements entities have in place”.

Risk mitigation

The review would give ASIC “a better understanding of the actual AI use cases being deployed and developed in the Australian market – and how they impact [on] consumers. We’re testing what risks to consumers licensees are identifying from the use of AI, and how they’re mitigating against these risks.”

While transparency and oversight were needed to prevent unfair practices, accidental or intended, Mr Longo was “not so sure” the current regulatory framework was adequate.

“Does it prevent blind reliance on AI risk models without human oversight that can lead to underestimating risks? Does it prevent failure to consider emerging risks that the models may not have encountered during training?”

Mr Longo said all participants in the financial system have a duty to balance innovation with the responsible, safe and ethical use of emerging technologies.

AI hallucinations

The Australian Signals Directorate, in a January report, outlined several challenges around AI use, including:

Data poisoning – adversarial manipulation of an AI model’s training data so it learns incorrect patterns and may misclassify data or produce inaccurate, biased or malicious outputs.
Input manipulation – an attack that inserts malicious instructions or hidden commands into an AI system. Prompt injection can enable a malicious actor to hijack an AI model’s output and jailbreak the AI system by evading content filters and other safeguards.
AI ‘hallucinations’ – outputs generated by AI systems may not always be accurate; generative AI systems are known to hallucinate information that is not factually correct.
Privacy and intellectual property concerns.

Key challenges

A new white paper from global insurtech Xceedance highlights four key AI challenges the insurance industry should focus on in 2024.

Prateek Vijayvergia, Xceedance Business Leader – Key Accounts in Australia, says the industry will advance how generative AI is applied across claims, underwriting and other insurance processes.

“As AI moves beyond task automation and machine learning, insurers and underwriting agencies must focus on testing, workflow transformation, customisation, and integration to ensure they realise the benefits of AI.

“Concurrently, they will focus on ways to build trust in the technology, make it more accessible to everyone in the organisation, and pinpoint the return on investment,” he says.

The four challenges the whitepaper identified are:

A significant focus on generative AI testing. Many insurers have achieved great results from machine learning and automating tasks in claims and underwriting. But, without taking the next steps, they’ll fail to realise AI’s full potential. The challenge is to be comfortable with computers making choices, from level and type of coverage to claims recommendations. Achieving that level of trust requires continuous trials and tests across business processes and lines of business.

Insurance workflows, many of which were designed more than 60 years ago, must be reinvented. Underwriting questions, which insurers ask clients before providing quotes or policies, haven’t changed significantly in decades. Technology raises both challenges and opportunities. Successful insurers will combine technology, data, and AI and revise their processes from the ground up.

AI will enable insurers to embrace the power of one. Insurance models have historically considered clusters of risks in determining rates and coverage. The ability to quickly analyse large volumes of data enables that to change and some insurers are starting to assess risks individually, no matter how small. In some cases, they’re reaching different conclusions, considering business they may not have previously, and identifying new market approaches.

Integration of information sources will drive use. Generative AI requires a lot of data. However, there is no shortage of data sources available to insurers and more are being created every day, from telematics to geospatial data. The challenge for insurers will be integrating data sources into AI platforms so the information can be used for decision making.

Centre of Excellence

Mr Vijayvergia says an important element is the need to groom AI models with correct data and information by filtering out data that’s inappropriate, inaccurate or biased.

The Xceedance white paper, Discover the future of property & casualty insurance: 2024 and beyond,was compiled from discussions with insurers and underwriting agencies around the globe.

In late 2023, Xceedance established the Centre of Excellence for Generative AI and invited insurance industry organisations worldwide to supply use cases to workshop within the centre.

“Generative AI has the potential to transform every business, but it’s important to move forward with trust and knowledge,” Mr Vijayvergia says.