What happens when the software everyone is rushing to deploy is so risky that no one can guarantee it? That’s about to become clear, as the Financial Times reports.
Major insurance companies including AIG, Great American and WR Barclay are asking US regulators for permission to exclude AI-related liabilities from corporate policies. One insurance company described the AI model output to FT as “too many black boxes.”
This story reminds us that the industry has good reason to be scared. Google’s AI Overview falsely accused solar companies of legal issues in March, leading to a $110 million lawsuit. Air Canada stalled last year on honoring discounts devised by chatbots. And last year, fraudsters stole $25 million from London-based design and engineering firm Arup by using digitally cloned versions of senior executives during video calls that appeared to be completely authentic.
What insurance companies are really afraid of is not a single huge payout. It is a systemic risk that if a widely used AI model were to intervene, thousands of claims would arise at the same time. One Aon executive said the insurance company could handle losses of $400 million to one company. What they can’t deal with is an agent AI mishap that causes a loss of 10,000 at a time.
