California implements new AI and software regulations for insurers

On September 28, 2024, California enacted SB 1120, which regulates the use of artificial intelligence (AI), an algorithm, or “other software tool” in utilization review and utilization management (UR/UM) functions by healthcare service plans (HCSPs) or disability insurers (and their contractors). SB 1120 amends California’s UR/UM laws governing HCSPs and disability insurers to define permissible uses of AI, algorithms, or other software tools in prospective, retrospective, or concurrent UR/UM functions.

Among other requirements, the amendments establish that (i) only a licensed physician or healthcare professional may make a medical necessity determination, (ii) no AI, algorithm or software tool may deny, delay, or modify healthcare services based, in whole or in part, on medical necessity, and (iii) such UR/UM decision must be based on any enrollee/insured’s own healthcare and medical record information, and not “solely on a group dataset.”

We take a closer look at SB 1120’s requirements below.

Overview of SB 1120’s amendments

SB 1120 amends both the California Health and Safety Code and the Insurance Code statutes governing prospective, retrospective, and concurrent UR/UM functions by HCSPs and disability insurers (and their contractors) for requests by providers for healthcare services for enrollees or insureds, based, in whole or in part, on medical necessity. Additionally, for HCSPs, this law also extends to delegation arrangements with medical groups, independent practice associations, or other contracting providers.

As amended by SB 1120, California law now imposes standards and restrictions on the use of AI, algorithms, and any “other software tool” (which is not defined) for the purpose of UR/UM functions.

In particular, the amendments establish that:

• Only a licensed physician or healthcare professional may make medical necessity determinations. That physician or professional must be competent to evaluate the specific clinical issues involved in the requested healthcare services.
• Any use of AI, algorithms, or other software tools must not deny, delay, or modify healthcare services based, in whole or in part, on medical necessity.
• Determinations made by AI, algorithms, or other software tools must be based on the enrollee or insured’s specific healthcare information and records, rather than solely on group datasets. The amendments limit the permissible information and records to the following, as applicable:
  • An enrollee or insured’s medical, or other, clinical history
  • Individual clinical circumstances as presented by the requesting provider, and
  • Other relevant clinical information contained in the enrollee or insured’s medical, or other, clinical record.

In addition to these rules, the amendments establish further requirements, including that:

• The criteria and guidelines for using these technologies comply with applicable state and federal law
• The technologies do not discriminate, directly or indirectly, against enrollees or insureds in violation of state or federal law
• The technologies are “fairly and equitably applied,” including in accordance with any applicable federal regulations and guidance, and
• The technologies must not directly or indirectly cause harm to the enrollee or insured.

The amendments also impose disclosure obligations, and subject the technologies to requirements concerning inspection and review.

These amendments will become effective January 1, 2025.

Please reach out to the authors for more information.