California lawmakers seek to lead in regulating business uses of AI

Following efforts by the Biden Administration and the European Union to manage the development of AI – through the Blueprint for an AI Bill of Rights and the Artificial Intelligence Act, respectively – California has now embarked on its own journey to regulate the use of AI tools.

Several provisions in the Golden State’s developments to rein in the Golden Age of AI have critical implications to businesses broadly. These include (1) mandatory pre-deployment disclosures to California regulators, (2) mandatory annual impact assessments and (3) a private right of action with monetary damages.

Summary

California’s version, Assembly Bill 331 (AB 331), would add a new chapter to its Business and Professions Code, requiring certain developers and deployers of AI to test and report on their algorithms annually to the California Civil Rights Department, starting in 2025. It would also require deployers to notify individuals when they are the subject of automated processing for “consequential decisions” (referred to in the bill as an “automated decision tool,” or ADT), giving individuals a right to opt out where those decisions are entirely automated and “if technically feasible.” 

The bill would authorize public attorneys, specifically the California Attorney General, city attorneys and local prosecutors, to bring civil suits for alleged violations of AB 331. In addition, beginning on January 1, 2026, private individuals would also be able to bring civil actions against deployers for alleged algorithmic discrimination, and may be awarded compensatory damages and attorney’s fees.

Most interesting, perhaps, are legislative revisions on the causation standard regarding such computerized discrimination. Early drafting prohibited use of ADTs “in a manner that contributes to” algorithmic discrimination. The latest version redefines a violation as an ADT “that results in” such harms.

Even as California explores its response to the brave new world, there is room for rebalancing in the quest to manage risks and benefits.

Key additional details

With its main provisions coming into effect on January 1, 2025, AB 331 would apply to “developers” and “deployers.” A developer is a person, partnership, state or local government agency, or corporation that designs, codes, or produces an ADT, or substantially modifies an AI system or service for the intended purpose of making, or being a controlling factor in making, “consequential decisions,” whether for its own use or for use by a third party. A deployer, meanwhile, is a person, partnership, state or local government agency, or corporation that uses an automated decision tool to make a consequential decision.

And, to summarize, the term “consequential decision” includes decisions with material impact on an individual’s employment, education, housing, essential utilities, family planning, healthcare, financial services, interactions with the criminal justice system or legal services and voting.

A developer or deployer would have to conduct impact assessments, annually and whenever there is a significant update, that explain an ADT’s purpose, benefits, data collection practices, uses and potential adverse impacts as well as a developer or deployer’s algorithmic discrimination safeguards. These requirements apply to all developers, but only those deployers with either at least 25 employees or whose ADT impacted at least 1,000 people in the prior calendar year.

Developers and deployers would also have to maintain a governance program, designate an employee to oversee and maintain compliance, and make a publicly available clear policy that summarizes the types of ADTs used and how algorithmic discrimination is managed. These requirements also apply to all developers, and again only to those deployers with either at least 25 employees or whose ADT impacted at least 1,000 people in the prior calendar year.

In addition, deployers would be required to accommodate opt-out requests if technically feasible and would be prohibited from using an ADT in a manner that results in algorithmic discrimination.

Lastly, the bill would allow the California Civil Rights Department to impose administrative fines, authorize public attorneys to bring civil actions against any violation of AB 331, and, beginning in 2026, provide a private right of action against deployers for algorithmic discrimination.

Key takeaways

• AB 331 goes well beyond regulation of businesses which specialize in developing ADTs. It would govern the use of ADTs by all but the smallest businesses.
  • This would include the use of ADTs in current commonplace contexts, such as employment (ie, hiring), insurance and consumer products (eg, Internet of Things devices).
• Under AB 331, civil regulators would be able to seek injunctions of the use of ADTs beginning in, and a private right of action would be available to individuals starting in, 2026.
• Companies should prepare for the potential of complex restraints on their use of AI as laws like AB 331 are enacted in other states.

For more information, please contact the authors.