In November 2023, the California Privacy Protection Agency (CPPA) released a set of draft regulations on the use of artificial intelligence (AI) and automated decision-making technology (ADMT).
The proposed rules are still in development, but organizations may want to pay close attention to their evolution. Because the state is home to many of the world’s biggest technology companies, any AI regulations that California adopts could have an impact far beyond its borders.
Furthermore, a California appeals court recently ruled that the CPPA can immediately enforce rules as soon as they are finalized. By following how the ADMT rules progress, organizations can better position themselves to comply as soon as the regulations take effect.
The CPPA is still accepting public comments and reviewing the rules, so the regulations are liable to change before they are officially adopted. This post is based on the most current draft as of 9 April 2024.
The California Consumer Privacy Act (CCPA), California’s landmark data privacy law, did not originally address the use of ADMT directly. That changed with the passage of the California Privacy Rights Act (CPRA) in 2020, which amended the CCPA in several important ways.
The CPRA created the CPPA, a regulatory agency that implements and enforces CCPA rules. The CPRA also granted California consumers new rights to access information about, and opt out of, automated decisions. The CPPA is working on ADMT rules to start enforcing those rights.
As with the rest of the CCPA, the draft rules would apply to for-profit organizations that do business in California and meet at least one of the following criteria:
Furthermore, the proposed regulations would only apply to certain uses of AI and ADMT: making significant decisions, extensively profiling consumers, and training ADMT tools.
The current draft (PDF, 827 KB) defines automated decision-making technology as any software or program that processes personal data through machine learning, AI, or other data-processing means and uses computation to execute a decision, replace human decision-making, or substantially facilitate human decision-making.
The draft rules explicitly name some tools that do not count as ADMT, including spam filters, spreadsheets, and firewalls. However, if an organization attempts to use these exempt tools to make automated decisions in a way that circumvents regulations, the rules will apply to that use.
The draft rules would apply to any use of ADMT to make decisions that have significant effects on consumers. Generally speaking, a significant decision is one that affects a person’s rights or access to critical goods, services, and opportunities.
For example, the draft rules would cover automated decisions that impact a person’s ability to get a job, go to school, receive healthcare, or obtain a loan.
Profiling is the act of automatically processing someone’s personal information to evaluate, analyze, or predict their traits and characteristics, such as job performance, product interests, or behavior.
“Extensive profiling” refers to particular kinds of profiling:
The draft rules would apply to businesses’ use of consumer personal data to train certain ADMT tools. Specifically, the rules would cover training an ADMT that can be used to make significant decisions, identify people, generate deepfakes, or perform physical or biological identification and profiling.
As a California law, the CCPA’s consumer protections extend only to consumers who reside in California. The same holds true for the protections that the draft ADMT rules grant.
That said, these rules define “consumer” more broadly than many other data privacy regulations. In addition to people who interact with a business, the rules cover employees, students, independent contractors, and school and job applicants.
The draft CCPA AI regulations have three key requirements. Organizations that use covered ADMT must issue pre-use notices to consumers, offer ways to opt out of ADMT, and explain how the business’s use of ADMT affects the consumer.
While the CPPA has revised the regulations once and is likely to do so again before the rules are formally adopted, these core requirements appear in each draft so far. The fact that these requirements persist suggests they will remain in the final rules, even if the details of their implementation change.
Before using ADMT for one of the covered purposes, organizations must clearly and conspicuously serve consumers a pre-use notice. The notice must detail in plain language how the company uses ADMT and explain consumers’ rights to access more information about ADMT and opt out of the process.
The company cannot fall back on generic language to describe how it uses ADMT, like “We use automated tools to improve our services.” Instead, the organization must describe the specific use. For example: “We use automated tools to assess your preferences and deliver targeted ads.”
The notice must direct consumers to additional information about how the ADMT works, including the tool’s logic and how the business uses its outputs. This information does not have to be in the body of the notice. The organization can give consumers a hyperlink or other way to access it.
If the business allows consumers to appeal automated decisions, the pre-use notice must explain the appeals process.
Consumers have a right to opt out of most covered uses of ADMT. Businesses must facilitate this right by giving consumers at least two ways to submit opt-out requests.
At least one of the opt-out methods must use the same channel through which the business primarily interacts with consumers. For example, a digital retailer can have a web form for users to complete.
Opt-out methods must be simple and cannot have extraneous steps, like requiring users to create accounts.
Upon receiving an opt-out request, a business must stop processing a consumer’s personal information within 15 days. The business can no longer use any of the consumer’s data that it previously processed. The business must also notify any service providers or third parties with whom it shared the user’s data.
Organizations do not need to let consumers opt out of ADMT used for safety, security, and fraud prevention. The draft rules specifically mention using ADMT to detect and respond to data security incidents, prevent and prosecute fraudulent and illegal acts, and ensure the physical safety of a natural person.
Under the human appeal exception, an organization need not enable opt-outs if it allows people to appeal automated decisions to a qualified human reviewer with the authority to overturn those decisions.
Organizations can also forgo opt-outs for certain narrow uses of ADMT in work and school contexts. These uses include:
However, these work and school uses are only exempt from opt-outs if they meet the following criteria:
None of these exemptions apply to behavioral advertising or training ADMT. Consumers can always opt out of these uses.
Consumers have a right to access information about how a business uses ADMT on them. Organizations must give consumers an easy way to request this information.
When responding to access requests, organizations must provide details like the reason for using ADMT, the output of the ADMT regarding the consumer, and a description of how the business used the output to make a decision.
Access request responses should also include information on how the consumer can exercise their CCPA rights, such as filing complaints or requesting the deletion of their data.
If a business uses ADMT to make a significant decision that negatively affects a consumer—for example, by leading to job termination—the business must send a special notice to the consumer about their access rights regarding this decision.
The notice must include:
The CPPA is developing draft regulations on risk assessments alongside the proposed rules on AI and ADMT. While these are technically two separate sets of rules, the risk assessment regulations would affect how organizations use AI and ADMT.
The risk assessment rules would require organizations to conduct assessments before they use ADMT to make significant decisions or carry out extensive profiling. Organizations would also need to conduct risk assessments before they use personal information to train certain ADMT or AI models.
Risk assessments must identify the risks that the ADMT poses to consumers, the potential benefits to the organization or other stakeholders, and safeguards to mitigate or remove the risk. Organizations must refrain from using AI and ADMT where the risk outweighs the benefits.
California’s draft rules on ADMT are far from the first attempt at regulating the use of AI and automated decisions.
The European Union’s AI Act imposes strict requirements on the development and use of AI in Europe.
In the US, the Colorado Privacy Act and the Virginia Consumer Data Protection Act both give consumers the right to opt out of having their personal information processed to make significant decisions.
At the national level, President Biden signed an executive order in October 2023 directing federal agencies and departments to create standards for developing, using, and overseeing AI in their respective jurisdictions.
But California’s proposed ADMT regulations attract more attention than other state laws because they can potentially affect how companies behave beyond the state’s borders.
Much of the global technology industry is headquartered in California, so many of the organizations that make the most advanced automated decision-making tools will have to comply with these rules. The consumer protections extend only to California residents, but organizations might give consumers outside of California the same options for simplicity’s sake.
The original CCPA is often considered the US version of the General Data Protection Regulation (GDPR) because it raised the bar for data privacy practices nationwide. These new AI and ADMT rules might produce similar results.
The rules are not finalized yet, so it’s impossible to say with certainty. That said, many observers estimate that the rules won’t take effect until mid-2025 at the earliest.
The CPPA is expected to hold another board meeting in July 2024 to discuss the rules further. Many believe that the CPPA Board is likely to begin the formal rulemaking process at this meeting. If so, the agency would have a year to finalize the rules, hence the estimated effective date of mid-2025.
As with other parts of the CCPA, the CPPA will be empowered to investigate violations and fine organizations. The California attorney general can also levy civil penalties for noncompliance.
Organizations can be fined USD 2,500 for unintentional violations and USD 7,500 for intentional ones. These amounts are per violation, and each affected consumer counts as one violation. Penalties can quickly escalate when violations involve multiple consumers, as they often do.
The draft rules are still in flux. The CPPA continues to solicit public comments and hold board discussions, and the rules are likely to change further before they are adopted.
The CPPA has already made significant revisions to the rules based on prior feedback. For example, following the December 2023 board meeting, the agency added new exemptions from the right to opt out and placed restrictions on physical and biological profiling.
The agency also adjusted the definition of ADMT to limit the number of tools the rules would apply to. While the original draft included any technology that facilitated human decision-making, the most current draft applies only to ADMT that substantially facilitates human decision-making.
Many industry groups feel the updated definition better reflects the practical realities of ADMT use, while privacy advocates worry it creates exploitable loopholes.
Even the CPPA Board itself is split on how the final rules should look. At a March 2024 meeting, two board members expressed concerns that the current draft exceeds the board’s authority.
Given how the rules have evolved so far, the core requirements for pre-use notices, opt-out rights, and access rights have a strong chance to remain intact. However, organizations may have lingering questions like:
Whatever the outcome, these rules will have significant implications for how AI and automation are regulated nationwide—and how consumers are protected in the wake of this booming technology.
Explore data compliance solutions
Disclaimer: The client is responsible for ensuring compliance with all applicable laws and regulations. IBM does not provide legal advice nor represent or warrant that its services or products will ensure that the client is compliant with any law or regulation.
The post What you need to know about the CCPA rules on AI and automated decision-making technology appeared first on IBM Blog.
TL;DR A conversation with 4o about the potential demise of companies like Anthropic. As artificial…
Whether a company begins with a proof-of-concept or live deployment, they should start small, test…
Digital tools are not always superior. Here are some WIRED-tested agendas and notebooks to keep…
Machine learning (ML) models are built upon data.
Editor’s note: This is the second post in a series that explores a range of…
David J. Berg*, David Casler^, Romain Cledat*, Qian Huang*, Rui Lin*, Nissan Pow*, Nurcan Sonmez*,…