by Eric Dinallo, Avi Gesser, Matt Kelly, Samuel J. Allaman, Anna R. Gressel, Melissa Muse, and Stephanie D. Thomas

On May 26, 2023, the Colorado Division of Insurance (the “DOI”) released its Revised Draft Algorithm and Predictive Model Governance Regulation (the “Revised Regulation”), amending its initial draft regulation (the “Initial Regulation”), which was released on February 1, 2023. The Revised Regulation imposes requirements on Colorado-licensed life insurance companies that use external consumer data and information sources (“ECDIS”), as well as algorithms and predictive models (“AI models”) that use ECDIS, in insurance practices. The Revised Regulation comes after months of active engagement between the DOI and industry stakeholders. In this Debevoise In Depth, we discuss the Revised Regulation, how it differs from the Initial Regulation, what additional changes should be considered, and how companies can prepare for compliance.

As discussed below, there are several significant changes in the Revised Regulation, including:

• Documentation. Removing many of the onerous documentation requirements that were in the Initial Regulation;
• Board Oversight. Including a requirement that the board or board committee must oversee the risk management framework;
• Focus on External Data. Clarifying in several places that the requirements apply to AI models that use ECDIS, rather than all models;
• Scoping Bias. Limiting the scope of unfair discrimination to race, presumably in recognition of the difficulties in obtaining or inferring data for other protected classes, such as national origin and sexual orientation;
• Risk Assessment. Requiring insurers to develop a rubric to asses and prioritize risks associated with the deployment of ECDIS and AI models that use ECDIS;
• Confidentiality. Adding a new section providing that any documents or materials disclosed to the DOI as result of the Revised Regulation are subject to S. § 10-3-1104.9(3)(d), meaning that they are not subject to disclosure under the Colorado Open Records Act or similar open records laws; and
• Certification. Requiring insurers that use ECDIS and AI models that use ECDIS to submit annual compliance reports that are signed by an identified officer, or provide a corrective action plan if the officer cannot attest to full compliance with the regulation.

The DOI will discuss the Revised Regulation at its upcoming stakeholder meeting on June 8, 2023, from 11:00 – 12:00 pm MT. The Revised Regulation is open for public comment (due by June 8, 2023) and, following the meeting, stakeholders will have additional opportunities to submit written and oral comments to the DOI.

Overview of the Revised Regulation

Like the Initial Regulation, the Revised Regulation requires life insurers that are authorized to do business in Colorado to implement AI governance and risk management measures that are designed to ensure that the use of ECDIS and AI models that use ECDIS, in insurance practices, does not result in unfair discrimination.

Definition of ECDIS

The Revised Regulation expands what is meant by ECDIS by adding the underlined text to the definition that appeared in the Initial Regulation:

ECDIS means, for the purposes of this regulation, a data or an information source that is used by a life insurer to supplement or supplant traditional underwriting factors or other insurance practices or to establish lifestyle indicators that are used in insurance practices. This term includes credit scores, social media habits, locations, purchasing habits, home ownership, educational attainment, licensures, civil judgments, court records, occupation that does not have a direct relationship to mortality, morbidity or longevity risk, consumer-generated Internet of Things data, and any insurance risk scores derived by the insurer or third-party from the above listed or similar data and/or information source. Section 4(C).

Interestingly, the Initial Regulation provided a definition of “traditional underwriting factors” that included medical information, family history, income, assets and several other well-established criteria for underwriting life insurance, but that definition has been removed from the Revised Regulation.

Scope of Unfair Discrimination

The Revised Regulation removed the definition of “Disproportionately Negative Outcome,” which may be an effort to align the regulation with existing definitions and to narrow the regulation’s scope to unfair discrimination with respect to race. It is unclear whether this was intended to change how the regulation is applied or was merely an effort to simplify the regulation and add clarity.

“Disproportionately Negative Outcome” was defined in the Initial Regulation as “a result or effect that has been found to have a detrimental impact on a group as defined by race, color, national or ethnic origin, religion, sex, sexual orientation, disability, gender identity, or gender expression, and that impact is material even after accounting for factors that define similarly situated consumers.” At the time, we noted that in our view this was an effort to define proxy discrimination in a way that does not appear to require any intention on the part of the insurer.

With that definition removed, the core obligation of the Revised Regulation has been narrowed to focus only on racial discrimination, as opposed to the Initial Regulation, which focused on all the protected classes set forth in C.R.S. § 10-3-1104.9. It now provides that:

Life insurers that use ECDIS, as well as algorithms and predictive models that use ECDIS in an insurance practice must establish a risk-based governance and risk management framework that facilitates and supports policies, procedures, and systems designed to determine whether the use of such ECDIS, algorithms, and predictive models result in unfair discrimination with respect to race. (Emphasis added) Section 5(A).

The term “unfair discrimination” remains defined by Section 10-3-1104.9, C.R.S. as:

[T]he use of one or more external consumer data and information sources, as well as algorithms or predictive models using external consumer data and information sources, that have a correlation to race, color, national or ethnic origin, religion, sex, sexual orientation, disability, gender identity, or gender expression, and that use results in a disproportionately negative outcome for such classification or classifications, which negative outcome exceeds the reasonable correlation to the underlying insurance practice, including losses and costs for underwriting.

The decision to limit the scope of discrimination to race in the Revised Regulation is likely a reflection of the difficulty that insurers would have in either collecting or inferring data for other protected classes such as national origin, religion, or gender expression. In contrast, there are some semi-reliable methods for inferring race from other data points, like Bayesian Improved First Name Surname Geocoding (BIFSG).

Governance and Risk Management Obligations

The Revised Regulation provides that the governance and risk management framework must include the following components (some of which are unchanged from the Initial Regulation, some of which are changed, and some of which are new). All changes can be viewed in this redlined version of the Revised Regulation.

• Guiding Principles (changed). Insurers must have documented governing principles that provide guidance for ensuring that ECDIS (and AI models that use ECDIS) are designed, developed, used, and monitored in a matter that is well-suited for effective oversight and management and do not lead to unfair discrimination. Section 5(A)(1).
• Board Oversight (new). The board of directors or appropriate board committee must oversee the risk management Section 5(A)(2).
• Senior Management Accountability (changed). Senior management must be responsible and accountable for “setting and monitoring the overall strategy” on the use of ECDIS and AI models that use ECDIS. This includes establishing clear lines of communication and regular reporting to senior management regarding ECDIS risks. Section 5(A)(3).
• Cross-Functional Governance Group (largely unchanged). Insurers must establish a cross-functional algorithm and predictive model governance group (the term “committee” was replaced by “group” in the Revised Regulation) that is composed of representatives from “key functional areas” including legal, compliance, risk management, product development, underwriting, actuarial, data science, marketing, and customer service, as applicable. Section 5(A)(4).
• Policies (largely unchanged). Insurers must have written policies and processes, including assigned roles and responsibilities, for the design, development, testing, deployment, use, selection and oversight of vendors (this criterion was added in the Revised Regulation), and ongoing monitoring of ECDIS and algorithms that use ECDIS to ensure that they are documented, tested, and validated. Section 5(A)(5).
• Training (largely unchanged). Insurers’ policies and procedures must include an ongoing supervision and training program for relevant personnel on the responsible and compliant use of ECDIS that addresses issues. Section 5(A)(5).
• Cybersecurity (removed). The requirement in the Initial Regulation that insurers must have internal security controls in place to prevent unauthorized access to AI models is not included in the Revised Regulation. (formerly Section 5(A)(7)).
• AI Incident Response Plan (removed). The requirement that insurers must have a plan for responding to and recovering from any unintended consequences of AI usage is also not included in the Revised Regulation. (formerly Section 5(A)(9)).
• Consumer Complaints and Inquiries (largely unchanged). Insurers must establish processes for addressing consumer complaints and inquiries about the use of ECDIS and models that use ECDIS in a manner that provides “sufficiently clear” information so that consumers can take meaningful action in the event of an adverse decision. Section 5(A)(6).
• Risk Assessments and Prioritization (new). Insurers must establish a rubric for assessing and prioritizing risks associated with the deployment of ECDIS, as well as models that use ECDIS, in insurance practices with appropriate consideration given to consumer impact. Section 5(A)(7).
• Outside Auditors (removed). The requirement that insurers engage outside experts to perform audits when internal resources are insufficient is not included in the Revised Regulation. (formerly Section 5 (A)(10)).
• Vendor Risk Management (changed). Insurers that use third-party vendors for their ECDIS and models that use ECDIS remain responsible for ensuring compliance with the requirements in the Revised Regulation and must establish a process for the selection and oversight of these vendors. Section 5(B).

Revised Documentation Obligations

Many of the documentation obligations that were part of the Initial Regulation have been either removed or changed in the Revised Regulation.

• Inventory of AI Models (changed). Insurers are required to maintain an up-to-date inventory, which includes version control, of all utilized ECDIS, as well as models that use ECDIS, a detailed description of each, its purposes, and the outputs generated through their use. The Revised Regulation limits the inventory to AI models that use ECDIS and removes the requirement that the inventory contain the problems the use of ECDIS is intending to solve and any potential risks and appropriate safeguards. Section 5(A)(8).
• Documentation of Material Changes (changed). Insurers are required to maintain documentation that explains any material changes in the inventory, as well as the rationale for the changes. Section 5(A)(9).
• Bias Assessments (largely unchanged). Insurers must have a description of any testing conducted to detect unfair discrimination resulting from the use of ECDIS and models that use ECDIS, including the methodology, assumptions, results, and steps taken to address unfairly discriminatory outcomes. Section 5(A)(10).
• Monitoring (largely unchanged). Insurers must document ongoing monitoring regarding the performance of AI models that use ECDIS. Section 5(A)(11).
• Vendor Selection (largely unchanged). Insurers must document the process used for selecting external vendors that supply ECDIS or AI models that use ECDIS. Section 5(A)(12).
• Regular Reviews (largely unchanged). Insurers must conduct regular reviews of the governance structure and risk management framework and make appropriate updates to the required documentation to ensure its accuracy. Section 5(A)(13).

Several other documentation requirements that were part of the Initial Regulation, including descriptions of inputs, limitations, training data, how the model makes predictions, potential risks, and decisions made regarding the use of ECDIS, are not included in the Revised Regulation. (formerly Section 6(A)(1,5,6,8, and 12)).

Certification of Compliance

Once the Revised Regulation is finalized and goes into effect, insurers using ECDIS and models that use ECDIS will have: (1) six months to provide a report to the DOI summarizing the progress made towards implementing the requirements of the Revised Regulation; and (2) one year to submit a report summarizing compliance. These requirements are largely unchanged from the Initial Regulation. The Revised Regulation does provide two significant new requirements with respect to reporting to the DOI.

First, the report summarizing compliance now must be submitted annually. Second, the report must include:

• The title of each individual responsible for ensuring compliance;
• The specific requirement for which that individual is responsible;
• A signature of an officer attesting to compliance with the Revised Regulation; and
• In the event an insurer is unable to attest to compliance with this regulation, the insurer must submit to the DOI a corrective action plan. Section 6(B).

Takeaways

• Comments. Insurers should closely review the Revised Regulation and consider providing comments before the June 8 deadline. The changes made to the Initial Regulation (as reflected in the Revised Regulation) demonstrates that the DOI is willing to seriously consider constructive suggestions.
• Gap Analysis & Road Map. Insurers should consider conducting a gap analysis between the requirements in the Revised Regulation and their current AI and data governance and compliance program. After the gap analysis, insurers should consider developing a road map to compliance. For some companies that are covered by the Revised Regulation, it may take significant time and resources to fully implement these requirements, and so they may want to start early. And even companies that are not subject to the Revised Regulation may consider conducting a gap analysis in anticipation that these rules, or similar ones, could be adopted by other regulators in the coming years or will come to be considered best practices for AI governance and compliance programs.
• Risk Assessment. The Revised Regulation requires that insurers develop a rubric to assess and prioritize risks. Insurers should consider creating a list of high-risk factors ECDIS uses to identify what are high, medium, or low risk use cases. Those criteria can then be used to identify the highest-risk ECDIS and AI models that use ECDIS applications for prioritization and help create the road map to compliance.
• Cross-Functional Group. The regulation calls for the creation of a cross-functional group. Determining which representatives from “key functional areas” should be in the group, how often the group should meet, what resources it needs, to whom it will report, how it will make decisions, and how its decisions will be implemented are all complicated considerations that will take time and discussion.
• Budget. A final version of the regulation will likely take effect sometime in the next year, and many components of its obligations could require some companies to significantly increase their compliance budgets and secure additional resources.

Eric Dinallo and Avi Gesser are Partners, Matt Kelly is Counsel, Samuel J. Allaman, Anna R. Gressel, Melissa Muse, and Stephanie D. Thomas are Associates at Debevoise & Plimpton LLP. This post first appeared on the firm’s blog.

The views, opinions and positions expressed within all posts are those of the author(s) alone and do not represent those of the Program on Corporate Compliance and Enforcement (PCCE) or of the New York University School of Law. PCCE makes no representations as to the accuracy, completeness and validity or any statements made on this site and will not be liable any errors, omissions or representations. The copyright or this content belongs to the author(s) and any liability with regards to infringement of intellectual property rights remains with the author(s).