HHS finalizes significant changes to the ONC Health IT Certification Program focusing on AI transparency and interoperability

On December 13, 2023, the US Department of Health and Human Services (HHS), via the Office of the National Coordinator for Health Information Technology (ONC), finalized the Health Data, Technology, and Interoperability: Certification Program Updates, Algorithm Transparency, and Information Sharing (HTI-1) rule. The final rule builds on the proposed HTI-1 rule released in April 2023, which attracted significant public comment. 

Along with new interoperability and enhanced information blocking requirements, the HTI-1 final rule establishes groundbreaking transparency requirements for artificial intelligence (AI) and other predictive algorithms in certified health IT. The algorithmic transparency requirements specify the provision of baseline algorithm information, designed to allow clinical users to evaluate algorithm fairness, appropriateness, validity, effectiveness and safety (FAVES). ONC is eager to adopt this approach, promoting responsible AI in ONC-certified health IT – IT which ONC reports supports care delivery by more than 96 percent of hospitals and 78 percent of office-based physicians in the US.

Given the growing importance and promise of AI in healthcare, this alert focuses on summarizing the new algorithmic transparency requirements for certified health IT included within the 916-page final rule.

HTI-1 rule and algorithm transparency

The HTI-1 final rule implements statutory[1] provisions and Executive Order[2] directives, designed to advance interoperability, health equity, and transparency in health system IT. The final rule makes numerous updates to the ONC Health IT Certification Program, a program which includes various standards, implementation specifications, and certification criteria for health IT developers and health IT modules. As such, the changes will be of great interest to IT developers who have certified a health IT module under the ONC Health IT Certification Program.[3]

Notably, the final rule does not apply to healthcare providers (ie, health systems) that self-develop health IT not offered to others.

While many of the finalized rule’s provisions reflect a steady evolution of the Health IT Certification Program, the decision support intervention (DSI) provisions, setting out algorithm transparency requirements, are a significant regulatory development.  Developers of certified health IT will be required to comply with the DSI requirements by the end of 2024.

The algorithm requirements in the finalized rule only apply to certified health IT (as opposed to software used in healthcare more generally), but, even with this tailored scope, a broad patient population will be impacted by these changes.

DSI certification criterion

The final HTI-1 rule adopts a DSI certification criterion as a revised version of the clinical decision support (CDS) certification criterion. This includes establishing a new definition for Predictive DSIs (namely “technology that supports decision-making based on algorithms or models that derive relationships from training data and then produce an output that results in prediction, classification, recommendation, evaluation, or analysis,” which notably is not tied to technology risk level or intended use) and expanding the set of required source attributes (categories of technical performance and quality information) in connection with evidence-based and Predictive DSIs.

Key requirements include:

• New demographic, SDOH, and health status assessment source attributes – making it known to users of health IT modules certified to the DSI criterion whether patient demographic, social determinants of health (SDOH), or health assessment data are used in evidence-based DSI. Users are to be informed if one or more of these data elements are included as inputs, or otherwise expressly relied upon to generate an output, in such DSI, and should be informed of which specific data element or elements are used. The goal is to enable individuals and organizations to better understand the nature of certified health IT, whether there may be inherent biases, and how best to use the technology for a specific patient population.
  
  
• New source attributes for Predictive DSIs – providing users of health IT modules certified to the DSI criterion with access to information about the design, development, training, and evaluation of Predictive DSIs. ONC coordinated with the US Food and Drug Administration (FDA) in an effort to ensure the source attribute information required is aligned with FDA’s approach to software, including the transparency information described in FDA’s September 2022 guidance on CDS software. The new source attributes required for Predictive DSIs include:
  
  
  • Details and output of the intervention, including whether the intervention output is a prediction, classification, recommendation, evaluation, analysis, or other type of output
    
    
  • Purpose of the intervention, including intended use, user, patient and decision-making role
    
    
  • Cautioned out-of-scope use, including known risks, inappropriate settings, inappropriate uses or known limitations
    
    
  • Intervention development details and input features, including training data criteria and fit to intended use
    
    
  • Process used to ensure fairness in development, including description of approaches to manage or eliminate bias
    
    
  • External validation process, including descriptions of environments and entities involved in such testing
    
    
  • Quantitative measures of performance, including validity and fairness with respecting to test data derived from the same source or a different source as the initial training data and references to evaluation of use with respect to patient outcomes
    
    
  • Ongoing maintenance of intervention implementation and use, including monitoring frequency and localization testing, and
    
    
  • Update and continued validation or fairness assessment schedule, including a description of the frequency of performance correction when validity and fairness risks are identified.
    
    
• Risk management practices for Predictive DSIs – the application, by developers, of risk analysis and mitigation practices related to validity, reliability, robustness, fairness, intelligibility, safety, security, and privacy, in respect of all Predictive DSIs comprising part of a developers certified health IT module. Additionally, developers will need to make summary information regarding these practices publicly available.
  
  
• Maintenance of certification – imposing an ongoing responsibility on certified health IT developers to review and update, as necessary, all required source attribute information, risk management practices, and related summary information.

The finalized rule does not include requirements for how to present source attribute information to users, other than mandating the availability of complete and up-to-date plain language descriptions of source attribution information.

The proposed rule suggested that certified health IT modules enable a user to review source attribute information “at a minimum via direct display, drill down or link out” from a health IT module. This language does not appear in the final rule, in consideration of comments stating that including too much information in the direct display can negatively impact usability, user adoption, and could unintentionally inhibit innovative user interfaces. The finalized rule does, however, require that certified health IT modules must enable a limited set of identified users to access, record, and change source attribute information for DSIs.

Other notable updates

The final rule revises certain information blocking definitions and exceptions and adds a new exception to encourage secure, efficient, standards-based exchange of electronic health information under the Trusted Exchange Framework and Common Agreement℠ (TEFCA).

In addition, the final rule adopts the United States Core Data for Interoperability (USCDI) Version 3 as the new baseline standard within the ONC Health IT Certification Program as of January 1, 2026.

Further, the final rule implements provisions of the 21^st Century Cures Act’s requirement to adopt a condition of certification for developers of certified health IT to report certain metrics as part of their participation in the Certification Program. Referred to as the “Insights Condition,” these metrics are intended to provide greater insight into how certified health IT is used in support of healthcare delivery.

Impact on health IT developers and healthcare providers

These requirements are not only important for certified health IT developers to understand and implement. Under the final rule, the DSI certification criterion will become the criterion required for healthcare providers with health IT that meets the Base Electronic Health Record (EHR) definition. This is significant for providers who intend to use Certified EHR Technology for the purposes of participating in certain Centers for Medicare & Medicaid Services programs.

While the final rule spaces out many of the new requirements over one, two, three, and four years to allow developers and the industry to implement changes over time, algorithmic transparency was identified as a key priority, and, accordingly, the DSI provisions have a one-year compliance mandate. Developers of AI/ML tools used within certified health IT will need to be aware of the DSI requirements and the one-year timeline for compliance.

Additional resources

The full text of the HTI-1 final rule and accompanying fact sheets are posted here, and ONC will host information sessions on the final rule in the new year.

To find out more about the implications of the final rule for your business, please contact any of the authors of this alert.