Real World Testing Result for Smile CDR

These are the Real World Test results for 2024 for the Smile CDR product. It provides real world test measurements and metrics that meet the intent and objectives of ONC’s Condition of Certification and Maintenance of Certification requirement for real world testing (§ 170.405 Real World Testing) to evaluate compliance with the certification criteria and interoperability of
exchanging electronic health information (EHI) within the care and practice setting which it is targeted for use.

This document provides the targeted testing measurements and metrics that Smile Digital Health will use to evaluate the Smile CDR product interoperability within typical production settings. For each measure, the associated testing methodology, ONC criteria, justification for measurement, and expected outcomes from the testing are presented.

General Information 

Test Report ID Number:
Developer Name: Smile Digital Health
Product Name: Smile CDR
Version Number: 2024
Certified Health IT Product List (CHPL) ID(s): 15.04.04.3129.Smil.24.02.0.241112
Developer Real World Testing Result Page URL: https://www.smiledigitalhealth.com/cdr-real-world-test

The 2024 Real World Testing Plan originally included Smile CDR v2023 (CHPL ID: 15.04.04.3129.Smil.02.00.0.221207). However, version 2023 was withdrawn, and its capabilities were fully inherited by Smile CDR version 2024. The 2024 version maintains all functionalities tested in 2023 while incorporating enhancements for interoperability, FHIR API performance, and compliance with ONC certification standards. This ensures that real-world testing remains aligned with previous implementation efforts.

Justification for Real World Testing

Real World Testing is intended to verify that deployed Certified Health IT continues to perform as intended, ensuring successful interoperability and data exchange across various clinical settings and practices. Smile CDR supports a broad range of use cases and can be utilized by many clinical practices, including ambulatory settings such as outpatient clinics. All measures reasonably align with the §170.315(g)(7) and §170.315(g)(10) certification criteria.

Developer Attestation

This Real World Testing result is complete with all required elements, including measures that address all certification criteria and care settings. All information in this result is up to date and fully addresses the health IT developer’s Real World Testing requirements.

Authorized Representative Name: James Agnew
Authorized Representative Email: compliance@smiledigitalhealth.com
Authorized Representative Phone: +1 (877) 537 3343
Authorized Representative Signature: James Agnew
DATE: January 2nd, 2025

Timeline and Milestones for Real World Testing 2024

• 2024-Q1: Draft test plan. Identify and allocate resources for RWT activities..
• 2024-Q2.Execute draft test plan and capture results for review. Create an internal review committee to review initial results.
• 2024-Q3. Complete RWT test plan according to ONC and ONC-ACB requirements and expectations. 2024 RWT Test Report will be prepared for submission before the end of the year.
• 2024-Q1. Submit RWT Test Results to ONC-ACB.
  
  

Standards Updates Standard and Version

Measurements

Measurement 2024-CDR-01

Associated Certification Criterion: 170.315 (g)(7) Application Access - Patient Selection

Measurement: Count the number of API queries for a patient search made in a real world setting

Description: Using the Smile CDR FHIR API, the product is able to demonstrate meeting the requirement (g)(7)(i) respond to requests for a single patient’s data according to the standard adopted in § 170.215(a)(1) and implementation specification adopted in §170.215(a)(2)

Justification: Demonstrating the ability Smile CDR API module to respond to (i)(A) requests

Expected Outcomes: Demonstrate the compliance of the standard by confirming that Smile CDR can process queries as expected and with less than 1% errors.

Testing Results:

Reporting Interval: 3 months (October 1, 2024 through December 31 2024)

Testing Measurement: Number of API queries are 550,575 counts and 4 errors. The error count is below the 1% intended threshold.

Measurement 2024-CDR-02

Associated Certification Criterion: 170.315 (g)(10) Standardized API of Patient and Population Services

Measurement: Count the number of API queries for the category of clinical data set made in a real life setting. 

Description: During the testing period, a report will be generated which tracks and logs the FHIR API queries made for specific data category requests in the real world.

Justification: Demonstrate the capability of Smile CDR to process FHIR API queries with a patient as a subject and fetch that patient-specific data by category.

Expected Outcomes: Demonstrate the compliance of the standard by confirming that Smile CDR can process queries as expected and with less than 1% errors.

Testing Results:

Reporting Interval: 3 months (October 1, 2024 through December 31 2024)

Testing Measurement: Number of API queries are 887 counts and 0 errors. The error count is below the 1% intended threshold.

Measurement 2024-CDR-03

Associated Certification Criterion: 170.315 (g)(10) Standardized API of Patient and Population Services

Measurement: Count the number of API queries for the category of obtaining valid access token from Smile CDR Authorization server in the real world.

Description: During the testing period, a report will be generated which tracks and logs the FHIR API queries made to access the authorization server to obtain a valid token to access other Smile CDR APIs for retrieving Patient data.

Justification: Demonstrate the capability of Smile CDR  to process FHIR API queries intended for our SMART on FHIR Authorization Server.

Expected Outcomes: Demonstrate the compliance of the standard by confirming that Smile CDR can process queries as expected and with less than 1% errors.

Testing Results:

Reporting Interval: 3 months (October 1, 2024 through December 31 2024)

Testing Measurement: Number of API queries are 11,664 counts and 2 errors.  The error count is below the 1% intended threshold.