Page 38

Wyoming Pregnant by Choice CMS Extension Approved: April 7, 2020; Effective through December 31, 2027 Amendment Approved: April 30, 2025 Page 34 of 38

Appendix C: Preparing the Evaluation Report

Introduction For states that are testing new approaches and flexibilities in their Medicaid programs through section 1115 demonstrations, evaluations are crucial to understand and disseminate what is or is not working and why. The evaluations of new initiatives seek to produce new knowledge and direction for programs and inform Medicaid policy for the future. While a narrative about what happened during a demonstration provide important information, the principal focus of the evaluation of a section 1115 demonstration should be obtaining and analyzing data on the process (e.g., whether the demonstration is being implemented as intended), outcomes (e.g., whether the demonstration is having the intended effects on the target population), and impacts of the demonstration (e.g., whether the outcomes observed in the targeted population differ from outcomes in similar populations not affected by the demonstration). Both state and federal governments need improved quantitative and qualitative evidence to inform policy decisions.

Expectations for Evaluation Reports Medicaid section 1115 demonstrations are required to conduct an evaluation that is valid (the extent to which the evaluation measures what it is intended to measure), and reliable (the extent to which the evaluation could produce the same results when used repeatedly). To this end, the already approved Evaluation Design is a map that begins with the demonstration goals, then transitions to the evaluation questions, and to the specific hypotheses, which will be used to investigate whether the demonstration has achieved its goals. States should have a well- structured analysis plan for their evaluation. With the following kind of information, states and CMS are best poised to inform and shape Medicaid policy in order to improve the health and welfare of Medicaid beneficiaries for decades to come. When conducting analyses and developing the evaluation reports, every effort should be made to follow the approved methodology. However, the state may request, and CMS may agree to, changes in the methodology in appropriate circumstances. When submitting an application for renewal, the interim evaluation report should be posted on the state’s website with the application for public comment. Additionally, the interim evaluation report must be included in its entirety with the application submitted to CMS.

Intent of this Attachment Title XIX of the Social Security Act (the Act) requires an evaluation of every section 1115 demonstration. In order to fulfill this requirement, the state’s submission must provide a comprehensive written presentation of all key components of the demonstration, and include all required elements specified in the approved Evaluation Design. This appendix is intended to assist states with organizing the required information in a standardized format and understanding the criteria that CMS will use in reviewing the submitted Interim and Summative Evaluation Reports. The format for the Interim and Summative Evaluation reports is as follows: A. Executive Summary; B. General Background Information; C. Evaluation Questions and Hypotheses;