Page 27

Demonstration Approval: November 1, 2023, through October 31, 2028 23

used repeatedly). The already-approved Evaluation Design is a map that begins with the demonstration goals, then transitions to the evaluation questions, and to the specific hypotheses, which will be used to investigate whether the demonstration has achieved its goals. When conducting analyses and developing the evaluation reports, every effort should be made to follow the methodology outlined in the approved Evaluation Design. However, the state may request, and CMS may agree to, changes in the methodology in appropriate circumstances.

When submitting an application for extension, the Interim Evaluation Report should be posted on the state’s website with the application for public comment. Additionally, the Interim Evaluation Report must be included in its entirety with the application submitted to CMS.

CMS expects Interim and Summative Evaluation Reports to be rigorous, incorporate baseline and comparison group assessments, as well as statistical significance testing. Technical assistance resources for constructing comparison groups and identifying causal inferences are available on Medicaid.gov: https://www.medicaid.gov/medicaid/section-1115- demonstrations/1115-demonstration-monitoring-evaluation/1115-demonstration-state- monitoring- evaluation-resources/index.html. If the state needs technical assistance using this outline or developing the evaluation reports, the state should contact its demonstration team.

Intent of this Attachment Title XIX of the Social Security Act (the Act) requires an evaluation of every section 1115 demonstration. In order to fulfill this requirement, the state’s evaluation report submissions must provide comprehensive written presentations of all key components of the demonstration, and include all required elements specified in the approved Evaluation Design. This Attachment is intended to assist states and territories with organizing the required information in a standardized format and understanding the criteria that CMS will use in reviewing the submitted Interim and Summative Evaluation Reports.

Required Core Components of Interim and Summative Evaluation Reports The Interim and Summative Evaluation Reports present research and findings about the section 1115 demonstration. It is important that the reports incorporate a discussion about the structure of the Evaluation Design to explain the goals and objectives of the demonstration, the hypotheses related to the demonstration, and the methodology for the evaluation. The evaluation reports should present the relevant data and an interpretation of the findings; assess the outcomes (what worked and what did not work); explain the limitations of the design, data, and analyses; offer recommendations regarding what (in hindsight) the state would further advance, or do differently, and why; and discuss the implications on future Medicaid policy. The format for the Interim and Summative Evaluation reports is as follows: A. Executive Summary; B. General Background Information; C. Evaluation Questions and Hypotheses; D. Methodology;