Page 85

Pennsylvania Keystones of Health Section 1115 Demonstration Approval Period: December 26, 2024 through December 31, 2029 Page 72 of 90 the methodology outlined in the approved Evaluation Design. However, the state may request, and CMS may agree to, changes in the methodology in appropriate circumstances. When submitting an application for renewal, the Interim Evaluation Report should be posted on the state’s website with the application for public comment. Additionally, the Interim Evaluation Report must be included in its entirety with the application submitted to CMS. CMS expects Interim and Summative Evaluation Reports to be rigorous, incorporate baseline and comparison group assessments, as well as statistical significance testing. Technical assistance resources for constructing comparison groups and identifying causal inferences are available on Medicaid.gov: https://www.medicaid.gov/medicaid/section–1115– demonstrations/1115–demonstration–monitoring–evaluation/1115–demonstration–state– monitoring–evaluation–resources/index.html. If the state needs technical assistance using this outline or developing the evaluation reports, the state should contact its demonstration team. Intent of this Attachment Title XIX of the Social Security Act (the Act) requires an evaluation of every section 1115 demonstration. In order to fulfill this requirement, the state’s evaluation report submissions must provide comprehensive written presentations of all key components of the demonstration, and include all required elements specified in the approved Evaluation Design. This Attachment is intended to assist states with organizing the required information in a standardized format and understanding the criteria that CMS will use in reviewing the submitted Interim and Summative Evaluation Reports. Required Core Components of Interim and Summative Evaluation Reports The Interim and Summative Evaluation Reports present research and findings about the section 1115 demonstration. It is important that the reports incorporate a discussion about the structure of the Evaluation Design to explain the goals and objectives of the demonstration, the hypotheses related to the demonstration, and the methodology for the evaluation. The evaluation reports should present the relevant data and an interpretation of the findings; assess the outcomes (what worked and what did not work); explain the limitations of the design, data, and analyses; offer recommendations regarding what (in hindsight) the state would further advance, or do differently, and why; and discuss the implications on future Medicaid policy. The format for the Interim and Summative Evaluation reports is as follows: A. Executive Summary; B. General Background Information; C. Evaluation Questions and Hypotheses; D. Methodology; E. Methodological Limitations; F. Results; G. Conclusions; H. Interpretations, and Policy Implications and Interactions with Other State Initiatives; I. Lessons Learned and Recommendations; and, J. Attachment(s).