Page 32

Demonstration Approval Period: October 1, 2024 through September 30, 2029 Page 32 of 43

ATTACHMENT A Developing the Evaluation Design

Introduction For states that are testing new approaches and flexibilities in their Medicaid programs through section 1115 demonstrations, evaluations are crucial to understand and disseminate what is or is not working and why. The evaluations of new initiatives seek to produce new knowledge and direction for programs and inform Medicaid policy for the future. While a narrative about what happened during a demonstration provides important information, the principal focus of the evaluation of a section 1115 demonstration should be obtaining and analyzing data on the process (e.g., whether the demonstration is being implemented as intended), outcomes (e.g., whether the demonstration is having the intended effects on the target population), and impacts of the demonstration (e.g., whether the outcomes observed in the targeted population differ from outcomes in similar populations not affected by the demonstration). Both state and federal governments need rigorous quantitative and qualitative evidence to inform policy decisions.

Expectations for Evaluation Designs All states with Medicaid section 1115 demonstrations are required to conduct an evaluation, and the Evaluation Design is the roadmap for conducting the evaluation. The roadmap begins with the stated goals for the demonstration followed by the measurable evaluation questions and quantifiable hypotheses, all to support a determination of the extent to which the demonstration has achieved its goals. When conducting analyses and developing the evaluation reports, every effort should be made to follow the approved methodology. However, the state may request, and CMS may agree to, changes in the methodology in appropriate circumstances.

The format for the Evaluation Design is as follows: A. General Background Information; B. Evaluation Questions and Hypotheses; C. Methodology; D. Methodological Limitations; and E. Attachments.

Submission Timelines There is a specified timeline for the state’s submission of Evaluation Design and Reports. (The graphic below depicts an example of this timeline). In addition, the state should be aware that section 1115 evaluation documents are public records. The state is required to publish the Evaluation Design to the state’s website within 30 days of CMS approval, as per 42 CFR 431.424(e). CMS will also publish a copy to the Medicaid.gov website.