Page 22

Demonstration Approval: November 1, 2023, through October 31, 2028 18

B. Evaluation Questions and Hypotheses – In this section, the state should: 1. Identify the state’s hypotheses about the outcomes of the demonstration, and discuss how the evaluation questions align with the hypotheses and the goals of the demonstration. 2. Address how the hypotheses and research questions promote the objectives of Titles XIX and/or XXI. 3. Describe how the state’s demonstration goals are translated into quantifiable targets for improvement, so that the performance of the demonstration in achieving these targets can be measured. 4. Include a Logic Model or Driver Diagram to visually aid readers in understanding the rationale behind the cause and effect of the variants behind the demonstration features and intended outcomes. A driver diagram, which includes information about the goals and features of the demonstration, is a particularly effective modeling tool when working to improve health and health care through specific interventions. A driver diagram depicts the relationship between the goal, the primary drivers that contribute directly to achieving the goal, and the secondary drivers that are necessary to achieve the primary drivers for the demonstration. For an example and more information on driver diagrams: https://innovation.cms.gov/files/x/hciatwoaimsdrvrs.pdf. 5. Include implementation evaluation questions to inform the state’s crafting and selection of testable hypotheses and research questions for the demonstration’s outcome and impact evaluations and provide context for interpreting the findings. Implementation evaluation research questions can focus on barriers, facilitators, beneficiary and provider experience with the demonstration, the extent to which demonstration components were implemented as planned, and the extent to which implementation of demonstration components varied by setting.

C. Methodology – In this section, the state is to describe in detail the proposed research methodology. The focus is on showing that the evaluation meets the prevailing standards of scientific and academic rigor, that the results are statistically valid and reliable, and that it builds upon other published research, using references where appropriate. The evaluation approach should also consider principles of equitable evaluations, and involve partners—such as community groups, beneficiaries, health plans, health care providers, social service agencies and providers, and others impacted by the demonstration who understand the cultural context—in developing an evaluation approach.

This section also provides evidence that the demonstration evaluation will use the best available data. The state should report on, control for, and make appropriate adjustments for the limitations of the data and their effects on results, and discuss the generalizability of results. This section should provide enough transparency to explain what will be measured and how, in sufficient detail so that another party could replicate the results. Table A below is an example of how the state might want to articulate the analytic methods for each research question and measure.

Specifically, this section establishes: 1. Methodological Design – Provide information on how the evaluation will be designed. For example, whether the evaluation will utilize pre/post data comparisons, pre-test or post-test only assessments.