PERSPECTIVES ON INNOVATION
MARCH 2015
A State Medicaid Approach to Evaluation
Given the central role that Medicaid plays in health care, it is understandable that a high level of interest exists in
the success of new payment and delivery system reforms like health homes, accountable care organizations
(ACOs), episodic payment, and other innovations. This is especially true for the states themselves. Like the federal
government and other payers, states are investing their own resources to stand up innovation and want to make
sure there is value for their public dollar. States are eager to implement successful models.
When state Medicaid Directors evaluate innovative delivery system reforms, the biggest questions are whether
the reform is improving health outcomes relative to what preceded it, and whether there is a foundation for future
improvement and success. Since states must make a sustained investment in both time and resources to bring
positive change to their health systems, it is vital we also consider the appropriate timeframes to see whether a
Medicaid program has met its goals. As many states, payers, and providers have just begun their efforts,
institutions new to innovation will need time to develop the organizational knowledge, skill, and experience to
implement reform. Going forward, assessing real world implementation and measuring outcomes of reforms are
essential, but must be paired with an appraisal of whether the reform has created a framework for future success.
The Right Infrastructure
In Medicaid programs across the country, many of the building blocks for value-based health care are being
created through the actual implementation of innovative reforms. States are using payment and delivery system
reform as the impetus to create the data systems and analytics, hire the personnel, and develop the internal
policies they need to be able to administer innovative programs. Many of the federal initiatives and programs
that support state-led reform, like the State Innovation Model (SIM) grants, Delivery System Reform Incentive
Payment (DSRIP) Program, 1115 Medicaid Waiver demonstrations, enhanced matching funds, and other
opportunities, are the vehicles that help provide Medicaid programs with the resources needed to create this
infrastructure.
A broader definition of success that should be applied to delivery system reforms should take into account
whether Medicaid is developing the expertise and infrastructure needed to put these foundational elements in
place. In the long term, states have to equip payers, providers, and beneficiaries with the tools they need to realize
value-based care. This is particularly true if one of the objectives of innovation is not simply creating payment
models for a particular disease or provider type, but rather applying the principles of a learning health system
culture, one that seizes opportunities to improve itself and raise the health outcomes, to the role of the payer itself.
One of the goals of reform is helping states find alignment between payment and outcomes whenever possible –
often across payers – as well as ensuring a state infrastructure that is nimble enough to modify its approach in
order to push innovation forward. Having this reform capacity in place will actually help Medicaid programs be
more responsive to value-based models that inevitably will emerge in the future.
This publication was made possible by the support of The Commonwealth Fund.
Foundation for Value
As noted in a recent Commonwealth Fund blog post, the incubation period for health care innovation can take
years, even during periods of substantial effort. The authors posit that we have to apply common sense to
interventions as the evidence develops and continue to support initiatives supporting reforms that are likely to
create value because their benefit is apparent or early evidence suggests success. Some Medicaid programs have
found that it can take three-to-four years or more, from the design of a program to finishing an evaluation with
sufficient evidence, to see whether a reform is successful. These timeframes are not always in alignment with the
demonstration programs and other opportunities that support reforms.
As states are investing their own dollars, and tapping resources like SIM and DSRIP, they are beginning to
demonstrate early successes that augur well for the future. States like Missouri and Oregon that have been on the
forefront of reform are beginning to post preliminary evaluations. While these frontline states have only had a
handful of years of experience, early evidence suggests that they have made headway both in infrastructure
development and enhancing health outcomes.
Initial evaluations demonstrate that innovation is still in a foundational phase, where the groundwork is being
laid that will pay dividends over the long term. As NAMD has written elsewhere, the administrative structure
the state must put into place to oversee payment and delivery system reforms takes time to develop, in terms of
staff, technology, and outreach. States are asking providers, plans, and beneficiaries to rethink and reorder their
approach to health care, and the full effects of innovation may take time to be reflected in health and expenditure
outcomes. The time horizon to see results is tied to how innovation is diffused and established at these plan,
provider and beneficiary levels. This can be an iterative process and reflects how payment and delivery system
reform is helping to lead to the application of the learning health system concept that applies data-driven,
evidence-based decision-making to Medicaid programs themselves.
Flexibility and Easing Burdens
It is often said in the Medicaid world that “if you have seen one Medicaid program, you have seen one Medicaid
program” – a concept that is confirmed by the various approaches to delivery reform. No “one-size-fits-all”
solution will be viable. Instead, states will choose to address different populations or delivery system features in
a variety of ways. This variation is appropriate, as there are significant differences in covered populations,
political culture, budgetary, legislative, and administrative infrastructures, stakeholders, provider capacity, and
a host of other idiosyncratic factors. Medicaid programs are not different because of any singular particular factor,
but rather they are different because of a host of internal and external factors.
Despite the range of different options, states do understand the value of learning from each other’s efforts. While
assessment of innovation across states is challenging, it is still important. However, with the proliferation of
payment and delivery system reform programs, there is a real strain being placed on this burgeoning reform
infrastructure to produce impact and evaluation analyses. When possible, evaluation should rely on existing
sources data and processes to minimize the stress on plans, providers, and state systems. Furthermore, reporting
requirements on state agencies and providers can be burdensome if various evaluation efforts are not aligned.
For instance, when a state 1115 waiver is the authorizing and funding vehicle for a policy innovation created in a
2
SIM grant, it may be most advantageous to policymakers at all levels to align the measurement strategies. This
would bring greater cohesion to the evaluation effort.
With the existing slate of reporting requirements and the advent of the Transformed Medicaid Statistical
Information System (T-MSIS), the federal government’s greatly expanded effort to collect state Medicaid data for
reporting and research purposes, it will be essential to minimize the capacity challenges to Medicaid programs in
order to maximize the impact of evaluation result. Given the already considerable capacity constraints faced by
Medicaid programs, attempts to expand data collection for the sake of more data – or otherwise place greater
reporting demands on state agencies – should be limited to circumstances where there is consensus that the
additional burden has a high value to states as well as to others.
The Year Ahead
The year 2015 will see many evaluations, including SIM and DSRIP assessments, released. Already in January,
CMS released evaluations on the Comprehensive Primary Care (CPC) initiative and the Multi-payer Advanced
Primary Care Practice (MAPCP) Demonstration, while the Patient-Centered Primary Care Collaborative issued a
report on the Patient-Centered Medical Home’s Impact on Cost and Quality. The authors of these reports note
that results are preliminary, but do show evidence of early success and remind us that innovative practices will
take time to materialize into predictable and scalable gains.
Medicaid policymakers will want to assess the impact of specific policy advancements, and make adjustments
where needed. However, these decisions will likely be made at a point of time when a particular reform is just
beginning to take hold. As these early results come in, it will be important to consider not just the success of
particular initiatives, but also whether they have helped states make crucial investments in personnel, systems
and other features, and are improving the delivery system in ways that promise enhanced outcomes in realistic
timeframes. Evaluation efforts should take into account this operational perspective for innovation in order to
assess the real-world effects of payment and delivery system reform and present a more holistic appraisal of the
changes brought on by innovation.
3
States Show Progress in Major Initiatives
Progress can be found in a number of evaluations that have already been released by states. While some reforms have
matured, even the most advanced have only had a few years of experience. However, the body of evaluative research has
begun to demonstrate that some real world progress has occurred in both developing infrastructure, establishing the
framework for reform, and demonstrating improved outcomes.

California’s Delivery System Reform Incentive Payment (DSRIP) has met a number of quality and infrastructure
milestones in their 1115 Bridge to Reform waiver according to a California Association of Public Hospitals and
Health Systems report, including some gains like reduced ER visits, reductions in avoidable admissions,
reductions in admissions and bed days for high-risk patients, and reductions in adverse events. Another
evaluation delves deeper into the progress the state and health systems have made in operationalizing the waiver.
California also has submitted an interim DSRIP evaluation to CMS that emphasizes the value of tying financial
incentives to process measures investing in infrastructure building and demonstrates the state’s move towards
outcome measures in later years of the demonstration.

The SIM Design Grant process has helped states leverage federal dollars to plan and jump start reform.
Connecticut received a Design grant in 2013 and a Testing grant in 2014. Under Connecticut’s Testing design, the
state will develop and implement a shared savings program with competitively procured Federally Qualified
Health Centers and advanced networks (the Medicaid Quality Improvement Shared Savings Program, MQISSP),
build on existing Medicaid Person Centered Medical Home (PCMH) strategies to promote primary care practice
transformation under the Advanced Medical Home (AMH) initiative, frame and implement a population health
initiative, and develop Health Information Technology (HIT) strategies to support the same. The SIM design
process has helped the state to cultivate a commitment to value-based payment across payers, has accelerated the
organization of providers into ACO-like entities, which are becoming a more prevalent feature of Connecticut’s
healthcare delivery system, and has created new pathways for a population health approach across state agencies
and their community partners.

Iowa received a SIM Design grant which has helped the state focus on the use of ACO and a multi-payer
framework to track quality and support incentive payments. The goal is to expand the design into the entire
Medicaid population, including Long Term Care and Behavioral Health and looks to build community care teams
and enhance the use of HIT and HIE. Iowa is focused on addressing social determinants of health through
community integration efforts and by developing mechanisms to incentivize healthy behaviors and drive positive
health outcomes for beneficiaries.

Missouri released a progress report for its Community Mental Health Center (CMHC) Healthcare Homes, and
found that there was a 12.8 percent reduction in hospital admissions and 8.2 percent reduction in emergency
room use per 1000 enrolled in CMHC. For a population of 6,156, there were $2.4 million in savings. This reflects
a 29.5 percent reduction in expenditures for these individuals over the previous year.

The Oregon Health Authority (OHA) regularly publishes the results of its Coordinated Care Organization (CCO)
quality measures on its website. In its 2013 Performance Report, OHA detailed how it decreased emergency
department visits and emergency department spending; increased primary care utilization and spending;
increased rates of developmental screenings during the first 36 months of life; decreased hospitalizations for
chronic conditions; and increased adoption of electronic health records. All CCOs showed improvements on
some measures and 11 out of 15 met 100 percent of their improvement targets. SIM funding, among other forms
of support that helped spread the CCM model, also enabled the creation of the Oregon Health Authority’s
Transformation Center, which supports CCOs by providing technical assistance, best practices, and other forms
of assistance to providers to embrace the state’s reform model.
4