Healthcare quality: Be careful what you measure for
There’s a lot of discussion focusing on high-value healthcare these days. Strategies like alternative networks or Centers of Excellence steer plan members to certain providers to get care, and the steerage mechanism is usually linked to provider quality – wanting to send folks to the best providers whenever possible. When provider quality becomes a lynchpin of your healthcare strategy, it’s important to be informed about where the market is in terms of measuring healthcare quality.
There are a number of tools designed to measure healthcare quality but there is no industry standard and each one has a different focus area. When you are evaluating a tool for use, keep in mind whether the tool was developed for use by consumers or healthcare professionals. The tool will be designed to work with different data sets depending on what type of questions it is focused on answering. For example, consumers may want to know the outcomes for hip and knee replacement surgeries, while a healthcare professional may be looking at how some facilities or provider groups in their health plan’s network compare to others.
Three examples of tools used to evaluate quality are CMS’ Care Compare, Leapfrog’s Safety Grade (both publicly available), and Mercer’s QualPic. Here we lay out what distinguishes each of these tools and what questions each is best designed to answer.
What distinguishes each of these tools?
None of the tools measure quality or define quality the exact same way. To start, the tools do not rely on the same metrics in their evaluation. Below are just a few examples of the various measures that go into each method for measuring healthcare quality, whether quality of experience, outcomes, or a combination.
Sometimes these metrics are what’s called “risk adjusted”; for example, all of the metrics within QualPic above are on a risk-adjusted basis. This means that the measure has been calibrated to what would be expected for that facility, given the patients that were treated. Raw measures taken in aggregate could be unfairly punitive to hospitals with especially sick patient populations, and certainly the opposite is also true.
In addition to the measures themselves, data sources vary widely: for CMS and QualPic, data comes from Medicare and commercial claims information while Leapfrog data comes from hospital survey results. The sources can affect how the data is reported and analyzed, since claims-based measures can usually be scrutinized in more detail and leave less room for qualitative thinking to enter the measurement process. Conversely, a lot of “mathematical maneuvering” can be achieved with claims-based metrics, so it’s important to dig into the details of the calculations to understand their strengths, weaknesses, and relevance to your population.
How does each tool report its findings?
Each entity uses a scoring paradigm that’s entirely different from the others. CMS has a 5-star rating system, Leapfrog has letter grades, and QualPic has scores from 0-100. Below is an example for one hospital:
With the alphabet soup of stars, letters, and numbers, it can be tough to perform any sort of robust reconciliation of results. One helpful characteristic of all three is that when you’re looking at results, they allow you to compare some of the output to a benchmark, to get a sense of how other facilities compare in that category. This allows for making thoughtful comparisons in context.
What can I do about all of this?
There are a number of valid approaches out there for measuring quality, but since there isn’t an industry standard none of them are exactly the same. First, we should keep in mind who the end audience might be. If you are a consumer shopping for where to get a knee or hip replacement, CMS has got you covered so long as you are similar to the types of patients captured in their data. If you really want to understand the prevalence of infections, CMS or Leapfrog are great places to start. If you’d like to synthesize how some facilities or provider groups in your health plan’s network compare to others, whether you’re doing a marketing campaign or not, QualPic is a great way to evaluate that.
Regardless of your use case, I’m sure we can agree that determining whether providers and facilities are high quality is an important part of the value proposition we can bring to our benefit programs.