CMS star ratings: The methodology behind the quality reports

A leading healthcare expert tried to explain the methodology a federal agency uses to arrive at star ratings for hospitals Thursday during a webinar where officials also outlined upcoming changes to the rating system.

Hospital star ratings, which are published on the Centers for Medicare & Medicaid Services' Hospital Compare website, operate under five guiding principles, said Arjun Venkatesh, M.D., director of quality and safety research and strategy in the Department of Emergency Medicine at the Yale University School of Medicine. 

Those guidelines include: simplicity/accessibility, inclusivity, scientific rigor, stakeholder engagement and consistency. Venkatesh clarified that as CMS adds and removes measures each quarter, star ratings will reflect those changes.

Venkatesh also explained the ratings methodology itself, which comprises five steps. CMS will:

  • Select measures
  • Group the selected measures in their respective categories, such as safety, mortality or patient experience
  • Calculate group scores based on latent variable models
  • Generate a weighted summary score for each hospital
  • Assign a final star rating

As of April 2016, Venkatesh said, CMS has removed 14 voluntary measures, all of which fell under the Effectiveness of Care and Timeliness of Care groups. In April, outcomes measures for 20-day mortality and readmission coronary artery bypass graft were added, with two colonoscopy measures to join them in July.

CMS officials also addressed the delay of its latest round of star ratings until July. At a to-be-determined date, CMS will provide hospital-specific reports along with Hospital Compare data, said CMS Technical Advisor Kristie Baus, R.N. These reports will provide hospitals with additional details, such as individual standardized measure scores, she said. While star rating updates will be released quarterly for the reminder of the year, she said that beginning in 2017 the agency will release them twice a year, in July and December.

Baus also addressed frequently asked questions pertaining to CMS quality ratings, such as the recurring concern that they fail to incorporate vital data on socioeconomic and demographic factors. The agency, she said, is conducting research on risk adjustment for sociodemographic factors and will issue a report to Congress by October. 

To learn more:
-  here are the webinar slides (.pdf)