Nursing homes’ publicly reported star ratings have increased substantially since the adoption of the Five-Star Quality Rating System. But there hasn’t been a corresponding drop in preventable hospitalizations, according to a new study, and “teaching to the test” could possibly be a culprit.

That’s the result of a new study by researchers with University of Pennsylvania and University of Chicago, published in the Journal of Health Care Organization, Provision and Financing.

Pouring over Medicare claims data from 2007 to 2010, investigators aimed to track the association between nursing homes’ ratings and the number of potentially preventable hospitalizations per 30 days.

Researchers found that, before the Five-Star system began in 2008, there was a consistent relationship between star rating and hospitalization rates, with the higher-quality facilities demonstrating lower numbers of readmits. After the release in December 2008, however, the gradient between star rating and hospitalizations reduced.

“This aligns with expectations that the ratings are becoming less meaningful over time as a broad indicator of quality,” researchers wrote. The adjusted rates of preventable hospitalizations from 2- to 5-star rates facilities were higher after the release.

One possibility is providers “teaching to the test” and making superficial changes that don’t actually stick. Another thought is patient selection, where providers may be cherry-picking patients that skew the ratings. While researchers did not observe differences in distribution of patients in nursing homes before and after the ratings were released, they argued it’s still worth further examination.

“Nevertheless, it is possible that for-profit nursing homes with higher 5-star ratings were incentivized to select high-risk patients (by partnering with hospitals willing to share their readmission incentives, for example). Selection of lower risk patients by poorly rated facilities would also explain the decrease in hospitalization rates among 1-star facilities after 5-star release,” they wrote.