Study Casts Doubt on Influential Hospital Safety Survey

TUESDAY March 31, 2009 -- The task just got harder for patients trying to figure out which hospital will provide the best care.

A new study casts doubt on a hospital safety rating conducted by the Leapfrog Group, a nonprofit business coalition.

Hospitals ranked in the highest quartile on the safety survey had about the same rate of in-hospital deaths as those in the lowest quartile, according to a study in the April 1 issue of the Journal of the American Medical Association.

"It takes time and money to assess quality of care," said senior study author Dr. R. Adams Dudley, an associate professor of medicine and health policy at University of California, San Francisco. "If we're going to do that, we need to make sure we put those resources into something that gives us a true signal of quality. Unfortunately, this doesn't seem to be doing that."

The Leapfrog Hospital Survey is a widely publicized report that assesses the safety, quality and efficiency of care at 1,200 U.S. hospitals, according to the Leapfrog Group's Web site.

The survey measures these main aspects of hospital quality: the extent to which physicians use computers to order prescriptions, which has been shown to cut down on medical errors; whether hospital intensive care units are properly staffed; and how well hospitals perform a selection of complex medical procedures.

Previous research has shown that death rates for complex procedures are lower at hospitals with lots of experience doing them, Dudley said.

In 2004, Leapfrog added a fourth measure: the Safe Practices Survey, which asks hospitals to supply detailed information about 27 measures of hospital safety, including how well it promotes a safety culture, ensures an adequate nursing workforce, provides anti-coagulation services and requires hand washing.

Hospitals are then ranked in quartiles based on their self-reported answers.

Dudley and his colleagues looked at 155 hospitals that took part in the 2006 survey and found little difference among those in the highest and lowest quartiles. The in-hospital death rate from lowest to highest quartile was: 1.97 percent, 2.04 percent, 1.96 percent and 2 percent.

Last year, the survey was reduced to questions about 13 practices. The streamlined survey also did not correlate to patient death rates.

Hospital administrators have complained that the Safe Practices Survey is too cumbersome and the questions too vague to assess safety.

"We strongly believe some of the Leapfrog issues don't correlate to patient outcome," said Debby Rogers, vice president for quality and emergency services at the California Hospital Association, which represents 400 hospitals in the state.

Nancy Foster, vice president for quality and patient safety policy for the American Hospital Association, which has 5,000 member hospitals, said several survey questions, such as whether a hospital has an informed consent procedure, would have limited impact on death rates.

"Most of these safe practices are the right thing to do. They could avert an error, but they are unlikely to have led to the prevention of death," Foster said.

HealthDay was unable to reach Leapfrog for comment.

Previous research has shown it's not easy for patients to find solid information about hospitals or individual physicians. Last year, a study in the Archives of Surgery that looked at six hospital-comparison Web sites found there were inconsistent results, and the sites used inappropriate or incomplete standards to measure a center's quality.

The sites included the U.S. government's "Hospital Compare," "Quality Check" from the Joint Commission on Accreditation of Healthcare Organizations and Leapfrog's survey.

Even though the Leapfrog Safe Practices Survey has problems, Dudley said prior research has validated the first three portions of the survey.

With health-care costs soaring and consumers demanding more information, determining how to best measure hospital quality is growing in importance.

"We need to do more of it," said Dudley, founder of CalHospitalCompare.org, a site that rates California hospitals. "We need to develop new tools. Unfortunately, this one appears not to be a effective tool."

More information

The Agency for Heathcare Research and Quality has more on hospital quality.

Posted: March 2009


View comments

Hide
(web4)