The folly of comparative effectiveness reviews
By Robert M. Goldberg
Comparative effectiveness—which is supposedly the “science” of comparing two treatments for the same illness and determining which one provides the best outcome for the least amount of money—is something that at least on the surface should be a process we can all agree on. Who wouldn’t want to use baking soda in warm water for an upset stomach instead of a four-dollar pill?
But these homely comparisons are not why a collection of interests—including insurance companies, managed care plans, government bureaucrats, advocates of single-payer health plans and experts from government-run health systems from Canada, Britain and Australia—have spent millions lobbying for the inclusion of $1.1 billion to create a government run agency to conduct comparative effectiveness studies. And it is not because, as the advocates for this mega-agency promise ( with a budget exceeding the Food and Drug Administration’s allocation on regulating new drugs, vaccines and devices) any decisions would be legally binding on doctors, Medicare, Medicaid or any health plan that would be regulated under Obamacare. In fact, Britain’s National Institute for Health and Clinical Excellence (NICE) has no authority to control what doctors do or what its National Health Service (NHS) pays for. (The managed care lobby, America’s Health Insurance Plans or AHIP, supports giving a comparative effectiveness agency such authority.)
Yet, the NHS now obliges itself to follow NICE comparative effectiveness decisions. And so doctors and patients have to wait years for NICE judgments or more accurately for NICE to say that paying for drugs for osteoporosis, Alzheimer’s, arthritis and cancer just isn’t worth it. Even worse, the whole comparative effectiveness decision-making process now overshadows everything. Any evidence, biological or otherwise that does not go through the government comparison mill lacks “kosher certification” and is regarded as not authoritative. To express pain, pose scientific questions, challenge questionable or sloppy assertions, one must become part of the bureaucratic apparatus or the parasitic lobbying necessary to obtain a “seat at the table.” The establishment of such an agency is dangerous if not done with great humility and humanity. My opposition is based not only on this concern but my experience on how it grinds human life into dust.
Several years ago my daughter battled bulimia. She was hospitalized three times for one month or less. Her discharge had nothing to do with treatment success. It had everything to do with her managed behavioral health plan following the logic of the comparative effectiveness review of the Agency for Healthcare Research and Quality (AHRQ) which will be responsible for turning the $1.1 billion into more reports. Or rather, it was how such reports are written and used that gave the insurer the running room to toss Sara and others like her out of the inpatient setting irrespective of whether they were clinically ready.
For instance, an AHRQ technology assessment, “Management of Eating Disorders,” published in 2006 concluded that the evidence for the effectiveness of combination of treatments for bulimia remains “weak.” It also went on to note that “few factors were found to be consistently related to outcomes.”
It was the process of taking the opinion of a group of health care consultants that evidence was “weak” and giving it the imprimatur of government authority that allowed health plans to limit coverage for eating disorders, claiming they are psychological instead of biological or truly measurable. This happened even without a law linking reimbursement to an explicit recommendation.
In fact, the reasons that “factors” are not “related to outcomes” have nothing to do with the implication that longer treatment is ineffective. The “technology assessment” never states what is well known: that the disease is still not well understood. Worse, it never acknowledges that the 30 day or less discharge contributes to the problem. When Sara was discharged she found herself fighting the urge to binge or purge the sense of shame and drive to perfection common among people with bulimia was amplified and was, sadly, reinforced by parents anxious to avoid a relapse or doing anything wrong.
Sara was discharged the first time after she did not binge or purge in a controlled setting for a few days in a row. Twice more she was discharged not because she was able to get control of her illness but because her blood pressure had been stabilized. Similarly, one of her friends at her eating disorders program had a perforated stomach from so much vomiting. Against the medical advice of her doctor, her insurance company kicked her out after a week because her heart rate was “normal.”
In 2007, Magellan, Aetna and Blue Cross and Blue Shield, three groups leading the push to expand AHRQ’s reach (and whose comparative effectiveness centers would do much of AHRQ's work) were sued. Susan Pisano, the spokeswoman for AHIP, said “there is no research that shows that longer treatment produces better results. In today's environment, the real question has to be: What does the evidence show?" I wonder where she got that excuse.
Advocates of comparative effectiveness claim that studies will be used differently here than in Britain. That’s a lie. In every other setting in the United States, comparative effectiveness reviews have been used to restrict access to new drugs or deny coverage to life-saving treatment more often than not. Comparative effectiveness research as currently construed is not about what’s best for people. It’s about saving money for political purposes. Don’t take my word for it. Ask my daughter.
By Robert M. Goldberg
Comparative effectiveness—which is supposedly the “science” of comparing two treatments for the same illness and determining which one provides the best outcome for the least amount of money—is something that at least on the surface should be a process we can all agree on. Who wouldn’t want to use baking soda in warm water for an upset stomach instead of a four-dollar pill?
But these homely comparisons are not why a collection of interests—including insurance companies, managed care plans, government bureaucrats, advocates of single-payer health plans and experts from government-run health systems from Canada, Britain and Australia—have spent millions lobbying for the inclusion of $1.1 billion to create a government run agency to conduct comparative effectiveness studies. And it is not because, as the advocates for this mega-agency promise ( with a budget exceeding the Food and Drug Administration’s allocation on regulating new drugs, vaccines and devices) any decisions would be legally binding on doctors, Medicare, Medicaid or any health plan that would be regulated under Obamacare. In fact, Britain’s National Institute for Health and Clinical Excellence (NICE) has no authority to control what doctors do or what its National Health Service (NHS) pays for. (The managed care lobby, America’s Health Insurance Plans or AHIP, supports giving a comparative effectiveness agency such authority.)
Yet, the NHS now obliges itself to follow NICE comparative effectiveness decisions. And so doctors and patients have to wait years for NICE judgments or more accurately for NICE to say that paying for drugs for osteoporosis, Alzheimer’s, arthritis and cancer just isn’t worth it. Even worse, the whole comparative effectiveness decision-making process now overshadows everything. Any evidence, biological or otherwise that does not go through the government comparison mill lacks “kosher certification” and is regarded as not authoritative. To express pain, pose scientific questions, challenge questionable or sloppy assertions, one must become part of the bureaucratic apparatus or the parasitic lobbying necessary to obtain a “seat at the table.” The establishment of such an agency is dangerous if not done with great humility and humanity. My opposition is based not only on this concern but my experience on how it grinds human life into dust.
Several years ago my daughter battled bulimia. She was hospitalized three times for one month or less. Her discharge had nothing to do with treatment success. It had everything to do with her managed behavioral health plan following the logic of the comparative effectiveness review of the Agency for Healthcare Research and Quality (AHRQ) which will be responsible for turning the $1.1 billion into more reports. Or rather, it was how such reports are written and used that gave the insurer the running room to toss Sara and others like her out of the inpatient setting irrespective of whether they were clinically ready.
For instance, an AHRQ technology assessment, “Management of Eating Disorders,” published in 2006 concluded that the evidence for the effectiveness of combination of treatments for bulimia remains “weak.” It also went on to note that “few factors were found to be consistently related to outcomes.”
It was the process of taking the opinion of a group of health care consultants that evidence was “weak” and giving it the imprimatur of government authority that allowed health plans to limit coverage for eating disorders, claiming they are psychological instead of biological or truly measurable. This happened even without a law linking reimbursement to an explicit recommendation.
In fact, the reasons that “factors” are not “related to outcomes” have nothing to do with the implication that longer treatment is ineffective. The “technology assessment” never states what is well known: that the disease is still not well understood. Worse, it never acknowledges that the 30 day or less discharge contributes to the problem. When Sara was discharged she found herself fighting the urge to binge or purge the sense of shame and drive to perfection common among people with bulimia was amplified and was, sadly, reinforced by parents anxious to avoid a relapse or doing anything wrong.
Sara was discharged the first time after she did not binge or purge in a controlled setting for a few days in a row. Twice more she was discharged not because she was able to get control of her illness but because her blood pressure had been stabilized. Similarly, one of her friends at her eating disorders program had a perforated stomach from so much vomiting. Against the medical advice of her doctor, her insurance company kicked her out after a week because her heart rate was “normal.”
In 2007, Magellan, Aetna and Blue Cross and Blue Shield, three groups leading the push to expand AHRQ’s reach (and whose comparative effectiveness centers would do much of AHRQ's work) were sued. Susan Pisano, the spokeswoman for AHIP, said “there is no research that shows that longer treatment produces better results. In today's environment, the real question has to be: What does the evidence show?" I wonder where she got that excuse.
Advocates of comparative effectiveness claim that studies will be used differently here than in Britain. That’s a lie. In every other setting in the United States, comparative effectiveness reviews have been used to restrict access to new drugs or deny coverage to life-saving treatment more often than not. Comparative effectiveness research as currently construed is not about what’s best for people. It’s about saving money for political purposes. Don’t take my word for it. Ask my daughter.
