SaveYourself.ca • Sensible advice for aches, pains & injuries
 

published 07/25/09, updated 1/14/11

The “Impress Me” Test

Most controversial therapies are fighting over scraps of scientific evidence

by Paul Ingraham, Vancouver, Canada BIO
Credentials & qualifications. I am a science journalist, and I was a massage therapist for ten years. I’m close to the end of a Health Sciences degree — 2 courses left! — and I am on the editorial team of Science-Based Medicine. I have spent many years studying therapy science, and my work is greatly enriched by thousands of conversations with readers and experts from around the world. I make a living from this website, selling some of my most detailed tutorials as ebooks. For more, see Who Am I to Say?

Readers and patients are forever asking me what my “hunch” is about a therapy: does it work? Is there anything to it? I’m honoured that my opinion is so respected, but I usually won’t take the bait. Like Carl Sagan, “I try not to think with my gut.”

It’s okay not to know. It’s okay for the jury to be out.

And it had better be, because there’s still a great deal of mystery in musculoskeletal health science. Most of the scientific evidence that I interpret for readers of SaveYourself clearly fails the “impress me” test. Even when that evidence is positive, it’s often only sorta positive. Even when there’s evidence that a therapy works, it’s usually weak evidence: some studies concluded that maybe it helps some people, some of the time … while other studies, almost always the betters ones, showed no effect at all. I’m supposed to get excited about this? To justify confidence in a therapy, we want really good evidence, evidence that makes you sit up and take notice, evidence that ends arguments because it’s just that clear.

Anything less is kind of lame. Anything less is underwhelming. Anything less fails to impress!

Thus, “controversy” about many popular therapies is much ado about nothing, and proponents are getting almost hysterical fighting over scraps of evidence. After decades of study, the effectiveness of a therapy should be clear and significant in order to justify its continued existence or “more study.” If it’s still hopelessly mired in controversy after so many years — more than a century in some cases — how good can it possibly be? Why would anyone — patient or professional — feel enthusiastic about a therapy that can’t clearly show its superiority in a fair scientific test? Where’s the value in even debating a therapy that is clearly not working any miracles, that has a trivial benefit at best?

Science, as they say, really delivers the goods: missions to Mars, long lives, the internet. A therapy has to deliver the goods. It’s got to help most people a fair amount and most of the time … or who cares?

Until it impresses you, it’s just some idea that hasn’t yet showed much promise.

We must somehow find a way to make peace with limited information, eagerly seeking more, without being dogmatic about premature conclusions.

Science Science And The Game Of 20 Questions, by Val Jones