In response to the kerfuffle over the Friends of Science in Medicine, The Conversation responded with an article from the alternative therapists’ point of view: “Alternative medicine can be scientific, say besieged academics“.
The article’s uncritical he-said-she-said reporting – normal practice for ‘unbiased’ journalism and a site which is trying to provoke a ‘conversation’ – is a bit disappointing when you consider that The Conversation was set up by CSIRO and universities to be an independent news source based on evidence. In the words of Prof Peter Doherty (according the to Herald Sun, anyway):
Whether in science, history or the arts, those who have the capacity to discuss complex issues from the basis of evidence rather than opinion tend to be found in Australia’s universities and scientific research institutes. … That’s why I think that the idea behind The Conversation is terrific.
Why am I disappointed? Surely an important feature of an evidence-based discussion is to hyperlink to the peer-reviewed research, so readers can check out the evidence – it’s a great advantage the internet has over print publications. It should be what distinguishes the opinion on The Conversation, from, say, an Andrew Bolt opinion column.
However, the only links in the article are in relation to a controversy over whether or not Steve Jobs took alternative therapies for his cancer – which is an anecdote, not scientific evidence. Helpfully, a commenter below the article did post a link to the cited acupuncture study, allowing me to investigate further.
What the alternative therapist said in The Conversation article:
However, Dr Myers said that CAM research at RMIT was conducted in a thoroughly scientific manner, with the NHMRC funding clinical trials of alternative medicines. …
The NHMRC had also granted A$400,000 for a project in which the university was collaborating with three Melbourne hospitals on a three-year clinical trial of acupuncture for pain management in emergency departments, Dr Myers said. “The project follows the promising results of pilot studies by RMIT researchers, in which more than 1,000 patients received acupuncture treatment for acute pain relief at the emergency department of the Northern Hospital.”
I searched on Pubmed and I couldn’t find that emergency department pilot study, but I’ve got the commenter’s handy link:
A Prospective, Randomised Control Trial of Acupuncture for Select Common Conditions within the Emergency Department
Acupuncture “control”
The study’s introduction says:
In regard to scientific trials, there is the obvious difficulty of finding an appropriate placebo. To address this, some trials have used a sham needle (1).This is where a superficial needling technique is used at non-acupuncture points. The problem with this is that these non-acupuncture points may still lie on a channel and of course the sun luo and fen luo, the little capillary network of vessels of the jingluo, connect the surface to the interior jingluo/channels and this may indeed induce a response. Other researchers have used a sham needle that does not penetrate the skin at all, and although it is apparently realistic (for the patient), since some acupuncture techniques only lightly stimulate the point, even this might be enough to exert an effect.
Image via Wikipedia
Hahahaha … sorry, I’ll try to take that seriously. If stimulating the point without needing to stick a needle into you patient has the same effect, then shouldn’t this be a wonderful giant leap forward for acupuncture? And if sticking a needle in anywhere will have the same effect, then what’s the point of acupuncture points?
Sham acupuncture is a commonly used control in studies, and ‘sham’ shallow needling not on acupuncture points has been previously used as a control by the acupuncture research group at RMIT, so there’s really no excuse for not having a control group.
What could this study conclude about the effectiveness of acupuncture?
Absolutely nothing, because there was no sham acupuncture control group. The supposed ‘control’ group was patients given conventional emergency department treatment. The acupuncture patients were given an extra “consultation lasting 30-40minutes”. This is a problem because:
- Response to acupuncture is highly susceptible to the placebo effect; sham acupuncture ‘treats’ self reported pain/symptoms significantly better than a placebo pill. (BMJ 2006)
- A supportive interaction with a practitioner is one of the most potent components of the placebo effect and produces a clinically significant improvement in the patient’s health. The consultation placebo effect may be due to factors such as warmth, empathy, duration of interaction, and the communication of a positive expectation. (BMJ 2008)
What does the study find?
There was no significant difference in any of the outcome measures (pain score, respiratory rate, heart rate, and systolic and diastolic blood pressure), between the normally treated group and the acupuncture group. At best this shows that acupuncture isn’t any improvement on current practices.
What can you conclude from this study?
Nothing – you should treat someone using standard practice in an emergency department. (Or you could add in a nice chat and a cup of tea and have … exactly the same effect. I’m really a bit disappointed that the massive potential placebo effect, from taking the patients out for a 40 minute consult and offering them the positive expectation of a therapy, didn’t produce some improvement.)
What did they conclude?
• These results demonstrate that acupuncture and conventional treatment are equally effective to reduce pain, however the acupuncture group received a significantly higher rate of patient satisfaction with the treatment.
• This type of trial could be applied to other departments within the public hospital system such as the orthopaedic, neurological and maternity wards.
• Given that the results have proven positive for the treatment of acute pain, this style of trial could be applied in frontline emergency relief situations with practitioners working alongside Westerns doctors with triage nurses.
• The results of this trial paved the way for RMIT acupuncture students to provide treatment for pain relief in the emergency department of the Northern Hospital.
Groan.
**For further discussion of the apparent equal effectiveness of acupuncture in this study, see the comments below by AJ and sansscience(me).
Okay, they did electroacupuncture on humans, but the cat photo is so much cuter. Image: holisticvetmed.co.uk
NHMRC funding
I couldn’t find an emergency department acupuncture pilot study at Northern Hospital on Pubmed, so it looks like the NHMRC has provided $400,000 based on an uncontrolled pilot study published in a journal which isn’t indexed on Pubmed and has no ISI impact factor.
This paper appears to be the study design of their new NHMRC-funded study, and it also has no sham acupuncture control group – so the study has no control for any of the important acupuncture/consultation placebo effects, discussed above. If this is correct, then this study will be absolutely useless for determining the effectiveness of acupuncture (which should also be disappointing for acupuncture proponents, because even if it does appear to show a positive result for acupuncture, it won’t be possible to know if it’s real or a placebo effect).
If this is the sort of pilot data on which NHMRC funding was based, and the sort of uncontrolled study it’s funding, then it’s an appalling waste of $400,000 of NHMRC funds, which could be spent on so many other worthy (and placebo controlled) projects. [We get so annoyed about this not just because it's a waste of money, but because it's taking money away from other funding. We all know good scientists with lots of quality publications who didn't have their NHMRC renewed, and right now their postdocs & RAs are out of work, wasting decades of training and expertise, and some of them are leaving science because they don't want to have to put up with the horrendous uncertainty of our funding system, playing Russian roulette with their job every few years.]
RMIT Chief Investigator Prof Charlie Xue says: “The importance of delivering effective analgesia to patients with acute emergency presentations and the lack of evidence for acupuncture in this setting points to a gap in the current knowledge base, which our research aims to fill”.
Alternative therapy of the gaps
There have been criticisms of the billions of dollars the NIH’s NCCAM has spent investigating alternative therapies, for no major positive outcomes. Part of the problem with alternative therapies is that they tend to claim to be able to treat every disease under the sun. They can quite correctly claim that their particular therapy hasn’t been scientifically tested against a long list of diseases and therefore scientists can’t say it doesn’t work against them (ie. Test XXX against arthritis. Does it work better than a placebo? No. Try diabetes. Try irritable bowel syndrome. Try lower back pain. etc. etc.) Instead of God of the gaps, it’s CAM of the gaps.
Many scientists say that unconventional treatments hold promise and deserve serious study, but that the federal center needs to be more skeptical and selective. “There’s not all the money in the world and you have to choose” what most deserves tax support, said Barrie Cassileth, integrative medicine chief at Memorial Sloan-Kettering Cancer Center in New York. – ABC news
Friends of Science in Medicine
Friends of Science in Medicine
Their position:
We are not trying to stop public access to alternative therapies if they are fully informed about their lack of, or minimal evidence for, safety and efficacy.
We are in favour of discussing the place of alternative therapies, their placebo effect and the testing of these therapies in well-designed trials.
A naturopathy academic in The Conversation article:
The world of CAM is not a “homogenous entity”, said Dr Wardle. “There is a lot of crap, but there’s good stuff, and treating it like it’s all the same thing is very, very fraught.
Very occasionally alternative therapies are shown to work by rigorous research and become part of medicine – what is not acceptable is for entire Australian university degrees to be teaching disproved nonsense as ‘science’. The naturopath also argues that CAM degrees at universities teach basic health science making practitioners less dangerous – making them a bit less dangerous doesn’t make it okay to teach university science courses in pseudoscience, like homeopathy, iridology and kinesiology. And if he isn’t willing to publicly delineate these courses as pseudoscience, then how can he expect all alternative medicine academics not to be lumped into the same basket?
For example, at Australian universities students are taught that homeopathy can treat serious, or sometimes deadly, medical conditions, such as asthma, autism, diabetes, mental illnesses, and influenza and other infectious diseases. It can’t be argued they’re being taught to think about CAM critically, because these examples come from the university health clinics where students are being taught to put this dangerous nonsense into practice on real patients.
Either students are being taught to think critically about evidence-based science, in which case they’re doing a whole degree in fairy studies and coming out the other end understanding that most/all of their course was a waste of time (cognitive dissonance, much?), or these universities are teaching students to believe in and practice woo and mumbo jumbo.
There’s a summary of acupuncture treatment effectiveness, with references, on the NIH NCCAM site.
And in my travels on Pubmed I found this amusing emergency department case report: Pain in the Neck: the Enigmatic Presentation of an Embedded Acupuncture Needle .
beej
29/01/2012
I’m not sure what’s more impressive: $400,000 of NHMRC funds being allocated to acupuncture research, or the sheer number of links you’ve managed to get into this article. Yay for you, it’s like you’re giving The Conversation a lesson in how to write and reference an article!
You should post the link to this page in the comments section of the Conversation article, just stating ‘This is my opinion.’ It would be funny.
AJ Haido
01/02/2012
I have to admit you’ve done a good job with this, but I’d like to point out something:
“There was no significant difference in any of the outcome measures (pain score, respiratory rate, heart rate, and systolic and diastolic blood pressure), between the normally treated group and the acupuncture group. At best this shows that acupuncture isn’t any improvement on current practices”
Actually when you think about it a supervised Chinese medicine student (not even a graduate yet) inserts a few cheap disposable needles and has the same outcome as an experienced doctor in the emergency department. That’s a pretty impressive result! Not to mention that conventional pain medications often have potential adverse effects (aspirin and NSAIDs-stomach ache, paracetamol-liver damage, codeine-constipation etc)
You do have a point about the lack of placebo control in this one, although I wonder how ethical it would be to use a placebo on a patient looking for relief from acute pain at their local hospital? Don’t you think it at least deserves further investigation?
I agree you have done a much better job than the conversation at least.
sansscience
02/02/2012
Hi AJ, yeah, it’s a good point about the apparent equal effectiveness, which I thought about when I was writing the article (I swear I really did!) but decided not to discuss it in the blog post it since the evidence for it in the paper was so confusing and the blog post was too long already!
First though, your other points:
1. Ethics If the ethics committee is happy with these patients delaying their medication to receive acupuncture, then they’d probably be fine with them having an equally effective ‘sham’ acupuncture control group (they excluded patients with medical emergencies and acute pain – it would be totally unethical to treat them with acupuncture).
2. Side effects You’re right that the idea of treating people with a side-effect-free placebo that leaves them feeling better is a great idea, but the problem is medical ethics. The placebo medication/therapy will only work if you don’t tell the patient that it’s a placebo. But do you think it’s ethical to give someone a placebo without telling them it’s not real medication?
Also, check out the acupuncture side effects (nocebo?):
Okay, so the numbers weren’t significant, but there was no significant difference in pain in treatment vs control and that was proclaimed in the abstract and discussion – funny how they didn’t mention the equal side effects ever again!
3. Apparent equal effectiveness
While theoretically they recruited patients in triage categories 3-5, which includes fractures and heavily bleeding cuts, they actually excluded patients in “acute pain” and “haemodynamic instability or suspected critical illness”. That left them with patients who didn’t require emergency medical attention; they specify that they only recruited patients with conditions such as headaches, panic attacks, colds, menstrual pain and back pain.
But I can’t tell you what conditions the patients had – it’s ridiculous that they don’t provide ‘patient characteristics’ (made up example: in the control group 9/12 patients had headaches, 3/12 back pain; in the treatment group 2/12 had headaches, 10/12 had back pain). Randomisation isn’t perfect, especially with small sample sizes, and couldn’t different conditions respond differently to pain treatment?
Since recruited patients didn’t need emergency medical treatment you would expect both the acupuncture and control groups should have similar medical outcomes (although the acupuncture group might be a bit more relaxed).
That just leaves subjective self-reported pain. Patients with lower-grade pain and minor injuries, are possibly more likely (than severe acute pain) to respond to the placebo effect of a 40 min appointment with a nice acupuncturist who listens to your problems and tells you they’re going to make you pain better.
Nowhere in there does it say who or how many in each treatment group received what medication and when. They say:
If they didn’t give any pain medication before the acupuncture, how did it affect their recruitment? If I offer an acupuncture trial to someone in severe pain, they might be more likely to say ‘No thanks, I’ll take the morphine’. You’re also more likely to get participation from people who already believe that acupuncture could help (in which case the placebo effect is stronger, darn, I’ve got a reference for that effect somewhere …).
We can’t conclude if the acupuncture was similarly/more/less effective than the conventional pain management – because the study doesn’t give any information about what, if any, pain management medication was given to the patients.
Like I said before, the paper was abstruse, so maybe you can find something I missed/misunderstood in the paper that explains some of this, but all in all, I don’t think they provide enough information so that you could conclude anything about the effectiveness of acupuncture compared to normal treatment. The hospital was probably just grateful for them getting people with headaches out of the emergency department.
AJ Haido
03/02/2012
Hi again sansscience,
Placebo ethics is a funny one, since we know placebo has profound effects in any kind of medical intervention, but I would guess that it is unethical to intentionally give any patient a placebo instead of a “real” treatment when they are seeking medical aid – even if the placebo is as good as the real acupuncture, which is as good as the standard hospital treatment.
And thanks for informing me about “nocebo” which I wasn’t familiar with – and is a whole new kettle of fish for me to think about.
In regards to the patients recruited suffering “headaches, panic attacks, colds, menstrual pain and back pain”, these can all range in severity from mild inconvenience to rolling around in bed crying and wanting to die.
I would think if a patient has bothered to wait in line at their hospital they would be closer to the latter end of the spectrum…although you are right it could have been clearer.
I think the great thing about this project is that it shows that if we adopt a model in our hospitals similar to standard practice in China that we can have equally good results for our patients but also save money (acupuncture is very cheap) and clear up the queues for the other patients who would not be expected to benefit from acupuncture.
sansscience
03/02/2012
Hi AJ, Yeah, I agree I was being a bit flippant – people with those things can be feeling really ill – but the article does itself a disservice by not giving any indication of what illness participants had or their severity.
I agree that things like this can be good ideas – but the problem is they remain ideas until someone actually does a decent study to show whether they work or not.
As for ethics committees and placebos – it’s unethical to give a placebo when there is already a treatment that has been shown to work, so the patient’s health will significantly suffer from not getting it. But that’s not the case in this study – they’re just delaying giving them Panadol by 40 minutes. Otherwise placebos are usually fine and a good thing – an ethics committee would far rather you did a trial with a placebo and produce a meaningful result, than do a shoddy trial and waste everyone’s time. If you want some examples of this happening in real life, here’s a Pubmed search for placebo and emergency http://www.ncbi.nlm.nih.gov/pubmed?term=placebo%5BTitle%5D%20AND%20emergency%5BTitle%5D
or at the MJA http://www.mja.com.au/cgi-bin/s.cgi?np=0&o=1&q=placebo+emergency&s=RPD
Standard medical practice in Chinese hospitals is something we’d want to adopt?? – I have no idea how well Chinese public hospitals work, but it is a (rapidly) developing country run by an authoritarian regime, which has a life expectancy of 73.1 years compared to 81.5 years here in Australia (Source: World Bank, World Development Indicators). I know lots of other factors like poverty contribute to that, but I don’t think you can argue they clearly have wonderfully better health outcomes than us when they’re dying younger. I’d be interested to see good independent [independent of the Chinese govt, which hates more than anything to lose face] evidence that health outcomes are better in China to back up your assertion, before we discuss it seriously.
AJ Haido
04/02/2012
Hi sansscience, It’s too hot today for me to try to come up with a clever argument, and Chinese national health statistics isn’t my strong point either…
Anyway, my real motive for writing in the first place is that I don’t want to see Friends of Science in Medicine succeed in getting CAM removed from Universities or health insurance refunds, but there are so many comments about it already, for example here!… htp://www.change.org/petitions/protect-university-education-in-natural-medicine
By the way thanks for posting, I feel we’ve actually had a “conversation” compared to some other websites I’ve looked at where there’s a lot of:
“You are all biassed and can’t see the truth!”
“No, YOU are all biassed and can’t see the truth!”
sansscience
04/02/2012
Thanks, and I responded because I thought you sounded like you could have a reasonable conversation!
Up until now most of the comments on my little blog have been a couple of friends who read it – but the comments from strangers have gone a bit crazy over on the ‘New Friends of Science in Medicine’ post, and there doesn’t seem any point engaging with most of them – like what you said “You’re all biassed and can’t see the truth!”. Sigh.