The Challenge of Studying Acupuncture
Below is the text of an article that I collaborated on. My two colleagues provided the scientific heavy lifting while I contributed my fledgling editorial skills, along with my experience of providing acudetox over the past 20 years. Plus, funnily enough, I participated in the study being examined. I have learned a lot, both from being in the study and then looking at the study from a critical eye. My understanding of evidence based decision making certainly is enhanced!
An edited version of this article was published in NADA’s (National Acupuncture Detoxification Association) newsletter “Guidepoints”. Go to acudetox.com for more info.
QUESTION THE DESIGN, NOT THE ACUPUNCTURE
Don Himmelman RT; Elizabeth B. Stuyt, MD; David Wurzel, PE, MAc, LAc
The Journal of Substance Abuse Treatment will soon publish, in print form, the results of a research study that has sparked yet another debate about the efficacy of acu detox. This article will examine the study in detail and present why such misleading conclusions are not warranted. In addition, some of the inherent challenges of conducting randomized trials of acu detox will be identified.
Critics of natural health modalities like acupunctures often site studies such as this wherein the outcomes did not show a statistically significant difference between the acupuncture intervention and the other treatment control groups to which it was compared. What is not often pointed out is that failure to show statistical relevance does not prove the treatment did not work, but rather the study’s design was not able to show significant differences between the treatment under investigation and the interventions to which it was compared.
The Research Study
Determining the efficacy of auricular acupuncture for reducing anxiety in patients withdrawing from psychoactive drugs – has been published electronically and was a combined effort of Addiction Prevention and Treatment Services, Capital Health and School of Health Sciences, Dalhousie University Halifax, Nova Scotia, Canada. This study was the team’s first attempt at examining the effects of acu detox.
The study’s goal was to determine, in a randomized, placebo-controlled fashion, if anxiety in patients in acute withdrawal from psychoactive substances was affected by any of three different treatment interventions. All study participants also received conventional addiction treatment (e.g. individual and group counseling, etc.). The three interventions studied were:
• The 5-point acu detox protocol
• Sham acupuncture (5 needles inserted into the helix of each ear)
• A “treatment setting control (relaxation)”
Study participants were randomly assigned to one of the three groups. Each participant was to attend his or her assigned group three times over a two-week period. All study interventions were offered in the same room with dimmed lights and soft, soothing music and each participant was given herbal tea.
The researchers hypothesized that auricular acupuncture, “reduces the anxiety associated with withdrawal from psychoactive drugs”. However, the study design and subsequent data analysis precluded their ability to prove this hypothesis.
Challenges with methodology/design
The authors initially calculated that at least 22 subjects per treatment group would provide statistical significance and all three randomized groups started with many more participants than this minimum. However, due to dropouts common to this population, in none of the three groups did at least 22 participants receive the required three interventions. By the authors’ own calculations this did not provide sufficient power to determine significance and thus no conclusions should have been drawn from these data.
It is unclear what supporting data the researchers used to base their decision to give only three treatments total over a two-week period. Depending on the psychoactive substance from which a person is withdrawing, symptoms such as anxiety can last for several weeks. Pharmacological-based treatments for anxiety such as benzodiazepines are usually given several times a day for an average of 3 to 7 days. Many NADA-style programs offer acu detox one or more times daily to individuals in acute withdrawal. Thus, the infrequency of the treatment interventions for all three groups most likely played a roll in the study failing to prove its hypothesis.
In addition to the frequency of treatment, there are also anecdotal reports that acupuncture has a cumulative effect. He et al. (Effect of acupuncture on smoking cessation or reduction: an 8-month and 5-year follow-up study. Preventive Medicine 2001 (33): 364-372) noted that in the studies reporting no lasting effect of acupuncture for tobacco cessation, the treatment was given at most once per week. These authors concluded the lack of effect found in these studies was a consequence of too long a time between each treatment. Most studies in addiction treatment demonstrating benefit of the NADA protocol utilize 3-5 treatments per week for several weeks. This highlights the need for studies to determine the optimal treatment dosage and frequency for acu detox for individuals in acute withdrawal.
Control Group Composition
The control group for this study was the relaxation group. But because all patients in addiction treatment programs receive some level of treatment this was not a true “no treatment” control. The authors referenced, “several randomized controlled trials have shown a single application of AA [auricular acupuncture] capable of significantly reducing perioperative anxiety in a variety of clinical settings.” These trials compared receiving acupuncture to a no treatment control group where acupuncture treatment was found effective compared to no treatment. This study lacked a control group in which the participants either received no intervention (i.e. just conventional addictions treatment) or were put in a “wait-list” group (one in which no treatment at all was given). Including a proper control group may have elicited the differences the authors were hoping to see.
Challenges with measurement
Anxiety levels were determined both by clients’ self-report and via physiological indictors. Clients filled out The Spielberger State-Trait Anxiety Inventory (STAI; Y-1 and Y-2) for adults when recruited into the study and again within 5 minutes before and after treatment. This 40 items scale differentiates between “state anxiety” (acute anxiety “in the moment”) and “trait anxiety” (chronic background level of anxiety). Heart rate and blood pressure were collected at the same time as the physiological indicators.
Upon recruitment only trait anxiety data was provided , but not state anxiety. Conversely, only state anxiety was analyzed pre- and post-treatment but not trait anxiety. This is unfortunate, as An analysis of trait anxiety throughout the study might have revealed differences due to the cumulative effects of treatment, including the conventional treatment each study participant received.
In the text of the article the researchers report that among, “the four major primary presenting problems (alcohol, cannabis, cocaine, or nicotine)” there, “are significant differences in the baseline trait anxiety scores” between those who reported cannabis or nicotine as their primary presenting problem. However, later in the article the authors state, “There were no statistically significant differences in the baseline trait anxiety scores among the four groups”. A discrepancy such as this calls into question the reliability of the entire study.
The study did report that all three groups showed statistically significant reductions of state anxiety over the 45 minutes of treatment interventions but there was no difference between the groups. Because The Spielberger state anxiety scores are indicative of the individual’s anxiety level in the moment, most likely all the authors measured was the effect of having people sit in a darkened room with soft, soothing music sipping herbal tea for three-quarters’ of an hour.
In addition, depending on the herbs in the tea, its effects might supersede those of any acupuncture treatment. This phenomenon highlights the challenges in measuring the effects of NADA-style treatments when the acupuncture is combined with herbal tea and/or ear seeds or magnets.
Challenges with sham/placebo controls
There is a debate in the field of acupuncture research as to the use of sham controls. As Moffet (Moffet HH. J Alternative and Complementary Med 2009; 15(3): 213-216) has so eloquently pointed out, sham acupuncture may be as efficacious as true acupuncture. In this study, one of the control groups had 5 needles inserted into the helix of both ears. It should be noted that the helix is not an inert area of the ear with no acupuncture points. While the helix points are not generally used for relaxation, there are points in that region that are used to increase alertness, decrease sexual compulsion, reduce fever, decrease blood pressure and treat skin disorders to name a few. Furthermore the auricular branches of the trigeminal nerve and cerebral cervical plexus nerves innervate the the helix and these nervous system branches can certainly be stimulated by the insertion of an acupuncture needle. It appears the authors were only referring to the NADA protocol as “auricular acupuncture” in the title of their article, but in all likelihood the sham group also received auricular acupuncture albeit with different points.
We commend this group’s attempt to analyze acu detox more closely in a randomized, placebo controlled fashion. While acupuncture will always have its critics, research utilizing tools and techniques sensitive enough to measure the subtle internal changes in individuals receiving acu detox will go a long way in this technique being accepted by the greater addiction treatment community. The conclusion of any study is only as strong and valid as its design and methodology.
It is also vital to remember that scientific study is but one quarter of the evidence required for proper evidence based decision-making. The other three factors, all of equal importance, are: the clients’ or patients’ experience, the clinicians’ experience, and the cultural relevance of the intervention (is it exclusive to certain populations or does it have universal applications?). Becoming overly focused on generating measurable scientific results can lead to a myopic view that excludes other equally important evidence.