Pharma bias destroys ITC quit smoking medication study
What happens when the senior researcher on a study analyzing the real-world value of approved stop smoking products is also a quitting product salesman serving on Pfizer's speakers bureau, and a paid consultant to GlaxoSmithKline?
I have no doubt that K. Michael Cummings, PhD, MPH, is a good, honest and decent man. But what's it like inside a mind that generates medical journal research findings and conclusions regarding the exact same quit smoking products for which the pharmaceutical industry is paying them consultation fees?
Do these researchers feel or appreciate the conflicts they live? If necessary, how many have the courage and independence to bite the hand that feeds them?
Here's the problem. Until now, while approved quitting products have clobbered placebo controls inside randomized clinical trials,, trials which by now all know were not blind as claimed, they get clobbered just as badly by cold turkey quitters in real-world use. In fact, until now, nearly every long-term population level finding since 2000 has found approved quitting products no more effective than quitting without them, and in most cases significantly worse. Here's a recent example.
The new study co-authored by K. Michael Cummings is featured online in Addiction, a journal whose editor-in-chief is Robert West, arguably the UK's most financially conflicted quitting product researcher. The new study is entitled "Effectiveness of stop-smoking medications: findings from the International Tobacco Control (ITC) Four Country Survey."
Below is the full-text of a critical review of the new ITC study by physician-professor Michael Siegel of the Boston University School of Public Health. Dr. Siegel raises a number of concerns regarding the study, which if valid effectively destroy its worth in valuing the population level worth of NRT, bupropion and varenicline. Readers can comment on Dr. Siegel's "The Rest of the Story" blog at his website. Dr. Siegel wrote a follow-up review on August 21 questioning the assertion that the study was "prospective."
As Dr. Siegel's initial review notes, after an earlier International Tobacco Control Policy Evaluation Project (ITC) survey study found cold turkey smoking cessation twice as effective as gradual weaning/tapering, that finding gets totally ignored. Instead, Cummings and his co-authors present ITC survey data in a selective fashion that makes Chantix, Zyban and the nicotine patch appear vastly superior to a "no medication" control group that includes gradual weaning/tapering quitters.
One billion smoking related deaths projected before century's end, it's my hope that sharing Dr. Siegel's review motivates financially conflicted smoking cessation researchers to reflect upon the natural biases created by working for the quitting product industry.
New Study Reports Effectiveness of Smoking Cessation
Medication But is Biased Towards Finding an Effect;
Financial Conflict of Interest Present
Professor Michael Siegel, M.D.
A new study published online ahead of print in the journal Addiction reports survey data purporting to show that quit attempts using medication were more successful than unaided attempts. The study uses data from an international survey conducted in the United Kingdom, Canada, Australia and the United States to study smoking and quitting behavior.
The study methods are described as follows: "A total of 7436 adult smokers (18+ years) selected via random digit dialing and interviewed as part of the International Tobacco Control Four Country Survey (ITC-4) between 2002 and 2009. Primary analyses utilized the subset of respondents who participated in 2006 or later (n=2550)." The main outcome of interest was successful quitting at either 1 month or 6 months from the quit attempt.
The chief results of the paper were reported as follows: "Among participants who recalled making a quit attempt within 1 month of interview, those who reported using varenicline, bupropion or nicotine patch were more likely to maintain 6-month continuous abstinence from smoking compared to those who attempted to quit without medication [adjusted odds ratio (OR) 5.84, 95% confidence interval (CI) (2.12-16.12), 3.94 (0.87-17.80), 4.09 (1.72-9.74), respectively]; there were no clear effects for oral NRT use."
The study concludes as follows: "Consistent with evidence from randomized controlled trials, smokers in the United Kingdom, Canada, Australia and the United States are more likely to succeed in quit attempts if they use varenicline, bupropion or nicotine patch. Previous population studies that failed to find an effect failed to control adequately for important sources of bias."
The Rest of the Story
This is a complicated study and there are many aspects to the rest of the story, so let me take them one at a time. The study procedures are extremely complex because of its retrospective design, so I will do my best to explain these important flaws.
1. Many Medication Users Were Thrown Out of the Study
The most important aspect of the study for readers to understand is that its methods were biased so as to find a higher effectiveness of smoking cessation drugs. How was this achieved?
It was achieved by throwing out from the study a significant proportion of medication users, but not using any similar procedure to exclude non-medication users, all of whom were included in the study.
Any time that you treat the intervention group different than the control group, you introduce a bias into the study and in this case, that bias works towards finding medication use to be more effective.
Specifically, here is what happened in the study: Rather than simply including all medication users (that is, reporting the quit rates for all smokers who used medication in their most recent quit attempt), the study excluded certain smokers who used medication based on their response to a question about their reasons for using smoking cessation medication.
The study explains this exclusion as follows: "During each survey wave, respondents were asked to recall their use of medications since the last survey, and those who reported using medications were asked a series of questions regarding the medications indicated, including: 'What was the main reason you used [the medication]?'. Only those who reported using medication in an attempt to stop smoking completely were considered to be medication users for the purpose of these analyses."
In other words, the study did not define medication users as being "medication users." Instead, it defined medication users as subjects who, after having tried medication to try to quit smoking and potentially failed, retrospectively report their use of medication was to intended to help them quit smoking.
Remember that the only indicated purpose for the use of smoking cessation drugs is to quit smoking. These drugs are not approved for any other purpose. So why exclude anyone who reported using smoking cessation medications? And if you are going to exclude smoking cessation medication users based on their intended purposes (reported after the fact), then why not also exclude unassisted quitters based on their reported intentions, also after the fact?
The way the human mind works, people are likely to alter their perceptions of why they used a particular approach after they observe the result of that approach. For example, suppose I am a two pack-per day smoker and I try to quit using NRT. I fail. Instead, I am able to cut down a little. Then, a few months later, I am asked to report the reason I used the NRT. To save face, I might (subconsciously) decide that the real purpose of my using the NRT was to cut down on the amount I smoke, rather than to quit completely.
Importantly, the same phenomenon might occur with a non-medication user. A person might initially decide that they are going to try to quit (without medication). But they might find it too difficult. So instead, they decide to just cut down. When asked a few months later why they decided to make a "quit" attempt, they might respond that they were actually trying to cut down rather than to try to stop smoking completely.
The effect of this procedure would be to systematically exclude smokers who have used smoking cessation medication unsuccessfully. In other words, this procedure would artificially bolster the observed smoking cessation rate for medication users.
To make matters worse, the question about the purpose of using smoking cessation drugs referred to any use of these drugs during the past year, not to the most recent quit attempt. Therefore, someone might have used NRT to try to cut down about six months ago and then used NRT to try to quit more recently. But they might still answer the question about the use of NRT as indicating that their intent was not necessarily to quit completely.
The bottom line is that smoking cessation medications are smoking cessation medications. Any study which excludes users of smoking cessation medications from the analysis, especially retrospectively, is going to bias the results towards finding an artificially high cessation rate for medication users.
2. Cold Turkey Quitters May Have Been Included in the Medication Group
For wave 5 and earlier of the survey, the study classified users of medication based on whether they had used any smoking cessation medication since the previous survey. However, it does not appear to have assessed the method used to quit during the most recent quit attempt. This question does not appear to have been added to the survey until wave 6. Therefore, it seems entirely possible that for wave 5 and earlier, a person might have tried and failed a few times to quit using NRT and then decided to go cold turkey for their most recent quit attempt. But because this person is not asked to report the method used in their most recent quit attempt, they would be classified as a medication user. If they were successful in their cold turkey attempt, that would go down in the results as a success for the use of drugs, not a success for unassisted quitting.
While the study describes itself as a "prospective cohort" study, it really is a "retrospective cohort" study because the assignment of exposure status is being made after the fact.
What might be the effect of this misclassification of exposure? If it were true that cold turkey quitting is more effective than NRT use, then this misclassification would mask that effect. It would result in successful quit attempts going down as being attributed to medication when they are really due to an unaided quit attempt.
In my view, the inability of the study to determine definitively the method used for the most recent quit attempt makes it suspect. Much more useful would be true cohort studies in which the classification of exposure is meaningful. An example of such an approach is the study reported by the UK National Health Service. In that study, callers were assigned to receive or not receive medication and then followed up to observe their cessation rates. The results, as I reported last week, showed no significant advantage to the use of smoking cessation drugs.
3. The Study Results Depend on the Exclusion of Anyone Other than Those Making Quit Attempts in the Past Month or Two
The study finds that the effectiveness of medication greatly increases among subjects who report having made their most recent quit attempt in the past month or two. When all quit attempts are considered, there is no significant effect of nicotine gum, the nicotine patch, or bupropion.
The study justifies the exclusion of the bulk of its data on the premise that unsuccessful cold turkey quitters are more likely than unsuccessful medication quitters to forget that they have attempted to quit smoking. On its face, this seems implausible. Would not a person making a cold turkey attempt remember that they made such an attempt? More importantly, the data purported to demonstrate this effect fail to do so.
The premise that failed cold turkey quit attempts are not remembered as easily as failed medication quit attempts is based on the finding that failed quitters who used medication are more likely to report a greater time since their quit attempt than failed quitters who did not use medication. The assumption is that the medication users made additional attempts to quit without medication more recently but forgot about those quit attempts. However, the paper provides no evidence to back up this assertion. It only demonstrates that those failed quitters who used medication reported more remote failed quit attempts than failed quitters who did not use medication.
There is a very reasonable alternative explanation for this finding, which is that when medication users fail in their quit attempts, they are very discouraged from making additional quit attempts. In contrast, cold turkey quitters do not face the same discouragement because they can always rationalize that they can do things differently. But if medication fails, it may be more likely to lead to a sense of hopelessness, which could well deter quit attempts for quite some time.
4. Non-Cold Turkey Quitters Were Included in the Unassisted Cessation Group
The real question of interest is not simply whether medication-assisted quitting is more effective than unassisted quitting, but whether medication-assisted quitting is more effecting than unassisted cold turkey quitting. I am not aware of anyone advocating for unassisted quitting using a gradual reduction approach. The question of interest is how an abrupt attempt at smoking cessation that is unassisted compares with medication-assisted quit attempts.
Unfortunately, this study does not report the effectiveness rates for cold turkey, unassisted quitting. Instead, it lumps together all unassisted attempts to quit, even those in which the smoker cut down gradually, an approach that has been shown to be ineffective and which no one is advocating. By including these attempts and reporting them in the unassisted quitting figures, the study again biases the results towards finding a greater effectiveness of medication-assisted quitting.
The comparison which nobody seems to want to make is smoking cessation drugs versus cold turkey quitting without drugs. But that is the relevant research question.
Importantly, a large proportion of the "unaided" quitters in the ITC Four-Country Survey used a gradual reduction, rather than a cold turkey approach. In fact, approximately one-third of smokers used a gradual reduction approach. Therefore, the inclusion of these subjects creates a substantial bias which artificially lowers the reported effectiveness rate of unassisted quitting, or at least fails to provide the key data of interest to the research question.
Putting it All Together
It strikes me that those who are trying to explain away the consistent finding that drugs do not appear to be much better than cold turkey quitting in population based studies (outside the context of clinical trials) are now grasping at straws, making valiant efforts at reasoning away the clear findings of these population-based studies.
Here, the study has created a premise for the entire analysis (that there is differential recall of failed cold turkey vs. medication quit attempts), a premise which is based not on the demonstration that cold turkey quit attempts tend to forget their quit attempts, but on the observation that medication users recall more remote failed quit attempts than unassisted quit attempters. Not considered is the possibility that the findings are real (rather than based on memory defects). Is it not possible that failed smoking cessation attempts are likely to deter further quit attempts for a longer period of time than failed unassisted quit attempts?
Furthermore, the study introduces two sources of potential bias, both of which would bias the results toward finding a higher rate of effectiveness of medication. The more troubling of the two is the exclusion of medication users based on a question, asked retrospectively, about the reasons for their use of stop smoking medication. In my view, once you start excluding certain users of medication, you are no longer playing fair. Smoking cessation drugs are smoking cessation drugs and to exclude failures based on smokers retrospectively reporting that they actually didn't have the desire to completely quit smoking is not a fair and balanced analytic approach.
Perhaps the most telling fact about this study is that, as you might expect based on the various biases in the study, one of the co-authors has a significant financial conflict of interest with companies that manufacture smoking cessation drugs.
According to the study's disclosure statement, one of the study investigators: "has served as a paid consultant on smoking cessation to Pfizer and Novartis, [and] has received payment from Pfizer and GlaxoSmithKline for lectures on smoking cessation to health professionals."
I am not suggesting that there is any conscious bias but subconsciously, these kinds of financial connections are going to influence the slant with which one conducts, analyzes and reports results. In this case, there is ample evidence from an analysis of the study that there is indeed bias towards finding an effect of smoking cessation medications. There is nothing wrong, since the conflict was appropriately disclosed. However, readers should take this conflict into account when interpreting the study results.
1. Fiore MC, et al, Clinical Practice Guideline - Treating Tobacco Use and Dependence: 2008 Update May 2008 PHS; see critical review at: Polito, JR, U.S. quit smoking policy integrity drowns in pharmaceutical influence, WhyQuit.com, May 13, 2008
3. Doran CM, Valenti L, Robinson M, Britt H, Mattick RP. Smoking status of Australian general practice patients and their attempts to quit. Addictive Behaviors, May 2006, Volume 31(5), Pages 758-765.
4. Hartman AM. What does US national population survey data reveal about effectiveness of nicotine replacement therapy on smoking cessation? National Cancer Institute paper presented at World Conference on Tobacco or Health, 12-15 July 2006, Washington, DC. FREE Full Text (see charts on PDF pages 35-37); Alberg AJ, Patnaik JL, May JW, Hoffman SC, Gitchelle J, Comstock GW and Helzlsouer KJ, Nicotine replacement therapy use among a cohort of smokers, Journal of Addictive Diseases 2005, Volume 24(1), Pages 101-113. Abstract; Pierce JP, Cummins SE, White MM, Humphrey A, Messer K, Quitlines and Nicotine Replacement for Smoking Cessation: Do We Need to Change Policy?, Annu. Rev. Public Health 2012. 33:12.1–12.16 Abstract; Ferguson J, Bauld L, Chesterman J, Judge K, The English smoking treatment services: one-year outcomes, Addiction. 2005 Apr;100 Suppl 2:59-69. Abstract; Pierce JP, Gilpin EA, Impact of over-the-counter sales on effectiveness of pharmaceutical aids for smoking cessation, JAMA. 2002 Sept. 11;288(10):1260-4. FREE Full Text; Alpert, HR, Connolly GN, Biener, L, A prospective cohort study challenging the effectiveness of population-based medical intervention for smoking cessation, TC Online First, January 10, 2012. Abstract
6. Addiction, Ethical Policy, Editors' Declarations of Interest, Robert West, http://www.addictionjournal.org/pages/ethical-policy, Last Updated: Unknown; Site Viewing Date: August 20-21, 2012. Interestingly, West's Addiction disclosure states, "He undertakes lectures and training in smoking cessation methods which have led to payments to him personally and to his institution," without indicating who made the payments. Additionally, a 2011 disclosure by West notes that he holds "a patent for a novel nicotine
delivery device" (see Addiction, 2011, Volume 106, Page 680).
7. Kasza KA, Hyland AJ, Borland R, McNeill AD, Bansal-Travers M, Fix BV, Hammond D, Fong GT, Cummings KM, Effectiveness of stop-smoking medications: findings from the International Tobacco Control (ITC) Four Country Survey, Addiction. August 14, 2012. doi: 10.1111/j.1360-0443.2012.04009.x. [Epub ahead of print], PubMed Abstract
8. Siegel, M, New Study Reports Effectiveness of Smoking Cessation Medication But is Biased Towards Finding an Effect; Financial Conflict of Interest Present, The Rest of the Story: Tobacco News Analysis and Commentary, August 20, 2012 http://tobaccoanalysis.blogspot.com/2012/08/new-study-reports-effectiveness-of.html
9. Siegel, M, New Study on Effectiveness of Smoking Cessation Medications is Really a Retrospective, Rather than Prospective Cohort Study; Bias in Defining Exposure Groups Threatens Validity of Results , The Rest of the Story: Tobacco News Analysis and Commentary, August 21, 2012 http://tobaccoanalysis.blogspot.com/2012/08/new-study-on-effectiveness-of-smoking.html
10. Cheong Y, Yong HH, Borland R, Does how you quit affect success? A comparison between abrupt and gradual methods using data from the International Tobacco Control Policy Evaluation Study, Nicotine & Tobacco Research, 2007 August; Volume 9(8), Pages 801-810. PubMed Abstract; Free Full Text
Any factual error will be immediately corrected upon receipt of credible authority
in support of the writer's contention. E-mail comments to firstname.lastname@example.org
Share this article
PDF copy for saving or printing
Related Links and Articles
- Cold turkey clobbers e-cigs, NRT and Chantix, John R. Polito, 07/18/18
- JNCI study finds pharmaceutical stop smoking aids ineffective, John R. Polito, 12/22/17
- Nicotine cessation: lost in the deserts of Abu Dhabi, John R. Polito, 03/16/15
- 10 studies screaming "leave replacement nicotine alone!" John R. Polito, 01/01/15
- The Surgeon General's Reports: 50 years of cessation ignorance, John R. Polito, 01/20/14
- Australia's half-baked quit smoking campaign, John R. Polito, 11/3013
- The Great American quit smoking sham, John R. Polito, 11/18/13
- BMJ's non-publication policy should extend to pharma's nicotine, John R. Polito, 10/19/13
- Gallup Poll: cold turkey 48 times smarter than Nicorette, John R. Polito, 08/26/13
- Cold turkey quitting still king, John R. Polito, 07/15/13
- Pharma bias destroys ITC quit smoking medication study, John R. Polito, 08/20/12
- Potent warnings, horrible quitting advice: what's the CDC smoking? John R. Polito, 03/17/12
- Replacement Nicotine's Killing Fields, John R. Polito, 02/11/12
- Chantix study joins negative findings omissions club, John R. Polito, 12/22/11
- Dying truths about quitting smoking methods, John R. Polito, 11/14/11
- Bias against cold turkey quitting apparent in literature, Dr. Michael Siegel, 11/03/11
- Suicidal behavior and depression in smoking cessation treatments, Moore TJ et al, 11/03/11
- Study finds cold turkey quitting more effective but concludes opposite, Dr. Michael Siegel, 11/01/11
- Tar wars over smoking cessation, BMJ Link, Professor Simon Chapman, 08/08/11
- Is a 14% Chantix success rate worth risking death?, John R. Polito, 06/14/11
- Chantix fails to defeat nicotine patch, John R. Polito, 03/30/11
- Chantix serious injuries exceed 8,500, John R. Polito, 01/28/11
- Champix - Chantix worth questioned - John R. Polito, 01/06/11
- GlaxoSmithKline Continues Defrauding Smokers - John R. Polito, 11/02/10
- Little communication in failed war against smoking - John R. Polito, 08/08/10
- Smokefree.gov really Buymeds.now - John R. Polito, 03/24/10
- Five facts Chantix ads keep hidden - John R. Polito, 12/29/09
- Pfizer's Chantix Continues Killing Quitters - John R. Polito, 10/22/08
- Chantix and Champix mental health safety assurances contrary to evidence - John R. Polito, 06/12/08
- Chantix blamed for 3,063 serious injuries and 78 deaths - John R. Polito, 05/13/08
- Strong Safety Signal Seen for New Varenicline Risks - ISMP, 05/21/08
- U.S. quit smoking policy integrity drowns in pharmaceutical influence - John R. Polito, 05/13/08
- Does updated tobacco treatment "Guideline" reflect sham science? - John R. Polito, 05/05/08
- FDA pulls video clip admitting Chantix suicide link - John R. Polito, 04/03/08
- New FDA Chantix suicide risk video clip raises concerns - John R. Polito, 04/02/08
- FDA Chantix Handling Betrayed Public Health - John R. Polito, 02/08
- Windows media clip of WhyQuit's candid views on Chantix & Champix: 18mb video for dial-up users | high res 53mb format | and 7mb audio only format, length 48 minutes, recorded 10/19/07 by Joel Spitzer
- Can Chantix Cause Depression? - The People's Pharmacy, 09/19/07
- Flawed research equates placebo to cold turkey - John R. Polito, 03/07
- Nicotine Fix - Behind Antismoking Policy, Influence of Drug Industry - Kevin Helliker, 02/07
- WSJ Highlights Financial Conflicts of Chair of Federal Guidelines Panel - Michael Siegel, MD 02/07
- Nixing the Patch: Smokers quit cold turkey - Ken Millstone, 02/07
- New Study Challenges Thinking on Use of NRT During Pregnancy - Michael Siegel, MD, 02/07
- Nicotine for the Fetus, the Infant and Adolescent - H.K. Ginzel, MD, 02/07
- The secret to quitting smoking - John R. Polito, 01/07
- Financial Ties Between Guideline Panels and Big Pharma Run Deep - Michael Siegel, MD, 01/07
- Chantix - an 8 in 10 failure rate or worse? - John R. Polito, 12/06
- Do physician's have a legal duty to ignore Guideline Recommendation 7? - John R. Polito, 12/06
- Is the U.S. government's quitting policy killing smokers? - John R. Polito, 10/06
- Will Chantix really help me quit smoking? - John R. Polito, 08/06
- Conference Sponsorship by Pharma Precludes Objective Symposium - Michael Siegel, MD 07/06
- 13th World Conference on Tobacco or Health Drenched in Nicotine - John R. Polito, 07/06
- Cold Turkey Twice as Effective as NRT or Zyban - John R. Polito, 05/06
- Nicotine Not Medicine, Its Use Not Therapy - John R. Polito, 04/06
- GlaxoSmithKline Attacks Cold Turkey Quitting - John R. Polito, 12/05
- UK Guidance for NRT use in pregnancy and by children - ASH London, 12/05
- June 2000 Guideline Chairman Michael Fiore's Testimony - see PDF pages 14 & 15, 05/05
- The Nicotine Patch, Gum and Lozenge - Mounting Evidence of a Sham - John R. Polito, 04/05
- 40 Years of Progress? - Joel Spitzer, 10/04
- Widespread Blinding Failures Put NRT Studies in Serious Question - John R. Polito, 06/04
- Nicotine Gum Maker's Concern Raises Concerns - John R. Polito, 05/04
- A Quitter's Dilemma: Hooked on the Cure - New York Times, 05/04
- Quebec CT Quitters Disprove "Double Your Chances" NRT Assertion - John R. Polito, 04/04
- Are nicotine weaning products a bad joke? - John R. Polito, 10/03
- Is CT Quitting More Productive & Effective than NRT? - John R. Polito, 07/03
- Are Teens Getting Hooked on NRT? - John R. Polito, 06/03
- March 2003 OTC NRT Meta-Analysis Finds 93% Midyear Relapse Rate - John R. Polito, 03/03
- Quitting Methods - Who to Believe? - Joel Spitzer, 2003
- JAMA Study Concludes NRT is Ineffective - John R. Polito, 09/02
- Real-World Nicotine Patch and Gum Rates - John R. Polito, 06/02
- Does the OTC Nicotine Patch Really Double Your Chances of Quitting? - John R. Polito, 04/02
- Is Nicotine Replacement Therapy The Smoker's Last Best Hope? - John R. Polito, 11/00
- Financial Disclosures for June 2000 Guideline Panel - U.S. Public Health Service, 06/00
- June 2000 Guideline Recommendation 7 - pharmacotherapy use by all quitters - 06/00
How to Quit Smoking
Our Free E-books
Read both & destroy fear driven anxieties!
Discover Smart Turkey Quitting
- WhyQuit.com - WhyQuit is the Internet's oldest forum devoted to the art, science and psychology of cold turkey quitting, the stop smoking method used by the vast majority of all successful long-term ex-smokers.
- Nicotine Addiction 101 - WhyQuit's basic guide to understanding nicotine dependency.
- Nicotine Cessation Topic Index - An alphabetical subject matter index to more than a thousand nicotine cessation articles, videos and support group discussions.
- Joel's Library - Joel Spitzer began presenting stop smoking clinics and seminars in 1976. WhyQuit's education director since 2000, Joel's Library is home to his life's work. It includes Joel's "Daily Quitting Lesson Guide," more than 100 original stop smoking articles, his free ebook "Never Take Another Puff," and to his ever growing collection of more than 400 stop smoking videos.
- "Freedom from Nicotine - The Journey Home" - Written by John R. Polito, a former 30-year heavy smoker and WhyQuit's 1999 founder, Freedom from Nicotine shares the science underlying nicotine dependency and successful abrupt nicotine cessation.
- Turkeyville - Imagine surrounding yourself with more than 10,000 cold turkey quitters. Turkeyville is a Facebook support group exclusively for cold turkey quitters.
- Freedom - Freedom was WhyQuit's original 1999 stop smoking support group. No longer accepting members, its 453,000 archived posts continue to share recovery insights.