What happens when the senior researcher on a study analyzing the real-world value of approved stop smoking products is also a quitting product salesman serving on Pfizer's speakers bureau, and a paid consultant to GlaxoSmithKline?

I have no doubt that K. Michael Cummings, PhD, MPH, is a good, honest and decent man. But what's it like inside a mind that generates medical journal research findings and conclusions regarding the exact same quit smoking products for which the pharmaceutical industry is paying them consultation fees?

Do these researchers feel or appreciate the conflicts they live? If necessary, how many have the courage and independence to bite the hand that feeds them?

Here's the problem. Until now, while approved quitting products have clobbered placebo controls inside randomized clinical trials,[1], trials which by now all know were not blind as claimed,[2] they get clobbered just as badly by cold turkey quitters in real-world use.[3] In fact, until now, nearly every long-term population level finding since 2000 has found approved quitting products no more effective than quitting without them, and in most cases significantly worse.[4] Here's a recent example.

The new study co-authored by K. Michael Cummings is featured online in Addiction, a journal whose editor-in-chief is Robert West,[5] arguably the UK's most financially conflicted quitting product researcher.[6] The new study is entitled "Effectiveness of stop-smoking medications: findings from the International Tobacco Control (ITC) Four Country Survey."[7]

Below is the full-text of a critical review of the new ITC study by physician-professor Michael Siegel of the Boston University School of Public Health. Dr. Siegel raises a number of concerns regarding the study, which if valid effectively destroy its worth in valuing the population level worth of NRT, bupropion and varenicline. Readers can comment on Dr. Siegel's "The Rest of the Story" blog at his website.[8] Dr. Siegel wrote a follow-up review on August 21 questioning the assertion that the study was "prospective."[9]

As Dr. Siegel's initial review notes, after an earlier International Tobacco Control Policy Evaluation Project (ITC) survey study found cold turkey smoking cessation twice as effective as gradual weaning/tapering[10], that finding gets totally ignored. Instead, Cummings and his co-authors present ITC survey data in a selective fashion that makes Chantix, Zyban and the nicotine patch appear vastly superior to a "no medication" control group that includes gradual weaning/tapering quitters.

One billion smoking related deaths projected before century's end, it's my hope that sharing Dr. Siegel's review motivates financially conflicted smoking cessation researchers to reflect upon the natural biases created by working for the quitting product industry.

Professor Michael Siegel, M.D.

A new study published online ahead of print in the journal Addiction reports survey data purporting to show that quit attempts using medication were more successful than unaided attempts. The study uses data from an international survey conducted in the United Kingdom, Canada, Australia and the United States to study smoking and quitting behavior.

The study methods are described as follows: "A total of 7436 adult smokers (18+ years) selected via random digit dialing and interviewed as part of the International Tobacco Control Four Country Survey (ITC-4) between 2002 and 2009. Primary analyses utilized the subset of respondents who participated in 2006 or later (n=2550)." The main outcome of interest was successful quitting at either 1 month or 6 months from the quit attempt.

The chief results of the paper were reported as follows: "Among participants who recalled making a quit attempt within 1 month of interview, those who reported using varenicline, bupropion or nicotine patch were more likely to maintain 6-month continuous abstinence from smoking compared to those who attempted to quit without medication [adjusted odds ratio (OR) 5.84, 95% confidence interval (CI) (2.12-16.12), 3.94 (0.87-17.80), 4.09 (1.72-9.74), respectively]; there were no clear effects for oral NRT use."

The study concludes as follows: "Consistent with evidence from randomized controlled trials, smokers in the United Kingdom, Canada, Australia and the United States are more likely to succeed in quit attempts if they use varenicline, bupropion or nicotine patch. Previous population studies that failed to find an effect failed to control adequately for important sources of bias."

The Rest of the Story

This is a complicated study and there are many aspects to the rest of the story, so let me take them one at a time. The study procedures are extremely complex because of its retrospective design, so I will do my best to explain these important flaws.

1. Many Medication Users Were Thrown Out of the Study

The most important aspect of the study for readers to understand is that its methods were biased so as to find a higher effectiveness of smoking cessation drugs. How was this achieved?

It was achieved by throwing out from the study a significant proportion of medication users, but not using any similar procedure to exclude non-medication users, all of whom were included in the study.

Any time that you treat the intervention group different than the control group, you introduce a bias into the study and in this case, that bias works towards finding medication use to be more effective.

Specifically, here is what happened in the study: Rather than simply including all medication users (that is, reporting the quit rates for all smokers who used medication in their most recent quit attempt), the study excluded certain smokers who used medication based on their response to a question about their reasons for using smoking cessation medication.

The study explains this exclusion as follows: "During each survey wave, respondents were asked to recall their use of medications since the last survey, and those who reported using medications were asked a series of questions regarding the medications indicated, including: 'What was the main reason you used [the medication]?'. Only those who reported using medication in an attempt to stop smoking completely were considered to be medication users for the purpose of these analyses."

In other words, the study did not define medication users as being "medication users." Instead, it defined medication users as subjects who, after having tried medication to try to quit smoking and potentially failed, retrospectively report their use of medication was intended to help them quit smoking.

Remember that the only indicated purpose for the use of smoking cessation drugs is to quit smoking. These drugs are not approved for any other purpose. So why exclude anyone who reported using smoking cessation medications? And if you are going to exclude smoking cessation medication users based on their intended purposes (reported after the fact), then why not also exclude unassisted quitters based on their reported intentions, also after the fact?

The way the human mind works, people are likely to alter their perceptions of why they used a particular approach after they observe the result of that approach. For example, suppose I am a two pack-per day smoker and I try to quit using NRT. I fail. Instead, I am able to cut down a little. Then, a few months later, I am asked to report the reason I used the NRT. To save face, I might (subconsciously) decide that the real purpose of my using the NRT was to cut down on the amount I smoke, rather than to quit completely.

Importantly, the same phenomenon might occur with a non-medication user. A person might initially decide that they are going to try to quit (without medication). But they might find it too difficult. So instead, they decide to just cut down. When asked a few months later why they decided to make a "quit" attempt, they might respond that they were actually trying to cut down rather than to try to stop smoking completely.

The effect of this procedure would be to systematically exclude smokers who have used smoking cessation medication unsuccessfully. In other words, this procedure would artificially bolster the observed smoking cessation rate for medication users.

To make matters worse, the question about the purpose of using smoking cessation drugs referred to any use of these drugs during the past year, not to the most recent quit attempt. Therefore, someone might have used NRT to try to cut down about six months ago and then used NRT to try to quit more recently. But they might still answer the question about the use of NRT as indicating that their intent was not necessarily to quit completely.

The bottom line is that smoking cessation medications are smoking cessation medications. Any study which excludes users of smoking cessation medications from the analysis, especially retrospectively, is going to bias the results towards finding an artificially high cessation rate for medication users.

2. Cold Turkey Quitters May Have Been Included in the Medication Group

For wave 5 and earlier of the survey, the study classified users of medication based on whether they had used any smoking cessation medication since the previous survey. However, it does not appear to have assessed the method used to quit during the most recent quit attempt. This question does not appear to have been added to the survey until wave 6. Therefore, it seems entirely possible that for wave 5 and earlier, a person might have tried and failed a few times to quit using NRT and then decided to go cold turkey for their most recent quit attempt. But because this person is not asked to report the method used in their most recent quit attempt, they would be classified as a medication user. If they were successful in their cold turkey attempt, that would go down in the results as a success for the use of drugs, not a success for unassisted quitting.

While the study describes itself as a "prospective cohort" study, it really is a "retrospective cohort" study because the assignment of exposure status is being made after the fact.

What might be the effect of this misclassification of exposure? If it were true that cold turkey quitting is more effective than NRT use, then this misclassification would mask that effect. It would result in successful quit attempts going down as being attributed to medication when they are really due to an unaided quit attempt.

In my view, the inability of the study to determine definitively the method used for the most recent quit attempt makes it suspect. Much more useful would be true cohort studies in which the classification of exposure is meaningful. An example of such an approach is the study reported by the UK National Health Service. In that study, callers were assigned to receive or not receive medication and then followed up to observe their cessation rates. The results, as I reported last week, showed no significant advantage to the use of smoking cessation drugs.

3. The Study Results Depend on the Exclusion of Anyone Other than Those Making Quit Attempts in the Past Month or Two

The study finds that the effectiveness of medication greatly increases among subjects who report having made their most recent quit attempt in the past month or two. When all quit attempts are considered, there is no significant effect of nicotine gum, the nicotine patch, or bupropion.

The study justifies the exclusion of the bulk of its data on the premise that unsuccessful cold turkey quitters are more likely than unsuccessful medication quitters to forget that they have attempted to quit smoking. On its face, this seems implausible. Would not a person making a cold turkey attempt remember that they made such an attempt? More importantly, the data purported to demonstrate this effect fail to do so.

The premise that failed cold turkey quit attempts are not remembered as easily as failed medication quit attempts is based on the finding that failed quitters who used medication are more likely to report a greater time since their quit attempt than failed quitters who did not use medication. The assumption is that the medication users made additional attempts to quit without medication more recently but forgot about those quit attempts. However, the paper provides no evidence to back up this assertion. It only demonstrates that those failed quitters who used medication reported more remote failed quit attempts than failed quitters who did not use medication.

There is a very reasonable alternative explanation for this finding, which is that when medication users fail in their quit attempts, they are very discouraged from making additional quit attempts. In contrast, cold turkey quitters do not face the same discouragement because they can always rationalize that they can do things differently. But if medication fails, it may be more likely to lead to a sense of hopelessness, which could well deter quit attempts for quite some time.

4. Non-Cold Turkey Quitters Were Included in the Unassisted Cessation Group

The real question of interest is not simply whether medication-assisted quitting is more effective than unassisted quitting, but whether medication-assisted quitting is more effecting than unassisted cold turkey quitting. I am not aware of anyone advocating for unassisted quitting using a gradual reduction approach. The question of interest is how an abrupt attempt at smoking cessation that is unassisted compares with medication-assisted quit attempts.

Unfortunately, this study does not report the effectiveness rates for cold turkey, unassisted quitting. Instead, it lumps together all unassisted attempts to quit, even those in which the smoker cut down gradually, an approach that has been shown to be ineffective and which no one is advocating. By including these attempts and reporting them in the unassisted quitting figures, the study again biases the results towards finding a greater effectiveness of medication-assisted quitting.

The comparison which nobody seems to want to make is smoking cessation drugs versus cold turkey quitting without drugs. But that is the relevant research question.

Importantly, a large proportion of the "unaided" quitters in the ITC Four-Country Survey used a gradual reduction, rather than a cold turkey approach. In fact, approximately one-third of smokers used a gradual reduction approach. Therefore, the inclusion of these subjects creates a substantial bias which artificially lowers the reported effectiveness rate of unassisted quitting, or at least fails to provide the key data of interest to the research question.

Putting it All Together

It strikes me that those who are trying to explain away the consistent finding that drugs do not appear to be much better than cold turkey quitting in population based studies (outside the context of clinical trials) are now grasping at straws, making valiant efforts at reasoning away the clear findings of these population-based studies.

Here, the study has created a premise for the entire analysis (that there is differential recall of failed cold turkey vs. medication quit attempts), a premise which is based not on the demonstration that cold turkey quit attempts tend to forget their quit attempts, but on the observation that medication users recall more remote failed quit attempts than unassisted quit attempters. Not considered is the possibility that the findings are real (rather than based on memory defects). Is it not possible that failed smoking cessation attempts are likely to deter further quit attempts for a longer period of time than failed unassisted quit attempts?

Furthermore, the study introduces two sources of potential bias, both of which would bias the results toward finding a higher rate of effectiveness of medication. The more troubling of the two is the exclusion of medication users based on a question, asked retrospectively, about the reasons for their use of stop smoking medication. In my view, once you start excluding certain users of medication, you are no longer playing fair. Smoking cessation drugs are smoking cessation drugs and to exclude failures based on smokers retrospectively reporting that they actually didn't have the desire to completely quit smoking is not a fair and balanced analytic approach.

Perhaps the most telling fact about this study is that, as you might expect based on the various biases in the study, one of the co-authors has a significant financial conflict of interest with companies that manufacture smoking cessation drugs.

According to the study's disclosure statement, one of the study investigators: "has served as a paid consultant on smoking cessation to Pfizer and Novartis, [and] has received payment from Pfizer and GlaxoSmithKline for lectures on smoking cessation to health professionals."

I am not suggesting that there is any conscious bias but subconsciously, these kinds of financial connections are going to influence the slant with which one conducts, analyzes and reports results. In this case, there is ample evidence from an analysis of the study that there is indeed bias towards finding an effect of smoking cessation medications. There is nothing wrong, since the conflict was appropriately disclosed. However, readers should take this conflict into account when interpreting the study results.

References:

1. Fiore MC, et al, Clinical Practice Guideline - Treating Tobacco Use and Dependence: 2008 Update May 2008 PHS; see critical review at: Polito, JR, U.S. quit smoking policy integrity drowns in pharmaceutical influence, WhyQuit.com, May 13, 2008

2. Polito JR Smoking cessation trials, Canadian Medical Association Journal, Feb. 29, 2008.

3. Doran CM, Valenti L, Robinson M, Britt H, Mattick RP. Smoking status of Australian general practice patients and their attempts to quit. Addictive Behaviors, May 2006, Volume 31(5), Pages 758-765. Abstract

4. Hartman AM. What does US national population survey data reveal about effectiveness of nicotine replacement therapy on smoking cessation? National Cancer Institute paper presented at World Conference on Tobacco or Health, 12-15 July 2006, Washington, DC. FREE Full Text (see charts on PDF pages 35-37); Alberg AJ, Patnaik JL, May JW, Hoffman SC, Gitchelle J, Comstock GW and Helzlsouer KJ, Nicotine replacement therapy use among a cohort of smokers, Journal of Addictive Diseases 2005, Volume 24(1), Pages 101-113. Abstract; Pierce JP, Cummins SE, White MM, Humphrey A, Messer K, Quitlines and Nicotine Replacement for Smoking Cessation: Do We Need to Change Policy?, Annu. Rev. Public Health 2012. 33:12.1–12.16 Abstract; Ferguson J, Bauld L, Chesterman J, Judge K, The English smoking treatment services: one-year outcomes, Addiction. 2005 Apr;100 Suppl 2:59-69. Abstract; Pierce JP, Gilpin EA, Impact of over-the-counter sales on effectiveness of pharmaceutical aids for smoking cessation, JAMA. 2002 Sept. 11;288(10):1260-4. FREE Full Text; Alpert, HR, Connolly GN, Biener, L, A prospective cohort study challenging the effectiveness of population-based medical intervention for smoking cessation, TC Online First, January 10, 2012. Abstract

5. Addiction, About the Editorial Team, http://www.addictionjournal.org/pages/contacts, Viewing Date August 20-21, 2012.

6. Addiction, Ethical Policy, Editors' Declarations of Interest, Robert West, http://www.addictionjournal.org/pages/ethical-policy, Last Updated: Unknown; Site Viewing Date: August 20-21, 2012. Interestingly, West's Addiction disclosure states, "He undertakes lectures and training in smoking cessation methods which have led to payments to him personally and to his institution," without indicating who made the payments. Additionally, a 2011 disclosure by West notes that he holds "a patent for a novel nicotine delivery device" (see Addiction, 2011, Volume 106, Page 680).

7. Kasza KA, Hyland AJ, Borland R, McNeill AD, Bansal-Travers M, Fix BV, Hammond D, Fong GT, Cummings KM, Effectiveness of stop-smoking medications: findings from the International Tobacco Control (ITC) Four Country Survey, Addiction. August 14, 2012. doi: 10.1111/j.1360-0443.2012.04009.x. [Epub ahead of print], PubMed Abstract

8. Siegel, M, New Study Reports Effectiveness of Smoking Cessation Medication But is Biased Towards Finding an Effect; Financial Conflict of Interest Present, The Rest of the Story: Tobacco News Analysis and Commentary, August 20, 2012 http://tobaccoanalysis.blogspot.com/2012/08/new-study-reports-effectiveness-of.html

9. Siegel, M, New Study on Effectiveness of Smoking Cessation Medications is Really a Retrospective, Rather than Prospective Cohort Study; Bias in Defining Exposure Groups Threatens Validity of Results , The Rest of the Story: Tobacco News Analysis and Commentary, August 21, 2012 http://tobaccoanalysis.blogspot.com/2012/08/new-study-on-effectiveness-of-smoking.html

10. Cheong Y, Yong HH, Borland R, Does how you quit affect success? A comparison between abrupt and gradual methods using data from the International Tobacco Control Policy Evaluation Study, Nicotine & Tobacco Research, 2007 August; Volume 9(8), Pages 801-810. PubMed Abstract; Free Full Text

I, John R. Polito, am solely responsible for the content of this article. Any errors brought to my attention will be immediately corrected.

---

---