What If the Institute of Medicine Wrote a Report and Nobody Followed it? - the Case of the Standards for Developing Trustworthy Guidelines

For over 20 years, clinical practice guidelines (CPGs) have been touted to improve health care quality and control costs.  Enormous numbers of guidelines have been developed, but with seemingly little impact on health outcomes.  While some of those leading health care organizations have predictably blamed individual practitioners for obstinately ignoring or challenging guidelines, there is increasing evidence that maybe the guidelines themselves are part of the problem.

 An Example of a Guideline that Apparently was Not Trusted

One example Dr Wally Smith and I have taught in our recurring mini-course on why physicians fail to follow guidelines (and otherwise appear not to practice in accord with others' wishes) is that of the guidelines on management of depression in primary care.  Most existing guidelines urge physicians to screen patients for (presumably mild-to-moderate) depression and treat them aggressively, with emphasis on the use of the newer anti-depressants.  These guidelines, in turn, were based on numerous published randomized clinical trials that showed that these drugs were safe and efficacious.  Yet multiple studies showed that physicians failed to follow these guidelines, and various attempts to improve their adherence did little.  So for years the assumption was that physicians at best experienced practical and system barriers to follow these guidelines, and at worst were ill-informed or irrational. 


However, information that came out gradually during the early part of the 21st century suggested that perhaps the problem was within the guidelines, not the health care professionals.  First, documents produced during New York Attorney General Eliot Spitzer's lawsuit against GlaxoSmithKline about the marketing of one of these drugs (Paxil, paroxetine) suggested that the company had suppressed clinical trial data that reflected poorly on the drug (See Kondro W.  Drug company experts advised staff to withold data about SSRI use in children.  Can Med Assoc J 2004; 170: 783.  Link here.)    These suspicions were later fleshed out  by consideration of documents further disclosed in litigation (e.g., see this post and its links).  Then several studies, most particularly that by Erick Turner and colleagues, showed that numerous trials of new anti-depressants had been suppressed, that is, never published (Turner et al. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 2008; 358:252-260.  Link here).  When the results of these trials were added to those that were published, the efficacy of anti-depressants was no longer so clear.  So maybe the guidelines that physicians did not follow were not trustworthy, and should not have been followed in the first place.

Would IOM Standards to Improve Guideline Trustworthiness Help?

So in 2011, the prestigious Institute of Medicine released a report on the development of  better standards to produce more trustworthy guidelines (Clinical Practice Guidelines We Can Trust.  Link here.)  We posted about that report here, but noted that it was receiving little other attention, an example of the anechoic effect. 

A few weeks ago, an article appeared documenting a study meant to assess the the trustworthiness of clinical practice guidelines published soon after the IOM report.  Its title telegraphs the results. ( Kung J, Miller RR, Mackowiak PA. Failure of clinical practice guidelines to meet Institute of Medicine standards: two more decades of little, if any progress.  Arch Intern Med 2012.  Link here.)

Methods and Results

The investigators selected a random sample of 114 individual guidelines available during June, 2011 stratified by 26 clinical topics. The versions of the guidelines used were those archived in the National Guideline Clearinghouse (NGC) maintained by the Agency for Healthcare Research and Quality (AHRQ).


The goal of the study was to "examine adherence to the IOM standards" by guidelines published after the standards were published.  Actually, the study only assessed adherence to 18 of the 25 standards espoused by the IOM (because the remaining seven were "too vague and subjective to be analyzed.")

Furthermore, the criteria used to determine if a specific guideline met each of the three items above were rather lax:

In evaluating each guideline summary, care was taken to be as liberal as possible in considering that a standard was met when the individual guideline summary provided any information pertaining to that particular standard.

Nevertheless, using these lax standards to only evaluate adherence to 18/25 guidelines, the authors found that "the overall median number of IOM standards satisfied (out of 18) was 8 (44.4%) ....  Fewer than half of the guidelines surveyed met more than 50% of the IOM standards."

An examination of the details of the study's methods reveals things are even worse than that.

Analyzing the Study's Methods to Find that Things Are Worse Than They Seem

Review of the study's methods show that they provided a very optimistic view of adherence to the IOM standards.  As noted above, the study did not look for adherence to all of the IOM standards.  Moreover, those they did consider were simplified and made less rigorous.  For example, Standard 2 from the IOM on management of conflict of interest (COI) was:

  STANDARD 2
Management of conflict of interest (COI)

2.1
Prior to selection of the Guideline Development Group (GDG), individuals being considered for membership should declare all interests and activities potentially resulting in COI with development group activity, by written disclosure to those convening the GDG.
- Disclosure should reflect all current and planned commercial (including services from which a clinician derives a substantial proportion of income), non-commercial, intellectual, institutional, and patient/public activities pertinent to the potential scope of the CPG.

2.2
Disclosure of COIs within GDG
- All COI of each GDG member should be reported and discussed by the prospective development group prior to the onset of their work.
- Each panel member should explain how their • COI could influence the CPG development process or specific recommendations.

2.3
Divestment
- Members of the GDG should divest themselves of financial investments they or their family members have in, and not participate in marketing activities or advisory boards of, entities whose interests could be affected by CPG recommendations.

2.4
Exclusions
- Whenever possible GDG members should not have COI.
- In some circumstances, a GDG may not be able to perform its work without members who have COIs, such as relevant clinical specialists who receive a substantial portion of their incomes from services pertinent to the CPG.
- Members with COIs should represent not more than a minority of the GDG.
- The chair or co-chairs should not be a person(s) with COI.
- Funders should have no role in CPG development.

However, the study boiled all this down to three items:
-  COIs stated
-  Chair has COI
-  Co-chairperson has COI

Thus the study did not address standards requiring full and complete disclosure (not just some disclosure) of all COIs; consideration of how the COIs might influence the particular guideline; divestment of specific types of conflicts of interest, that is, financial investments, and cessation of participation in marketing activities or advisor boards; minimization of conflicts of all members of the committee; and barring of participation of funders in guideline development.

Even so, the guidelines assessed did a very poor job upholding even these few liberalized standards regarding conflicts of interest.  Of the guidelines assessed, less than half, 46.8% provided ANY disclosure of conflicts of interest.  Those written by sub-specialty societies were particularly opaque in this regard.  Less than one-third, 29.3%, provided any disclosure.  Thus the majority of guidelines assessed were not at all transparent about conflicts of interest affecting the guideline development process.

Furthermore, of the 46.8% of all the guidelines which made any disclosures of conflicts of interest, 71.4% admitted their chair people HAD a conflict of interest.  Thus, only (0.468 * [1 - .714]) = 13.3% provided assurance that they fulfilled the single requirement (from standard 2.4 above) that the chair person did not have a COI.  For the guidelines written by sub-specialty societies, by a similar calculation, only 12.2% provided an assurance that the chair had no COI.  (The proportions providing assurance that the co-chair people had no COI were even lower.)  Thus the vast majority of guidelines did not clearly follow two straight-forward standards for minimizing the effects of conflict of interest, that the guideline committee chair and co-chair should not have any relevant conflicts. 

Given the miserable results concerning even minimal adherence to some of the IOM report's conflict of interest standards, it is likely that almost no published guidelines from 2011 came close to fulfilling the full set of IOM standards.  Despite the best efforts of the IOM, it appears that guideline developers have not progressed at all towards providing trustworthy guidelines. 

Summary

An editorial (Shaneyfelt T. In guidelines we cannot trust.  Arch Intern Med 2012) accompanying the article by Kung and colleagues summarized its results thus:

The same problems that have plagued guideline development continue to plague guideline development; namely, their variable and opaque development methods, their often conflicted and limited panel composition, and their lack of significant external review by stakeholders throughout the development process.   As a result, the trustworthiness of guidelines is limited.

While guidelines may have seemed to be a promising method to improve health care in the early 1990s, they have failed to live up to that promise.  Shaneyfelt was not optimistic they would improve in the future:

I am not optimistic that much will improve.  No one seems interested in curtailing the out-of-control guideline industry.

On the other hand, in my humble opinion, it is not that on one is interested in better guidelines.  It would clearly be in the best interests of patients and the public, and of health care professionals who care about the quality of their practice and the outcomes of their patients to curtail that industry.  The issue is why patients', the public's, and professional's interests were ignored.

Neither Shaneyfelt nor Kung et al discussed why there has been so little attention to patients' and the public's health, and to health care professionalism in all this.  For a quick answer, we do not have to look far on Health Care Renewal. 


In fact, the IOM report on guideline development from 2011 was a serious challenge to the powers that be in health care.  In particular, it challenged the cozy relationships that had grown up among the organizations that undertook guideline development and the health care professionals on guideline panels on one hand and organizations that stand to gain were specific guidelines to favor their products, services, and agendas on the other.  The standards mandated transparency and honesty about conflicts of interest affecting guideline committees and the organizations which assembled them, and if upheld would have greatly reduced these relationships.

Now it turns out that the guideline standards have been honored mainly in the breach.  Of course, these standards, while increasing trustworthiness, would have cost a lot of medical societies considerable commercial funding, and would have cost a lot of health care professionals on guideline panels considerable personal wealth.  These standards would probably also have cost a lot of companies whose products and services were addressed by guidelines to lose revenue.  So it is not surprising that the IOM standards were ignored.  Their implementation would have cost too many people who are financially benefiting from the status quo too much money.  And these people, that is, leaders of professional societies dependent on commercial outside funding, health care professionals and academic used to financial support from commercial interests, and health care corporations are good at making sure their interests are not ignored, even if their interests conflict with those of patients, the public, and well-intentioned health care professionals.  

So, the flouting of the well reasoned IOM guideline standards adds one more reason for patients and the general public to distrust modern health care and all those who "deliver" it, even to distrust well-intentioned health care professionals who have not been able to distinguish themselves from their colleagues who are too happy to help commercial interests while taking commercial money.  If health care professionals want to regain the public's trust, they could do worse than publicly declaring their intention to show that their practice in the future will be guided by trustworthy guidelines based on clinical research evidence and knowledge of biomedical science, drawn up by health care professionals independent of commercial interests.

Marketers' Systemic Influence over Ostensibly Scholarly, Peer-Reviewed Publications: the Medtronic Infuse BMP-2 Example

On the heels of our discussion of how one pharmaceutical company employed a "publications strategy" to commission and control randomized controlled trials to serve marketing purposes, a US Senate committee has released a report about how a device/ biotechnology company "influenced the content of articles in peer-reviewed scientific publications to present... [its product] in the best possible light."

The report was summarized by the Wall Street Journal,

A report by the Senate Finance Committee based on thousands of documents it subpoenaed from Medtronic Inc raises new questions about the integrity of the medical research underpinning one of the medical-device maker's products.

Medtronic was 'heavily involved in drafting, editing and shaping the content of medical journal articles' about the product—a bone-growth protein used in spine surgery called Infuse—even as it was paying the physicians who wrote those articles a total of $210 million for unrelated work, the Senate report alleges.

In one instance, a Medtronic employee recommended to one of the physicians not publishing a list of side effects associated with Infuse in a 2005 journal article, company emails show. Medtronic marketing officials also urged inserting language into other journal articles touting the use of Infuse as better for patients than using bone harvested from their pelvises because of the pain associated with the latter, other company documents show.

Medtronic's influence extended to preparing a physician's 2002 speech to a panel advising the Food and Drug Administration on whether to approve the drug, the report alleges. The physician's disclosure to the panel at the time suggested his testimony was independent, but the company had in fact helped him draft it and paid him as a consultant the previous year, company documents show. Medtronic later hired the physician as an executive.

In response to the Senate report, Medtronic issued a statement saying it 'vigorously' disagreed with any suggestion that 'it improperly influenced or authored any of the peer-reviewed published manuscripts.' The company also denied that it 'intended to under-report adverse events' associated with Infuse.

The Extent of the Problem

The report itself is available here. It is worth discussing some of its sections about how the company is alleged to have influenced the peer-reviewed, scholarly clinical literature in more detail.

First, it asserted that "Medtronic employees, including employees working for its marketing department, collaborated with physician authors, many of whom had significant financial relationships with Medtronic, to draft" 11 specific articles in the literature published between 2002 and 2009.

Obfuscating Adverse Effects

The report provided detail about specific instances in which Medtronic employees appeared to influence publication to further marketing objectives.  The first was apparently to cloud discussion of adverse effects of its product [italics added for emphasis]:

documents indicate that a Medtronic employee involved in editing a draft of the 2005 Journal of Bone and Joint Surgery (JBJS) article by Burkus, et al. about a similar InFuse procedure involving allograft bone (a cage made from donated bone rather than the FDA-approved titanium), recommended that 'significant detail' concerning adverse event  data should not be published.

On June 16, 2004, Dr. Julie Bearcroft, Director of Technology Management in Medtronic’s Biologics Marketing Department, wrote an e-mail to other Medtronic employees, commenting on a draft of the study, 'I have made some significant changes to this document (some at the request of Dr. Burkus) both in format and content.  In this e-mail, she asked: 'How much information should we provide relative to adverse events? . . . You will see my [note] in the attached document but I don’t think significant detail on this section is warranted.'  The referenced note in the draft article stated: 'I don’t believe we want to report in the same manner as we do in IDE studies. I personally think it is appropriate to simply report the adverse events were equivalent in the two groups without the detail.' According to an internal e-mail, the adverse events were observed in the trial and formatted in a detailed table. But following the advice of Bearcroft, this table of adverse events was not included in the published paper.

On July 3, 2004, after Medtronic edited the paper, Dr. Burkus sent a draft to his co-authors writing that 'this manuscript documents the superiority in clinical and radiographic outcomes with the use of rhBMP2 in a study population of only 133 patients.'

According to the Carragee et al. Spine Journal article published in 2011, the 2005 JBJS article 'reported no complications, such as end-plate fracture, collapse, and implant migration associated with rhBMP–2 despite the clear radiographic findings in at least the one presented case.  The e-mail exchange indicates that, in addition to Medtronic editing the manuscript without attribution, the company was recommending that the article omit a complete accounting of adverse event data, including serious adverse event data that were already considered a documented concern by FDA in similar application.
 
These types of adverse events were disclosed in Table V of a 2009 follow-up article concerning the original IDE study. Studies published in 2007 revealed that InFuse is associated with 'a clinically important early inflammatory and osteoclastic effect of the rhBMP–2 in soft tissue and bone, respectively.  In other words, Medtronic recommended against including information in the study that was ultimately revealed to have an association between In-Fuse and weakening that could lead to collapse of the bone and implant and required that patients undergo additional surgery.

Note that in this example, someone explicitly associated with marketing apparently edited a draft of a scholarly article, suggested that detail about adverse effects of the company's product be omitted, and that the published article in fact omitted such detail.

The report also included an example in which another Medtronic employee apparently tried to "tone down" the discussion of adverse effects of the product, but did so too late to influence the published version. 

Emphasizing the Adverse Effects of an Alternative to Using the Company's Product

On the other hand, the report also included an example in which it alleged that a company employee suggested adding emphasis to the drawbacks of using a management strategy that did not include use of the company's product.

Documents show that Medtronic edited draft publications to stress the pain patients experienced from undergoing a bone graft procedure instead of receiving InFuse. Medtronic markets InFuse as a less painful alternative to bone graft procedures for patients undergoing spinal fusion surgery.

In particular,

After receiving a draft of an early InFuse study 52 to review in October 2001, Medtronic’s Neil Beals, whose 'primary job responsibility was to manage Biologics marketing programs and initiatives,'  recommended that the physician authors of the study emphasize pain experienced by patients who received the bone graft. The patients were divided into an investigative group that received InFuse and a control group that received a bone graft obtained from the iliac crest of their pelvis.  An October 31, 2001 e-mail shows that Beals suggested to Dr. Burkus that 'a bigger deal should be made of elimination of donor site pain with INFUSE . . . so that ‘equivalent’ results aren’t received as a let down.'   Again, after reviewing a later draft of the study, Beals asked Dr. Burkus on March 8, 2002, 'would it be appropriate to make a bigger deal out of donor site pain and include more discussion and references?  Subsequently, a sentence was inserted at the end of a later draft, and included in the published version of the article, that read, 'The use of rhBMP–2 is associated with high fusion rates without the need for harvesting bone graft from the iliac crest and exposing the patient to the adverse effects associated with that procedure.'

Medtronic also sought to include discussion of long-term pain in the Baskin, et. al. 2003 paper on InFuse in the cervical spine. In a draft of the publication that was being circulated on August 30, 2002, the authors wrote, '[b]y 12 months after surgery, the patients [sic] graft-site pain had resolved . . . and no patients complained about the graft-site appearance.'  Beals inserted comments after this sentence stating, 'ALTHOUGH THE PATIENTS DID NOT COMPLAIN ABOUT APPEARANCE DIDN’T SOME STILL EXPERIENCE PAIN AT THE DONOR SITE? SEEMS LIKE RESIDUAL EFFECTS OF DONOR SITE SHOULD BE NOTED.  [sic] [emphasis in original]. In an e-mail to his colleague, Beals wrote, 'I would also add in more discussion on donor site pain and need for osteogenetic graft material (plant seed of doubt for just using allograft by itself ).  A review of the final published article reveals that, after Beals made the suggestion to emphasize pain at the bone graft site, a sentence was added in the final version of the article that read,  '. . . even at the 24-month follow-up assessment, some patients continued to experience residual pain at the donor site, and rated the appearance of the site as only fair.'

Note that in these two examples, another marketing employee edited drafts of two scholarly articles, suggested that more emphasis be put on supposed adverse effects of the procedure that the articles compared to the procedure which used the company's product, and that the two articles included such emphasis. 

Deceptive Response to Peer Reviewers' Criticisms of Apparent Bias 

Finally, the report included an instance in which a company employee managed correspondence between an article's ostensible first author and journal editors to try to defend wording which reviewers had criticized for bias in favor of the company's product.

In summary,

E-mail exchanges between Dr. Burkus and Medtronic employees regarding a study of InFuse utilizing the posterior lumbar interbody fusion (PLIF) technique and published in The Spine Journal in 2004 demonstrates that Medtronic employees not only edited the draft manuscript to include comments supportive of InFuse,  they also covertly participated in the peer-review process by drafting responses to peer-reviewers on behalf of the physician authors named on the paper.

In particular, one employee, Rick Treharne, previously identified as "Senior Vice President of Clinical and Regulatory Affairs," wrote a very positive summary statement for the article's discussion section:

In a January 10, 2003, e-mail to Dr. Burkus, Rick Treharne wrote, 'In looking over the data, I was impressed with how well the BMP patients actually did. So much so that I added a few paragraphs at the end that you may not agree with.' 

However, the reviewers were skeptical,

One reviewer wrote: 'Unless the authors can discuss the results of this study in an unbiased manner, which they have been unable to do in its present form, this data should not be published.'  Another reviewer wrote: 'The manuscript is full of biased statements that are a reflection of the data evaluators—the company that markets the product.'  That reviewer recommended a discussion of potential bias in the text of the paper writing,  'As it stands it is an advertisement for a specific product without significant scientific merit.'

Then, Medtronic employees then took over the process of responding to this review, to wit,

E-mail correspondence on May 28, 2003, indicates that Medtronic’s Rick Treharne wrote and sent Dr. Burkus a draft letter to Dr. Tom Mayer, Editor-in-Chief of The Spine Journal, to address concerns raised by orthopedic surgeons tasked with peer-reviewing the submitted PLIF paper. A subsequent e-mail by Julie Bearcroft notes that she and Dr. Burkus collaborated further on the response to the peer-reviewers of this study during a Lumbar Spine Study Group event.

In response to the peer-reviewers’ concerns about bias in the manuscript, the response letter seemingly misled The Spine Journal by stating that 'To help eliminate any potential bias, only one of the co-authors was a clinical investigator—the other three were independent reviewers of all the data. Since these data are taken from a clinical IDE study sponsored by a company, only the company would have all the data in its database—data that is reviewed by FDA auditors. We don’t believe any discussion of bias is needed for the text.'  By the end of 2003, 'independent reviewers' Dr. Haid and Dr. Burkus would have received $7,793,000 and $722,000 from Medtronic, respectively. This draft letter, written at least in part by Medtronic on behalf of Dr. Burkus, did not disclose the company’s role in directly editing the paper nor did it disclose the magnitude of financial payments made to the supposed 'independent reviewers.'

Thus, in this case, a marketing employee apparently edited a draft of a scholarly article to exaggerate benefits of the company's product, directly adding text to the article, and when peer-reviewers suggested that the article showed bias towards the company's product, apparently two employees ghost-wrote a response letter that claimed that certain authors were "independent reviewers," obfuscating that they had previously received large payments from the company.

Summary

In 2010, we noted reporting that suggested Medtronic had paid huge amounts, millions of dollars, to spine surgeons for reasons that were not clear.  Later that year, we noted further reporting that surgeons who were getting amounts sometimes exceeding one million dollars from Medtronic were not disclosing these payments in scholarly articles about the company's BMP-2 product.

Now there appears a US Senate committee report alleging that Medtronic marketing employees systematically influenced the writing, editing, and publication of multiple ostensibly scholarly articles, ostensibly written by doctors, to favor the Medtronic In-Fuse BMP-2 product.  This adds to previous case studies suggesting that pharmaceutical, biotechnology, device and probably other kinds of company marketers may try to systematically manipulate the design, implementation, analysis, and dissemination of supposedly scholarly and unbiased clinical research to more effectively market their products and services.  

This is discouraging to a former full-time academic physician who now realizes that the difficulties I encountered getting my manuscripts published, given that they written entirely by academic investigators and uninfluenced by marketers, may have been due to the cacophony of competition from marketing influenced texts professionally promoted to journals to serve vested interests.

This is more discouraging to a proponent of evidence-based medicine who believes that medical decisions ought to be informed by critical review of clinical research obtained by systematic search in order to weigh the benefits and harms of management options,accounting for patients' values.  When that clinical research was being deliberately influenced and biased by people selling goods and services, it is not clear that even rigorous critical review will distill the truth from the injected bias.  Yet if physicians cannot depend on published research to guide their decision making for individual patients, what can they depend on?

I conclude as I did my last post,...   Thus health care professionals, policy makers, researchers, and the interested public need to be even more skeptical about arguments made to promote innovative treatments and other clinical interventions.  However, it is not clear that even rigorous skepticism can defend the integrity of evidence based medicine from marketing disguised as clinical research.

Going forward, we must consider erecting an impregnable barrier between clinical research and those whose primary interest is to make money by selling health care goods and services.  If we do not do that, we will forever need to worry that we really have no idea what "works in medicine," and whether any particular test, treatment, or program provides benefits that outweigh its harms. 

When Clinical Trials are Meant for Marketing, not Science

The development of the randomized controlled clinical trial (RCT) was one of the major scientific advances in clinical medicine.  RCTs provide a major part of the evidence underlying evidence based medicine.  RCTs provide a major source of data used by the US Food and Drug Administration, and similar agencies in other countries, to decide whether to approve drugs or devices to manage particular clinical problems.  Unfortunately, with the rise of the RCT came a rise in attempts to suppress and manipulate clinical trials by those with vested interests, often in selling the products and services the trials can evaluate.

Many of the examples we have discussed were of attempts at manipulation or suppression of clinical trials originally done to provide evidence for product approval, or even ostensibly to advance clinical science.  Yet a relatively new article covering evidence revealed from litigation about the drug gabapentin (Neurontin) originally made by Parke-Davis, first a part of Warner Lambert, now part of Pfizer suggests that many clinical trials may not be done to advance science, or even just to provide data to regulators, but only to market products.

The article is Vedula SS, Goldman PS, Rona IJ, Greene TM, and Dickersin K. Implementation of a publication strategy in the context of reporting biases: a case study based on new documents from the Neurontin litigation.  Trials 2012; 13: 136.  Link here.

Methods Summary

The article described a case study based on documents revealed in 2008 litigation.  The purpose of the study was:

to describe the implementation of a publication strategy for off-label marketing of gabapentin, within the context of reporting biases and spin of Pfizer and Parke-Davis’s clinical trial findings, for four off-label uses: migraine prophylaxis, treatment of bipolar disorders, neuropathic pain, and nociceptive pain. 

The study examined documents revealed through the 2008 litigation.  It concentrated on documents relating to the marketing of Neurontin for four off-label indications, that is, for four potential reasons to use the drug which had not yet been approved by the US FDA.  It focused on documents discussing strategies for marketing for these indications, and then documents about particular company sponsored RCTs done for these indications.

Strategies to Sell Drugs

The documents examined by the investigators showed that Parke-Davis prepared "marketing assessments" for four possible indications for Neurontin.  The assessments discussed two possible strategies.  The "indication strategy" would be to conduct trials for the purposes of providing data to the FDA in the hope that the FDA would then approve the new indication for the drug.  The "marketing strategy" would be, in the words of an internal company memo quoted in the study, to conduct clinical trials and

to disseminate the information as widely as possible through the world’s medical literature

In other words, the publication strategy involved using RCTs first for marketing, rather than scientific or regulatory purposes.

How the Trials were Controlled by Marketers 

The publication strategy involved not simply doing trials and publicizing their results, but controlling the messages conveyed by the trials to make sure they primarily supported marketing.  This was done by having the company's marketing department control the content of the trial reports.

 A Neurontin Publications Subcommittee (NTN PSC) was formed within Pfizer and Parke-Davis to implement a publication plan. Minutes from meetings between the NTN PSC and Medical Action Communications (MAC), a medical writing company, indicate that a list of key messages, guiding the content of published reports related to the trials of gabapentin for off-label indications, was developed based on a branding guide

Thus it appeared to be that marketers, not physicians or scientists were in control of supposedly scholarly research publications appearing in medical journals.

The marketers controlled content by controlling those who wrote it.

 A standard operating procedure related to publication of affiliate-driven manuscripts was identified in internal company documents dated October 16, 2002, and it sheds further light on the publication planning process (see Figure 3). (The term affiliate in this context refers to Pfizer’s foreign affiliates, that is, corporations related to Pfizer by either shareholdings or other means of control, including subsidiary, parent, or sibling corporations). According to the internal company documents, 'affiliate-driven manuscripts' were written for Pfizer and Parke-Davis by MAC and sent to the authors for approval. Each article was coordinated by a manuscript team, consisting of representatives from the medical and marketing divisions of the company. The documents also indicate that all affiliate-driven manuscripts should be forwarded to the NTN PSC for review. One of the objectives of manuscripts being reviewed by the NTN PSC was to ‘ensure that they are in-line with current product messages and areas of interest’

Manipulating the Dissemination (and Analysis) of Research Results

Note that the control of publication by marketers describe above involved ghost-writing of articles by medical education and communications companies (MECCs).  Later the article explained that these ghost writers were "not appropriately acknowledged" in the resulting published articles.

The Vedula et al study article provided examples of "spin" in the writing of again supposedly scholarly research publications used in fact to support marketing, and controlled if not composed by marketers.

 We identified spin in publications related to 8/12 trials included in our analysis.... We classified the following as spin: emphasis in the published report on outcomes that were not specified in the study protocol (Study 879–201) ...; conclusions that did not match study findings described in the internal company research report (Study 945–220) ...; extensive rationale to explain away statistically non-significant (unfavorable to the sponsor) findings (Study 945–209; 945–291; No study number - Gorson) ...; conclusion of treatment effectiveness from an uncontrolled study (Study 945–250) ...; emphasis on statistically significant secondary outcomes despite negative findings for the primary outcome (Study 945–271) ...; and an explicit description of an attempt to spin study findings (as described in internal company emails) (Study 945–306).

Note that while most of these examples of spin involved manipulation of the dissemination of study results, that is, doubtful, biased or fallacious arguments based on apparently unbiased data, they also involved manipulation of the analyses (in italics). 

The article also described manipulation of dissemination involving the timing of publication, including delaying publication of an article despite the wishes of the study's investigators because the results were not statistically significant, and hence not favorable for marketing. 

Summary

Documents revealed by litigation about Neurontin in 2004 provided insights about how pharmaceutical and presumably other kinds of health care corporations may conduct systematic, deceptive stealth marketing campaigns to promote their products and services (look here).  We noted that initial media coverage of documents revealed in the 2008 litigation also suggested the existence of a systematic "publication strategy" to control dissemination of results of particular trials, while suppressing trials whose results could not easily be spun to provide support of marketing objectives (look here).

Now the new paper by Verdula et al fill out our knowledge of this case.  The paper suggests that randomized controlled clinical trials may be done not to advance science, or even convince regulators, but primarily to market their sponsors' products.  Thus some significant proportion of the clinical research literature, the literature that physicians and other health professionals have relied upon to make evidence-based decisions for their patients, may exist mainly for marketing purposes.  Even the most rigorous methods used by clinical epidemiologists to review research are meant to discover problems that arose from human error or the inevitable trade-offs made when research is done in the real world, but not deliberately introduced biases and defects meant to promote vested interests.  Thus it is not clear that even the most "evidence-based" medical decisions are based on real scientific evidence rather than the spinning of marketers.

Thus health care professionals, policy makers, researchers, and the interested public need to be even more skeptical about arguments made to promote innovative treatments and other clinical interventions.  However, it is not clear that even rigorous skepticism can defend the integrity of evidence based medicine from marketing disguised as clinical research.

Going forward, we must consider erecting an impregnable barrier between clinical research and those whose primary interest is to make money by selling health care goods and services.  If we do not do that, we will forever need to worry that we really have no idea what "works in medicine," and whether any particular test, treatment, or program provides benefits that outweigh its harms. 

Continuing Dissection of the Infamous Study 329

On the 1BoringOldMan blog, semi-anonymous blogger Mickey has been dissecting in detail the infamous Study 329 that GlaxoSmithKline used to promote Paxil (Seroxat, paroxetine) as safe and effective for children and adolescents, when the study data, rigorously analyzed, showed that was not the case.  Study 329 was an important part of the US Department of Justice case against GSK that resulted in a record $3 billion settlement (see this post). 

The posts so far are:
A Pretty Rotten Era  
A Movement
To Make Distortion Possible
The Lesson of Study 329: the Basics
The Lesson of Study 329: Efficacy Drifts to Trends and 2s
The Lesson of Study 329: Conventions and Protocols
The Lesson of Study 329: Uh-oh
The Lesson of Study 329: Data Transparency
The Lesson of Study 329: the Authors

This does not seem to be the end of the series, but the last post yesterday did have a striking conclusion:

But, for the moment, I am stuck on the piece that is, to me, the most fundamental lesson from this story, and unfortunately many similar stories in the industry-funded clinical trial literature that fills our journals. The authors of this study did not function as physicians, clinicians, scientists, or even authors. Some were there because of their previous credentials. Others were there to build up future credentials. But they weren’t there to do what they presented themselves as doing. Researchers are practicing medicine too. I might be seen as naive to see this as the center of this story. If that’s the case, I’ll go to my grave being naive. Medicine has been my life, and this isn’t Medicine.

There’s no place or tradition for physicians, particular physicians in high places, taking part in this kind of bullshit – particularly all the while claiming otherwise. The fact that it has been a widespread practice isn’t an excuse. That’s also an even stronger indictment. There are many principled neuroscientists working in this difficult area of medical science who’ve been tarnished by this kind of pseudoscience. There are many psychiatrists and other practitioners who’ve actually trusted these deceitful articles. And there are untold numbers of patients who’ve been medicated un-necessarily or even directly harmed by this and other capricious advise coming out of these studies – this one in particular. Rather than functioning as the advisers of ill people, these were physicians who actively participated in capitalizing on patients’ illness.

However, despite the seriousness of these concerns, and their corroboration in the Justice Department document that pushed GSK to settle for so much money, my alma mater, Brown University, where Dr Martin Keller, the lead author of Study 329, was Chair of Psychiatry, has consistently refused to revisit the study, and consistently failed to publicly refute the accusations against Dr Keller, nor to accept them and take appropriate action.  The University supposedly conducted an internal investigation of the matter, but has not seen fit to make it public.  When asked again about this, Brown's new President, Christina Paxson, referred the inquiry to the public relations people:

'The recent announcement by the U.S. Department of Justice did not suggest that any further reviews of the paper by the university are immediately warranted,' Darlene Trew Crist said in a written response. 'We have no further statement to make at this time.'

Given the seriousness of the allegations, and the University's mission:

The mission of Brown University is to serve the community, the nation, and the world by discovering, communicating, and preserving knowledge and understanding in a spirit of free inquiry, and by educating and preparing students to discharge the offices of life with usefulness and reputation.

the logic of the administration's refusal to have a public discussion of this case escapes me. As I have said before (in 2008!), the appearance of continued stonewalling, now going on for years, can easily be interpreted to imply that the institution has something really big and bad to hide.

Also see comments by Alison Bass on the Alison Bass blog, and by Dr Howard Brody on the Hooked: Ethics, Medicine and Pharma blog.

ADDENDUM (4 September, 2012) - There have been two additional posts on the 1BoringOldMan blog on Study 329:
the lesson of Study 329: the hurdles…
the lesson of Study 329: naked Emperors, fractious Queens…

ADDENDUM (4 September, 2012) - See also this post on the Alison Bass blog.

ADDENDUM (5 September, 2012) - See this post on the retirement of Dr Keller by Ed Silverman on PharmaLot.

ADDENDUM (10 September, 201) - See the final two posts in the 1BoringOldMan series:
the lesson of Study 329: “we’re only as sick as our secrets”…
the final lesson of Study 329: epilogue…

His conclusion in the next to last post of the series:

Something terrible happened in psychiatry, an alliance between the pharmaceutical industry and a number of our leaders who allowed their academic credentials to be used for commercial purposes. It happened on a large scale, and in the process shamed all of us, whether we were involved or not. Many of the people we looked to to guide us fell into the role of key opinion leaders – a marketing term that meant that they could influence what we did and how we practiced. And in that new role, medical degrees, academic positions, the methods of science, and psychiatric experience became little more than marketing tools in a commercial campaign. Such was the case with Study 329 – an article that has become exemplary because it is now so painfully well known – a symbol for the failings of an era. It needs to be retracted simply because it was wrong on purpose – and we all know it. And yet no involved person, institution, or organization highlighted in that last paragraph has swallowed their pride, or guilt, or denial, or arrogance and called for retraction.

[Italics and bold typeface added for emphasis]

Giant GSK Settlement Provides Reminder of the Pervasiveness of Stealth Marketing

The latest  and biggest legal settlement involving health care to hit the news, that of GlaxoSmithKline (GSK) and the US government, has many familiar elements. As summarized by the New York Times,

In the largest settlement involving a pharmaceutical company, the British drugmaker GlaxoSmithKline agreed to plead guilty to criminal charges and pay $3 billion in fines for promoting its best-selling antidepressants for unapproved uses and failing to report safety data about a top diabetes drug, federal prosecutors announced Monday. The agreement also includes civil penalties for improper marketing of a half-dozen other drugs.

As was the case for nearly every other legal settlement we have discussed,

No individuals have been charged in any of these cases.

Thus, how well even such a large settlement will deter future wrong-doing is not clear.

Nonetheless, the documents released with it provide good documentation about how pervasive systematic, deceptive stealth marketing campaigns have become in health care. 

In particular, the official "complaint" filed by the US Department of Justice emphasized all these elements in the stealth marketing of paroxetine (Paxil, Seroxat in the UK) to adolescents.

Manipulation of Clinical Research

We have frequently discussed how health care corporations, particularly pharmaceutical, biotechnology and device companies, now sponsor  the majority of clinical research.  Their control of the design, implementation, analysis and dissemination of clinical research allows manipulation that increases the likelihood that the results will be favorable to their vested interests, usually the products and services they sell. 

We have previously discussed the manipulation of Study 329 to promote the marketing of Paxil (look here and here).  However, the US DOJ document makes these concerns more official.  It included:

Manipulation of Study Endpoints

Study 329 was a randomized controlled trial of Paxil vs imipramine vs placebo for depression in adolescents. The two primary endpoints pre-specified to the US Food and Drug Administration were "the degree to which a patient's Hamilton Rating Scale for Depression ('HAM-D') total score changed from baseline"; and "the patient's 'response' to medication, as defined as (a) a 50% or greater reduction in the patient's HAMD-D score, or (b) a HAM-D score of less than or equal to 8." However, initial analysis by GSK failed to show that Paxil improved either of these two end-points. The company concluded "it would be commercially unacceptable to include a statement that efficacy had not been demonstrated, as this would undermine the profile of [Paxil]."

So the analysis emphasized secondary outcomes, the "Study 329 investigators later added several additional efficacy measures not specified in the protocol. Paxil separate statistically from placebo on certain of these measures." Adding numerous post-hoc measures increased the likelihood of finding a difference on at least one due to chance alone.

Manipulation of Data

Initial analysis of the data suggested that patients given Paxil experienced 11 serious adverse events, including five that appeared related to suicidal ideation or action. When the FDA later reexamined the data, "upon closer examination the number of possible suicide-related events among the Study 329 Paxil patients increased beyond the five patients GSK described in the JACAAP article as having 'emotional lability.' While collecting saftey information for the FDA, GSK admitted that there were four more possible suicide-related events among Paxil patients in Study 329. In addition the FDA later identified yet another possible suicide-related event in the Study 329 Paxil patients, which was also not among the 11 serious advents listed in the JAACAP article. Thus, altogether, 10 of the 93 Paxil patients in Study 329 experienced a possible suicidal event, compared to one in 87 patients on placebo. This is a fundamentally different picture of Paxil's pediatric safety profile than the one painted by the JAACAP article...." 

Manipulation of Dissemination

The report describing the results of Study 329 (Keller MB, Ryan ND, Strober M et al.  Efficacy of paroxetine in the treatment of adolescent major depression: a randomized controlled trial.  J Am Acad Child Adolescent Psychiatry 2001; 40: 762-772.  Link here. ) was written under the control of GSK. "In April 1998, GSK hired Scientific Therapeutics Information, Inc (STI) to prepare a journal article about Study 329. GSK worked closely with STI on the article by providing a draft clinical report to 'serve as a template for the proposed publication.'"

The published report of Study 329 "mischaracterized the results." "Although the ... article identified the study's two primary endpoints in the abstract, the article did not explicitly state that Paxil failed to show superiority to placebo on either of the primary efficacy measures." Also, "the article did not explicitly identify the two protocol-specified primary outcome measures - or that Paxil failed to show superiority to placebo on these two measures. Instead the article claimed that there were eight efficacy measures and that Paxil was statistically superior to placebo on four of them." In addition, "while the article listed the five protocol-defined secondary endpoints, the text of the article omitted any discussion regarding three of the secondary measures on which Paxil failed to statistically demonstrate its superiority to placebo and instead focused on the five secondary measures that GSK added belatedly and never incorporated into the Study 329 protocol. The article claimed that these finve secondary measures had been identified 'a priori,' therefore incorrectly suggesting that all secondary endpoints had been part of the original study protocol." In other words, the final published articles contained multiple outright falsehoods about the drug's efficacy that exaggerated that efficacy.

Furthermore, initial analysis showed that patients given Paxil had more serious adverse events than others. An initial draft of the study article stated, "serious adverse events occurred in 11 patients in the paroxetine group, 5 in the imipramine group, and 2 in the placebo group." These included "headache during down-titration(1 patient), and various psychiatric events (10 patients): worsening depression (2); emotional lability (e.g., suicidal ideation/ gestures, overdoses), (5); conduct problems or hostility (e.g., aggressiveness, behavioral disturbance in school) (2); and mania (1)." As noted above, the number of suicide related events was actually double that noted in this draft as "emotional lability."  However, the published version of the report "falsely state[d] that only one of the 11 serious adverse events in Paxil patients was considered related to treatment...."  Nor did it mention the true number of events related to suicidal ideation or action.

The article only "listed at most five possibly suicidal events among Paxil patients, brushed those off as unrelated to Paxil, and conclude that treating children with Paxil was safe."

Later, GSK marketing materials described the results of the study thus,

This 'cutting-edge,' landmark study is the first to compare efficacy of an SSRI and a TCA with placebo in the treatment of major depression in adolescents. Paxil demonstrates REMARKABLE Efficacy and Safety in the treatment of adolescent depression."

Thus the conrol exerted by GSK over the published article, despite its apparent academic authorship, enabled it to promote a drug that was not efficacious and had major adverse events as remarkably safe and effective, a totally deceptive result that would mislead any health care professional who used the article to guide clinical practice. 

Suppression of Clinical Research

GSK sponsored two other studies of Paxil in pediatric populations, Studies 377 and 701. As stated in the Department of Justice's Criminal Complaint against GSK,

GSK Did Not Publicize the Results of Studies 377 and 701
43. GSK learned the results of Study 377 in 1998 and the results of Study 701 in 2001. Paxil failed to demonstrate efficacy on any of the endpoints of either study.
44. GSK did not hire a contractor to help write medical articles about the results of Studies 377 and 701, as it had with Study 329.
45. GSK did not inform its sales representatives about the results of Studies 377 and 701.

Thus, GSK managed to conceal the fact that the majority of the studies it sponsored about Paxil used for adolescent patients showed no evidence that the drug worked, again seriously distorting the evidence-base on which clinicians made decisions, and doubtless leading to the use of a dangerous, ineffective drug by numerous vulnerable patients.

Bribing Physicians to Prescribe

GSK's sales representative reflected in their call notes their use of money, gifts, entertainment and other kickbacks to induct doctors to prescribe GSK drugs....

One really creative way to pay physicians to be exposed to marketing:

For example, in or about 2000 or 2001, GSK used 'Reprint Mastery Training Programs' or 'RMTS' to further promote drugs by purporting to pay physicians to train sales representative to review reprints of studies. Although the training was purportedly for the representatives, in fact, the sales force was already familiar with the materials. GSK typically paid physicians $250 to $500 to review the reprints.

Thus GSK simply paid physicians to use its drug, a practice characterized as kickbacks in the official complaint.  An article in the Guardian noted that the US Attorney involved in the case put it even more bluntly,

The sales force bribed physicians to prescribe GSK products using every imaginable form of high-priced entertainment, from Hawaiian vacations [and] paying doctors millions of dollars to go on speaking tours, to tickets to Madonna concerts.

Use of Key Opinion Leaders as Disguised Marketers

GSK also created a group of national 'key opinion leaders' ('KOLs') who were paid generous consulting fees. GSK selected many of these physicians based on their prescribing habits and influence within the community and used the speaker fees paid to these physicians to induce and reward prescribing of GSK's products. GSK used these individual to communicate marketing messages focused on the drug's marketing campaigns at the time, including off-label uses. Some physicians on GSK's speaker's board have been paid more than a million dollars for speaking on behalf of the company and recommending its drugs.

Thus key opinion leaders were paid specifically to market drugs, and as a reward, a bribe for prescribing drugs.

Consulting Fees as Kickbacks

In general,

In order to induce physicians to prescribe and recommend its drugs, GSK paid kickbacks to health professionals in various forms, including speaking or consulting fees, travel, entertainment, gifts, grants, and sham advisory boards, training,....

In particular,

During 2000 and 2001 at least, GSK also utilized events termed 'advisory boards' or consultant meetings and forums to disseminate its promotional message. Although these boards were purportedly composed of 'thought leaders' for the purpose of obtaining advice from the physicians, in fact, the 'advisory boards' were little more than promotional events coupled with financial inducement to prescribing and influential physicians.

Also,

GSK typically paid the physician between $250 and $750 to attend each local 'advisory' meeting. The payments did not reflect the value of services. The physician was not required to do anything but show up. GSK had no legitimate business reason to hire thousands of 'advisors' to 'consult' with the company about a single drug.

Manipulation of Continuing Medical Education

GSK also used so-called CME and CME Express programs and other sham training for marketing purposes, and to promote off-labe uses for the GSK prescription drugs.

Furthermore,

These CME programs purported to be independent eduaction free of company influence, but in fact functioned as GSK promotional programs disguised as medical education. GSK maintained control and influence over the purportedly independent CME programs through speaker selection, and influence over content and audience, among other things. Although third party vendors were usually also involved, they served only as artificial 'firewalls' that did not insulate the program from GSK's influence.

Summary

The legal documentation of the GSK settlement demonstrated how one drug company used an integrated, systematic campaign incorporating deception and bribery to sell drugs. Its elements included manipulation and suppression of the clinical research it sponsored, paying key opinion leaders to be disguised drug marketers, outright payments to physicians to prescribe drugs, and manipulation, again using payments to physicians, of supposedly independent continuing medical education. 

Note that while I summarized the elements of the stealth marketing campaign to sell Paxil, particularly for use in pediatric patients, the US government complaint also documents similar activities used to sell other drugs.  Furthermore, other stealth marketing campaigns have come to light through legal action, and many other instances of manipulation and suppression of clinical research, use of KOLs as disguised marketers, kickbacks and bribes, and manipulation of CME have been documented.

This means that any claims that:
- commercially sponsored clinical research provides clear, unbiased data that should drive clinical decisions
- health care professionals and academics paid as consultants by commercial health care firms are not influenced by these payments, and can provide clear, unbiased opinions
- commercially sponsored medical education provides clear, unbiased teaching
unfortunately must be viewed with extreme skepticism. This is particularly unfortunate given that most clinical research is now supported by commercial sponsors, and the majority of influential academics in medicine get some form of payments from the health care industry (look here).

Of course, there are some physicians who consult for commercial firms who actually provide clinical or scientific advice or assistance, and some commercially sponsored activities are honest. But we must wonder what garden path all those advocates for increasing industry "collaboration" to promote "innovation," and who regard conflicts of interest as "inevitable" and "manageable" are taking us down (e.g., look here and here).

Although the current settlement will require a huge payment, as I have said many times before (as early as 2008, here), do not expect such settlements to deter future bad behavior like that listed above.  The cost of the settlement will actually be spread among all company shareholders, all company employees, and likely patients and taxpayers.  However, the settlement will entail no specific negative consequences to the people who authorized, directed, or implemented the bad behavior.  In particular, executives whose remuneration was swollen by proceeds from the sales of affected drugs, and the health care professionals who willingly accepted what the US Attorney called bribes will not pay any sort of penalty.  The bad behavior listed above was doubtless personally very profitable for some people.  Unless people who indulge in such behavior face the possibility of penalties worse than their expected gains, expect such bad behavior to continue.

In fact, as the New York Times reported,

critics argue that even large fines are not enough to deter drug companies from unlawful behavior. Only when prosecutors single out individual executives for punishment, they say, will practices begin to change.

'What we’re learning is that money doesn’t deter corporate malfeasance,' said Eliot Spitzer, who, as New York’s attorney general, sued GlaxoSmithKline in 2004 over similar accusations involving Paxil. 'The only thing that will work in my view is C.E.O.’s and officials being forced to resign and individual culpability being enforced.'

True health care reform would strive to eliminate important conflicts of interest affecting clinical research and medical education.  Specifically, it would prevent corporations that sell health care products or services from controlling clinical research meant to evaluate these products or services.  It would seek to eliminate serious conflicts of interest affecting health care professionals.  Finally, it would prevent vested interests from controlling medical education.  Not that I expect any such reform in the near future, it would be too threatening to those who have personally benefited from the current system.

Hat tip to Dr Howard Brody whose Hooked: Ethics, Medicine and Pharma blog scooped me on the details of the settlement relevant to study 329.

ADDENDUM (11 July, 2012) - see also this post on the 1BoringOldMan blog.

"More Marketing Than Science" - An Anonymous Confession About Deceptive Marketing Published in the British Medical Journal

The British Medical Journal just published an anonymous article by a pharmaceutical company insider that explained once again how pharmaceutical companies turn research studies, apparently scholarly articles, and medical education into stealth marketing efforts.  (See Anonymous.  Post-marketing observational studies: my experience in the drug industry.  Brit Med J 2012; 344: 28.  Link here.)

We have previously discussed examples of health care corporate insiders confessing their individual efforts to turn medical research and education into marketing.  For example, peruse this.  We have also discussed how documents made public through litigation have revealed marketing plans for specific drugs that used apparently academic, educational, or scholarly publications and venues to market without revealing this transformation.  For example, see the Neurontin marketing plan (see post here), and the Lexapro marketing plan (see post here).  We have also discussed numerous examples of manipulation of particular research studies by those with vested interests, and outright suppression of studies whose results did not favor such vested interests.

Yet I suspect majorities of health care academics and professionals, health care policy makers, and the public at large do not realize, or would not admit that the evidence base for making health care decisions, and the general academic and professional discourse has been so corrupted.  So it is worthwhile to review once again how an insider summarized this corruption.

Research Studies Designed Primarily as Marketing Vehicles

In general, the anonymous author suggested that at least some studies were done for marketing, not scientific purposes:

some of the studies I worked on were not designed to determine the overall risk:benefit balance of the drug in the general population. They were designed to support and disseminate a marketing message.

Whether it was to highlight a questionable advantage over a 'me-too' competitor drug or to increase disease awareness among the medical community (particularly in so called invented diseases) and in turn increase product penetration in the market, the truth is that these studies had more marketing than science behind them.

Furthermore, the studies were supervised not by physicians or scientists, but by marketers in the marketing department,

Although the medical department developed the publication plans, designed the study, performed the statistical analysis, and wrote the final paper (which when published was passed on to marketing and sales to be used as marketing material), the marketing team responsible for that product were directly involved in all stages. They also closely supervised the content of other educational 'scientific' materials produced in the medical department and intended for potential prescribers. Instructions from marketing to the medical staff involved were clear: to ensure that the benefits of the drug were emphasised and the disadvantages were minimised where possible.

Manipulation of Research Design, Implementation, or Analysis

The author described how the marketers manipulated research studies so they would produce the results desired from a marketing perspective, regardless of their underlying truth,

Since marketing claims needed to be backed-up scientifically, we occasionally resorted to 'playing' with the data that had originally failed to show the expected result. This was done by altering the statistical method until any statistical significance was found. Such a result might not have supported the marketing claim, but it was always worth giving it a go to see what results you could produce. And it was possible because the protocols of post-marketing studies were lax, and it was not a requirement to specify any statistical methodology in detail. On the other hand, the studies were hypothesis testing (such as cohort studies, case-control studies) rather than hypothesis generating (such as case reports or adverse events reports), so playing with the data felt uncomfortable.

Other practices to ensure the marketing message was clear in the final publication included omission of negative results, usually in secondary outcome measures that had not been specified in the protocol, or inflating the importance of secondary outcome measures if they were positive when the primary measure was not.

So to summarize, the marketers would control the statistical analyses, promoting multiple analyses to attempt to come up with the "right" result that would support the marketing message (although the more kinds of analyses one tries, the more likely one is likely to come up with false results by chance alone). Presumably the marketers did not care whether or not the results were really true, which is perhaps why even they felt "uncomfortable" in some circumstances.

They would also foster the suppression of negative results, and the dredging of data for extra outcome measures when analysis showed no advantage in terms of the real primary outcomes. Suppression of negative results could be viewed as plain lying. Deliberate analysis of multiple end-points again risks identifying random error as true results.

The Role of Key Opinion Leaders

The author described how key opinion leaders, that is, health care professionals thought to be especially influential on practice or policy, were hired to become marketers presumably without revealing this intention.

Every big international observational study had a large advisory board. This was critical since the success of a newly launched drug in the market would depend on how many key opinion leaders were part of the study. Not only would they add credibility to the results, but they would also be key in influencing decision makers and other prescribers. In regional studies with thousands of patients, the study’s advisory board was formed by at least one key opinion leader from each country in that region, ensuring that areas important in terms of possible sales were covered. The contributions of the key opinion leaders to the study were always positive, but in my experience more directed towards designing new studies to answer their specific clinical questions rather than critically appraising our results and conclusions. In general, the relationship was amicable. We took them to the best hotels and restaurants during our advisory board meetings, and they appeared as authors in our research. Later, they would act as the company’s 'ambassadors,' giving conferences, teaching doctors, or talking to the media about the benefits of the drug.

Note that even the anonymous author could not bring him or herself to call the key opinion leaders salespeople, or marketers, but used the ambiguous wterm "ambassadors." Nonetheless, the role of key opinion leaders described would clearly be that of marketers or salespeople. However, it is extremely doubtful that any of these KOLs felt they had to declare that they were paid salespeople when presenting at conferences, teaching doctors, or talking about the media.

I would suggest that their actions would therefore fit Transparency International's definition of ethical corruption, "abuse of entrusted power for private gain." The KOLs are entrusted to be professional, and in many cases, scholarly. Using a professional or scholarly guise to act as a salesperson appears to be abuse of that entrusted power, in my humble opinion. In nearly every case, KOLs are paid, often handsomely, by the companies whose products they are selling. Thus, key opinion leaders acting as described by the anonymous author appear to be ethically corrupt.

Summary

The evidence of unethical marketing practices by commercial health care firms is mounting. Although I am most familiar with evidence from the US, there is mounting evidence that these practices are global, often done by companies that are not US based, and meant to influence practice and policy world-wide.

As the evidence mounts, it becomes increasingly clear that many such marketing practices are corrupt, at least in an ethical sense. Whether they may break laws in particular countries is a question for someone else.

The latest BMJ article is a reminder how skeptical the shrinking group of health care professionals who do not have conflicts of interest, are not biased in favor of particular products, and who put patients' interests first must be about ALL published research, scholarly publications, and apparently educational activities. Advocates of true evidence-based medicine must be extremely careful to try to use the least biased and manipulated parts of the evidence-base.

The mounting evidence suggests that at a minimum, all research reports, scholarly articles, media reports, conferences, and educational programs should provide full, detailed disclosure of all conflicts of interest. Perhaps having to make a declaration like "I am paid 50,000 Euros a year by the marketing department of company X to help market drug Y" before a supposedly educational talk would make some health care professionals think twice about such relationships.

However, even such detailed disclosure may not be sufficient to hamper marketing practices that now appear overtly corrupt. In my humble opinion, it is time to consider a global ban on the funding or influencing of human research by companies and other organizations which stand to gain financially according to the results of the research, or whose products and services are subject of such research. It is also time to consider a global ban on the funding or influencing of health care education by such organizations.

However, so many people are making so much money from the current practices that I doubt such proposals would get much support, or even public notice beyond this humble blog.

Pseudo-Evidence Based Medicine Should be a Global Health Concern

We have frequently advocated for evidence-based medicine (EBM) , that is, medicine based on judicious use of the best available evidence from clinical research, critically reviewed and derived from systematic search, combined with biomedical knowledge and understanding of patients' values and preferences.  However, EBM risks being turned into pseudo-evidence based medicine due to systematic manipulation and distortion of the clinical research evidence base.  Dr Wally Smith wrote about pseudo-evidence based medicine, which he defined as "the practice of medicine based on falsehoods that are disseminated as truth," in the British journal Clinical Governance (Smith WR. Pseudoevidence-based medicine: what it is, and what to do about it. Clinical Governance 2007; 12: 42-52. Also see this post).  

Now it appears that this issue is causing concern in the Cochrane Collaboration, the main voluntary international group promoting EBM.  An article from December, 2011 in the Indian Journal of Medical Ethics outlined why we need to question the trustworthiness of the global clinical evidence base.  (See Tharyan P. Evidence-based medicine: can the evidence be trusted.  Ind J Med Ethics 2011; 8: 201-207.  Link here.)  It merits further review.

The Role of Vested Interests

Dr Tharyan, its author, emphasized that a major threat to the integrity of the clinical research data base is the influence on clinical research of those with vested interests in marketing particular products, e.g., drugs, devices, etc.

The motives for conducting research are often determined by considerations other than the advancement of science or the promotion of better health outcomes. Many research studies are driven by the pressure to obtain post-graduate qualifications, earn promotions, obtain tenured positions, or additional research funding; many others are conducted for financial motives that benefit shareholders, or lead to lucrative patents.

The Importance of Pervasive Conflicts of Interest

Early on, the author discussed how the distortion of the clinical research data base arises from the pervasive web of conflicts of interest in medicine and health care:

This hijacked research agenda perpetuates further research of a similar nature that draws more researchers into its lucrative embrace, entrenching the academic direction and position statements of scientific societies and academic associations. Funders and researchers are also deterred from pursuing more relevant research, since the enmeshed relationship between academic institutions and industry determines what research is funded (mostly drugs at the expense of other interventions), and even how research is reported; thus hijacking the research agenda even further away from the interests of science and society.

The article then goes on to show how the influence of vested interests may distort the design, implementation, analysis, and dissemination of research.

Distortion of Research Design and Implementation

The Research Question

Dr Tharyan noted

The majority of clinical trials conducted world-wide are done to obtain regulatory approval and a marketing licence for new drugs. These regulations often require only the demonstration of the superiority of a new drug over placebo and not over other active interventions in use. It is easier and cheaper to conduct these trials in countries with lower wages, lax regulatory requirements, and less than optimal capacity for ethical oversight. It is therefore not surprising that the focus of research does not reflect the actual burden of disease borne by people in the countries that contribute research participants, nor address the leading causes of the global burden of disease. Some 'seeding' trials, conducted purportedly for the purpose of surveillance for adverse effects, are often only a ploy to ensure brand loyalty among participating clinician-researchers.

Insufficient Sample Size

The article stated,

Many trials do not report calculations on which the sample size was estimated, often leading to sample sizes insufficient to detect even important differences between interventions (for primary, let alone secondary outcomes)

I would add that such small trials are particularly bad at detecting important adverse effects of the interventions being promoted.

Excessively Stringent Enrollment Criteria

As Dr Tharyan wrote,

Most RCTs funded by industry and academia are designed to demonstrate if a new drug works, for licensing and marketing purposes. In order to maximise the potential to demonstrate a 'true' drug effect, homogenous patient populations; placebo controls; very tight control over experimental variables such as monitoring, drug doses, and compliance; outcomes addressing short term efficacy and safety; and methods to minimise bias required by regulatory agencies are used to demonstrate if, and how, the drug works under ideal conditions.

The problem is that very few patients in clinical practice resemble those enrolled in such trials, so the generalizability of the trials' results is actually dubious. Ideally, clinicians practicing evidence-based medicine could refer to trials that include patients similar to those for whom they care.

Practical or pragmatic clinical trials are designed to provide evidence for clinicians to treat patients seen in day-to day clinical practice, and evaluate their effectiveness under 'real-world' conditions. These trials use few exclusion criteria and include people with co-morbid conditions, and all grades of severity. They compare active interventions that are standard practice, and in the flexible doses and levels of compliance seen in usual practice. They utilise outcomes that clinicians, patients, and their families consider important, such as satisfaction, adverse events, return to work, and quality of life). Recommendations exist on their design and reporting , but such trials are rare.

Comparisons to Placebo, not the Other Interventions Clinicians Might Realistically Consider

Per the article,

Industry sponsored trials rarely involve head-to head comparisons of active interventions, particularly those from other drug companies, thus limiting our ability to understand the relative merits of different interventions for the same condition.

The result may thus be a number of studies showing particular interventions appear to be better than nothing, but few if any studies that would help clinicians decide which intervention would be best for a particular patient.

Inappropriate Comparators

However,when studies are done comparing the intervention of interest to other active interventions, the details of the choice of comparators are often managed so that the comparators are likely to appear to be worse.

Even if active interventions are compared in industry-sponsored trials, the research agenda has devised ways in which the design of such trials is manipulated to ensure superiority of the sponsor’s drug. If one wants to prove better efficacy, then the comparator drug is a drug that is known to be less effective, or used in doses that are too low, or used in non-standard schedules or duration of treatment. If one wants to show greater safety, then the comparator is a drug with more adverse effects, or one that is used in toxic doses. Follow up, also, is typically too short to judge effectiveness over longer periods of time.

Meaningless Outcomes

A favorite tactic used in the design of trials influenced by vested interests is to choose outcomes that are likely to show results favorable to the product being promoted, but have no meaning for patients or clinicians. There are three ways this is commonly done.

The first is to use rating scales that are very sensitive to small perturbations, but not meaningful in terms of how patients feel or function:

The choice of outcome measures used often ensures statistically significant results in advance, at the expense of clinically relevant or clinically important results. Outcomes likely to yield clinically meaningless results include the use of rating scales (depression, pain, etc.). These scales yield continuous measures usually summarised by means and standard deviations, rather than the dichotomous measures clinicians use such as: clinically improved versus not improved. These rating scales, however extensively validated, are hardly ever used in routine clinical practice. A difference of a few points on these scales results in statistically significant differences (low p values), that have little clinical significance to patients.

Another is to use surrogate outcomes,

Other outcomes commonly used are surrogate outcomes; outcomes that are easy to assess but serve only as proxy indicators of what ought to be assessed, since the real outcome of interest may take a long time to develop. These are mostly continuous measures that require smaller sample sizes (blood sugar levels, blood pressure, lipid levels, CD4 counts, etc.). These measures easily achieve statistical significance but do not result in meaningful improvements (reduction in mortality, reduction in complications, improved quality of life) in patients’ lives, when the interventions are used (often extensively) in clinical practice.

Finally, there are composite outcomes,

The use of composite outcomes, where many outcomes (primary, secondary and surrogate outcomes) are clubbed together (e.g.: mortality, non-fatal stroke, fatal stroke, blood pressure, creatinine values, rates of revascularisation) as a single primary outcome, can also mislead. Such trials also require smaller sample sizes, and increase the likelihood of statistically significant results. However, if the composite outcome includes those of little clinical importance (lowered blood pressure, or creatinine values), the likelihood of real benefit (reduction in mortality, or strokes, or hospitalisation) and the potential for harm (increase in non-fatal strokes or all-cause mortality) are masked.

Distortion of Analysis

Relative Instead of Absolute Risks

A favorite trick is to emphasize relative risks rather than absolute risks. For example if a trial reduces the risk of a bad outcome from 2 of 10,000 patients (0.02%) to 1 of 10,000 patients (0.01%), the relative risk reduction is 50%, but only 1 of 10,000 patients experienced a benefit.

The use of estimates of relative effects of interventions, such as relative risks (RR) and odds ratios (OR) with their 95% confidence intervals, provides estimates of relative magnitudes of the differences and whether these exclude chance, as well as if these differences were nominal or likely to be clinically important.

However, even relative risks can be misleading since they ignore the baseline risk of developing the event without the intervention. The absolute risk reduction (ARR) is the difference in risk of the event in the intervention group and the control group, and is more informative since it provides an estimate of the magnitude of the risk reduction, as well the baseline risk (the risk without the intervention, or the risk in the control group). Systematic enquiry demonstrates that on average, people perceive risk reductions to be larger and are more persuaded to adopt a health intervention when its effect is presented as relative risks and relative risk reduction (a proportional reduction) rather than as absolute risk reduction; though this may be misleading.

Sub-Group Analysis

Another statistical trick used to present favourable outcomes for interventions is the use of spurious subgroup analyses, where observed treatment effects are evaluated for differences across baseline characteristics ( such as sex, or age, or in other subpopulations). While they are useful, if limited to a few biologically plausible subgroups, specified in advance, and reported as a hypothesis for confirmation in future trials; they are often used in industry-sponsored trials to present favourable outcomes, when the primary outcome(s) are not statistically significant.

A major statistical point is that the more ways one subdivides the study population, the more likely it is to find a difference between those receiving different interventions by chance alone.

Distortion of Dissemination

Poor Exposition

As noted by Dr Tharyan,

Evidence also shows that the published reports are not always consistent with their protocols, in terms of outcomes, as well as the analysis plan, and this again is determined by the significance of the results. Harms are very poorly reported in trials compared to results for efficacy; and are also often suppressed or minimised.

Ghost-Writing

Also, those with vested interests may try to ensure maximally favorable dissemination by employing ghost-writers,

Other tactics used to influence evidence-informed decision making include ghost-writing, where pharmaceutical companies hire public-relations firms who 'ghost-write' articles, editorials, and commentaries under the names of eminent clinicians; a strategy that was detected by one survey in 75% of industry-sponsored trials, where the ghost author was not named at all, and in 91% when the ghost author was only mentioned in the acknowledgement section. Detecting such conflicts of interest is difficult, since they are rarely acknowledged due to the secrecy that shrouds the nexus between academia and industry in clinical trials.

Industry-sponsored trials often place various constraints on clinical investigators on publication of their results; these publication arrangements are common, allow sponsors control of how, when, and what is published; and are frequently not mentioned in published articles.

Ghost-writers under the direct control of those with vested interests could more efficiently bias their writing in favor of their sponsors than could even academics constrained by contractual obligations to sponsors.

Suppression of Research

When all else fails, the most crude example of distorted dissemination is suppression of studies that despite all the manipulations noted above fail to produce favorable results for the product being promoted:

A considerable body of work provides direct empirical evidence that studies that report positive or significant results are more likely to be published; and outcomes that are statistically significant have higher odds of being fully reported, particularly in industry funded trials.

By the way, the latest demonstration of suppression of research was a study by Turner et al (Turner EH, Knoepflmacher D, Shapley L. Publication bias in antipsychotic trials: an analysis of efficacy comparing the published literature to the US Food and Drug Administration database. PLoS Medicine 2012; 9(3): e1001189. Link here) It showed that of 24 trials of atypical antipsychotics registered in the US FDA database, 15/20 (75%) of the published trials were positive, that is, had results in favor of the sponsors' drugs according to the FDA review, but only 1/4 (25%) of the unpublished trials were positive. In other words, 15/16 of positive trials were published, but only 5/8 non-positive ones were.

Summary

Thus, Dr Tharyan provided a nice summary of many of the ways that the clinical research data base can be manipulated or distorted to serve vested interests. Such distortions risk the transformation of evidence-based medicine into pseudo-evidence based medicine.

We have repeatedly discussed (see links above) most of these issues. Because we are located in the US, and speak mainly English, this blog may have given the impression that the lack of trustworthiness of the clinical research evidence base is primarily a US problem. The article by Tharyan emphasizes, however, that it is a global problem.

While this problem now seems to have the attention of the Cochrane Collaboration, it seems to be relatively anechoic in global health circles. It appears to be no more prominent on the agendas of global health organizations than is health care corruption (look here.) Yet the two issues are highly related. Most of the distortions in the global clinical evidence database may be driven by conflicts of interest. Conflicts of interest are risk factors for corruption. Distortions of the global clinical research data base that lead to use of expensive, but ineffective or dangerous interventions when other options would work as well or better leads to needless suffering and death, and by unnecessarily raising the costs of care, decreases access, especially for the poor.

Also note that conflicts of interest may be one reason that all these problems remain so anechoic (look here).

True global health care reform requires addressing health care corruption, but also conflicts of interest and their role in the distortion and manipulation of the clinical research data base. I still live in hope that that some academic health care institutions, professional societies, health care charities and donors, and patient advocacy groups will gain enough fortitude to stand up for accountability, integrity, transparency, and honesty in health care.