This example appeared in the journal Evidence-Based Mental Health [Gahemi SN. The failure to know what isn't known: negative publication bias with lamotrigine and a glimpse inside peer review. Evidence Based Mental Health 2009; 12: 65-68. Link here.]
The author was able to use the database of clinical trials provided by GlaxoSmithKline as part of a settlement of a suit by former New York state Attorney General Elliot Spitzer that charged that the company had suppressed information about the selective serotonin reuptake inhibitor paroxetine (Paxil). His interest was the use of lamotrigine (Lamictal) in depression. His main findings were:
Of the nine lamotrigine related bipolar disorder studies posted on the website (see table 1), two were positive and published supporting the FDA approved indication for delay of relapse in the long term treatment for bipolar disorder patients. A negative study in rapid cycling bipolar disorder and another in acute bipolar depression were published but both emphasised positive secondary outcomes as opposed to the negative primary outcomes. Five other negative studies involving rapid cycling bipolar disorder, acute bipolar depression and acute mania have not been published and are only available on the GSK website.
Failure to adequately publish these negative studies led to the creation of a clinical impression that lamotrigine is an 'antidepressant,' a view innocently expressed to me as recently as last week by an academic colleague. This mistaken impression occurred partly because the prophylactic benefits of lamotrigine for depressive episodes were confused with a presumed acute benefit. Partly it was due to the publication of one apparently positive study, and the non-publication of several negative studies.
The clinical relevance of the lamotrigine studies is notable: taking the negative outcomes into account, as of now, one might say that this agent is reasonably effective in maintenance treatment of bipolar disorder, particularly in prevention of depression. It is proven ineffective in acute mania, rapid cycling disorder and acute bipolar depression.
Note that Dr Ghaemi and colleagues had published a summary of the suppressed negative articles in Medscape Journal of Medicine in late 2008. [Ghaemi SN, Shirzadi AA, Filkowski M. Publication bias and the pharmaceutical industry: the case of lamotrigine in bipolar disorder. Medscape J Med 2008; 10(9):211. Link here.] That earlier article also documented the group's failed attempts to discover whether negative trials of other drugs for depression had been suppressed.
Dr Ghaemi's newer made some additional important points about suppression of clinical research. The first is that it is occurring with the acquiescence of government regulators. In particular,
It is worth mentioning that the [US] FDA [Food and Drug Administration] has encouraged this state of affairs, by viewing negative studies as uninformative, due to the possibility of being 'failed' rather than truly negative (ie, the sample may have simply been unresponsive, or dosing might have been too low and so on). Thus drugs could have two positive studies, and 10 or so negative ones (as did a number of selective serotonin reuptake inhibitors),8 and the FDA not only allowed approval but it did not require that the pharmaceutical industry publish its negative results. The pharmaceutical industry did the minimum necessary; the FDA set the minimum far below what should have been the acceptable scientific standard.
Furthermore, even though the FDA is supposed to allow access to information about all studies for drugs for which it provides an indication, in fact it still does not allow access to raw data from industry sponsored studies, which are viewed as confidential and proprietary. Rather, in my experience and those of colleagues who have attempted to access such studies, Freedom of Information Act requests are met with abstracted summary results. While summary results are better than no results, full access to scientific data should be the standard at the FDA.
The second is that medical journal reviewers and editors do not seem enthused about publishing articles about how research has been suppressed (an apparent example of the anechoic effect, or perhaps the anechoic effect squared, i.e., discussion of our failure to discuss research studies whose results did not please their sponsors is simply not done.)
In the course of trying to investigate and publicise these negative findings, we ran into numerous roadblocks, as I discussed in an interview for the Carlat Report last year
Instead of repeating what I have said before about the evils of suppressing clinical research, I will offer some concluding quotes from Dr Ghaemi.
Evidence based medicine—or, more simply put, the science of medicine—cannot be taken seriously, and is certainly not valid, if the evidence base is only partial. The scientific literature currently is like an under cooked meal which we think is ready to eat. We never know whether what we see in the evidence is correct or biased in one direction or the other. Meta-analyses of large published datasets are not as meaningful as they seem when unpublished data languish elsewhere.Manipulation and suppression of clinical research has likely misled doctors and patients into using and paying too much for minimally effective, ineffective, or even harmful drugs and devices. Manipulation and suppression of research is thus probably a major reason that health care costs much more than its value and accessibility warrants. To truly reform health care, we should not let those with vested interests in selling health care products or services control the clinical research that purports to evaluate what they are selling.
