What If the Institute of Medicine Wrote a Report and Nobody Followed it? - the Case of the Standards for Developing Trustworthy Guidelines

For over 20 years, clinical practice guidelines (CPGs) have been touted to improve health care quality and control costs.  Enormous numbers of guidelines have been developed, but with seemingly little impact on health outcomes.  While some of those leading health care organizations have predictably blamed individual practitioners for obstinately ignoring or challenging guidelines, there is increasing evidence that maybe the guidelines themselves are part of the problem.

 An Example of a Guideline that Apparently was Not Trusted

One example Dr Wally Smith and I have taught in our recurring mini-course on why physicians fail to follow guidelines (and otherwise appear not to practice in accord with others' wishes) is that of the guidelines on management of depression in primary care.  Most existing guidelines urge physicians to screen patients for (presumably mild-to-moderate) depression and treat them aggressively, with emphasis on the use of the newer anti-depressants.  These guidelines, in turn, were based on numerous published randomized clinical trials that showed that these drugs were safe and efficacious.  Yet multiple studies showed that physicians failed to follow these guidelines, and various attempts to improve their adherence did little.  So for years the assumption was that physicians at best experienced practical and system barriers to follow these guidelines, and at worst were ill-informed or irrational. 


However, information that came out gradually during the early part of the 21st century suggested that perhaps the problem was within the guidelines, not the health care professionals.  First, documents produced during New York Attorney General Eliot Spitzer's lawsuit against GlaxoSmithKline about the marketing of one of these drugs (Paxil, paroxetine) suggested that the company had suppressed clinical trial data that reflected poorly on the drug (See Kondro W.  Drug company experts advised staff to withold data about SSRI use in children.  Can Med Assoc J 2004; 170: 783.  Link here.)    These suspicions were later fleshed out  by consideration of documents further disclosed in litigation (e.g., see this post and its links).  Then several studies, most particularly that by Erick Turner and colleagues, showed that numerous trials of new anti-depressants had been suppressed, that is, never published (Turner et al. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 2008; 358:252-260.  Link here).  When the results of these trials were added to those that were published, the efficacy of anti-depressants was no longer so clear.  So maybe the guidelines that physicians did not follow were not trustworthy, and should not have been followed in the first place.

Would IOM Standards to Improve Guideline Trustworthiness Help?

So in 2011, the prestigious Institute of Medicine released a report on the development of  better standards to produce more trustworthy guidelines (Clinical Practice Guidelines We Can Trust.  Link here.)  We posted about that report here, but noted that it was receiving little other attention, an example of the anechoic effect. 

A few weeks ago, an article appeared documenting a study meant to assess the the trustworthiness of clinical practice guidelines published soon after the IOM report.  Its title telegraphs the results. ( Kung J, Miller RR, Mackowiak PA. Failure of clinical practice guidelines to meet Institute of Medicine standards: two more decades of little, if any progress.  Arch Intern Med 2012.  Link here.)

Methods and Results

The investigators selected a random sample of 114 individual guidelines available during June, 2011 stratified by 26 clinical topics. The versions of the guidelines used were those archived in the National Guideline Clearinghouse (NGC) maintained by the Agency for Healthcare Research and Quality (AHRQ).


The goal of the study was to "examine adherence to the IOM standards" by guidelines published after the standards were published.  Actually, the study only assessed adherence to 18 of the 25 standards espoused by the IOM (because the remaining seven were "too vague and subjective to be analyzed.")

Furthermore, the criteria used to determine if a specific guideline met each of the three items above were rather lax:

In evaluating each guideline summary, care was taken to be as liberal as possible in considering that a standard was met when the individual guideline summary provided any information pertaining to that particular standard.

Nevertheless, using these lax standards to only evaluate adherence to 18/25 guidelines, the authors found that "the overall median number of IOM standards satisfied (out of 18) was 8 (44.4%) ....  Fewer than half of the guidelines surveyed met more than 50% of the IOM standards."

An examination of the details of the study's methods reveals things are even worse than that.

Analyzing the Study's Methods to Find that Things Are Worse Than They Seem

Review of the study's methods show that they provided a very optimistic view of adherence to the IOM standards.  As noted above, the study did not look for adherence to all of the IOM standards.  Moreover, those they did consider were simplified and made less rigorous.  For example, Standard 2 from the IOM on management of conflict of interest (COI) was:

  STANDARD 2
Management of conflict of interest (COI)

2.1
Prior to selection of the Guideline Development Group (GDG), individuals being considered for membership should declare all interests and activities potentially resulting in COI with development group activity, by written disclosure to those convening the GDG.
- Disclosure should reflect all current and planned commercial (including services from which a clinician derives a substantial proportion of income), non-commercial, intellectual, institutional, and patient/public activities pertinent to the potential scope of the CPG.

2.2
Disclosure of COIs within GDG
- All COI of each GDG member should be reported and discussed by the prospective development group prior to the onset of their work.
- Each panel member should explain how their • COI could influence the CPG development process or specific recommendations.

2.3
Divestment
- Members of the GDG should divest themselves of financial investments they or their family members have in, and not participate in marketing activities or advisory boards of, entities whose interests could be affected by CPG recommendations.

2.4
Exclusions
- Whenever possible GDG members should not have COI.
- In some circumstances, a GDG may not be able to perform its work without members who have COIs, such as relevant clinical specialists who receive a substantial portion of their incomes from services pertinent to the CPG.
- Members with COIs should represent not more than a minority of the GDG.
- The chair or co-chairs should not be a person(s) with COI.
- Funders should have no role in CPG development.

However, the study boiled all this down to three items:
-  COIs stated
-  Chair has COI
-  Co-chairperson has COI

Thus the study did not address standards requiring full and complete disclosure (not just some disclosure) of all COIs; consideration of how the COIs might influence the particular guideline; divestment of specific types of conflicts of interest, that is, financial investments, and cessation of participation in marketing activities or advisor boards; minimization of conflicts of all members of the committee; and barring of participation of funders in guideline development.

Even so, the guidelines assessed did a very poor job upholding even these few liberalized standards regarding conflicts of interest.  Of the guidelines assessed, less than half, 46.8% provided ANY disclosure of conflicts of interest.  Those written by sub-specialty societies were particularly opaque in this regard.  Less than one-third, 29.3%, provided any disclosure.  Thus the majority of guidelines assessed were not at all transparent about conflicts of interest affecting the guideline development process.

Furthermore, of the 46.8% of all the guidelines which made any disclosures of conflicts of interest, 71.4% admitted their chair people HAD a conflict of interest.  Thus, only (0.468 * [1 - .714]) = 13.3% provided assurance that they fulfilled the single requirement (from standard 2.4 above) that the chair person did not have a COI.  For the guidelines written by sub-specialty societies, by a similar calculation, only 12.2% provided an assurance that the chair had no COI.  (The proportions providing assurance that the co-chair people had no COI were even lower.)  Thus the vast majority of guidelines did not clearly follow two straight-forward standards for minimizing the effects of conflict of interest, that the guideline committee chair and co-chair should not have any relevant conflicts. 

Given the miserable results concerning even minimal adherence to some of the IOM report's conflict of interest standards, it is likely that almost no published guidelines from 2011 came close to fulfilling the full set of IOM standards.  Despite the best efforts of the IOM, it appears that guideline developers have not progressed at all towards providing trustworthy guidelines. 

Summary

An editorial (Shaneyfelt T. In guidelines we cannot trust.  Arch Intern Med 2012) accompanying the article by Kung and colleagues summarized its results thus:

The same problems that have plagued guideline development continue to plague guideline development; namely, their variable and opaque development methods, their often conflicted and limited panel composition, and their lack of significant external review by stakeholders throughout the development process.   As a result, the trustworthiness of guidelines is limited.

While guidelines may have seemed to be a promising method to improve health care in the early 1990s, they have failed to live up to that promise.  Shaneyfelt was not optimistic they would improve in the future:

I am not optimistic that much will improve.  No one seems interested in curtailing the out-of-control guideline industry.

On the other hand, in my humble opinion, it is not that on one is interested in better guidelines.  It would clearly be in the best interests of patients and the public, and of health care professionals who care about the quality of their practice and the outcomes of their patients to curtail that industry.  The issue is why patients', the public's, and professional's interests were ignored.

Neither Shaneyfelt nor Kung et al discussed why there has been so little attention to patients' and the public's health, and to health care professionalism in all this.  For a quick answer, we do not have to look far on Health Care Renewal. 


In fact, the IOM report on guideline development from 2011 was a serious challenge to the powers that be in health care.  In particular, it challenged the cozy relationships that had grown up among the organizations that undertook guideline development and the health care professionals on guideline panels on one hand and organizations that stand to gain were specific guidelines to favor their products, services, and agendas on the other.  The standards mandated transparency and honesty about conflicts of interest affecting guideline committees and the organizations which assembled them, and if upheld would have greatly reduced these relationships.

Now it turns out that the guideline standards have been honored mainly in the breach.  Of course, these standards, while increasing trustworthiness, would have cost a lot of medical societies considerable commercial funding, and would have cost a lot of health care professionals on guideline panels considerable personal wealth.  These standards would probably also have cost a lot of companies whose products and services were addressed by guidelines to lose revenue.  So it is not surprising that the IOM standards were ignored.  Their implementation would have cost too many people who are financially benefiting from the status quo too much money.  And these people, that is, leaders of professional societies dependent on commercial outside funding, health care professionals and academic used to financial support from commercial interests, and health care corporations are good at making sure their interests are not ignored, even if their interests conflict with those of patients, the public, and well-intentioned health care professionals.  

So, the flouting of the well reasoned IOM guideline standards adds one more reason for patients and the general public to distrust modern health care and all those who "deliver" it, even to distrust well-intentioned health care professionals who have not been able to distinguish themselves from their colleagues who are too happy to help commercial interests while taking commercial money.  If health care professionals want to regain the public's trust, they could do worse than publicly declaring their intention to show that their practice in the future will be guided by trustworthy guidelines based on clinical research evidence and knowledge of biomedical science, drawn up by health care professionals independent of commercial interests.

Logical Fallacies in Defense of the European Society of Cardiology's Lenient Approach to Conflicts of Interest

The European Society of Cardiology just published its recommendations for interactions between medical societies and industry.(1)   The report emphasized the supposed benefits of industry relationships and funding, and suggested few restrictions on current practices.  Like other defenses of leniency towards conflicts of interest, its arguments seemed to rest on a series of logical fallacies.  

Wonderful "Innovations ... Through Productive Collaboration" - Appeal to Belief and the Fallacy of Division

The report made several arguments about the benefits of professional - industrial interaction.  These included:

In recent decades, cardiology has been a fast-moving medical speciality. Many advances have come from basic and clinical research conducted by universities and by pharmaceutical and medical device companies. Innovations have been realized in part through productive collaborations between clinicians, academia, and industry. Such links are essential and need to be encouraged and supported by appropriate investment if medical progress is to be sustained.

The notion that interactions between physicians and industry is a source, perhaps the most important source of wonderful "innovations" is often used to justify the sorts of interactions, involving payments by industry to academia and physicians, that are now prevalent. In this report, this first appears as an assertion without any evidence to justify it.

Later, the report noted:

The Association of American Medical Colleges has stated that there are benefits from effective partnerships between industry and academic medical centres.

This statement came with a citation, to a 2008 report entitled "Industry Funding of Medical Education."(2) This AAMC report, in turn, contained many similar assertions, e.g.,

An effective and principled partnership between academic medical centers and various health industries is critical in order to realize fully the benefits of biomedical research and ensure continued advances in the prevention, diagnosis, and treatment of disease.

However, the AAMC report also failed to cite any evidence in support of these assertions.

There are two major problems with these assertions, which form the foundations of both reports. The first is they do not seem to have a clear evidence base.

While the notion that current research efforts marked by substantial interaction between physicians, academia and industry are producing wonderful innovations is appealing, what evidence there is suggests that clinically important innovation is rare. For example, I quote a 2009 review of global drug discovery by Light(3) on whether new drugs represent advances over older treatments:

the best evidence of clinical quality comes from systematic efforts to assess therapeutic advantage and adverse effects compared with existing drugs. A detailed analysis of therapeutic quality in new drugs over the past twenty years found that 14 percent of all new chemical entities are either therapeutic breakthroughs or substantially superior to existing medications. Likewise, a comprehensive review of all new drugs approved between 1989 and 2000 in the United States concluded that 14.8 percent were new chemical entities that provided significant clinical improvement, and a Canadian review board concluded that 10.7 percent of new chemical entities in 2000–2004 did so.

Thus while many new drugs are introduced, only a few introduced recently were chemically unique or had effects different than older treatments.  I would argue that most of these will prove not to be truly clinically innovative, in that good clinical research will not show that they produce substantial improvements in clinical outcomes without major adverse effects for a good number of patients with not uncommon problems.  (It is not hard to think of really important innovations developed before the era of major industry-academic-physician interactions: antisepsis for surgery, anesthesia for surgery, antibiotics, hormone replacement therapy with insulin, thyroid hormone, etc, smallpox vaccination, polio vaccination, etc, etc)  However, it is actually very hard to think of more than a handful of new drugs or devices discovered in the last 20 years which were that hugely innovative.  (The closest might be multi anti-viral drugs used to treat HIV, and Gleevec for chronic myelogenous leukemia.)  If there has been very little really important innovation in the last 20 plus year era of enhanced industry-academic-physician interactions, these collaborations could not have produced tremendous innovation. 

So the report's arguments rests on an assertion that is not clearly justified, and which appears to be at best a huge exaggeration. Resting an argument on a belief that is not further supported by evidence amounts to a obvious logical fallacy.  It is an appeal to belief.

Furthermore, while both reports emphasize physician-industry collaboration or interaction, there are many ways interaction or collaboration can occur without involving substantial payments by the latter to the former. It is possible for two people or organizations to work together without one paying the other.  Yet both reports use the broad assertions about interaction and collaboration to justify interactions and collaborations in which industry makes substantial payments to physicians or academic institutions. This appears to be an example of the fallacy of division, that is, an unjustified assertion that "what is true of a whole must also be true of its constituents."

CME Would Wither if Financing Were Reduced, and Industry is the Only Possible Source of Such Money - A Slippery Slope and a False Dilemma

The ESC report made the argument that without industry funding, CME would wither. For example,

Should Europe choose to follow the strategy proposed in the USA, severing links between industry and medical societies, CME could be severely compromised. Relying completely on public funding is not a viable option for Europe at the moment. The removal of industry support for medical associations would be followed by increased fees and reduced attendance at congresses especially by clinical trainees and young fellows. It is the view of the ESC that in the absence of alternative funding, or until alternative funding is identified, maintaining links with industry is appropriate....

The argument is that CME would fail if financial support for it would be reduced, and that industry is the only possible source of financial support at the current level. The notion that CME must inevitably fail if financing of it were reduced is a slippery slope fallacy, that is, a statement that a inevitably causes b when such inevitability is not proven. Why could not adequate CME be done at a lower cost, even if the result were less luxurious? Furthermore, the notion that industry is the only source of funding is a false dilemma fallacy. In fact, there are other possibly sources of funding. Given that physicians are among the world's best paid professionals, why could they not pay for their own CME?

"Conflicts of Interest are Unavoidable and ... They Cannot Be Abolished" - False Dilemma

The third major argument on which the report bases its recommendations is encapsulated above, and was buttressed by:

The risk of bias in medical education is not restricted to activities that are supported by industry. It can affect any type of scientific communication, even an educational meeting organized independently by a university or medical association.

Underlying these assertions seem to be extremely broad definitions of bias and conflicts of interest, coupled with unwillingness to see a distinction between trivial and serious varieties of each. This again appears to be a false dilemma. The distinctions should be between no conflicts, trivial conflicts, and serious conflicts, not between no conflicts and any conflicts, even if trivial.  This also could be called an example of how "the perfect is the enemy of the good."  Even if perfectly preventing all conflicts of interest is impossible, does this imply that preventing serious conflicts of interest is not worthwhile.   

Summary

We have noted that logical fallacies are increasingly deployed to defend the status quo in health care, and particularly to defend the interests of those who are profiting the most from the current dysfunctional system.  We have noted that several defenses of the conflicts of interest generated by financial relationships between physicians and medical academics on one hand and commercial health care firms on the other, were based on logical fallacies.  (See examples here, here,  here, and here.)  I have yet to see a coherent, logical, fact-based argument that the benefits for patients' and the public's health of physicians and medical academics working part-time as consultants, advisers, speakers, and directors of health care corporations outweigh the obvious risks of biasing medical decision making, education and research in favor of vested interests.

So we add to our ongoing series how, based on a series of logical fallacies, the European Society of Cardiology provided a series of recommendations that allowed nearly any kind of relationship among CME speakers and selection committees and industry, as long as the relationships were disclosed.  The only relationships banned were those "which would represent a significant conflict of interest" for the Chairperson of a Congress Programme Committee.  Similarly, the only stipulations for the society's cardiology journals were that interests affecting editors and editorial board members must be declared, and board members and reviewers should decline reviews of manuscripts "relating to topics, drugs, or devices, in which they have significant commercial of academic interests."  The rules for guideline committees were somewhat more rigorous, but "receipt of consultancy fees or fees for lecturing would not debar an individual from being a member of a committee but must be fully disclosed."

It is discouraging that the web of conflicts of interest that currently enmeshes much of academic medicine and many medical professionals is so heavily defended.  It is more discouraging that its defenders include so many prominent academics and practicing physicians.  It is more discouraging that so many well trained people resort to logical fallacies to make their arguments, and do so in prestigious scholarly journals.

Our continuing series about how logical fallacies are used to support the status quo and the powers that be in health care suggests, if nothing else, that health care professional education ought to include courses in logic.

Finally,  in 2011, I noted, "I have also yet to see an argument in favor of conflicts of interest made by anyone who does not have such conflicts."At least, however, up to that point I had not noted any such arguments made by people who had much power to enforce their views, as opposed to the ability to just express them.  Last month, however, I discussed how the leader of one of the most acclaimed US medical schools made an argument in support of conflicts of interest based on logical fallacies,  Now we have just seen such arguments made by the leaders of European cardiology.  The new paper suggested that disclosure is the best way to manage conflicts of interest.  True to this belief, the paper included a disclosure section that took up an entire journal page.

It seems likely that number and magnitude of ongoing commercial interests so disclosed may have influenced the content of the position paper.  Yet while it may be unsurprising, it is most disappointing that conflicts of interest are now being uncritically and illogically publicly defended by people in positions to exert so much influence on health care.

The noted cognitive psychologists George Loewenstein, Sunita Sah, and Daylian Cain just asserted in JAMA(4):

Conflicts of interest, including fee-for-service arrangements, are at the heart of the astronomical increases in health care costs in the United States, and transparency is not substitute for more substantive reform.

True health care reform requires such substantive reform of the financial arrangements among corporations that sell health care services or products and health care professionals, others who make decisions about patients' or the public's health, and academic health care institutions. To decide how to accomplish such reform, we need a better discussion informed by logic and evidence, sans logical fallacies. Those who lead health care ought to be able to participate in this discussion under these conditions.

References
1. ESC Board. Relations between professional medical associations and the health-care industry, concerning scientific communication and continuing medical education. Eur Heart J 2012; 33: 666-674. Link here.
2. Association of Americian Medical Colleges. Industry Funding of Medical Education. Washington, DC: AAMC, 2008. Link here.
3. Light DW. Global drug discovery: Europe is ahead. Health Aff 2009; 28: w969-w977. Link here.
4. Loewenstein G, Sah S, Cain DM. The unintended consequences of conflict of interest disclosure. JAMA 2012; 307: 669-670. Link here.

The Institute of Medicine Releases Reports on Practice Guidelines and Systematic Reviews Which Generate Few Echoes

Two days ago, the prestigious US Institute of Medicine released two reports on important health care issues, clinical practice guidelines and systematic reviews.  Systematic reviews of the relevant clinical research have been advocated by evidence-based medicine proponents as the appropriate basis for clinical and policy decisions.  Clinical practice guidelines have been advocated by many health researchers, policy makers, and clinicians as the best way to encapsulate the evidence to inform clinical and policy decision making.  Both reports suggested series of standards for how systematic reviews and clinical practice guidelines should be developed. 

These topics are of general importance to clinicians, health services researchers, and health policy makers.  The Institute of Medicine, part of the US National Academy of Sciences, is one of the most authoritative sources of opinion on medicine and health care.  Therefore, one would think that these reports would have gotten wide notice, and would hardly required Health Care Renewal to create some echoes.

However, a Google News search today produced only six "hits" relevant to these reports, including the original press release.  All are in specialized medical/ health care news outlets.  None are in the national media, and none are from major medical/ professional journals or societies. 

Let me suggest a theory about why these two major reports have generated so few echoes so far.  Let me quote from the summary of the report on clinical practice guidelines:

Most guidelines used today suffer from shortcomings in development. Dubious trust in guidelines is the result of many factors, including failure to represent a variety of disciplines in guideline development groups, lack of transparency in how recommendations are derived and rated, and omission of a thorough external review process. To be trustworthy, clinical practice guidelines should:
• Be based on a systematic review of the existing evidence;
• Be developed by a knowledgeable, multidisciplinary panel of experts and representatives from key affected groups;
• Consider important patient subgroups and patient preferences, as appropriate;
• Be based on an explicit and transparent process that minimizes distortions, biases, and conflicts of interest;
• Provide a clear explanation of the logical relationships between alternative care options and health outcomes, and provide ratings of both the quality of evidence and the strength of recommendations; and
• Be reconsidered and revised as appropriate when important new evidence warrants modifications of recommendations.
Additionally, as reflected in the committee’s standards for developing trustworthy clinical practice guidelines, guideline development groups optimally comprise members without conflict of interest. The committee recognizes that in some circumstances, a guideline development group may not be able to perform its work without members who have conflicts of interest—for example, relevant clinical specialists who receive a substantial portion of their incomes from services pertinent to the guideline. Therefore, the committee specifies that members of the guideline development group who have a conflict of interest should not represent more than a minority of the group.

So it seems that the report on clinical practice guidelines emphasized two issues highly relevant to Health Care Renewal, the need for transparency in guideline development, and the need to avoid conflicts of interest affecting the development process. The two first standards for guidelines are about transparency and minimization of conflicts of interest.  Similarly, the report on systematic reviews also included fairly tough standards to minimize conflicts of interest.

We on Health Care Renewal go on and on about the need to maximize transparency in health care, and particularly in health care leadership and governance, and about the need to disassemble the now pervasive web of conflicts of interest that has entangled health care.  However, as we know, these are not popular topics among the leadership of health care, which includes many individuals who have greatly benefited from lack of transparency and pervasive conflicts of interest.  We know these topics make these leaders, and many of those who report to, or work or associate with them very uncomfortable.

So unfortunately, I am not surprised that the two new and likely authoritative reports from the Institute of Medicine, despite that organization's prestige, have started off relatively anechoic.  It also unfortunately likely that they will remain relatively anechoic. 

In 2009, the IOM issued an authoritative report on conflicts of interest in medicine and health care which suggested fairly tough standards to decrease such conflicts and their influence (also see post here).  Since 2009, I just found 53 citations to that report in the medical literature using the ISI Web of Science, for a rate of 27/year.  In 1999, the IOM issued a report on medical errors, "To Err is Human."  Since then, it has received 1374 citations, a rate of 115/year. 

As we noted above, the topic of conflicts of interest seems to make the powers that be in health care very uncomfortable.  In contrast, "To Err is Human" was widely interpreted to mean that physicians make a lot of dangerous errors, and the best way to decrease them is to impose more controls by bureaucrats, managers, and executives (even if that was not its intent).  Thus, that report could be twisted to fit the talking points of the powers that be, and hence has been anything but anechoic.

So while Health Care Renewal is hardly a powerful tool for creating publicity, I thought we should try to get the word out about the new IOM reports on clinical practice guidelines and systematic reviews.  Every little bit helps.

Meanwhile, the deathly quiet reception these reports have gotten so far emphasizes the need to combat the anechoic effect.  As long as the powers that be can command billions of dollars to influence the health care conversation through their marketing, public relations, and lobbying departments, expect the discussion not to question what they do, and how they benefit from the status quo to the financial and health detriment of patients and the population.

We will not be able to truly reform health care until we can speak openly about what threatens health care values and what needs to be done about these threats.

Key Opinion Leader Services Companies: the Creation of Useful Idiots and Usefully Idiotic Organizations

In researching the conflicts of interest of the University of California "36," I stumbled upon a fascinating corner of the pharmaceutical/ biotechnology/ medical device marketing universe, the companies that find and manage key opinion leaders (KOLs), also known as "thought leaders."  Reviewing their own marketing materials reveals how KOLs truly are health care corporate marketing's useful idiots.

I found three companies which seem entirely devoted to the adoption, care and feeding of KOLs, plus numerous companies, including some medical education and communication companies (MECCs) that provide KOL-related products and services.  I will first describe the companies briefly, then draw upon their marketing materials to underline what KOLs are really about.

Leadership in Medicine Inc

This is the company I found first, because one of its directors is a member of the UC 36, the group of top university leaders who threatened to sue the university to increase their already generous pensions.

Leadership in Medicine Inc's web-site describes its reason for being thus:

IF YOU NEED TO KNOW who are the most prominent, admired, and influential actors in healthcare, how they are interconnected, and why, you need our expertise.

Given how vastly complex are the relationships among providers, researchers, and other significant actors in healthcare, it is vital to focus on key opinion leaders (KOLs) at local, regional, and global levels, and to understand the ties among them.

Its clients are:

Over 80 client companies
* All of the top 15 largest pharmaceuticals
* 8 of the 10 largest biotechs
* 5 of the 10 largest medical device companies

A graphic on its "experience" page listed the following companies: Baxter, Wyeth, Lilly, Roche, Gilead, GlaxoSmithKline, Pfizer, Abbott Laboratories, Genzyme, Bristol-Myers-Squibb, Medtronic, Johnson and Johnson, Genentech, and Covidien.

KOL LLC

Company description:

As our name implies, we are a company devoted to providing Key Opinion Leader software and Key Opinion Leader Management services for pharmaceutical, biotechnology and device companies.

The company's graphic client list included: Cephalon, Scios, Novartis, Schering-Plough King Pharmaceuticals, Pfizer, Genentech, Reliant Pharmaceuticals, McNeil (division of Johnson and Johnson, Jazz Pharmaceuticals, Endo Pharmaceuticals, Cytogen, Health Products Research, Shire, Reckitt Benckiser, Protein Design Labs, and Odyssey Pharmaceuticals.

Thought Leader Select

The relevant parts of the company description:

Thought Leader Select is a Chapel Hill, NC-based private research and consulting firm serving the biopharmaceutical and healthcare industries.

and

We serve these industries and the medical community at large by assessing medical experts (known as 'thought leaders' and 'key opinion leaders')....

Perusal of the materials used by these companies, and other companies which market KOL or thought leader related services makes the nature of the relationship between KOLs and commercial health care firms, and the purpose of employing KOLs clear.

KOLs are Employed by Marketing Departments to do Marketing

The best example comes from a description of a KOL information technology application sold by Nagarro:

A web-based application was developed by Nagarro to help the marketing department of a global pharmaceutical company exploit Key Opinion Leader (KOL) information in order to promote products and remain ahead of its competition.

Problem Description

In today s fast-paced competitive environment, pharmaceutical companies cannot solely rely on superior products to succeed. Well organized marketing departments help sales departments reach goals and give companies an edge over competition, but without access to valuable resources, like KOLs, they are ineffective. KOLs influence the medical community and ultimately the end users of pharmaceutical products.

Pharmaceutical companies that are able to identify and work with KOLs will be better positioned to compete....

Benefits

[include]
Creation of market intelligence from highly specialized and customizable reports containing previously unavailable aggregate data....

Ability to group KOLs by product knowledge and associations in order to better promote products

Ability to maximize ROI from KOL related events

Enhanced sales and marketing productivity through streamlining of complex multi-source information

That makes it crystal clear that marketers use KOLs to market, to sell products. While at times KOLs might actually be used to advise health care corporations about clinical or scientific issues, that is hardly their major point. KOLs are almost always hired to market by marketing departments.

In case someone might argue that this is only one example, let us look at materials from the other companies.

KOL LLC described the usefulness of KOLs thus:

Everyone recognizes the value of opinion leaders (OL), or thought leaders. While national level OLs may not write many prescriptions they influence thousands of prescribers and hence prescriptions through their research, lectures, publications and their participation on advisory boards, committees, editorial boards, professional societies and guidelines/consensus document development. Regional level OLs are often involved in state societies or legislative initiatives in addition to their speaking and publications. While local level OLs may not publish, they provide advice to local colleagues and may speak at grand rounds. And who are the ‘rising stars’ in your therapeutic area?

It is imperative that you know the OLs in your market at a national, regional and local influence level as well as those ‘rising stars’.

This is a bit more indirect, but it is clear that the goal is to "influence prescribers" to prescribe, not provide scientific or clinical advice to the company.

Leadership in Medicine Inc's materials also continually emphasized the point of KOLs is to influence, for example, they boasted of pioneering analyses "to assess paths of influence in healthcare," developed a particular tool called "Centrality Ranking" to "provide fine-grained ratings of KOLs' influence," and claimed to "have identified, profiled, and mapped the influence of tens of thousands of individual KOLs...." The clear implication is that KOLs' influence is the central consideration, and what else is this influence good for other than to sell products, and perhaps advocate for corporations in general?

When KOLs are Involved, Many Activities that Appear to be Educational or Scientific Really Are For Marketing

Strikingly, KOL LLC claimed its role in guideline development:

Guidelines produced by national societies are optimal, but often can be a slow, painful and expensive process to develop. KOL, L.L.C. can provide a faster alternative. We have experience convening a panel of experts in a therapeutic area. We serve a project management role to ensure the timelines and deliverables are met. We have access to a medical writing team of 25 healthcare professionals who can write the initial drafts, as our experience tells us it is easier for experts to edit, than to write from scratch.

There has been growing realization that guidelines may be biased by commercial sponsorship and by the participation by individuals with conflicts of interest. The KOL LLC marketing materials suggest, however, that guidelines have become purpose-built marketing vehicles through the participation of selected KOLs with allegiances to drug, device and biotechnology companies. As an aside, note that guideline-development services includes the participation of a team of ghost-writers who will write the first drafts, a function that naive academics might have thought should be that of clinical and scientific experts.

For another example, KOL LLC asserted it could manage "investigator meetings," :

We’ll help you better plan to maximize the communication of the trial results through targeted abstracts, posters, publications and lectures.

so

Due to the time constraints placed on Clinical Research Departments, many times ‘research mills’ are selected as the trial sites. This is fine, but who is going to publish the results and stand up and present the results at national, regional and local meetings. We can provide you advice and counsel about how to involve your KOLs effectively, while maintaining your aggressive timelines.

The goal of KOL management here is for the company to control how the research is disseminated. Note also the cynical view of "research mills," which likely refers to contract research organizations. Do we really think that CROs are used by commercial firms because they do better research?

Key Leading Organizations

A bonus from reading through the offerings on KOL management was to discover another related business that has not been subject of polite conversation before. Leadership in Medicine Inc put it this way:

Equally essential is recognizing the roles played by key leading organizations (KLOs) such as medical institutions, payers, professional organizations, patient groups, government entities, and journals in structuring KOL activities and relationships, since those are the stages on which KOLs perform.

Key Leading Organizations (KLOs) apparently include influential organizations, e.g., academic medical institutions, medical societies, and patient advocacy groups that can be deliberately turned into organizations of useful idiots for marketing purposes. Note that we and others have discussed how institutional conflicts of interest and conflicts of interest affecting leaders of of such organizations can lead to bias in favor of commercial interests. But what Leadership in Medicine Inc has written suggests that such organizations can be deliberately taken over to function as industry's fellow travelers.

Similarly, Thought Leader Select advertised services to manage organizations to support "thought leadership":

Through Centers, all of our research and assessment skills culminate in our evaluation of universities, influential clinics, and research foundations for a holistic approach to thought leadership in the medical community. With centers of excellence assessments we take a drill-down approach, starting at the academic medical centers, then moving into affiliated hospitals and clinics....

Summary

Industry spokespeople and key opinion leaders tout themselves as clinical, educational, and/or scientific experts chosen for their expertise to advance medicine, science and public health.  There are documented instances (e.g., see posts here and here) in which defectors from marketing departments of commercial health care corporations described KOLs as salespeople who could be more influential hidden within their professional or academic cloaks.  Even some physicians paid to be speakers on behalf of pharmaceutical corporations have acknowledged their role as salespeople in fancy dress (see post here).  There are cases of documents revealed by discovery in legal actions that show how companies planned organized stealth marketing efforts for drugs that included activities by KOLs (e.g., see post here about marketing of Lexapro, and here about Neurontin).

However, the marketing materials used by KOL service companies (for lack of a better name) show that KOLs are largely meant to be stealth marketers, and hired for that purpose, that KOLs participate as marketers in the sorts of activities that to the naive appear to be educational or scientific, and that marketers try to recruit whole organizations, such as medical schools, research organizations, medical societies, and patient advocacy groups as disguised sales organizations.

This goes beyond the problem of bias of physicians, or individual health professionals due to their financial relationships.  It goes beyond the problem of bias of organizations due to their sources of financial support or the financial relationships of their leaders.  It looks like there has been a massive campaign by health care corporate marketers to make useful idiots out of possibly a majority of medical academics and academic, professional, and supposedly patient-centered organizations.  This appears to be a massive, cynical effort to hollow out our once respected health care institutions and professionals in the service of marketing.

A final word to any individuals reading this who are paid by corporate marketers to be KOLs.  If you think that you are paid for educational or scientific purposes, you likely have been made into a chump.  The people who did this to you were likely not acting in your best interests, or those of society, but to cynically market their product and increase their own earnings.  If you doubt this, look at the materials cited above.  You really don't want to continue being chumps, do you?

High Costs and Poor Outcomes - A Lesson About the Suppression of Research

Last week, blogger Alison Bass was one of the few to comment on a new article on the risks of antidepressant medications published electronically last week by the British Medical Journal.[1] The article adds to the clinical evidence about these specific drugs. However, it also should remind us about why our health care costs seem to continue to inflate disproportionate to any improvements in health care outcomes.

Background - Why Don't Doctors Diagnose and Treat Depression According to the Evidence?

Here is the background. Since the early 1990s, based on numerous published clinical trials that showed that the newer (mainly SSRI, that is, selective serotonin reuptake inhibitor) antidepressants seemed to be quite safe and effective, authoritative guidelines have urged physicians to screen patients for depression, and have a low threshold for beginning treatment, particularly with these then new drugs. (For example, see the 1993 AHCPR guidelines,[2] and the Ambulatory Care Quality Alliance Recommended Starter Set[3].) Such guidelines targeted primary care physicians as well as psychiatrists. As a primary care physician, I often felt guilty that I didn't screen all patients, including those who did not outwardly appear depressed, for depression, and was reluctant to be the primary prescriber of antidepressants for depressed patients. I was not alone. There was evidence that many primary care physicians failed to follow these apparently evidence-based guidelines.

Suspicions that SSRIs Were Not as Effective or Safe as Thought

However, in 2004, internal documents from one drug company (Glaxo Smith Kline), published by the Canadian Medical Association Journal, suggested that the company had suppressed a study which failed to show its SSRI was effective for adolescent patients.[4] Also, then New York state attorney general Elliot Spitzer sued the company alleging it had suppressed such data.[5] Considerable anecdotal information thereafter came to light suggesting that drug companies may have suppressed data from clinical research (that is, studies done on patients, not in the laboratory) which tended to show the drugs were less effective, or more dangerous than previously thought. Many began to suspect that the drugs increased the risk of suicidal thoughts or behavior, even though one would think that if they were effective in treating depression, they should decrease such risks.

How Does Uncovering Suppressed Trials Change the Evidence About the Benefits and Harms of SSRIs?

Then, an analysis of trials of SSRIs given to adolescent patients, which included unpublished as well as published trials, did show they increased the risk of suicide.[6] But what about adult patients?

In 2008, Erick Turner and colleagues published a meta-analysis that included numerous trials of SSRIs and other antidepressants given to adult patients that had never been published, but whose results had been submitted to the US Food and Drug Administration (FDA).[7] When those trials were included in the analysis, the drugs seemed less effective than was previously believed.

The new meta-analyis by Stone et al[1] suggested how adding evidence from the suppressed trials improves our knowledge about the possible harms of these drugs. Stone et al specifically addressed whether the drugs increased the risk of suicidal thoughts or actions for adult patients. It was based on data from 372 trials that included 99,231 patients. "Most of the studies were unpublished; those that had been published in some form seldom contained information concerning suicidality in the publication." Its main result was that SSRI drugs approximately doubled the risk of suicidal behavior for patients less than 25 years old, but decreased the risk for older patients.

Suppressing Trials Distorted the Evidence-Base

Thus, the combined results of several meta-analyses suggests that SSRIs are neither as safe nor as effective for adult patients as they were previously touted to be. The drugs used to appear to be more safe and effective because the drug companies that sponsored (paid for) clinical trials of their own products suppressed the publication of studies whose results did not show their products in a favorable light. The guidelines once thought to be so authoritative are probably much to enthusiastic about the use of drugs to treat depression. It may be that those physicians who resisted pressure from these guidelines and other efforts by would-be quality improvers to diagnose more depressed patients and treat them with SSRIs weren't so dumb.

Health Policy Implications

The recently published meta-analysis of the risk of suicidal ideation or behavior due to SSRIs for adult patients[1] adds not just to the clinical evidence about the treatment of depression, but to the evidence that suppression of research by those with vested interests in its results may not be rare. There is no reason to think that suppression of research about other medical interventions used for other clinical problems does not occur just as often.

This underscores why we as a society should not have handed over control of how new drugs and devices are evaluated to the companies that stand to profit by selling them. The apparent benefits and safety of most drugs and devices brought to the market in recent years may have been exaggerated by the ability of the corporations who sponsored the research to suppress those studies that were least favorable to their products. Thus, it is very likely that doctors have over prescribed newer (and more expensive) drugs and devices based on erroneous beliefs about their safety and effectiveness, beliefs supported by an evidence base which was distorted by suppression of studies whose results did not show the results their commercial sponsors wanted to see. Furthermore, we have likely collectively paid too much for these drugs and devices based on these same erroneous beliefs. Thus, the distortion of the clinical evidence base caused by putting the makers of medical products in charge of their evaluation has likely led to increased costs and worse outcomes for patients.

Health care reformers serious about wanting to improve patient outcomes and reduce costs ought to be at the forefront of a movement to make clinical research more honest. All clinical trials completed and submitted to government regulators ought to be published in some form, so that the data they contain is available to guide clinical decisions. Going forward, all studies should be published in some form in a timely manner to prevent future research suppression. There ought to be better regulation of research sponsored, and de facto run by companies who have vested interests in the results being favorable to their products. It would not be unreasonable to prevent companies with such vested interests from running or otherwise directly influencing clinical research meant to evaluate their own products.

But here in the States, we are probably too diverted by yelling at each other about whether a "public option" is vital or nefarious to worry about such issues.. Meanwhile, expect our health care to continue to get more expensive, less accessible, and less good for patients.

ADDENDUM (25 August, 2009) - Also, see the comments by Dr Howard Brody on the Hooked: Ethics, Medicine and Pharma Blog

References

1. Stone M, Laughren T, Jones ML et al. Risk of suicidality in clinical trials of antidepressants in adults: analysis of proprietary data submitted to US Food and Drug Administration. Brit Med J 2009; 339: b2880. Link here.
2. Depression Guideline Panel. Depression in Primary Care: Volume 2 - Treatment of Major Depression. Rockville, MD: US Department of Health and Human Services, Public Health Service, Agency for Health Care Policy and Research, 1993. Link here.
3. Ambulatory Care Quality Alliance. Recommended Starter Set: Clinical Performance Measures for Ambulatory Care. http://www.ahrq.gov/qual/aqastart.htm
4. Kondro W. Drug company experts advised staff to withhold data about SSRI use in children. Can Med Assoc J 2004; 170: 783. Link here.
5. Wadman M. Sptizer sues drug giant for deceiving doctors. Nature 2004; 429: 589. Link here.
6. Hammad T, Laughren T, Racoosin JA. Suicidality in pediatric patients treated with antidepressant drugs. Arch Gen Psychiatry 2006;63:332-9. Link here.
7. Turner EH, Matthews AM, Linardatos E et al. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 2008; 358: 252-260. Link here.