Ray Moynihan: Let’s stop the burning and the bleeding at Cochrane—there’s too much at stake

Comment SIN-NL
A most important acknowledgment of the fact that medical harms (iatrogenic harm) is underreported and under-investigated in medical studies.
Quote…..”This is particularly true when it comes to harms such as autonomic dysfunction syndromes and other syndromes that are not reported” …..
It is a very sad and wrong development that Peter Gøtzsche, a highly critical scientist and supporter of evidence-based medicine, has been expelled from Cochrane.
As known Sophie Hankes LLM Chair of foundations SIN-NL and IEU-Alliance suffers from autonomic dysfunction caused by experimental neurosurgery on her brainstem, without informed consent, serious medical harm!
Please read statement by Peter Gøtzsche sept. 14 2018: Moral crisis in Cochrane 
Please read website www.deadlymedicines.dk


To see the future of Cochrane threatened foreshadows a disaster for all of us

Sometimes it feels necessary to state the bleeding obvious—particularly when there’s blood on the floor. The tens of thousands of people around the world who create Cochrane and its summaries of evidence are contributing to one of humankind’s greatest scientific endeavors.

For 25 years researchers, clinicians, consumers, policy-makers and others have been using the rigorous new tools of an evidence-informed approach, to collectively produce systematic reviews about what works and what doesn’t in healthcare. Notwithstanding limitations and uncertainties, the Cochrane Review has rightly become something of a gold standard in evidence, and whenever a friend or family member has a question about their care—including questions of life or death—it’s the first place I send them.

What’s at stake in the current bloody fight unfolding within Cochrane’s Governing Board, is not just the credibility of individuals or organisations, it’s the future of reliable trustworthy evidence in a world of increasing falsity and fake news. [1,2] To see this future threatened foreshadows a disaster for all of us.

Unlike almost everything I’ve ever written, this piece is a desperate personal plea to the many clever and cool heads within the extended Cochrane family to try and calm this crisis, heal the rifts, and turn this challenge into an opportunity to enhance public trust, not squander it. No one asked me to write this opinion piece and writing it may mean I lose my role hosting The Recommended Dose podcast—funded by Cochrane Australia and co-published with The BMJ—but I’ve spent too long watching this tree grow to stand by and watch it burn to the ground.

When I first started reporting on healthcare almost 25 years ago, reading Archie Cochrane’s simple book Effectiveness and Efficiency was life-changing. [3] The searing scrutiny of evidence-based medicine was starting to reveal that the benefits of many medical interventions were being routinely exaggerated, and their harms played down. Twenty years ago the television documentary that accompanied my first book, took me to Welsh coal mining districts where Archie cut his epidemiological teeth, to the tiny first Cochrane centre in Oxford, and to McMaster in Canada, where this radical new approach was incubating.

This piece is not about who is right in the current fight, but about reinforcing the rightness of the giant global collaborative project that is Cochrane. My personal views on the dispute over the HPV vaccine review are unimportant and beyond the scope of this opinion piece—suffice to say what appears to be some overstatement of criticism has precipitated what looks like an overreaction. [4] I would respectfully suggest that if there was a pill—or a course of cognitive behavior therapy—that meaningfully reduced the symptoms of hubris and promoted humility, I would recommend key players take a strong dose. Starting yesterday.

More seriously, I would wish every success to those at the Cochrane Colloquium in Edinburgh this week who must be striving through late-night and early-morning meetings to try and heal the wounds as quickly as possible. Potential short-term solutions to the crisis will be complicated and of uncertain benefit—familiar challenges for those dealing with healthcare evidence.

What’s more certain is that the current challenge can serve as an opportunity. The much bigger crisis here is the threat to the reliability of healthcare evidence and public trust posed by the unhealthy financial entanglement between industry and those who evaluate and use its products. [5]

Fifteen years ago, the Cochrane Collaboration was at a crossroads in its relationship with pharmaceutical companies. [6] It opted then to tighten its policy and firmly reject the idea of companies sponsoring Cochrane reviews. Yet Cochrane policy, renewed again in 2014, still allows individuals with financial ties to pharmaceutical companies to review evidence about those same companies’ products—if they constitute a minority of the review team.

Given what we know about the systemic bias introduced into industry-sponsored studies, the egregious nature of much of industry’s marketing behavior, including its work with key opinion leaders, it’s an anathema that conflicted individuals should be reviewing what is often conflicted evidence to start with. Cochrane has an opportunity to provide global leadership by cleaning up this mess—as The BMJ is attempting to do with its new policy of seeking out non-conflicted researchers to author influential educational material. [7]

Related to this reform should be explicit new ways to address and investigate the under-investigation and under-reporting of harms in medical studies—already flagged by Cochrane leadership in recent correspondence* about the current controversy. [8] And thirdly, as part of a drive to enhance public trust and get closer to the truth about which particular groups of people might benefit from medical interventions, Cochrane reviews might throw a lot more explicit scrutiny on the sometimes controversial definitions of disease on which they rely—highlighting the inappropriately lowered diagnostic thresholds which can drive overdiagnosis and overtreatment.

These words are offered respectfully, not as partisan criticisms to support one side or the other in the current dispute—but rather with a hope for calm, cool, conflict-resolution—from a long-term observer with a strong interest in seeing this global collaboration survive and thrive. Like many of us, I want to see this tree continue to grow and bear fruit for a long-time to come.

Ray Moynihan, senior research fellow, Centre for Research in Evidence Based Practice, Bond University, Australia.

Conflict of interest statement: Ray Moynihan has a contract to present The Recommended Dose podcast, funded by Cochrane Australia. He is also a long-time contributor to BMJ.

References:

1. Marcus A, Oransky I. Turmoil erupts over expulsion of member from leading evidence-based medicine group. Stat News. September 16, 2018.
2. Koster M, Burton M. Message from the Governing Board. Cochrane (website) September 15, 2018
3. Cochrane A. Effectiveness and Efficiency: random reflections on health services. Cambridge: Cambridge University Press; 1972.
4. Hawkes N. Cochrane HPV vaccine review: BMJ journal defends “inconvenient criticisms” BMJ 2018;362:k3927.
5. Lo, B, Field M.(eds) Institute of Medicine (US) Committee on Conflict of Interest in Medical Research, Education, and Practice. Washington DC: National Academies Press; 2009
6. Moynihan R. Cochrane at crossroads over drug company sponsorship. BMJ 2003;327:924
7. Mabel Chew M, Brizzell C, Abbasi K, Godlee, F. Medical journals and industry ties. BMJ 2014;349:g7197.
8. Tovey D, Soares-Weiser K. Cochrane’s Editor in Chief responds to a BMJ Evidence-Based Medicine article criticizing the Cochrane Review of HPV vaccines. Cochrane (website) September 3, 2018.


see www.cochrane.org 
p. 7

Cochrane’s Editor in Chief responds to the BMJ Evidence-Based Medicine article criticizing the
Cochrane Review of HPV vaccines
7
The review included published and unpublished data on serious harms, and reported the findings on mortality transparently and responsibly.
In making their assessment of serious harms, the Cochrane Review authors identified and included
unpublished data, and compared these with data from published trial reports. Jørgensen et al claim
that the review authors made an error in their reporting of serious adverse events in relation to the
PATRICIA study. This is not the case. We have checked the data presented in the Cochrane
Review against the reports on ClinicalTrials.gov and the GSK Study Register, and the figures
accurately match the number of women experiencing one or more serious adverse events.
In addition, asJørgensen et al note,the review authors identified and reported the excess of
deaths in the older vaccinated women, in both relative and absolute terms, within the Abstract of
the review as well as in the main body of the text. We judged it important to present the data
transparently, but also to provide further context to ensure responsible reporting.
The assessment by World Health Organization experts and the
data onthe causes of death provide no clear causal mechanism or link with the vaccine. We judged that readers would find this information useful and that its inclusion was appropriate.
Otherwise the reporting of other harms was, as described in the protocol, limited to the published
peer-reviewed reports from randomized controlled trials. This is not unusual for systematic reviews
from Cochrane or elsewhere.
In relation to harms more generally, we acknowledge that there is a case for including other forms
of evidence. The ‘Discussion’section of theCochrane Review and the accompanying Editorial both
noted the importance of national surveillance programmes to identify and report harms. 2,8
This is particularly true when it comes to harms such as autonomic dysfunction syndromes and other syndromes that are not reported (positively or negatively) in most of the journal-published reports, but about which concerns have been raised subsequently from observational reports.
This underlines the importance of systematic reviews being used in conjunction with the evidence from national surveillance programmes.
Finally, we believe that this Cochrane Review has raised broader questions for Cochrane in relation to reporting harms. We propose to initiate work aimed at providing updated guidance for
author teams on identifying and reporting harms in the current and future data and research
environment, as part of our ongoing implementation of Cochrane’s content strategy.
—————-

Trish Greenhalgh: The Cochrane Collaboration—what crisis?

Last week, the Governing Board of the Cochrane Collaboration voted to expel one of its members, Peter Gøtzsche, for activities which allegedly threatened to bring the organisation into disrepute. Four of the 13 members resigned in solidarity with Professor Gøtzsche, and two appointed trustees (who did not support Gøtzsche) volunteered to step down for administrative reasons (essentially, to allow the reduced Board to continue to function). [1] In a single day, the Collaboration lost over half its Board, four of whom (including Gøtzsche) are leads of national Cochrane Centres. Is there a crisis—and if so, of what nature?

The dispute is still unfolding and involves legal material that is not in the public domain. With that important caveat, the “crisis” is both philosophical (relating to the nature of facts) and political (relating to organisational governance).

Gøtzsche’s three-page statement did not name a specific incident. [2] Rather, it appears that his overall approach to overseeing and commenting on Cochrane reviews was deemed (by some but not all members of the Board) unacceptable.

Gøtzsche might be classified as an evidence-based medicine purist. Described by one publication as “a fearlessly outspoken defender of integrity in medicine,” he is a co-author on the CONSORT, PRISMA, STROBE and SPIRIT statements on how to undertake and publish research. [3] He has argued—controversially—that content experts may not be required on systematic review teams since assessing methodological quality is fundamentally a scientific task. [4] He campaigns passionately against bias in research and against policies that rely on what he views as biased evidence. [5,6]

Why would Gøtzsche be asked to leave the Cochrane Collaboration, of which he was a founder member and whose mission and values appear so closely to reflect his own?

The philosophical explanation is that facts are not self-interpreting; they are theory- and value-laden. [7,8] Even when there are agreed criteria for including or excluding a study or for assigning a particular score to an aspect of the methods, multiple subjective judgments need to be made. [9] What if one trial used a slightly different version of a vaccine, or a new (better) test for a primary end-point, than the one named in the protocol? What counts as a placebo? Because of the need for judgement on such questions, two systematic review teams can produce different findings even when both teams are expert and use identical checklists and statistical methods. [10] This important point is often overlooked by those who view systematic review as an entirely technical process (and for whom there are only “good” or “bad” reviewers). [11] One person’s intellectual rigour is another’s intellectual rigidity. Meticulous application of the Cochrane Handbook is thus likely to generate new kinds of disagreements rather than a single, uncontested truth. [8] Some methodological hard-liners view such disagreements as errors to be corrected rather than as a philosophical inevitability. And this, I think, is the stone in Cochrane’s shoe.

Gøtzsche and colleagues recently published a detailed critique of a newly-published Cochrane review, claiming that the authors had failed to identify numerous studies and misapplied the risk-of-bias tools, resulting in a review that was itself biased. [12,13] This prompted a news report in The BMJ which suggested that the famous Cochrane kite mark had been tarnished, as well as an editorial defending Gøtzsche’s team in BMJ Evidence-Based Medicine, though at least one leading scholar considered their analysis flawed and Cochrane’s Editor-in-Chief ruled that what had been described as “omissions” were actually the result of defensible judgements that took account of clinical, scientific and policy realities.[9,14,15,16]

The political explanation for Cochrane’s crisis relates to the tension between governing an organisation and respecting individual members’ academic freedom to express dissent. The four Board members who resigned have claimed the moral high ground, but if they do not accept the statutory decision-making processes of the organisation they agreed to govern, why did they cast a vote at all? [17]

In the 26 years since its inception, the Cochrane Collaboration has grown from a tiny academic network run on collegiality and small donations to a sprawling trans-national bureaucracy with numerous sub-committees, a thick tome of standard operating procedures and a multi-million dollar annual turnover along with an address book of philanthropists on whose contributions its work depends. 21st-century science is an intersectoral endeavour that necessarily occurs in dialogue with society. [18] Maintaining—and funding—the “view from nowhere” requires delicate navigation of tricky political spaces and sometimes accepting hard-won compromises. [19] Board members are presumably expected not to spit in the soup (especially when using Cochrane letterhead).

At this stage in a fast-unfolding story, I am not convinced that the Cochrane Collaboration is experiencing a crisis of either morality or democracy. Its brand, now as ever, stands for rigour, independence, and a commitment to using science to achieve high-quality patient care and social justice. We should cut it some slack while it gets its house in order.

Trish Greenhalgh is Professor of primary care health sciences at the University of Oxford.

Twitter: @trishgreenhalgh

Competing interests: TG works in the same department as Carl Heneghan who is editor of the journal BMJ Evidence Based Medicine. In writing this article, she drew on comments and links posted on social media and circulated via an email list of academics who are members of, or connected to, the Cochrane Collaboration. She has attended the Cochrane Colloquium as a keynote speaker in the past, claiming economy class air fare or second-class train fare, but not receiving a fee. Her work is part-funded by the National Institute for Health Research Oxford Biomedical Research Centre, grant number BRC-1215-20008 to the Oxford University Hospitals NHS Foundation Trust and the University of Oxford.

References:

  1. Koster M, Burton M. Message from the Governing Board, 15th September 2018. Accessed 15.9.18 on https://www.cochrane.org/news/message-governing-board. Edinburgh: Cochrane Collaboration. 2018.
  2. Gøtzsche P. A moral governance crisis: the growing lack of democratic collaboration and scientific pluralism in Cochrane. Accessed 15.9.18 on https://nordic.cochrane.org/sites/nordic.cochrane.org/files/public/uploads/moral_crisis_in_cochrane.pdf. . Copenhagen: Nordic Cochrane Centre 2018.
  3. Anonymous. Peter Gøtzsche receives 2016 HealthWatch Award. HealthWatch Newsletter Accessed 15918 on https://wwwhealthwatch-ukorg/images/Newsletters/Number_103_Highlightspdf 2016;Autumn:2.
  4. Gøtzsche PC, Ioannidis JP. Content area experts as authors: helpful or harmful for systematic reviews and meta-analyses? BMJ: British Medical Journal (Online) 2012;345
  5. Gøtzsche P. Mammography Screening: Truth, Lies and Controversy. London: CRC Press 2012.
  6. Gøtzsche P. Deadly Medicines and Organised Crime: How Big Pharma Has Corrupted Healthcare. London: CRC Press 2013.
  7. Kelly MP, Heath I, Howick J, et al. The importance of values in evidence-based medicine. BMC medical ethics 2015;16(1):69.
  8. Wieringa S, Engebretsen E, Heggen K, et al. Rethinking bias and truth in evidence‐based health care. Journal of evaluation in clinical practice 2018
  9. Bastian H. The HPV vaccine: a critique of a critique of a meta-analysis. . Absolutely Maybe at PLOS Blogs: https://blogs.plos.org/absolutely-maybe/2018/08/25/the-hpv-vaccine-a-critique-of-a-critique-of-a-meta-analysis/. Accessed 15.9.182018.
  10. Low J, Ross JS, Ritchie JD, et al. Comparison of two independent systematic reviews of trials of recombinant human bone morphogenetic protein-2 (rhBMP-2): the Yale Open Data Access Medtronic Project. Systematic reviews 2017;6(1):28.
  11. Uttley L, Montgomery P. The influence of the team in conducting a systematic review. Systematic reviews 2017;6(1):149.
  12. Arbyn M, Xu L, Simoens C, et al. Prophylactic vaccination against human papillomaviruses to prevent cervical cancer and its precursors. Cochrane Database of Systematic Reviews 2018(5)
  13. Jørgensen L, Gøtzsche PC, Jefferson T. The Cochrane HPV vaccine review was incomplete and ignored important evidence of bias. BMJ evidence-based medicine 2018:bmjebm-2018-111012.
  14. Hawkes N. HPV vaccine safety: Cochrane launches urgent investigation into review after criticisms: British Medical Journal Publishing Group, 2018.
  15. Heneghan C, Onakpoya I. Editors’ response to concerns over the publication of the Cochrane HPV vaccine review was incomplete and ignored important evidence of bias. BMJ Evidence-Based Medicine 2018 doi: 10.1136/bmjebm-2018-111108
  16. Tovey D, Spares-Weiser K. Cochrane’s Editor in Chief responds to BMJ EBM article criticizing HPV review. Published 3.9.18. Accessed 16.9.18 on https://www.cochrane.org/news/cochranes-editor-chief-responds-bmj-ebm-article-criticizing-hpv-review. Oxford: Cochrane Collaboration 2018.
  17. Gartlehner G, Hammerstein D, Meerpohl J, et al. Why we resigned. Letter submitted to members of the Cochrane Collaboration, 15.9.18. 2018.
  18. Nowotny H, Scott P, Gibbons M. Mode 2 revisited: The new production of knowledge. Minerva 2003;41(3):179-94.
  19. Shaw SE, Russell J, Parsons W, et al. The view from nowhere? How think tanks work to shape health policy. Critical Policy Studies 2015;9(1):58-77.