Comment
Millar’s editorial comments¹ in this journal “…the primary source of medical pharmacological knowledge stems from application of controlled clinical trials” and “…alternative sources of information such as uncontrolled trials or personal experience deserve little standing” unfortunately adhere to the widely accepted but faulty dictum “If the study was not randomized, we would suggest that you stop reading it and go on to the next article”.²  In contrast to the sole reliance on randomised controlled trials (RCTs) promulgated by Millar, we previously published³ that

1.  By the mid-1990’s, the social sciences had already concluded that RCTs were overvalued.  Walach et al. promoted the use of “a multiplicity of methods, [with] different designs, counterbalancing their individual strengths and weaknesses to arrive at pragmatic but equally rigorous evidence which would provide significant assistance in clinical and health systems innovation. Such evidence would better inform national health care technology assessment agencies and promote evidence-based health reform”.⁴  They further declared that “Rather than postulating a single ‘best method’ this view acknowledges that there are optimal methods for answering specific questions, and that a composite of all methods constitutes best scientific evidence…The important point is not whether a study is randomized, but whether it uses a method well suited to answer a question and implements this method with optimal scientific rigor…Methods that are high in internal validity, such as placebo controlled RCTs…tend to be lower in external validity.  Thus, their results need to be balanced by large and long-term observational studies which document the use, safety, and effectiveness of the intervention in clinical practice”.⁴

2.  Some 20 years after the social sciences had downplayed the importance of RCTs, the medical researcher Thomas Frieden reiterated many of those same concerns and added: “The increasingly high costs and time constraints of RCTs can also lead to reliance on surrogate markers that may not correlate well with the outcome of interest… These limitations and the fact that RCTs often take years to plan, implement, and analyze reduce the ability of RCTs to keep pace with clinical innovations; new products and standards of care are often developed before earlier [ones] complete evaluation. These limitations also affect the use of RCTs for urgent health issues, such as infectious disease outbreaks, for which public health decisions must be made quickly on the basis of limited and often imperfect available data”.⁵  Early on during the COVID-19pandemic, large population studies, including some involving tens of millions of subjects,  demonstrated the efficacy of repurposing ivermectin for both prophylaxis and treatment,⁶ and they were largely and tragically ignored.

3.  In 2016, the US government stated that, for the purpose of the 21^st Century Cures Act, real world evidence (RWE) means “…data regarding the usage, or the potential benefits or risks, of a drug derived from sources other than randomized clinical trials,” and furthermore mandated that RWE should be used “to help support the approval of a new indication for a drug [previously] approved,” that is, for repurposing drugs.⁷

4.  In 2021, Angus Deaton, the recipient of the 2015 Nobel Prize in Economics, and his colleague Nancy Cartwright analyzed RCTs and determined that they had serious limitations including failure to balance confounders, and finding little practical value of unbiasedness compared to precision.⁸  They concluded that “…RCT results can serve science but are weak grounds for inferring ‘what works’ clinically.”

5. Earlier in 2024, we noted that “poorly designed or poorly executed RCTs [are routinely] accepted as high quality, not because of their actual scientific merit, but because of [their unjustified acceptance as the clinical Gold Standard]…A priori, quality studies of the same disease should yield similar results independent of study design.³  And in fact they do, as Anglemyer et al (2014) stated: “On average, there is little evidence for significant effect estimate differences between observational studies and RCTs, regardless of specific observational study design, heterogeneity, or inclusion of studies of pharmacological intervention”.⁹

Doidge’s 2020 review describes studies from the 1990’s reporting that observational studies obtained results similar to those of RCTs.¹⁰  Doidge stated “ignoring real-world evidence (RWE) and other reports … undermines the original intent of evidence-based medicine (EBM) to use signals from all types of study designs”.¹⁰

Added to all this, there are conditions where it is unethical to perform RCTs and other methods must be used to test efficacy.  The classic example is a trial for parachute efficacy where the placebo of human subjects jumping out of planes with nonworking parachutes would be unethical.  A satirical RCT of such parachute efficacy has been published by Robert Yeh et al.¹¹ A common real example is the premature termination of an RCT trial when treated patients continue to live while untreated ones continue to die.

To address the above limitations, we introduced the T-EBM Qualitative Wheel (Fig. 1),³ which includes all peer-reviewed study types. The wheel classifies studies into four types (each with its own subtypes), namely: 1) Structured analyses (systematic reviews, meta-analyses, and clinical guidelines); 2) Clinical studies (RCTs, cohort studies, case-control studies, case series, case reports, population studies); 3) Mechanisms of action (in-silico: computer-based studies, in-vitro: lab-based studies and in-vivo: animal studies); and 4) Non-systematic reviews (narrative reviews, editorials, and expert opinions).  The wheel aligns with the core intent of EBM, which is to use the best available evidence to guide clinical decisions, rather than adhering to a singular reliance on RCTs. In the case of treating COVID-19, such reliance has resulted in ignoring a wide variety of evidence that could have attenuated the pandemic early on and thereby saved countless lives.

Provenance:  Comment on previous Tasman Medical Journal publication, not externally reviewed
Acknowledgments:  None
Funding: Not required
Ethical approval:  Not required

Corresponding author:  Dr Jerome Dancis, Associate Professor Emeritus, Mathematics University of Maryland, College Park, Md 20742-4015, USA.  Email address: jnd@umd.edu.

Editor’s note:  This paper is the third of three manuscripts received in response to our editorial review (Reference 1).  As the topic of hydroxychloroquine in SARS-CoV-2 infection and repurposed drugs in general is of some importance the Journal has made its columns available to the authors of these papers without comment.  Further comment is welcome.