Tasman Medical Journal

ISSN: 2652-1881

The “Fragility” of Stroke Thrombolysis

Daniel M Fatovich

View / Download PDF

ABSTRACT

The “fragility index” for a clinical trial reports how robust the results of the trial are. It calculates the minimum number of patients who would need to have a different primary outcome to change the measured p value from < 0.05 to > 0.05. A low fragility index means that only a small number of patients need to have a different outcome for the trial to lose traditional statistical significance. This increases the probability that the trial result was due to the play of chance. The application of this principle is described here for placebo-controlled trials of thrombolysis with alteplase (tPA) in acute stroke.
Tasman Medical Journal 2020; 2(1): 6-10

Full Text

Introduction

One of the negative consequences of the premature adoption of novel therapies is that subsequent research assumes its value and enhances it with faster and wider implementation. Stroke thrombolysis has promoted new systems of care for stroke, including the development of stroke centres, Telestroke services and stroke ambulances with CT scanners. The mantra behind this is the phrase “time is brain”, which was borrowed from “time is muscle” for acute myocardial infarction.1 The basis for the phrase which propelled this juggernaut was the NINDS trial, published in 1995. In addition, a 2006 paper presenting a mathematical calculation based on rodent brain tissue promoted another mantra: 2 million neurons are lost per minute. This was fuel for a campaign using marketing-based medicine.1

Many doctors are simply unaware that of the 12 stroke thrombolysis trials that led to this situation, only two are reported as positive.  Of the 10 that failed to find benefit for stroke thrombolysis, 4 had to be stopped early for harm (increased mortality). That leaves only 2 supportive studies (NINDS and ECASS-III) for our current situation.

NINDS

The National Institute of Neurologic Disorders and Stroke (NINDS)2 study was the trial that started this controversy. This is the only placebo-controlled trial reporting a benefit for acute ischaemic stroke, if the trial drug (tPA) is given within 3 hrs of onset of symptoms.  The trial consisted of two parts. Part 1 (n = 291) found no difference in the National Institutes of Health Stroke Severity score criterion (improvement by 4 points), or resolution of deficits at 24 hrs. Part 2 (n = 333) was an extension to Part 1, added for unexplained reasons.  This “part” had an originally planned primary outcome of 20% improvement in functional and stroke scores, but this was changed post-hoc to a dichotomous favourable (modified Rankin Scale 0-1) vs unfavourable (modified Rankin Scale 2-6) outcome at 3 months. The original planned primary outcome was not reported, and it is reasonable to question this decision.  The results were that 39% of tPA patients achieved the new endpoint vs 26% on placebo, an absolute difference of 13% (relative difference 33%). However, this was counterbalanced by symptomatic intracranial haemorrhage (0.6% placebo, 6.4% tPA; relative difference 1067%), of which 45% in the tPA group died.

In NINDS, there was an imbalance in baseline stroke severity, as the placebo group had more severe strokes (27.5% vs 18.3% tPA) and more mild strokes in the tPA group (19% vs 4.2% placebo).3  It is well known that stroke severity is a major determinant of outcome, and that this imbalance could explain the result.4  The study methodology also required selective trial enrolment, with equal numbers in the 0-90 min time frame, and the 91-180 min time frame. In a re-analysis using the original data and controlling for stroke severity, the authors found no benefit with tPA and no support for the time is brain hypothesis.5  This re-analysis result is consistent with the totality of the stroke thrombolysis trial results. Un-fortunately, there has been no replication study that validates the original result. Yet replication is a fundamental part of the scientific process.

In summary, the NINDS study (part 1) was negative. The extension study (part 2) was subject to unexplained protocol changes, and reported major drug-related adverse effects, including a 10-fold increase in intracranial (including fatal) haemorrhage that offset the modest advantage in outcome in survivors.

ECASS

In 2008, ECASS-III was published, reporting a 7% absolute improvement of tPA vs placebo, in the 3-4.5 hr time frame.6  Again, there was an imbalance in baseline stroke severity, with Shy reported on an updated online result that meant there was a significant difference between the groups, with placebo again having more severe strokes (placebo 14.1% vs tPA 7.7%).7  This results in no benefit for tPA in the adjusted analysis. The original version of the paper reported that there was no baseline difference between the groups. Similarly, if the primary endpoint in ECASS-III is recategorized from a modified Rankin Scale (mRS) 0-1 vs 2-6 to a more realistic mRS 0-2 vs 3-6, the supposed benefits of tPA disappears.8

The inter-rater reliability of outcome assessments using the mRS are moderate at best.9,10  A clinical trial has internal validity if and only if the imbalance between groups and bias in the assessment of outcome and chance, have been excluded as possible explanations for the difference in outcomes.11 Newer concepts such as the fragility index (FI) reinforce this.

FRAGILITY INDEX APPLIED TO NINDS AND ECASS

The FI is a method to understand how statistically reproducible a study is. It is the minimum number of patients who would need to have a different outcome to change the p value from < 0.05 to > 0.05, ie from statistically significant to insignificant.12 The 0.05 threshold as a measure of statistical significance is problematic, but this is not discussed in the present analysis.13  A low FI means that only a small number of patients would have to have their outcome change for the trial to lose statistical significance. It is simply calculated by repeatedly applying Fisher’s exact test, while successively reallocating patients, one at a time, from the favourable outcome group to the other (control) group. Thus the size of the trial remains unchanged. This can be done with online FI calculators, which add events to the group with a smaller number of events (either placebo or active).  The FI can be applied only to trials in which there is a significant difference by treatment group in the primary outcome.

The FI of the NINDS trial is 3.12  This means that only 3 patients would have to have had a different outcome to change the study from positive to negative. The FI of ECASS-III is 1.14  When these results are considered in light of the methodological flaws described above and the lack of internal validity, it defies credibility to understand how one unreplicated study had the effect of changing international practice. It has been argued that the case for thrombolysis is built on a house of cards.15  Data from stroke registries and community studies, used for post-marketing monitoring in the real world of clinical practice, supports the case that treatment of stroke with tPA is worse than placebo.15

DISCUSSION

IST-3 Trial and meta-analyses

The IST-3 trial, published in 2012, is the largest clinical trial of stroke thrombolysis (n = 3035), up to 6 hrs after stroke onset.16 This was a negative trial for its primary endpoint. The first 276 patients were double-blind and in this group no advantage of tPA over placebo was observed. Hence the FI cannot be calculated. The subsequent patients were open-label and despite the study’s negative result, it was concluded to be positive by using a secondary outcome measure (ordinal shift in the Oxford Handicap Score) that has itself been the subject of controversy.17  In the 3-4.5 hr subgroup (n = 1177), no difference in functional outcomes in those randomized to tPA vs placebo (32% tPA vs 38% placebo (OR 0.73) was reported. However, this was obtained by using a 99% confidence interval (0.50-1.07).  Readers will be aware that the conventional reporting approach is the 95% CI.  Use of a 95% CI results in a statistically significant association between placebo and good functional outcome.18

Emberson et al (2014) published a meta-analysis (MA) which concluded that tPA significantly improves the overall odds of a good stroke outcome when delivered within 4.5 hrs of onset.19  This was irrespective of age or stroke severity and despite an increased risk of fatal intracranial haemorrhage. About half their included patients were from IST-3, the largest stroke thrombolysis trial, which was negative on its primary outcome. IST-3 found no consistent relationship between time-to-treatment and outcome. It is difficult to understand how the negative finding of IST-3 did not move the MA needle in a more negative direction. If most clinical trials including IST-3 were negative, but the MA is positive, trials other than IST-3 are driving the outcome.20

Although Emberson et al19 promoted their MA as using individual patient data, this does not negate the problem of heterogeneity. Trials must be clinically and statistically homogeneous to be validly pooled.21 One needs a p value for test of heterogeneity > 0.1 to remove heterogeneity as a problem. Emberson et al did not report an assessment of heterogeneity and this was not provided when requested.22  Their time-to-treatment analysis is also invalid because there is no evidence that time-to-treatment functions as a continuous variable.  Attempting to compare patient outcome based on time-to-treatment, therefore, degenerates into a comparison between heterogeneous study populations. Despite this, the authors forced all data into a regression equation, which implied that thrombolysis is beneficial up to five hours after stroke. This relationship is likely due to confounding factors.23

The Emberson et al paper also had multiple conflicts of interest. It has been shown that industry supported studies are associated with more favourable conclusions than independent sources.24 The Institute of Medicine has published guidelines on MA and recommends exclusion of “individuals with a clear financial conflict of interest”.25  The multiple conflicts of interest listed in the Emberson et al paper show that these guidelines were not followed. A 2016 MA by Donaldson et al reported no conflicts of interest, and reached a more skeptical conclusion that promoted replication studies.26

In 2018, the author of the original phrase “time is brain” wrote: “It is no longer reasonable to believe that the effect of time on the ischaemic process represents an absolute paradigm. It is increasingly evident that the volume of injured tissue within a given interval after the estimated time of onset shows considerable variability in large part due to the beneficial effect of a robust collateral circulation.”27  The effect of any intervention in stroke is due to factors such as the precise automated measurement of a small ischaemic core, a large reversible penumbra, and the use of thrombectomy in highly selected patients with detailed consent,1 coupled with absence of treatment detriment. The endovascular trials have demonstrated is that it is possible to identify a cohort of patients who will benefit from revascularization, independent of time of onset, when perfusion imaging is paired with an effective treatment modality. This has not been achieved for stroke thrombolysis using tPA. It is time to recognise the “fragility” of the stroke thrombolysis literature and ask again whether treatment with tPA has a real benefit, when the harms are proven.

Review:  Reviewed internally by TMJ
Ethics approval:  Not required (literature-based)
Conflicts of interest:  None declared

Correspondence: Professor D M Fatovich, Department of Emergency Medicine, Royal Perth Hospital, GPO Box X2213, Perth WA 6847, Australia. Daniel.Fatovich@health.wa.gov.au

REFERENCES

  1. Mosely M. IV tPA is Still a Naked Emperor. Emergency Medicine News 2019; 41(7): 30.
  2. Tissue plasminogen activator for acute ischemic stroke. The National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group. N Engl J Med 1995; 333: 1581-7.
  3. Marler JR, Tilley BC, Lu M, et al. Early stroke treatment associated with better outcome: the NINDS rt-PA stroke study. Neurology 2000; 55: 1649-55.
  4. Adams HP, Jr., Davis PH, Leira EC, et al. Baseline NIH Stroke Scale score strongly predicts outcome after stroke: A report of the Trial of Org 10172 in Acute Stroke Treatment (TOAST). Neurology 1999; 53: 126-31.
  5. Hoffman JR, Schriger DL. A graphic reanalysis of the NINDS Trial. Ann Emerg Med 2009; 54: 329-36, 36 e1-35.
  6. Hacke W, Kaste M, Bluhmki E, et al. Thrombolysis with alteplase 3 to 4.5 hours after acute ischemic stroke. N Engl J Med 2008; 359: 1317-29.
  7. Shy BD. Implications of ECASS III error on emergency department treatment of ischemic stroke. The Journal of emergency medicine 2014; 46: 385-6.
  8. Ferguson C. Use of thrombolysis in acute ischaemic stroke. Best Evidence Topics (BETs) http://www.bestbets.org/bets/bet.php?id=717 (Accessed  21/12/19).
  9. Quinn TJ, Dawson J, Walters MR, Lees KR. Reliability of the modified Rankin Scale: a systematic review. Stroke 2009; 40: 3393-5.
  10. Wilson JT, Hareendran A, Hendry A, Potter J, Bone I, Muir KW. Reliability of the modified Rankin Scale across multiple raters: benefits of a structured interview. Stroke 2005; 36: 777-81.
  11. Penston J. stats.con How we’ve been fooled by statistics-based research in medicine: The London Press. 2010.
  12. Farkas J. What is the fragility index of the NINDS trial? PulmCrit (EMCrit), 5 July 2016. http://emcrit.org/pulmcrit/fragility-index-ninds/ (Accessed 12 July 2016).
  13. Fatovich DM, Phillips M. The probability of probability and research truths. Emerg Med Australas 2017; 29: 242-4.
  14. Farkas J. Mirror Mirror on the Wall, Who’s the Most Fragile of Them All? Assessing the Fragility Index of ECASS III. PulmCrit (EMCrit), 16 August, 2016. http://empharmd.blogspot.com.au/2016/08/mirror-mirror-on-wall-whos-most-fragile.html (Accessed 9 December, 2016).
  15. Johnstone C. Thrombolysis for acute ischemic stroke: does it work?-the con position. CJEM 2015; 17(2): 180-3.
  16. Sandercock P, Wardlaw JM, Lindley RI, et al. The benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the third international stroke trial [IST-3]): a randomised controlled trial. Lancet 2012; 379: 2352-63.
  17. Radecki RP, Chathampally YG, Press GM. rt-PA and Stroke: Does IST-3 Make It All Clear or Muddy the Waters? Answers to the November 2012 Journal Club Questions. Ann Emerg Med 2013; 61: 489-98.
  18. Brown MD, Burton JH, Nazarian DJ, Promes SB. Clinical Policy: Use of Intravenous Tissue Plasminogen Activator for the Management of Acute Ischemic Stroke in the Emergency Department. Ann Emerg Med 2015; 66: 322-33.e31.
  19. Emberson J, Lees KR, Lyden P, et al. Effect of treatment delay, age, and stroke severity on the effects of intravenous thrombolysis with alteplase for acute ischaemic stroke: a meta-analysis of individual patient data from randomised trials. Lancet 2014; 384: 1929-35.
  20. Fatovich DM. Truth, thinking and thrombolysis. Emerg Med Australas 2016; 28(5): 490-2.
  21. Thompson SG. Why sources of heterogeneity in meta-analysis should be investigated. BMJ (Clinical Research) 1994; 309: 1351-5.
  22. Emberson J, Lees KR, Lyden P, Baigent C, Sandercock P, Hacke W. Thrombolysis in acute stroke–authors’ reply. Lancet 2015; 385: 1396.
  23. Farkas J. The fallacy of time-to-intervention studies. PulmCrit (EMCrit), August 12, 2014. https://emcrit.org/pulmcrit/the-fallacy-of-time-to-intervention-studies/ (Accessed 21/12/19).
  24. Lundh A, Sismondo S, Lexchin J, Busuioc OA, Bero L. Industry sponsorship and research outcome. The Cochrane database of systematic reviews 2012; 12: Mr000033.
  25. Graham R, Mancher M, Wolman D, Greenfield S, Steinberg E. Clinical Practice Guidelines We Can Trust. Committee on Standards for Developing Trustworthy Clinical Practice Guidelines; Institute of Medicine 2011. https://www.ncbi.nlm.nih.gov/books/NBK209539/pdf/Bookshelf_NBK209539.pdf (Accessed 21/12/19).
  26. Donaldson L, Fitzgerald E, Flower O, Delaney A. Review article: Why is there still a debate regarding the safety and efficacy of intravenous thrombolysis in the management of presumed acute ischaemic stroke? A systematic review and meta-analysis. Emerg Med Australas 2016; 28: 496-510.
  27. Gomez CR. Time Is Brain: The Stroke Theory of Relativity. J Stroke Cerebrovasc Dis 2018; 27: 2214-27.

Publication Categories

AUTHOR INFORMATION

MBBS FACEM PhD Daniel M Fatovich

OCCUPATION
Consultant Emergency Physician, Centre for Clinical Research in Emergency Medicine, Department of Emergency Medicine, Royal Perth Hospital, Western Australia, Australia
INSTITUTIONAL AFFILIATIONS
Harry Perkins Institute of Medical Research, University of Western Australia, Perth, Western Australia, Australia