The use of RWD/RWE have become increasingly common and relevant, especially in oncology, because there is a growing recognition that RCTs might not be sufficiently representative of the entire patient population that is affected by cancer, and that specific clinical research questions might be best addressed by RWD/RWE [1].

RWE use cases include:

• • • Identify patients and patient subpopulations
    • Population bridging
    • Comparative effectiveness
    • External controls
    • Inform trial design
    • Establish endpoints

The FDA has recently indicated it is aware and supportive of the fact that pharma needs use RWD in drug discovery. The industry now needs to create the interoperability, standards, and methods to ensure that data can be included in regulatory submissions. This evolution may be akin to the critical path initiative. When the FDA embraced the idea of the critical path and allowing more in silico modeling of clinical trial design and development, it took the industry almost 10 years to adopt and apply the guidance [2].

According to the EMA 2025 Vision for RWE Use in EU Medicines Regulation [3]:

“…by 2025 the use of real-world evidence will have been enabled and the value will have been established across the spectrum of regulatory use cases”

Use of RWE to Support Regulatory Decisions – EU

In a recent publication (Flynn et al., 2021) [4], the EMA assessed the contribution of RWE to marketing authorization applications (MAAs) made to the European Medicines Agency in 2018–2019.  

According to Flynn et al., [4] there is widespread use of RWE to support evaluation of MAAs and extension of indications (EOIs) submitted to the EMA.  RWE can have a substantial impact on regulatory decision making, for example by:

• • • Informing on the natural history of disease and standards of care
    • Contextualizing results of uncontrolled trials when used as comparator groups of patients for single arm trials, or
    • Collecting follow-up data to generate post-authorization evidence on long-term safety and effectiveness of medicinal products

Interestingly, even though there is widespread use if RWE to support the evaluation of MAAs and EOIs, the majority of uses are still limited to post-authorisation safety studies, as indicated below:

• • • 40% of initial marketing authorisation applications and 18% of applications for extension of indication for products currently on the market contained RWE
    • 86% of RWE submitted as part of an MAA were Post-Authorisation Safety Studies (PASS)

DARWIN EU

The EMA  is creating a Data Analysis and Real-World Interrogation Network (DARWIN EU).  The aim of DARWIN EU is to [5]:

• • • Deliver real-world evidence from across Europe on diseases, populations and the uses and performance of medicines
    • Enable EMA and national competent authorities to use these data whenever needed throughout the lifecycle of a medicinal product.
    • Support regulatory decision-making by:

• • • • • establishing and expanding a catalogue of observational data sources for use in medicines regulation;
        • providing a source of high-quality, validated real world data on the uses, safety and efficacy of medicines;
        • addressing specific questions by carrying out high-quality, non-interventional studies, including developing scientific protocols, interrogating relevant data sources and interpreting and reporting study results.

DARWIN EU will support the implementation of the European Health Data Space (EHDS), which aims to leverage EU-wide health data for (amongst other things) research purposes.

International Collaboration to Integrate Real-World Evidence into Regulatory Decision Making

Recently (July 2022), the EMA, FDA and Health Canada published a statement on international collaboration to integrate real-world evidence into regulatory decision-making [6].  

According to the statement, the following collaboration opportunities exist:

• • • Harmonisation of RWD and RWE terminologies:

• • • • • Generate common operational definitions of RWD and RWE, with clear scope and level of
        • granularity (e.g., pertaining to RCTs and observational studies)
        • Leverage existing ICH activities, such as M14 on “General principles on planning and
        • designing pharmacoepidemiological studies that utilize real-world data for safety assessment of a medicine”

• • • Convergence on RWD and RWE guidance and best practice, including:

• • • • • Common principles for RWD quality;
        • Metadata to enable characterisation and discoverability of RWD
        • Suitable scenarios where RWE may contribute to regulatory decision-making, building on
        • existing use-cases
        • Templates for study protocols/reports that can be used in multiple regulatory jurisdictions

• • • Readiness

• • • • • Through the strengthening of international regulatory collaboration on RWE, enable the rapid creation of expert groups on specific topics of interest, including in case of emerging health threats
        • Foster collaboration on governance and processes to enable the efficient conduct of studies based on RWD from different countries to address important public health challenges

• • • Transparency

• • • • • Define common principles and practices for the systematic registration of pre-specified study
        • protocols (including description of feasibility assessments) and study results in publicly
        • available registries
        • Promote publication of study results in open-source, peer-reviewed journals

NICE (UK) RWE Framework 2022 

NICE makes recommendations that guide decisions in health, public health, and social care.  We want to make greater use of real-world data to learn from the delivery of health and social care and patient health and experiences to improve these recommendations [7].

According to NICE’s RWE Framework, randomised controlled trials (RCT) re the preferred source of evidence on the effects of interventions. However, randomised trials are sometimes unavailable or are not directly relevant to decisions about patient care in the NHS, for reasons including [7]:

• • • Randomisation is considered unethical, for instance because of high unmet need 
    • Patients are unwilling to be allocated to one of the interventions in the trial 
    • Healthcare professionals are unwilling to randomise patients to an intervention which they consider less effective 
    • A small number of eligible patients 
    • Financial or technical constraints on studies 
    • Not all treatment combinations (including treatment sequences) can be directly assessed

Even if randomised evidence is available, it may not be sufficient for decision making in the NHS for several reasons including [7]: 

• • • The comparator does not reflect standard of care in the NHS 
    • Relevant population groups are excluded 
    • There are major differences in patient behaviours, care pathways or settings that differ from implementation in routine practice 
    • Follow up is limited 
    • Unvalidated surrogate outcomes are used 
    • Learning effects are present 
    • Trials were of poor quality

Real-world data is already widely used to inform NICE guidance to, for example [7]: 

• • • Characterising Health Conditions and and Patient Outcomes – characterise health conditions, interventions, care pathways and patient outcomes and experiences 
    • Economic Models – design, populate and validate economic models (including estimates of resource use, quality of life, event rates, prevalence, incidence and long-term outcomes) 
    • Digital Health Technologies – develop or validate digital health technologies (for example, digital technologies may use a clinical algorithm developed using real-world data) 
    • Health Inequalities – identify, characterise and address health inequalities 
    • Safety – understand the safety of medical technologies including medicines, devices and interventional procedures 
    • Impact Assessments – assess the impact of interventions (including tests) on service delivery and decisions about care 
    • Trial Data Generalisability/ Relevance – assess the applicability of clinical trials to patients in the NHS. 

Real-world data that represents the population of interest is NICE’s preferred source of evidence for most of these applications. Such data is regularly used for these purposes in NICE guidance, but its use could be more commonplace, especially of routinely collected data. Real-world data could be used more routinely to fill evidence gaps and speed up patient access. For this promise to be realised, real-world evidence studies must be performed transparently and with integrity, use fit-for-purpose data, and address the key risks of bias. 

Accordingly, NICE are communicating their view on best practices for the conduct of real-world evidence studies to ensure they are generated transparently and are of good quality. This is essential to improving trust in real-world evidence studies and their use in decision making. 

External Controls

Real-world evidence (RWE) is increasingly being used to support and inform the clinical development of medicinal products. RWE use cases include external controls.

An external control arm is an umbrella term referring to any control that is not a randomized control and includes control data collected concurrently or historically from previous clinical trials or RWD [8].

External Controls – Uses and Trends [9]:

• • • Accelerated approval programs for high morbidity and high unmet need diseases have driven the use of single-arm studies, studies that do not include placebo or active comparator arms (ie, no concurrent control), for drug development. 
    • Single-arm studies speed up patient access to innovative treatments because they often require fewer patients than randomized controlled trials (RCTs) and use intermediate or surrogate endpoints (eg, objective response rate [ORR]). 
    • Oncology drug development has been increasingly relying on single-arm studies: from 1992 to 2017, 67% of the Food and Drug Administration’s (FDA) accelerated approvals were based on single-arm trials.1 
    • Similar trends have been observed in health technology assessment (HTA) submissions. From 2000 to 2016, 22 submissions to the National Institute for Health and Care Excellence (NICE) in the UK were based on nonrandomized data, and oncology drugs accounted for more than half of these submissions. 
    • A consequence of using single-arm studies is generally less evidence on the therapeutic benefits of the product at market launch [Drives the need for confirmatory trials post-approval]. 
    • In the absence of the preferred evidence standard (i.e., large-scale RCTs with a true state of clinical equipoise) in submission packages, real-world evidence (RWE)—or evidence generated from real- world data (RWD)—is increasingly being considered by decision makers. 
    • External control arms (ECAs) generated from RWD, or “data relating the patient health status and/or the delivery of healthcare routinely collected” from insurance claims, electronic health re- cords (EHRs), registries, etc, are emerging to contextualize single-arm trial data by exploring what would happen if single-arm trial patients did not receive the study drug. 

External Controls – Challenges [8]:

• • • There are numerous challenges to decision makers’ adoption of RWE, especially in the context of ECAs, as a trusted source of evidence. One main concern is the potential to introduce bias that is often reduced through randomization 
    • 2 common methodological challenges are:

• • • • • Confounding = Systematic differences in patient characteristics across treatment groups that can affect effectiveness
        • Selection Bias = Individuals in a study differing systematically from the population of interest, leading to a systematic error in an association or outcome

The FDA has said it recognizes the importance of RWD, but that acknowledgement has resulted in few approvals. Looking at the use of synthetic control arms and RWD in regulatory submissions over the last five years shows just 10 submissions and all were in oncology. Only one was a successful submission, and the rest were rejected because of lack of completeness of the data [2].

According to the FDA review of the External Control for Xpovio (selinexor) [9]:

 “Due to major methodological issues (including immortal time bias, selection bias, misclassification, confounding, and missing data), the FDA does not consider these results adequate to support regulatory decision making.” (2019 NDA Review)

External Control – Rozlytrek (Entrectinib)  – List of key study limitations identified by the FDA [10] [11] [12]:

• • • Post-hoc analysis (lack of prior FDA review of the protocol and SAP)
    • Selection Bias
    • Missing Data Among Covariates and Missing Covariates
    • Statistical Modeling…limited by sample size
    • Measure of Study Outcomes – Not adequate to allow a robust comparison of treatment outcomes between arms

Impact = Label limitations (FDA label of Rozlytrek excluded TTD, PFS, and OS outcomes, and only referenced improvements in ORR) [12].

External Control – Abecma (idecabtagene vicleucel/ide-cel) – List of Key Issues Identified by the FDA [13]:

• • • Selection criteria issues
    • Missing data
    • RWD not fit for purpose
    • Impact
    • ECA not used
    • Systematic literature review accepted but of ‘limited utility’ due to lack of patient-level data

FDA Summary = “Given the methodological limitations discussed above, we conclude that the evidence generated from the RW analysis is not adequate to provide context or comparison for the outcome of MM-001 study” [14]

FDA feedback on the systematic literature review: “Given the differences in the patient characteristics, the definitions of outcomes, and other unreported differences across the studies, the treatment effect estimate may be biased limiting the utility of this analysis” [15]

External Control – Abecma (idecabtagene vicleucel/ide-cel) – List of Key Issues RWE Study Identified by the EMA [16]:

• • • The rather long time period (up to 60 days from the index date) allowed for the collection of baseline data
    • The overlapping recruitment periods for the RWS and the MM-001 at the same study centres
    • The large proportion of missing data (up to 30%) for some included co-variates
    • Several co-variates excluded from the PS model due to >30% missing data

According to the EMA, despite the limitations of the indirect treatment comparisons, the results indicate that ide-cel treatment is associated with responses that are well above those reported with current standard of care [16].

Summary of FDA Findings/Limitations for Reviewed External Controls 

ECA evidence appears to have limited impact on regulatory and HTA decision making [8].

List of key external control limitations identified by the FDA:

• • • Post-hoc analysis (lack of prior FDA review of the protocol and SAP)
    • Selection Criteria Issues (selection bias, misclassification, and confounding)
    • Index Date Issues (immortal time bias)
    • Comparability Issues (lack of comparability, missing data, confounding bias)
    • Limited Sample Size (Lack of statistical power due to insufficient real world cohort size)
    • RWD not fit for purpose

Suggestions for optimising the acceptability of external controls for regulatory decision making:

• • • Design the external control to emulate the preferred randomised controlled trial – use a “target trial approach”
    • Use appropriate RWD sources to optimise comparability, relevance and generalizability of the control group population
    • Seek early FDA review of the protocol and SAP (a priori rather than post hoc)
    • Assess and limit bias using the appropriate methodological/ statistical designs
    • Use the START-RWE templates to demonstrate the what, how and why of the data sources, curation and analysis
    • List the limitations of the RWD/ RWE in the protocol

Using RWE to Support Regulatory Decisions – Evidence Considerations and Opportunities

The healthcare real world evidence (RWE) regulatory framework has rapidly evolved over the past few years (see below), such that RWE is increasingly being used to support and inform the clinical development of medicinal products. RWE use cases include:

• • • Identifying patients and patient subpopulations
    • Population bridging
    • Comparative effectiveness
    • External controls
    • Informing trial design
    • Establishing endpoints

Despite this rapid evolution, the regulatory acceptance of RWE is limited, with the majority of RWE submitted to the EMA being preemptive post-authorisation safety studies (PASS).  Furthermore, FDA acceptance of RWE in the form of external controls is limited by poor data quality, bias and inappropriate use of real world data sources. 

Moving forward, real-world evidence studies need to be conducted so that they are transparent and reproducible and the data generated (RWE) are relevant, reliable and of good quality. This is essential to improving trust in real-world evidence studies and their use in regulatory decision making.

Much progress has been made since 2016…it is clearly evident that there is significantly more work needed before the RWE generated during drug development are of acceptable quality and reliability to support MAA, NDA and BLA submissions.

References

1. Batra A, Cheung WY. Role of real-world evidence in informing cancer care: lessons from colorectal cancer. Curr Oncol. 2019 Nov;26(Suppl 1):S53-S56. doi: 10.3747/co.26.5625. Epub 2019 Nov 1. PMID: 31819710; PMCID: PMC6878934.

Link: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6878934/ 

2. The Next Three Years Of Clinical Trials: DCTs, RWE, And Beyond. Ed Miseta. Clinical Leader (18 April 2022)

Link: https://www.clinicalleader.com/doc/the-next-three-years-of-clinical-trials-dcts-rwe-and-beyond-0001 

3. EMA – A vision for use of real-world evidence in EU medicines regulation (24 November 2021)

Link: https://www.ema.europa.eu/en/news/vision-use-real-world-evidence-eu-medicines-regulation  

4. Flynn, R., Plueschke, K., Quinten, C., Strassmann, V., Duijnhoven, R.G., Gordillo-Marañon, M., Rueckbeil, M., Cohet, C. and Kurz, X. (2022), Marketing Authorization Applications Made to the European Medicines Agency in 2018–2019: What was the Contribution of Real-World Evidence?. Clin. Pharmacol. Ther., 111: 90-97. https://doi.org/10.1002/cpt.2461

Link: https://ascpt.onlinelibrary.wiley.com/doi/10.1002/cpt.2461  

5. EMA – Data Analysis and Real World Interrogation Network (DARWIN EU)

Link: https://www.ema.europa.eu/en/about-us/how-we-work/big-data/data-analysis-real-world-interrogation-network-darwin-eu   

6. EMA – Global regulators call for international collaboration to integrate real-world evidence into regulatory decision-making

Link: https://www.ema.europa.eu/en/news/global-regulators-call-international-collaboration-integrate-real-world-evidence-regulatory-decision  

7. NICE Real-World Evidence Framework – Corporate document [ECD9]Published: 23 June 2022

Link: https://www.nice.org.uk/corporate/ecd9/chapter/overview  

8. Ashley Jaksa, Anthony Louder, Christina Maksymiuk, Gerard T. Vondeling, Laura Martin, Nicolle Gatto, Eric Richards, Antoine Yver, Mats Rosenlund. A Comparison of Seven Oncology External Control Arm Case Studies: Critiques From Regulatory and Health Technology Assessment Agencies. Value in Health, 2022. ISSN 1098-3015, doi.org/10.1016/j.jval.2022.05.016. (25 June 2022)

Link: https://www.sciencedirect.com/science/article/pii/S1098301522020046  

Link2: https://www.valueinhealthjournal.com/article/S1098-3015(22)02004-6/fulltext?_returnURL=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS1098301522020046%3Fshowall%3Dtrue   

9. Section 7.2.6 (KS-50039 (Retrospective observational study using real-world data)) (page 84) – FDA – NDA/BLA Multi-disciplinary Review and Evaluation, NDA 212306, XPOVIO® (selinexor) (July 2019) 

Link: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2019/212306Orig1s000MultidisciplineR.pdf 

10. FDA approves entrectinib for NTRK solid tumors and ROS-1 NSCLC (August 2019) 

Link: https://www.fda.gov/drugs/resources-information-approved-drugs/fda-approves-entrectinib-ntrk-solid-tumors-and-ros-1-nsclc  

11. Drugs@FDA: FDA-Approved Drugs – Rozlytrek (NDA 212725)

Link: https://www.accessdata.fda.gov/scripts/cder/daf/index.cfm?event=overview.process&ApplNo=212725 

12. Section 8 – Appendix (page 30) – CDER – Review of Study Report No WO40977: Comparative analysis of ROS1-positive locally advanced or metastatic non-small cell lung cancer between patients treated in entrectinib trials and crizotinib treated patients from real world data (11 July 2019)

Link: https://www.accessdata.fda.gov/drugsatfda_docs/nda/2019/212725Orig1s000,%20212726Orig1s000OtherR.pdf  

13. Section 2.5 (Summary of Pre- and Post-submission Regulatory Activity Related to the Submission)(page 16) – FDA BLA Clinical Review Memorandum – STN 125736/0 (27 July 2020)

Link: https://www.fda.gov/media/147740/download 

14. Section 9.2 (Aspect(s) of the Clinical Evaluation Not Previously Covered- Systemic Literature Review/ Available therapies)(page 115) – FDA BLA Clinical Review Memorandum – STN 125736/0 (27 July 2020)

Link: https://www.fda.gov/media/147740/download 

15. Section 9.2 (Aspect(s) of the Clinical Evaluation Not Previously Covered- Systemic Literature Review/ Available therapies)(page 115) – FDA BLA Clinical Review Memorandum – STN 125736/0 (27 July 2020)

Link: https://www.fda.gov/media/147740/download 

16. Section 2.5.3 (Discussion on clinical efficacy) (page 113) – EMA – Abecma – EPAR

Link: https://www.ema.europa.eu/en/medicines/human/EPAR/abecma