RWR Insights | Quality Standards for Registries – EUnetHTA REQUEST Tool
APRIL 2022 – This is the third (3rd) and last in the series exploring current quality standards for registries and registry-based studies. This month we focus on quality standard expectations when using real world data (RWD) from registries to support Health Technology Assessment. Specifically, we’ll look in detail at the ‘Registry Evaluation and Quality Standards Tool (REQueST)’ tool developed by the European Network for Health Technology Assessment (EUnetHTA) [1] [2].
The use of registries is becoming increasingly common in health technology assessment (HTA) and regulation. There is a growing interest in the role of observational data in complementing experimental data. This project aims to support best practice in the collection, use and re-use of real world data, and explore options to support sustainable multi-stakeholder collaboration (as per Paragraph 1 of the EUnetHTA JA3 Vision Paper on REQueST Tool, September 2019) [3].
EUnetHTA – REQUEST Tool
For HTA purposes, the Registry Evaluation and Quality Standards Tool (REQueST) is designed to be used in 3 steps (as per Paragraph 13 of the EUnetHTA JA3 Vision Paper on REQueST Tool, September 2019):
-
- Step 1: Initial screening of a potential registry for suitability. Standards currently presented in the ‘Methodological Information’ section of the tool are intended to enable a user to assess whether a registry can provide data that fulfil their needs6.
- Step 2: All registries that are potentially suitable should then be assessed against ‘Essential Standards’ relating to registry quality.
- Step 3: Some registries will require assessment against additional criteria for specific purposes (e.g., international collaboration on data collection will require interoperability). This is assessed in the ‘Additional Requirements’ section of the tool.
The registry owner carries out a self-assessment by completing the ‘registry owner’ column in the ‘Essential Standards’ worksheet of REQueST. This information together with a summary of the registry methodological information and the minimum key documents (listed in paragraph 32) are presented on the registry’s web site. These may be reviewed at any point by organisations considering whether to use the data in evidence development for HTA and regulatory monitoring to check if the information meets their needs (as per Paragraph 23 of the EUnetHTA JA3 Vision Paper on REQueST Tool, September 2019).
Periodic Review of Registries
In general, assessment of the quality of a registry cannot be done as a one-off event; ongoing quality needs to be demonstrated and the tool output will require periodic review (as per Paragraph 30 of the EUnetHTA JA3 Vision Paper on REQueST Tool, September 2019).
Criteria for the frequency of registry review could include the (as per Paragraph 30 of the EUnetHTA JA3 Vision Paper on REQueST Tool, September 2019):
-
- Maturity of the registry and technology (e.g., national joint registries dealing with established technologies and governance would require less frequent review).
- Production of peer-reviewed publications based on the registry (e.g., if at least one peer-reviewed publication per technology is produced in an acceptable time interval, the REQueST review could be less frequent).
- Purpose of the registry (e.g., if it is bespoke, that is, to meet specific regulatory or technology assessment objectives, quality should be assessed at the beginning and at the point of data use).
It should be noted that the ‘Methodological Information’ and ‘Additional Requirements’ sections of REQueST include questions that relate to specific uses of a registry. Users of REQueST may therefore need to run the tool more than once for an individual registry, and only the ‘Essential Standards’ would be transferrable between assessments (as per Paragraph 31 of the EUnetHTA JA3 Vision Paper on REQueST Tool, September 2019).
All phases require the registry owner to produce and make publicly available the following ‘minimum key documents’ (as per Paragraph 32 of the EUnetHTA JA3 Vision Paper on REQueST Tool, September 2019):
-
- Registry aims and methodology including minimum data set and data security policies.
- Declarations of relevant interests.
- Demonstration of continuous and comprehensive data collection (exact format and periodicity to be agreed but this is likely to include regular reporting on coverage, completeness and validation of data). Where a registry is federated between many countries, a report would be required from every participating registry.
- Safety statement detailing any alerts that have been raised (initiated by the registry owner and jointly publicised with the regulator/assessor).
Methodological Information – Screen for registries whose data and methodology match the requirements of the HTA/regulatory study or research question(s)
‘Methodological Information’ refers to the research methodology and which information is collected (research question, protocol and observational methods). This section provides an opportunity for the HTA agency to gather information about the data collected by the registry. Methodological information will be used to assess whether a registry is ready and able to answer a specific research question. There are 8 ‘Methodological Information’ items covering the following areas (as per Appendix A of the EUnetHTA JA3 Vision Paper on REQueST Tool, September 2019):
• Type of registry
• Use for registry-based studies and previous publications
• Geographical and organisational setting
• Duration
• Size
• Inclusion and exclusion criteria
• Follow-up
• Confounders
Essential Standards – Assessment of registry governance to assure general data quality and protection
‘Essential Standards’ are the minimum requirements for every registry. They are essential elements of good practice and evidence quality that can be used in the evaluation of the registry. Unless all essential criteria are demonstrably fulfilled, the HTA agency should not use the registry for evidence evaluation. There are 12 ‘Essential Standards’ items covering the following areas (as per Appendix A of the EUnetHTA JA3 Vision Paper on REQueST Tool, September 2019):
1. Registry aims and methodology
2. Governance
3. Informed consent
4. Data dictionary
5. Minimum data set
6. Standard definitions, terminology and specifications
7. Data collection
8. Quality assurance
9. Data cleaning
10. Missing data
11. Financing
12. Protection, security and safeguards
Additional Requirements – Specific requirements for the evidence questions
‘Additional Requirements’ are elements of good practice and evidence quality which are not always practical or feasible to achieve but are useful to consider in planning and evaluating registries. Evaluation of the ‘Additional Requirements’ depends on the requirements of an individual HTA agency and the specific context or registry use (e.g., an international collaboration on data collection will require registry interoperability). There are 3 ‘Additional Requirements’ items covering the following areas (as per Appendix A of the EUnetHTA JA3 Vision Paper on REQueST Tool, September 2019):
• Interoperability and readiness for data linkage
• Data sources
• Ethics
References
1. Registry Evaluation and Quality Standards Tool (REQueST)
Link: https://www.eunethta.eu/request-tool-and-its-vision-paper/
2. European Network for Health Technology Assessment (EUnetHTA)
Link: https://www.eunethta.eu/
3. EUnetHTA JA3 Vision Paper on REQueST Tool (September 2019)
