Abstract
In the European Union (EU), the delivery of health services is a national responsibility but there are concerted actions between member states to protect public health. Approval of pharmaceutical products is the responsibility of the European Medicines Agency, while authorising the placing on the market of medical devices is decentralised to independent ‘conformity assessment’ organisations called notified bodies. The first legal basis for an EU system of evaluating medical devices and approving their market access was the Medical Device Directive, from the 1990s. Uncertainties about clinical evidence requirements, among other reasons, led to the EU Medical Device Regulation (2017/745) that has applied since May 2021. It provides general principles for clinical investigations but few methodological details – which challenges responsible authorities to set appropriate balances between regulation and innovation, pre- and post-market studies, and clinical trials and real-world evidence. Scientific experts should advise on methods and standards for assessing and approving new high-risk devices, and safety, efficacy, and transparency of evidence should be paramount. The European Commission recently awarded a Horizon 2020 grant to a consortium led by the European Society of Cardiology and the European Federation of National Associations of Orthopaedics and Traumatology, that will review methodologies of clinical investigations, advise on study designs, and develop recommendations for aggregating clinical data from registries and other real-world sources. The CORE–MD project (Coordinating Research and Evidence for Medical Devices) will run until March 2024. Here, we describe how it may contribute to the development of regulatory science in Europe.
Cite this article: EFORT Open Rev 2021;6:839-849. DOI: 10.1302/2058-5241.6.210081
Introduction
Regulations concerning medical devices are shared across the European Union (EU) but verification that manufacturers conform to their requirements is devolved to independent notified bodies overseen by their national regulatory agencies (or ‘competent authorities’). The role of the European Commission is to implement the Medical Device Regulation (MDR)1 together with the national regulatory agencies. Unlike some other jurisdictions worldwide, however, it lacks its own central scientific department with medical expertise in each major clinical field.
High-risk implantable medical devices are essential for clinical care. Scrutiny of clinical evidence before their market access is also essential. Uncertainty about the standards being applied2 and concerns that some devices had been granted access despite insufficient clinical evidence, contributed to agreement that new EU regulations were needed.3 That perception was reinforced by problems relating to heart valves, metal-on-metal hip replacements, breast implants, and surgical meshes. Perhaps in consequence, the MDR has increased requirements for clinical evidence before a new high-risk device can be approved. Previously, market access was faster in the EU than in the USA,4 but now device developers are concerned that the EU system may become less predictable and more time-consuming.
The optimal way to determine what levels of clinical evidence should be required before approval of new devices is by scientific enquiry. Thus to obtain expert advice on appropriate methodologies and standards for clinical investigations, the European Commission published a research call in 2020 entitled ‘Developing methodological approaches for improved clinical investigation and evaluation of high-risk medical devices’.5 This paper explains the rationale and objectives of the project that was awarded this grant. We outline the main tasks that are planned and describe how colleagues can contribute to its activities.
Regulatory context and rationale
The MDR aims to provide ‘a robust, transparent, predictable and sustainable regulatory framework for medical devices which ensures a high level of safety and health whilst supporting innovation’ (Recital 1).1 Application of the MDR has increased technical and administrative requirements relating to the generation of clinical data, but new policies for clinical studies have not been fully developed.
Medical devices are assessed in Europe under the ‘new approach framework’6 which applies uniform standards for approved products in each sector. One challenge from a clinical policy perspective is that the legal text provides only high-level common principles which can be applied to determine whether products should enter or leave the system (Annex I MDR).1,7 Requirements relating to overall benefit/risk are not linked to validated methods of generating evidence for different technologies. The application of generic rules in a decentralised system can lead to significant differences in the clinical evidence which supports individual medical devices, due to variations in interpretation and inconsistency in application of the legal rules. When these factors are combined with the potential for irreversible effects from implanted devices with a critical function, then the public health consequences can be significant.
Many medical devices have entered the market in Europe with minimal or no clinical data relating to the device itself, by utilising evidence from devices claimed to be equivalent. The MDR has changed the rules concerning equivalence (see Annex XIV), with a contract now being required between manufacturers of high-risk devices and with the need for an equivalent device to satisfy precise criteria.1 The legal definition of equivalence does not refer to non-inferiority studies, which is a different concept.
Pre-defined evidence requirements for predictable European market access do not exist for individual devices or technology groups, which may have a negative impact on safety and on the innovation of new technologies. The option provided by the MDR for the European Commission to develop a voluntary system for developers to consult scientific expert panels about the clinical development strategy for their high-risk devices has not yet been taken up.
The EU relies on international standards after they have been harmonised to EU laws,8 but those do not always prescribe clinical evidence requirements. The MDR makes it possible for common specifications to be introduced by the European Commission when relevant harmonised standards are insufficient or when there is a need to address public health concerns (MDR Article 9.1).1 That process will require considerable resources and expert knowledge, and to date no pathway has been implemented for its introduction in a systematic way. In the meantime, adherence to harmonised European standards confers a “presumption of conformity” to EU legal requirements, but their application by manufacturers is voluntary. Regulatory agencies and notified bodies have no legal power to require adherence to any harmonised standard, only to the ‘state of the art’.
The EU regulation concerning clinical trials of medicinal products9 does not apply to investigations of medical devices.10 The international guideline concerning good clinical practice (GCP) that is currently under revision “may also be applied to other clinical investigations that may have an impact on the safety and well-being of human subjects” but it was similarly not developed for trials of devices.11 Standard EN ISO 14155 does describe GCP relating to the evaluation of devices12 but it was developed independently and it has not yet been harmonised with the MDR.
Differences in how drugs and devices are regulated in the EU (Table 1) leave a gap for more specific guidance. As stated in the research call, “medical devices have particularities that make the conduct of clinical investigations difficult” and “there is a need for methodologies that enable to generate improved clinical evidence”.5
Some key differences between the regulatory evaluation of pharmaceutical products (drugs) and high-risk medical devices in the European Union
| Pharmaceutical products | Medical devices | |
|---|---|---|
| Organisation responsible for granting market access | European Medicines Agency (about 90%) National authorities (≈ 10%) |
Notified bodies |
| Types of organisations which bring products to market | Mostly large and established pharmaceutical companies | Variable: many start-ups and small and medium enterprises, as well as large medical technology companies |
| Time when clinical evidence is generated | Generally pre-market | Both pre- and post-market studies |
| Clinical development phases | Highly standardised (phases 1–4) |
Less standardised
Product-dependent |
| Clinical study design | Highly standardised Double-blind randomised controlled trial expected |
Less standardised
Pivotal trials often done after CE-marking |
| Irreversible effects on study subjects | Rare | Common, particularly with permanent implants |
The italicised text indicate areas that will be considered by CORE–MD (Coordinating Research and Evidence for Medical Devices).
CE = Conformité Européenne.
Overview of the CORE–MD project
The Coordinating Research and Evidence for Medical Devices (CORE–MD) project began in April 2021 and will run until March 2024. It comprises a unique collaboration with wide geographical distribution across Europe, importantly including national regulatory agencies and notified bodies as well as public health institutes, medical professional associations, academic institutions, and patients (Table 2). The project is led by the European Society of Cardiology (ESC) in partnership with the European Federation of National Associations of Orthopaedics and Traumatology (EFORT), and it is supported by an advisory board of international regulators and academic experts. Manufacturers’ trade associations have been invited to participate as advisers.
Partners in the CORE–MD consortium
 European medical professional associations • European Society of Cardiology (ESC) • The European Federation of National Associations of Orthopaedics and Traumatology (EFORT) • Biomedical Alliance in Europe (BioMed Alliance) • European Academy of Paediatrics (EAP) Academic institutions • Leiden University Medical Center, The Netherlands • The University of Oxford, UK • Uppsala Clinical Research Center, Sweden • Royal College of Surgeons of Ireland, Dublin, Ireland • Bern University Hospital, Switzerland • Katholieke Universiteit Leuven (KU Leuven), Belgium • UMIT – University for Health Sciences, Medical Informatics and Technology, Hall in Tirol, Austria • University of Gothenburg, Sweden • Politecnico di Milano, Italy • Ludwig-Maximilians-Universität, Munich, Germany (for EAP) National regulatory authorities of EU member states • Health Products Regulatory Authority (HPRA), Dublin, Ireland • Lægemiddelstyrelsen / Danish Medicines Agency (DKMA), Copenhagen, Denmark • Urząd Rejestracji Produktów Leczniczych, Wyrobów Medycznych i Produktów Biobójczych / Office for Registration of Medicinal Products, Medical Devices and Biocidal Products (URPL), Warsaw, Poland National public health institutes • Rijksinstituut voor Volksgezondheid en Milieu / National Institute for Public Health and the Environment (RIVM), Bilthoven, The Netherlands • Istituto Superiore di Sanità / Italian National Institute of Health (ISS), Rome, Italy Institutes for health technology assessment • Austrian Institute for Health Technology Assessment (AIHTA), Vienna, Austria • Fundación Pública Andaluza Progreso y Salud / Andalusian Health Technology Assessment Unit (AETSA), Seville, Spain European umbrella organisation for patients’ groups • European Patients’ Forum (EPF), Brussels, Belgium Trade association for European Notified Bodies • The European Association for Medical Devices of Notified Bodies (TEAM NB), Brussels, Belgium |
The most important objectives of CORE–MD are to review and develop methodologies for the clinical investigation and evaluation of high-risk medical devices. The scope does not extend to in vitro diagnostic medical devices or to health technology assessment. Specific tasks of CORE–MD are listed in Table 3 and described below. There will be a focus on cardiovascular, orthopaedic, and diabetes devices, since together those represent the majority of high-risk medical devices and because they exemplify devices used to reduce mortality and morbidity.
Overview of tasks in CORE–MD work packages 1–3
| Understanding methods used to generate clinical evidence for high-risk medical devices | |
| 1.1 | Methodologies in published clinical studies of high-risk medical devices |
| 1.2 | Statistical methods for medical device studies |
| 1.3 | Regulatory utility of patient-reported outcome measures |
| 1.4 | Published regulatory guidance and expert recommendations for clinical investigations |
| New methods for generating clinical evidence | |
| 2.1 | Providing evidence during the early development of high-risk medical devices |
| 2.2 | New designs for randomised clinical trials and studies of high-risk medical devices |
| 2.3 | Developing guidance for the evaluation of artificial intelligence and standalone software in medical devices |
| 2.4 | Recommendations concerning high-risk medical devices in children |
| Extracting maximal value from medical device registries and real-world evidence | |
| 3.1 | Aggregating insights from registries, big data, and clinical experience |
| 3.2 | Development of a mashup for collecting clinical reports of devices from accessible trusted web sources |
| 3.3 | Study of the use of conditions for clinical evidence generation after market access |
Methods used to generate clinical evidence for high-risk medical devices
Methods used to evaluate the clinical safety and efficacy of devices and to monitor their performance should be rigorous and transparent. Before the CORE–MD consortium can advise which methodologies are reliable to generate sufficient evidence, and also proportionate to risk, it will be important to understand how devices are currently being evaluated in the EU.
Methodologies applied in published clinical studies of high-risk medical devices
A systematic review will be performed according to recommended standards (PRISMA-P),13 to evaluate critically the methodologies used in clinical studies of high-risk medical devices in cardiovascular medicine, orthopaedics and diabetic medicine.
For cardiovascular devices (Table 4), peer-reviewed reports of studies of prospective design will be included. For orthopaedic devices, prospective and retrospective studies (cohort, registry-based cohort, case-control, randomised controlled trial (RCT), case series or reports) and annual reports from national registries will be evaluated, to obtain insights into descriptive data and revision rate estimates. For diabetic devices, literature will be reviewed for observational and experimental designs including RCTs, non-randomised trials, cohort studies, case-control studies, and case series.
Device types selected for systematic reviews of clinical trial methodologies
| Cardiovascular disease | Orthopaedic surgery | Diabetic medicine |
|---|---|---|
| – Drug eluting stents – Bioresorbable vascular scaffolds – Transcatheter aortic valve replacement – Transcatheter mitral valve repair – Left atrial appendage occluder – Leadless pacemaker – Subcutaneous implantable cardioverter defibrillator |
– Hip joint replacement (total hip cups, heads, liners and stems) – Knee joint replacement (total knee systems, unicompartmental knee systems) |
– Implantable continuous glucose monitoring systems – Implantable insulin pumps – Automated insulin delivery devices |
The focus will be on devices that are most relevant to clinical practice in the EU rather than those that receive a CE mark but may not be commonly used. Since there is not yet a European portal where certificates of conformity with their dates can be accessed, the CE-marking dates of the assessed devices will be identified from sources such as press releases, contact with manufacturers and notified bodies, and national databases of European countries. Pre-specified information on study designs performed before and after CE-marking, patient population, indexed intervention, and primary outcome(s) will be abstracted on study level. Systematic gender-specific analyses, reporting on gender-dimension usage in the reviewed study designs, and statistical methods will be considered. Differences in accumulated clinical evidence will be assessed within and across classes of medical devices.
Statistical methods for medical device studies
Recommendations for statistical methods, reporting, and levels of confidence that are appropriate for trial designs specific to high-risk medical devices and for new study designs other than traditional RCTs are needed. Key concepts are bias, confounding, precision and validity. Specific challenges include achieving appropriate comparability, for example by propensity matching or designing efficient adaptive studies, and adjusting for time-series data and operator experience or learning curves. Within this task, selected questions will be considered, to advise on analytical approaches that are appropriate for building up evidence.
One consequence of approving new devices on the basis of limited evidence is that some risk is accepted. A higher risk would probably be accepted by patients with a critical condition for which other devices are not available, while a much lower risk will be preferred when there are alternatives. Information about the relationship between the cumulative experience of a medical device (such as total follow-up years) and the probability of known or unknown adverse effects should be publicly available and disclosed, using a tool that is intelligible to patients and approved by assessors in notified bodies.
Methodology for applying objective performance criteria (OPC) was proposed more than 30 years ago14 and adopted for surgically implanted prosthetic heart valves.15 Other methods have been used to identify outlier performance of orthopaedic devices. Guidance will be provided for manufacturers and regulators on the applicability of these benchmarks, and on standard methodology that might encourage wider uptake.
Regulatory utility of patient-reported outcome measures
Medical devices are usually assessed using measures such as implant survival, mortality and complications, but many are designed to restore or preserve function, improve health-related quality of life, and relieve symptoms. Patient-reported outcome measures (PROMs) are therefore being used increasingly to assess patients’ self-perceived health status.16,17 Many tools are available that measure general health or aspects specific to a medical condition, disease, part of the body or intervention. Ideally, PROMs should focus not only on the disease treated by a device, but also on its overall impact according to the International Classification of Functioning (ICF).18
Despite this shift towards including PROMs, there is a lack of knowledge and consensus about which instruments have been validated and should be used and about how to interpret differences or adjust for confounding comorbidities and other patient-related factors. There are no EU standards regarding implant performance in terms of patient-reported outcomes. This task will address how PROMs can contribute to discriminating between well-performing and under-performing devices; the typical sample size to assess a device in terms of PROMs; the interpretation of change in terms of minimum clinically important difference; and what follow-up intervals would be recommended for assessing PROMs for specific interventions.
A systematic review will assess how PROMs have been used in trials and studies for regulatory purposes and post-market surveillance, and investigate their utility for monitoring the safety and efficacy of new implants. A Delphi panel will be convened among patients to understand the most valued PROMs and their feasibility and acceptance. Representatives from notified bodies will provide insights from their experience of evaluating evidence on PROMs submitted by manufacturers. A framework of regulatory requirements will be proposed.
Published regulatory guidance and expert recommendations for clinical investigation
Many principles of trial design used for pharmaceutical products apply also for medical devices, but some characteristics of high-risk devices imply a need for different guidance.19 Incremental development with short life-cycles, the physical mode of action, the complexity of interventions, and dependency on contextual factors may lead to device modifications during the course of clinical investigations, hinder blinding, or imbalance recruitment to study arms due to provider and patient preferences. Individual and institutional learning curves may have to be taken into account when quantifying the effect of an intervention.20 Guidance on methods for generating clinical evidence for medical devices needs a comprehensive scientific basis and expertise to ensure its implementation.
The objectives of this systematic review are, firstly, to identify guidance on the design, conduct, analysis and reporting of confirmatory pivotal clinical trials and other clinical investigations for high-risk medical devices, from regulators, international standardisation organisations, medical professional associations, academic consortia, and health technology assessment agencies; secondly, to compare their similarities and differences; and, thirdly, to identify gaps for further research on trial methodology.
The recommendations will be summarised in thematic sections using a similar approach to a former overview.21 Recommended methodologies will be compared with published studies (from Task 1.1).
New trial and study designs for high-risk medical devices
EU regulators are seeking expert consensus recommendations on gaps in standards and guidance, and on a hierarchy of methodological approaches. These should set an appropriate balance between the need for evidence and the need to avoid unnecessary regulation.22,23
Providing evidence during the early development of high-risk medical devices
The IDEAL framework (Idea, Development, Exploration, Assessment, Long-term study) is an attempt to develop an integrated evaluation pathway for complex therapeutic interventions throughout their lifecycle.24 It emphasises the need to adapt the focus and format of studies to the stage of evolution of the device, in order to answer the most relevant questions for that stage. The most innovative IDEAL proposals concern early clinical evaluation, where modified cohort studies with specific goals are used to study important aspects such as learning curves and device modifications. In this task the IDEAL Collaboration will work with innovators to apply the IDEAL Recommendations to the early clinical evaluation of novel devices in a series of case studies. Analysing the performance of IDEAL in these exemplars will improve understanding of its potential as a standard methodology to allow evaluation to progress safely from proof of principle to definitive pivotal assessment of comparative effectiveness.
New designs for randomised clinical trials and studies of high-risk medical devices
Prospective trials with enrolment of broad representative patient populations are needed in order to be able to compare the performance and outcomes of medical devices reliably.25 These trials need to be pragmatic, with rapid enrolment and completion in order to provide clinically relevant results. Preferably, they should be integrated into the clinical care of patients with minimal extra work for investigators and limited extra obligations for patients.26 Trials also need to be affordable, and acceptable from a methodological perspective for regulatory authorities. A pragmatic trial can successfully collect most of the necessary data using existing digital infrastructures such as registries, medical files and claims databases.
Trials built upon clinical registries, labelled registry-based randomised clinical trials (RRCTs), have been conducted successfully to study many devices and drugs. One of the first was TASTE (Thrombus Aspiration in ST-Elevation, myocardial infarction in Scandinavia), which used a nationwide clinical registry for randomisation, collection of baseline and procedural variables, and documentation of outcomes, in order to evaluate the strategy of using a medical device (thrombus aspiration catheter) in addition to standard primary percutaneous coronary intervention.27 Its results had a rapid impact on medical practice.28 Currently, INFINITY-SWEDEHEART is using a similar approach for a head-to-head comparison of a novel device against an established product, and there are RRCTs in orthopaedic29 and cardiothoracic surgery.30 The term ‘nested trials’ has typically been used to describe trials using other sources such as electronic medical records.31
More pragmatic trials to evaluate new devices should be performed using existing digital infrastructures, directly integrated into routine clinical care. They could be undertaken as a collaboration between professional societies, manufacturers, and regulatory authorities, to inform decision makers, doctors and patients about the best care. In CORE–MD the use of RRCTs will be reviewed and recommendations will be prepared for their wider application. Essential features of trial designs will be considered together with the Good Clinical Trials Collaborative.32
Developing guidance for the evaluation of artificial intelligence and standalone software
Artificial intelligence (AI) is increasingly prevalent in diagnostic imaging, decision support for high-risk medical decisions, robotics, and implantable devices with an electronic component. AI software is often self-learning, or it needs frequent adaptation to shifting applications to avoid bias and improve accuracy. A lengthy approval methodology would be inappropriate for a rapidly evolving software solution. At the same time, AI can be a ‘black box’ without explainability, that cannot simply be tested in silico if existing test databases are unrepresentative of the population in which the AI will be used. The MDR was not designed to accommodate this situation, so approval processes need to be adapted to the different types of applications of AI.
The challenge for evaluating software as a medical device within the new MDR is to provide practical guidelines that combine scientific rigour, support for innovation, efficient procedures, and protection of patients and caregivers. A risk assessment is the first step to identify software that carries the largest potential for inappropriate or incorrect results impacting on the outcomes of patients. For high-risk AI, scrutiny of the data used for creating, testing and validating the software is essential. Then there is a need for clinical testing, which can be stratified according to the level of explainability of the software and its reliance on tested and validated pathophysiological evidence. Either a typical RCT may be needed, or early release with rigorous real-world follow-up could be appropriate.
Guidelines, standards and definitions at the European level are essential to guarantee broad applicability. An ethical framework as outlined in the EU publication of 8 April 2019 is needed, focussing on transparency as a major requirement to make AI trustworthy and a useful addition to the health system.33 Related questions are being considered by many groups worldwide; within CORE–MD, essential principles for EU regulatory evaluation of high-risk AI as a medical device or within medical devices will be expounded.
Recommendations concerning high-risk medical devices in children
Implementing an evidence-based regulatory policy for medical devices used in infants, children and adolescents is challenging since few high-risk devices are needed, sample sizes are small, and patients are heterogeneous. For an individual child the availability of a state-of-the-art device can be life-saving, but paediatric patients often suffer from delayed and limited availability of new devices.34 It can be difficult for manufacturers to develop and evaluate paediatric devices if only small numbers can be sold to recover their investment. In the EU, implementation of the MDR for children requires special precautions to achieve an adequate balance between full documentation of safety and efficacy, and ensuring access to innovative devices. This is essential to secure the rights of children to get the highest possible standard of healthcare, as agreed by the world’s nations.35
Almost all devices approved in the USA between 2008 and 2011 for use in children were studied only in adults and in non-randomised open-label trials with surrogate effectiveness end-points and mandated post-marketing studies.36 Most had not been approved for children.37 Off-label drug use in children is associated with adverse events38 and risks may also occur from off-label use of devices. In CORE–MD, the evaluation of evidence for high-risk medical devices in children will serve as an example of special considerations particular to orphan devices.39
A systematic review of methodological approaches to evaluate medical devices in children will address pre-clinical, clinical and post-market evaluations using controlled trials, open-label studies, historical controls, patient and parent/caregiver reported outcomes, and extrapolation of data from adult patients. It will evaluate strengths and limitations of different identified approaches. Another systematic review will assess ethical aspects of the clinical evaluation of medical devices in paediatric patients. An evidence-based and feasible regulatory policy for paediatric medical devices is an urgent priority; recommendations will be developed in a joint workshop.
Medical device registries and real-world evidence
The third major objective of CORE–MD is to study aggregation methods that can exploit all sound data available from different sources. This will include real-world data adapted for the needs of conformity assessment and evaluation throughout the lifetime of a device.
Aggregating insights from registries, big data, and clinical experience
Trials on high-risk medical devices present methodological challenges related to the choice of comparator, randomisation, blinding, the learning curve of implantation, the timing of assessments, and difficulties determining all relevant outcomes during limited follow-up.40 Registries provide important insights into the performance and safety of medical devices in daily clinical practice and evidence of outcomes and risks that are rarely encountered during trials, especially if they enrol all patients in whom a device is implanted including those with multiple comorbidities. Differences in design, organisation (by the manufacturer or an independent body), methods and end-points determine the quality of the evidence and its utility for clinicians, patients, industry, and national and EU regulators.
CORE–MD will update and expand an earlier study41 that identified European registries in orthopaedics and cardiology. More in-depth characterisation of the quality of data collected and new evidence on outcomes and safety will allow comparisons between device types, focusing on hip and knee arthroplasty and on stents and heart valves. The latest available reports from registries will be reviewed and supplementary data may be requested. A systematic review will characterise variables that would be reliable as performance criteria and useful for benchmarking. For orthopaedic devices these will relate to the total construct, including different combinations of cup and stem for hip implants. It will also consider how to identify safety concerns related to implant failure or other adverse events.
The data obtained and reports concerning objective performance criteria will be used as inputs for a Delphi expert panel, with the aim to develop a decision framework for assessing the quality of a device registry and the safety and performance of devices after market access. Criteria will be proposed to ensure that outcomes are estimated reliably (such as the minimum number of patients or devices required) and that they are considered against appropriate comparator devices. Time-points for benchmarking against similar devices will be recommended. Methods will be developed to combine patient-reported and clinical outcomes into a single benchmark. Distributed network analysis and meta-analysis of database-specific estimates will be employed to allow for combination of data from different sources.
Development of a mashup for collecting clinical reports of devices
Post-market responsibilities are attributed to manufacturers and national regulatory agencies (see MDR Articles 83-100).1 The primary responsibility for surveillance is with the manufacturer, which must establish a systematic procedure for proactively collecting and reviewing experience gained from any high-risk medical device that it places on the market. Periodic safety update reports are submitted to the European Databank on Medical Devices (EUDAMED), a central database run by the European Commission.42 Regulatory agencies assess ongoing risks related to reported incidents and alerts and some also maintain independent publicly available databases of device information in searchable web portals.
The MDR obliges notified bodies in certain cases to submit the clinical evaluation assessment reports that they generate during conformity assessment of high-risk devices to the relevant expert panel (or ‘Expamed’) for an opinion on the sufficiency of the clinical evidence. In order to decide whether an independent scientific opinion is needed, the expert screening panel applies three criteria, the last of which is evidence of a significantly increased rate of serious incidents for that specific category or group of devices (MDR Article 54 and Annex IX Section 5.1).1 Expert panels, however, will not have access to this information in EUDAMED; if available it should be provided by the Commission secretariat. It could be helpful to develop methods for automatically collecting and integrating information on alerts and recalls of high-risk medical devices, in order to capture possible trends in respect of a specific category or group of devices. The output could be displayed in a dedicated dashboard and then used not only for scientific analysis but also to inform and assist the expert panels.
The aim of this task is to develop a mashup tool (a web page or web application that uses content from more than one source to create a single new service displayed in a single graphical interface) to produce enriched results, fed with information obtained by periodic extraction of web data (web scraping) from the websites of EU national competent authorities, non-EU regulatory authorities, and other trusted sources such as peer-reviewed literature, following a process of harmonisation into the English language and using a taxonomy of medical devices and reporting alerts. Manual validation of the automatically generated extraction on specific searches will serve as comparison. The resource will be open-access.
Clinical evidence generation after market access
Worldwide, evidence-development schemes may be specified by regulatory authorities. The need for conditional approvals pending further clinical investigation and re-evaluation has been illustrated by accelerated access during the coronavirus pandemic.43 The MDR does not provide for this explicitly, but notified bodies have authority to place a condition on a certificate of conformity, which might achieve similar aims. Little is known about how they have applied such conditions, however, because certificates will be available publicly for the first time only when the EUDAMED database becomes fully operational.
Challenges associated with conditional access schemes for medical devices have been described, but not specifically for high-risk medical devices. This task will review conditional evidence development schemes for high-risk medical devices, building on earlier analyses.44 It will identify and synthesise how schemes are operated worldwide, drawing lessons from the experience of other bodies including those regulating pharmaceutical products such as the European Medicines Agency (which does have a conditional approval process) and health technology assessment (HTA) bodies where appropriate.
Notified bodies in the CORE–MD consortium will participate in a prospective study to document their use of conditions on certificates of conformity. The results and insights will be used to shed light on how they apply conditions for evidence generation after market access and how such conditions are followed up, and to formulate recommendations. Common principles that are shared between notified bodies would make the process transparent and fair, and assure that sufficient clinical evidence is obtained for all high-risk devices.
Synthesis and dissemination of results
Accompanying the implementation of the new EU medical device regulations, the distinct communities that are engaged along the lifecycle of medical devices are encouraged to build an ecosystem of mutual learning and exchange. Innovators, clinical trialists and clinical experts collaborate in the generation of clinical evidence, notified bodies and regulators in the appraisal of this clinical data, and health technology assessors and payers in the evaluation of the clinical benefit for decisions on market access and reimbursement. Alignment among academic trialists, regulators (national competent authorities), notified bodies and HTA will benefit manufacturers by sharing of a reliable set of common principles and methodological approaches with the potential for seamless introduction of safe and beneficial high-risk medical devices responding to public health needs.45,46
CORE–MD will contribute to the development of ‘regulatory science’ and the quality of clinical care in Europe by fostering interactions, promoting exchange of best practices, and supporting networking activities among developers and clinical scientists. Within Work Package 4, an ethics charter for medical device innovation, will be developed incorporating essential principles and guidance on addressing ethical challenges such as unknown risks, requirements for consent, and disclosures of interests. Secondly, recommendations for a hierarchy of methodologies for the clinical evaluation of high-risk devices will be prepared, based on the results of the earlier tasks. This may be of value to manufacturers when setting up their pre-market clinical investigations, as well as to regulators and the clinical community when reviewing evidence and designing post-market studies. Finally, a roadmap of needs for training, education, capacity-building and regulatory research will be formulated, based on feedback from clinical experts working in Expamed and informed by advice from medical associations within the Biomedical Alliance in Europe, regulatory agencies, notified bodies, and members of manufacturers’ research and development divisions.
Advocacy
CORE–MD is an ambitious project with a broad scope (Fig. 1). A particular strength is the wide membership of the consortium, representing all major stakeholder groups including patients. It includes experts from notified bodies, which must approve the clinical data before any high-risk device is placed on the market48 and which can withdraw a product from the market if performance and safety data are not confirmed or collected adequately. A potential limitation for the CORE–MD consortium is the resources available, but the project is being conducted within an intense ecosystem where many other researchers, institutes, EU-funded programmes, and advisory and regulatory bodies already operate. Members of the consortium are engaged in several initiatives which have overlapping objectives, and other colleagues who would be interested in collaborating with specific CORE–MD tasks as described in this paper are invited to follow its activities at www.core-md.eu where there are details of how to contact the organising team. In the spirit of promoting global regulatory convergence for medical device regulations and standards, as espoused by the International Medical Device Regulators’ Forum (IMDRF) and endorsed by the European Commission, the consortium has appointed advisers based in other jurisdictions.
CORE–MD envisages hybrid functions including primary and secondary research, but its most important goal is to prepare expert consensus recommendations for methodologies that should be used for the clinical evaluation of high-risk medical devices. There is a need to update guidance developed for the EU medical device directives.49 Recommendations from the CORE–MD consortium will be presented at the Working Group on Clinical Investigation and Evaluation of the Medical Device Coordination Group, which is the statutory committee of all EU medical device regulatory agencies and which is coordinated by the European Commission (Unit B6 Medical Technology, in the Directorate General for Health, DG SANTE).
Potential contributions of CORE–MD across the lifecycle of medical device development and evaluation
The central horizontal panel illustrates the standard sequence for evaluating the evidence for a high-risk medical device in the EU. It does not show all the responsibilities of notified bodies. Figure adapted from Fraser et al.47
The green boxes correspond to specific tasks in CORE–MD.
CORE–MD, Coordinating Research and Evidence for Medical Devices; DG SANTE, Directorate General for Health; CEAR, clinical evaluation assessment report; CE, Conformité Européenne.
Citation: EFORT Open Reviews 6, 10; 10.1302/2058-5241.6.210081
Appendix: CORE–MD Investigators
ESC: Alan Fraser, Piotr Szymański, Chris Gale, Aldo Maggioni, Elisabetta Zanon, Christina Dimopoulou, Cinzia Ceccarelli, Polyxeni Vairami, Anett Ruszanov
EFORT: Per Kjærsgaard-Andersen, Rob Nelissen, Adrian Ott
BioMed Alliance: Elizabeth Macintyre, Loredana Simulescu, Marieke Meijer
EAP: Berthold Koletzko, Sarah Wieczorek, Adamos Hadjipanayis, Stefano Del Torso
Leiden University Medical Center: Perla Marang-van de Mheen, Lotje Hoogervorst, Ewout W. Steyerberg, Bas Penning De Vries
The University of Oxford: Peter McCulloch, Martin Landray, Daniel Prieto Alhambra, James Smith, Anne Lubbeke-Wolf
Uppsala Clinical Research Center: Stefan James, Sergio Buccheri
Royal College of Surgeons of Ireland: Robert Byrne, Laurna McGovern
Bern University Hospital: Stephan Windecker, Andre Frenk, Georgios Siontis, Christoph Stettler, Arjola Bano, Lia Bally
KU Leuven: Frank E. Rademakers, Jan D‘hooge, Anton Vedder, Elisabetta Biasin, Erik Kamenjasevic
UMIT: Petra Schnell-Inderst, Felicitas Kühne
University of Gothenburg: Ola Rolfson, Joel Jakobsson, Amanda Tornsö
Politecnico di Milano: Enrico G. Caiani, Lorenzo Gianquintieri, Cinzia Cappiello, Maristella Matera
HPRA: Tom Melvin, Niall MacAleenan, Ria Mahon, Michèle Meagher, Gearóid McGauran
DKMA: Thomas Wejs Møller, Ann-Sofie Sonne Holm-Schou
URPL: Jan Szulc
RIVM: Robert E. Geertsma, Jantine W.P.M. van Baal, Joëlle M. Hoebert, Susana L.F. Cabaço
ISS: Paola Laricchiuta, Marina Torre, Filippo Boniforti, Eugenio Carrani, Stefania Ceccarelli
AIHTA: Claudia Wild, Sabine Ettinger
AETSA: Juan Antonio Blasco Amaro, Juan Carlos Rejón Parrilla, Agnieszka Dobrzynska
University of Granada: David Epstein
EPF: Valentina Strammiello, Hannes Jarke, Kaisa Immonen
Team NB: Françoise Schlemmer, Sabina Hoekstra, Marianna Mastroroberto, Christoph Ziskoven, Michael Hahn, Erman Melikyan, Richard Holborow, Suzanne Halliday, Alexey Shiryaev, Gero Viola, Harry van Vugt
[Note: Abbreviations of consortium members explained in Table 2].
Open access
This article is distributed under the terms of the Creative Commons Attribution-Non Commercial 4.0 International (CC BY-NC 4.0) licence (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed.
None of the authors declare any conflict of interest.
European Union Horizon 2020 Research and Innovation Programme (Grant number 965246).
This paper is being published jointly in the European Heart Journal - Quality of Care and Clinical Outcomes.
This article is distributed under the terms of the Creative Commons Attribution-Non Commercial 4.0 International (CC BY-NC 4.0) licence (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed.
References
- 1.↑
European Union. Regulation (EU) 2017/745 of the European Parliament and of the Council of 5 April 2017 on medical devices, amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Regulation (EC) No 1223/2009 and repealing Council Directives 90/385/EEC and 93/42/EEC. http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:OJ.L_.2017.117.01.0001.01.ENG&toc=OJ:L:2017:117:TOC (date last accessed 22 September 2021).
- 2.↑
Fraser AG, Butchart EG & Szymański Pet al. The need for transparency of clinical evidence for medical devices in Europe. Lancet 2018;392:521–530.
- 3.↑
Commission Staff Working Document Impact Assessment on The Revision of The Regulatory Framework For Medical Devices, accompanying the documents Proposals for Regulations of the European Parliament and of the Council on Medical Devices, and amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Regulation (EC) No 1223/2009 and on in vitro diagnostic medical devices. SWD/2012/0273 final. https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX:52012SC0273 (date last accessed 16 June 2021).
- 4.↑
Hwang TJ, Sokolov E, Franklin JM, Kesselheim AS. Comparison of rates of safety issues and reporting of trial outcomes for medical devices approved in the European Union and United States: cohort study. BMJ 2016;353:i3323.
- 5.↑
European Commission. SC1-HCO-18-2020. Developing methodological approaches for improved clinical investigation and evaluation of high-risk medical devices. https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/topic-details/sc1-hco-18-2020 (date last accessed 17 May 2021).
- 6.↑
Council Resolution of 7 May 1985 on a new approach to technical harmonization and standards (85/C 136/01). OJ C 136, 4.6.1985, pp.1–9. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:31985Y0604%2801%29 (date last accessed 22 September 2021).
- 7.↑
European Commission. Commission Notice of 26 July 2016: The ‘Blue Guide’ on the implementation of EU products rules (2016/C 272/01). https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52016XC0726(02)&from=BG (date last accessed 21 June 2021).
- 8.↑
European Commission. Lists of harmonised standards for medical devices. Official J EU 2020;63:1–39. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=OJ:L:2020:090I:TOC (date last accessed 21 June 2021).
- 9.↑
European Union. Regulation (EU) No 536/2014 of the European Parliament and of the Council of 16 April 2014 on clinical trials on medicinal products for human use, and repealing Directive 2001/20/EC. https://ec.europa.eu/health/sites/default/files/files/eudralex/vol-1/reg_2014_536/reg_2014_536_en.pdf (date last accessed 17 June 2021).
- 10.↑
Clinical Trials Regulation (EU) No 536/2014 – Questions & Answers Version 2.3. Annex I: Decision tree to establish whether a trial is a “clinical trial”. February 2021. https://ec.europa.eu/health/sites/health/files/files/eudralex/vol-10/regulation5362014_qa_en.pdf (date last accessed 2 July 2021).
- 11.↑
International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH). Integrated Addendum to ICH E6(R1): Guideline for Good Clinical Practice E6(R2). Current Step 4 version dated 9 November 2016. https://database.ich.org/sites/default/files/E6_R2_Addendum.pdf (date last accessed 21 June 2021).
- 12.↑
International Standardization Organisation. ISO 14155:2020. Clinical investigation of medical devices for human subjects: good clinical practice. https://www.iso.org/standard/71690.html (date last accessed 21 June 2021).
- 13.↑
Moher D, Shamseer L, Clarke Met al; PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015;4:1.
- 14.↑
Gersh BJ, Fisher LD & Schaff HVet al. Issues concerning the clinical evaluation of new prosthetic valves. J Thorac Cardiovasc Surg 1986;91:460–466.
- 15.↑
Wu Y, Butchart EG, Borer JS, Yoganathan A, Grunkemeier GL. Clinical evaluation of new heart valve prostheses: update of objective performance criteria. Ann Thorac Surg 2014;98:1865–1874.
- 16.↑
Organisation for Economic Co-operation and Development. Health at a glance. OECD Indicators, 2019:chap 2. https://doi.org/10.1787/4dd50c09-en (date last accessed 22 September 2021).
- 17.↑
Weszl M, Rencz F, Brodszky V. Is the trend of increasing use of patient-reported outcome measures in medical device studies the sign of shift towards value-based purchasing in Europe? Eur J Health Econ 2019;20:133–140.
- 18.↑
International classification of functioning, disability, and health: ICF. Geneva: World Health Organization, 2001.
- 19.↑
Motte AF, Diallo S & van den Brink Het al. Existing reporting guidelines for clinical trials are not completely relevant for implantable medical devices: a systematic review. J Clin Epidemiol 2017;91:111–120.
- 20.↑
Sedrakyan A, Marinac-Dabic D, Normand SL, Mushlin A, Gross T. A framework for evidence evaluation and methodological issues in implantable device studies. Med Care 2010;48:S121–S128.
- 21.↑
Schnell-Inderst P, Hunger T, Conrads-Frank A, Arvandi M, Siebert U. Recommendations for primary studies evaluating therapeutic medical devices were identified and systematically reported through reviewing existing guidance. J Clin Epidemiol 2018;94:46–58.
- 22.↑
Reith C, Landray M & Devereaux PJet al. Randomized clinical trials: removing unnecessary obstacles. N Engl J Med 2013;369:1061–1065.
- 23.↑
Gribben J, Macintyre E & Sonneveld Pet al. Reducing bureaucracy in clinical research: a call for action. HemaSphere 2020;4:e352.
- 24.↑
McCulloch P, Altman DG, Campbell WBet al; IDEAL Collaboration. No surgical innovation without evaluation: the IDEAL recommendations. Lancet 2009;374:1105–1112.
- 25.↑
Collins R, Bowman L, Landray M, Peto R. The magic of randomization versus the myth of real-world evidence. N Engl J Med 2020;382:674–678.
- 26.↑
James S, Rao SV, Granger CB. Registry-based randomized clinical trials: a new clinical trial paradigm. Nat Rev Cardiol 2015;12:312–316.
- 27.↑
Fröbert O, Lagerqvist B, Olivecrona GKet al; TASTE Trial. Thrombus aspiration during ST-segment elevation myocardial infarction. N Engl J Med 2013;369:1587–1597.
- 28.↑
Buccheri S, Sarno G & Fröbert Oet al. Assessing the nationwide impact of a registry-based randomized clinical trial on cardiovascular practice. Circ Cardiovasc Interv 2019;12:e007381.
- 29.↑
Wolf O, Sjöholm P, Hailer NP, Möller M, Mukka S. Study protocol: HipSTHeR – a register-based randomised controlled trial – hip screws or (total) hip replacement for undisplaced femoral neck fractures in older patients. BMC Geriatr 2020;20:19.
- 30.↑
Ragnarsson S, Janiec M & Modrau ISet al. No-touch saphenous vein grafts in coronary artery surgery (SWEDEGRAFT): rationale and design of a multicenter, prospective, registry-based randomized clinical trial. Am Heart J 2020;224:17–24.
- 31.↑
Ioannidis JP, Adami HO. Nested randomized trials in large cohorts and biobanks: studying the health effects of lifestyle factors. Epidemiology 2008;19:75–82.
- 32.↑
The Good Clinical Trials Collaborative: new proportionate guidance for randomised controlled trials (RCTs). https://www.goodtrials.org/ (date last accessed 22 September 2021).
- 33.↑
European Commission. Brussels, 8.4.2019. COM(2019) 168. Building trust in human-centric artificial intelligence. https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52019DC0168&from=BG (Date last accessed 25 June 2021).
- 34.↑
Almond CS, Chen EA & Berman MRet al. High-risk medical devices, children and the FDA: regulatory challenges facing pediatric mechanical circulatory support devices. ASAIO J 2007;53:4–7.
- 35.↑
United Nations General Assembly. Convention on the Rights of the Child. November 20, 1989. Resolution 25, session 44. New York: The Office of the United Nations High Commissioner for Human Rights (OHCHR), 1989.
- 36.↑
Hwang TJ, Kesselheim AS, Bourgeois FT. Postmarketing trials and pediatric device approvals. Pediatrics 2014;133:e1197–e1202.
- 37.↑
American Academy of Pediatrics Policy Statement. Section on cardiology and cardiac surgery, section on orthopedics. Off-label use of medical devices in children. Pediatrics 2017;139:e20163439.
- 38.↑
Gore R, Chugh PK, Tripathi CD, Lhamo Y, Gautam S. Pediatric off-label and unlicensed drug use and its implications. Curr Clin Pharmacol 2017;12:18–25.
- 39.↑
Fraser AG, Burch M. Establishing evidence for high-risk medical devices in orphan diseases. Eur J Cardiothorac Surg 2013;44:840–842.
- 40.↑
Neugebauer EAM, Rath A & Antoine SLet al. Specific barriers to the conduct of randomised clinical trials on medical devices. Trials 2017;18:427.
- 41.↑
Niederländer C, Wahlster P, Kriza C, Kolominsky-Rabas P. Registries of implantable medical devices in Europe. Health Policy 2013;113:20–37.
- 42.↑
European Commission website. Medical Devices – EUDAMED: Overview. https://ec.europa.eu/health/md_eudamed/overview (date last accessed 22 September 2021).
- 43.↑
Fraser AG, Szymański P, Macintyre E, Landray M. Regulating drugs, medical devices, and diagnostic tests in the European Union: early lessons from the COVID-19 pandemic? Eur Heart J 2020;41:2140–2144.
- 44.↑
Reckers-Droog V, Federici C, Brouwer W, Drummond M. Challenges with coverage with evidence development schemes for medical devices: a systematic review. Health Policy Technol 2020;9:146–156.
- 45.↑
Wild C, Sauerland S, Schnell-Inderst P. Closing the gap between regulatory and HTA requirements for approval and reimbursement of high-risk medical devices in Europe. J Med Dev Reg 2017;14:27–40.
- 46.↑
Erdös J, Ettinger S, Mayer-Ferbas J, de Villiers C, Wild C. European Collaboration in Health Technology Assessment (HTA): goals, methods and outcomes with specific focus on medical devices. Wien Med Wochenschr 2019;169:284–292.
- 47.↑
Fraser AG, Byrne RA & Kautzner Jet al. Implementing the new European Regulations on medical devices – clinical responsibilities for evidence-based practice: a report from the Regulatory Affairs Committee of the European Society of Cardiology. Eur Heart J 2020;41:2589–2596.
- 48.↑
Medical Devices Coordination Group. MDCG 2020-13: Clinical evaluation assessment report template, July 2020. https://ec.europa.eu/health/sites/default/files/md_sector/docs/mdcg_clinical_evaluationtemplate_en.pdf (date last accessed 22 September 2021).
- 49.↑
European Commission. Clinical evaluation: a guide for manufacturers and notified bodies under Directives 93/42/EEC and 90/385/EEC. MEDDEV 2.7/1 revision 4. June 2016. http://ec.europa.eu/DocsRoom/documents/17522/attachments/1/translations/ (date last accessed 4 October 2017)).
