Identification of implant outliers in joint replacement registries

Introduction

Total hip and knee replacement are effective operations for the management of end-stage arthritis. There are increasing numbers of these operations being performed, and the rate of increase is anticipated to continue into the future (1, 2, 3, 4). There is also a substantial rise in the lifetime risk of a person receiving a total hip or knee replacement and this has been shown in several countries (5, 6, 7, 8). There are a large number of joint replacement prostheses on the market available for use and not all perform the same. Many have no published outcomes. Joint replacement registries provide an appropriate way to monitor the real-world outcomes of these procedures and can provide comparative data on the rates of revision for specific prostheses. Revisions are generally defined in joint registries as a removal or exchange of a prosthesis or part thereof and provide an unambiguous record of a problem with the joint replacement. There are many factors that affect revision rates. Non-device-related issues may include patient factors, surgical technique, surgeon experience and volume of cases. Device-related factors may contribute to the variation in the rates of revision with individual prostheses.

Prosthesis outcomes have received closer attention following the high rate of revision and subsequent recall of the ASR Hip Resurfacing System and ASRXL Acetabular System. Over 93 000 patients were implanted with these prostheses and the outcome of these was shown to be device-related, independent of multiple other possible causes of a higher rate of revision (9). Concerns regarding the outcomes of the ASR Resurfacing System were first identified by the Australian Orthopaedic Association National Joint Replacement Registry (AOANJRR) in 2007 (10) and the ASRXL Acetabular System in 2008 (11). Following confirmation by the National Joint Registry for England, Wales, Northern Ireland, Isle of Man and the States of Guernsey (NJR) (12), these prostheses were withdrawn worldwide in 2010. This resulted in tighter regulations regarding the introduction of monitoring for new implants (13) demonstrating the critical role of registries in post-market total joint replacement surveillance.

While many joint replacement registries report comparative revision rates of prostheses, it is much less common for registries to publicly highlight specific prostheses or prostheses combinations that are performing outside of the expected norm. The purpose of this review is to determine if joint replacement registries identify prostheses that have a higher than expected rate of revision, to describe the current outlier methodologies and discuss the consequences of this and recommend ways in which the international registry community may cooperate to enhance future surveillance opportunities.

Methods

A detailed search was performed of all joint replacement registries listed on the official websites of the International Society of Arthroplasty Registries (ISAR) (14) and Arthroplastywatch (15). In addition, all links from those registries were also evaluated to include smaller regional registries. Available online reports were reviewed to determine if individual registries specifically identified prostheses with a higher than expected rate of revision and, if so, the method by which this was performed. Those registries that did not have an accessible online document were contacted to obtain a hard copy version or, if not available, personal communication was made to the relevant registry contact to determine if they identified prostheses, but the information was not publicly available. The registries that were accessed are listed in Appendix 1.

Results

There were a total of 36 registries with websites identified, with 5 reporting prostheses with higher rates of revision than expected. These were the AOANJRR, the NJR, the New Zealand National Joint Registry, Swedish Arthroplasty Registry and the Swiss National Hip and Knee Joint Registry (SIRIS) that identified prostheses in their report and the methods by which this was performed. There were three registries that identified prostheses internally without the information being publicly available. These were the Dutch Arthroplasty Register (LROI), the Michigan Arthroplasty Registry Quality Initiative (MARCQI) and the Kaiser Permanente National Total Joint Replacement Registry. The first two plan to publish the results in their next publicly available report.

The AOANJRR has previously reported on a method for identifying prostheses with a higher than expected rate of revision (16). This involves a three-stage process commencing with an automated screening test to identify prostheses that have twice the rate of revision per 100 observed component years of all other prostheses in the same class. The second stage involves a more detailed analysis of the identified prostheses by the AOANJRR Registry staff. Age- and gender-adjusted hazard ratios are calculated using Cox regression models. If the hazard ratio of a prosthesis, compared to all others in the same class combined, is statistically significant, then the prosthesis progresses to stage 3. In this stage, a panel of independent orthopaedic specialists from the Australian Orthopaedic Association Arthroplasty Society analyses all the data and determines which prostheses will be identified in the annual report. Prostheses or prosthetic combinations are then listed as ‘Identified and not used’, ‘Identified and still used’ and ‘Newly Identified’.

The NJR (17) has an Implant Performances Sub-Committee whose brief is to analyse and assess confidential data on potential outliers. This analysis is performed on a patients time incidence rate (PTIR), which is the revision ratio per 100 observed component years, compared against the prosthesis group, as first introduced by the AOANJRR. Notification for an unacceptably high rate of revision is a PTIR of at least twice the group PTIR, allowing for confidence intervals (level 1 notification). When this occurs, a report is filed with the Medicines and Healthcare Products Regulatory Agency (MHRA) and the manufacturer is notified. When the PTIR is 1.5 times the group PTIR, a warning letter is sent to the manufacturing company (level 2 notification).

The New Zealand Joint Registry has a table titled Revision versus Hip Prosthesis combinations. This lists contain prostheses combinations that have a minimum of 50 registered primary arthroplasties and they are sorted in order of descending revisions per 100 observed component years along with the number of procedures performed in the previous year. It forms the basis for identifying devices with a higher rate of revision and these are identified in the report (18).

The Swedish Knee Arthroplasty Registry (SKAR) reports on factors that influence the revision rate of knee replacements (19). When the implant model is the factor, the Cox regression analysis adjusts for differences in gender, age and diagnosis and uses the latest 10-year period for the analysis. The SKAR for many years used the AGC as the reference model with a risk of one to which other implants were compared. In the 2014 report, the reference model was changed to the PFC Sigma-MBT. In 2021, the Swedish Arthroplasty Registry combined both hip and knee data and introduced a similar method of evaluating acetabular cups and femoral stems as SKAR had previously done.

The Swiss National Hip and Knee Joint Registry uses similar methodology to the AOANJRR with the comparator being all prosthesis combinations that have similar features to the device being examined, i.e. if the device is a cementless stem/cup combinations in patients with procedures performed for primary osteoarthritis. The outlier alert boundary is set at more than twice the reference revision rate. All potential outliers are evaluated and discussed by the SIRIS Scientific Advisor y Board, and for each of these implants, a separate outlier analysis is conducted and an outlier report was written. When the results of the analyses suggest a justifiable need for action, the SIRIS Scientific Advisory Board changes the outlier’s status from ‘potential outlier’ to ‘confirmed outlier’ (20).

A table listing all total hip and total knee implants identified by the above registries to date is provided in Supplementary Tables 1 and 2 (see section on supplementary materials given at the end of this article). However, acurrent list of devices that have been identified can be found in the respective annual reports of the above registries.

The Dutch Arthroplasty Register has performed an Outlier Procedure for prostheses from 2019 (21). Possible outliers were detected using funnel plots with 99.8% control limits with the prosthesis volume on the x-axis and the outcome measure on the y-axis. The outcome measures used were the 3-year overall and major (exchange of at least one of the fixed components) revision rate using LROI data. Possible outliers were defined as prostheses above the upper control limit of the funnel plot. All total knee and hip arthroplasties for osteoarthritis in the period 2007–2019 were included. A detailed report was made based on LROI data. The manufacturing company was informed about the possible outlier status of their prosthesis and an explanation was requested. A subcommittee of experts from the LROI and the Committee of Orthopaedic Implants of the Netherlands Orthopaedic Association (NOV) examined the LROI results as well as the manufacturers’ explanation and the LROI board was informed. The LROI board advises the NOV board whether the outcome should be communicated.

MARCQI utilizes the published AOANJRR methodology for outlier detection and is supplementing this with risk-adjusted funnel plots by hospital and surgeon (22). An expert panel has been assembled to review the first round of analyses, and the registry plans to publish the results in the MARCQI Annual Report. The registry makes this report publicly available on their web site. MARCQI has been coordinating this effort with the Advanced Medical Technology Associate (AdvaMed), the trade group of manufacturers of medical implants, to ensure consistent communication across individual manufacturers.

The Kaiser Permanente Registry has a medical device surveillance committee which oversees all devices and, for joint replacement, uses a similar methodology to the AOANJRR providing reports of prostheses performance to surgeons within the organization to guide implant selection and organizational product contract decision-making (23).

One other registry, the Scottish Arthroplasty Project (SAP) does not formally collect implant data but monitors operations and subsequent complications. As outlier status is frequently associated with poor implants, SAP states that their methods are applicable for indirect implant surveillance (24).

Discussion

The main objective of the study was to determine if joint replacement registries identified prostheses with higher than expected rate of revision and how this was performed. There were five registries with publicly available information on the devices and the threshold for reporting. While most registries report the survivorship or rates of revision of individual prostheses and combinations, there is no formal policy for the identification of specific prostheses. A post-market surveillance system for medical devices should provide the following functions: readily identify underperforming devices, characterize and disseminate information about real-world performance and provide data that can be used to support pre-market clearance or approval of new devices (25). Not all implants are identified across all the registries for a variety of reasons. These include the diverse use of prostheses among different countries, variation in the use of fixation methods and enough numbers of devices implanted to allow appropriate identification.

Surgeons rely on many sources of information when deciding which prosthesis to use and these include, but are not limited to, experience in training, colleague interaction, peer-reviewed literature, scientific meetings, company-sponsored events, and joint replacement registries. A considerable proportion of prostheses available have no readily available evidence of clinical effectiveness to support their use (26). Joint replacement registries, with their continuous surveillance, provide the best data for the use and outcomes of a device in the general population (27, 28, 29). Careful interpretation of this can help guide prosthesis selection in the absence of published evidence (30, 31, 32), and identification of specific outlier prostheses, as opposed to listing comparative rates of revision, highlights prostheses that are not performing as well as others within their class.

The need for registries to identify outliers along with well-performing implants is therefore important, but there are problems with current approaches. As revision is a relatively rare occurrence, some prostheses may be used in low volumes and there may be insufficient power to detect differences.

Moreover, the minimum number of implants required for the publication of implant-specific revision rates in an annual report varies widely (e.g. NJR 2500 procedures; AOANJRR 500 procedures; SIRIS 50 procedures), limiting the result aggregation of implants, which are newly introduced or generally used in lower numbers (e.g. implants for more specific indications such as dual-mobility cups). There is also a lack of standardization of implants and their attributes which may hamper international comparisons. Registries may have different prostheses in their databases, limiting the ability to link data. The comparator and the threshold for identification also differ. Four registries use twice the rate of revision of all other devices in the same class for initial listing, one uses 1.5 times, one uses a single prosthesis as the comparator and one lists in order of revisions per 100 observed component years. Finally, both the timing and the best methods to disseminate findings to the relevant stakeholders need to be addressed.

There are several potential solutions to the above issues to further improve the identification of outlier prostheses and post-market surveillance of implants. Registries can consider pooling data to increase the numbers available for statistical analysis. The Nordic Arthroplasty Register Association was established in 2007 by Sweden, Norway and Denmark to improve collaboration and was joined by Finland in 2010. This enabled a greater number of prostheses with a longer-term follow-up (1995–2011) to be analysed (33). While there are examples of registries pooling data to examine the outcomes of specific prostheses (34), this has proven more difficult with regards to outlier identification. The International Consortium of Orthopaedic Registries commenced in 2011 and focused on two major goals: research and surveillance for hip and knee implants and worldwide implant harmonization. The consortium involved over 30 orthopaedic registries and has performed multinational investigations of total hip replacement bearing surfaces, prosthesis fixation and total knee replacement outcomes with respect to mobile and fixed bearings and stabilization. This initiative has demonstrated that registries worldwide can cooperate to monitor and improve the outcomes of joint replacements (35).

The ISAR can play a significant role in coordinating data to aid in the early identification of outliers. If international registries are to compare results, then it is essential that a similar minimum dataset is collected (36) and there is harmonization of the device catalogues between registries. One of the current objectives of the registry community is the development of a standardized component catalogue that is made available to all registries thereby allowing the comparison of similar devices. This initiative will lead to an improvement in early signal detection by close operation and the sharing of data on prostheses that have been potentially flagged by individual registries but need larger numbers for accurate analysis. Equally essential is the harmonization of registry reporting (a minimum common output regarding implant-specific outcomes) to improve comparison, aggregation and/ or linkage (when feasible). This is a further requisite for more efficient early signal detection.

The most common comparator used is all prostheses in the same class. The use of the whole implant class or group may lead to a partial masking of prostheses that are at the higher end of the revision scale (as they are themselves part of the class), and a comparison to a group of the best-performing implants may be more appropriate. Using a single implant as comparator may also present some difficulties. Comparing to the most commonly used prosthesis is one method, but this may not necessarily be the best-performing implant in a registry. Also, the use of implants changes with time, as can the performance, and another device will need to be chosen as the comparator.

There is also a concern that the organization of implants by category or class could result in the poor result of one subset of those implants being hidden by the overall results of the rest of the group. This has been termed ‘camouflaging’ of implants and range analysis will become more important. Recent data to support this have found higher revision rates for certain sizes of a particular implant (37). Other implants have different surface finishes available for the same implant that may not be differentiated in the implant library. Variations in manufacturing or surface finish could also lead to this effect.

The timing of release of information on outlier prostheses is also an important factor to consider. The release of information in an annual report may come many months after a decision has been made on prostheses with higher than expected rate of revision. Websites that provide real-time data for surgeons, industry and regulators on the performance of prostheses allow for closer monitoring of joint replacement rates of revision and may alert users to seek further, more detailed reports. The timing of release of outlier prosthesis identification requires a consistent approach to be certain of the accuracy of the data, while at the same time, being aware that a delay in notification may put patients at risk and arthroplasty watch is such a website devoted to timely release of information regarding issues with all types of joint replacement. It was developed as an information project and opened in February 2013 with the purpose of collecting data on arthroplasty safety issues from a wide range of information sources on the internet and disseminating this in one single, publicly accessible site (15). The sources include arthroplasty registries, reports from regulatory authorities, manufacturers and scientific publications, but there is no formal method for outlier identification.

There is no question that outlier identification plays an important role in improving the outcomes of joint replacement. This has been demonstrated by the marked reduction or cessation of use for most prostheses identified by the national registries. The consequence of this identification has been the reduced exposure of patients to devices with higher than expected revision rates. Only a small percentage of devices are identified and this does not impact surgeon choice, as there is ample evidence of many prostheses with long-term low rates of revision that surgeons can use for their patients. When registries identify devices, there are variable arrangements to report these to the appropriate regulatory bodies. The AOANJRR works closely with the Therapeutic Goods Administration (TGA) which is the regulatory body for reporting devices in Australia, and every device identified in the annual report is investigated further by the TGA. The NJR has a similar process whereby devices are reported to both the industry and also to the Medicines and Healthcare Products Regulatory Agency (MHRA). Other registries do not have specific mechanisms to report to their respective bodies though this may alter in the future.

Another solution to avoid using devices with higher than expected revision rates would be to only use prostheses with good long-term outcomes. The Orthopaedic Data Evaluation Panel (ODEP) was set up as part of the United Kingdom National Health Service Purchasing and Supply Agency to monitor data for primary hip replacement. Prostheses are classified according to the level of evidence spanning a time period, with the highest rating being a 15A (38). To qualify for this rating, a hip prosthesis needs to have at least 15 years of outcomes data, the letter A indicates that at a minimum 10 years follow-up, the Kaplan–Meier revision rate is <10% or better and A* indicates a revision rate of <5% at that time point in a cohort study of a minimum 500 prostheses. The AOANJRR also lists THRs and TKRs with 15- and 20-year rates of revision which can provide a guide to well-performing implants in the long term. However, if this approach is followed, it may not allow for innovation. New prostheses can still be introduced if they participate in a post-marketing surveillance programme such as Beyond Compliance, the initiative in the United Kingdom (39). The International Society of Arthroplasty Registries has developed a method for benchmarking devices, and this is available on the ISAR web site (40).

There are some limitations to this review on registry identification of outlier prostheses. Despite a thorough search, there may be small regional registries that have local publications which are not readily available for review. A comprehensive attempt was made to read reports or contact all registries identified. Also, registries that were reviewed may not publicly identify prostheses but may do so internally and communicate results to hospitals and surgeons thereby influencing outcomes at a local or regional level. There may also be medicolegal issues in some countries that may impact the ability to identify prostheses in an annual report.

Conclusion

The registry experience is that early signal detection of prostheses with a higher than expected rate of revision can lead to markedly reduced use and withdrawal of these devices from the marketplace. Consistent reporting of outlier prostheses from registries across countries would make it less likely this was due to patient or surgeon factors. Further research is needed to determine the optimum methods for identification, including the threshold, the comparator and the numbers required for notification of devices. Collaboration and co-operation of registries at a global level will enhance this process thus reducing adverse outcomes for patients.

Supplementary materials

This is linked to the online version of the paper at https://doi.org/10.1530/EOR-22-0058.

ICMJE Conflict of Interest Statement

The authors declare that there is no conflict of interest that could be perceived as prejudicing the impartiality of the research reported.

Funding

No benefits in any form have been received or will be received from a commercial party related directly or indirectly to the subject of this article.