Literature Reviews for Medical Devices: 6 Expert Tips

Catarina Carrão, freelance medical writer on Kolabtree, outlines the importance of literature reviews for medical devices and best practices to follow. 

A clinical evaluation is an ongoing process, conducted throughout the life cycle of a medical device. It is usually first performed during the development phase of the medical device, in order to identify the data that it needs to be granted market access. In the European Union, for an initial CE-marking, a Clinical Evaluation Report (CER) is mandatory, and it must be actively updated continuously afterwards[1]. In the United States, a Pre-Market Approval (PMA) [2] is the Food and Drug Administration (FDA) process for scientific and regulatory review, to evaluate the safety and effectiveness of a medical device (Class III), so that it can reach the consumer. It also uses an evidence-based review system for scientific evaluation of medical devices.

This process of clinical evaluation is fundamental, because it ensures the safety and performance of the device based on abundant clinical evidence, throughout the lifetime of the medical device on the market. It enables Notified Bodies (NBs) and Competent Authorities to read through the clinical evidence to demonstrate the conformity of the device with the essential requirements, not just for initial marketing, but throughout its lifetime (e.g., fulfilment of post-market surveillance and reporting requirements) [1].

Literature Reviews for Medical Devices

Literature reviews are crucial to the success of a CER and PMA, because a solid and systematic literature research strategy fortifies every stage of the medical device life cycle process: from concept and design, through clinical trials to release of the medical device and reimbursement [3]. So, more than just a wise investment, the screening of the literature to comply with regulatory authorities during the approval process and for post-market surveillance, is fundamental to the global success of any marketed medical device.

For many companies, especially Small and Medium Enterprises (SMEs), the data retrieved from literature searches will represent most, if not all, of the data collected. As such, this search identifies sources of clinical data for establishing the current knowledge or “the-state-of-the-art” that describes the clinical background in the corresponding medical field; the clinical data that is relevant to the device under evaluation, or to an equivalent device (if equivalence is claimed in a CER1, or 510K[4]); and, the identification of potential clinical hazards. That’s why it is so critically important to develop a literature search strategy that is robust, and can be replicated during subsequent updates by any person.

1. Search protocol (Stage 1)

The searching strategy should be thorough and objective, i.e. it should identify all relevant favourable and unfavourable data; and, should be carried out based on a search protocol [1]. The search protocol documents the planning of the search before execution. Once the searches have been executed, the adequacy of the searches should be verified, and a literature search report should be compiled to present details, with any deviations from the literature search protocol documented, together with the results of the search. It is important that the literature search is documented to such degree that the methods can be appraised critically, the results can be verified, and the search reproduced if necessary.

According to the regulations [1], [2], the literature search protocol should include the following elements [5]:

• Sources of data used (e.g., MEDLINE/PubMed, Embase, Google Scholar, ResearchGate, internet searches, etc.);
• The methodology used for the searches;
• The exact search terms and parameters used to search scientific databases (e.g., dates);
• Specific selection or exclusion criteria along with justifications for each;
• How was duplication of data from multiple sources addressed;
• How was data integrity ensured (e.g., Quality Control Methods or second reviewers);
• How each data source was appraised, and its relevance for the specific device;
• Analysis and data processing handling.

The search strategy must be broad enough to ensure that no essential information is missed, but still allow precise identification of relevant results. This can involve the use of search features such as filters to narrow down the result set; sub-headings based on key concepts such as device adverse effects, or device comparison; and triage and analysis methods to identify the most relevant literature. The results themselves are generally in the form of a list of citations or data, with descriptive indexing tags and other key information.

Abstracts lack sufficient detail to allow issues to be evaluated thoroughly and independently, but may be sufficient to allow a first evaluation of the relevance of a paper [1]. Good research informatics solutions allow both flagging of the citation and annotation of the article text, so that teams can work closely on individual items [3]. Copies of the full text papers should be included in the final files. The literature search protocol(s), the literature search report(s), and full text copies of articles and relevant documents, become part of the final technical documentation for the medical device.

2. Possible errors

A precise literature search provides accurate evidence; but, unless implemented correctly, the result can be misleading, time consuming, or even useless [6]. There are errors related to the volume of evidence, relevance of the data, tone of evidence, and its value to the research topic, that might undermine even a high-skilled researcher. It is necessary to focus the literature search on precise topics, and obtain relevant evidence within a stipulated time, otherwise outcomes might deviate.

Usually, errors have their origin in an incorrect use of primary attributes of literature search, viz., keywords, Boolean, and database [6]. For example, the evaluator could create errors in setting eligibility criteria (type of literature and databases); or errors in selecting keywords and Boolean logics; or even, errors in setting up search phrases in the database.

These attributes can lead to errors of inclusion (too much data, partly not relevant to the issue); or, exclusion of important data, because of too stringent keyword use. But it can also lead to errors of “inclusive exclusions”, due to bias by literature professionals in the searches; and, “exclusive inclusions”, with the use of highly specific key-terms with inadequate Booleans, or even the exclusion of synonyms for the same medical terminology. An error of “exclusive exclusions“ is also called the „error of limited relevance“, and it happens in many cases. This error is a combination of bias and specific exclusiveness; where the search phrases constructed will be biased to only one-sided data trends, and the terms selected will be too exclusive to return sufficient information [6].

Besides possible errors in retrieving important clinical data, uncertainty of the final literature review also arises from two sources: the methodological quality of the data, and the relevance of the data to the evaluation of the device in relation to the different aspects of its intended purpose [1]. Both sources of uncertainty should be analysed, in order to determine a weighting for each data set. As such, a balanced assessment of the quality of the data is essential to the success of the literature review search.

Also read: Clinical evaluation for EU MDR Compliance: 5 Dos and Don’ts 

3. Appraisal of the clinical data

When appraising the data generated by the database search (Stage 2), the evaluator is looking to make sure it has statistically significant data sets, uses proper statistical methods, has adequate controls, and properly collects mortality and/or serious adverse event data. It is essential that the correct assessment is done based on the complete text of the publications found, not just by reading the abstracts or summaries. For each document appraised, there needs to be a documentation of the appraisal to the point that it could be reasonably reviewed by others. The appraisal results should also support conclusions about the clinical safety and performance of the finished device (e.g., citing non-device-related literature would be ranked low for appraisal)5. There are some red flags provided by the regulation in order to appraise the medical publications, for example:

• The article lacks basic information such as the methods used, number of patients, identity of products, etc.;
• Has data sets that are too small to be statistically significant;
• Contains data that applies improper statistical methods;
• Employs studies that lack adequate controls;
• Has an improper collection of mortality and serious adverse event data;
• Depicts a misrepresentation by the authors;

The evaluators should verify whether clinical investigations have been defined in such a way as to confirm or refute the manufacturer’s claims for the device; and, whether these investigations include an adequate number of observations to guarantee the scientific validity of the conclusions [1]. Some papers considered unsuitable for demonstration of adequate performance because of poor elements of the study design or inadequate analysis, may still contain data suitable for safety analysis, or vice versa.

Typically, clinical data should receive the highest weighting, when generated through a well designed and monitored randomized controlled clinical investigation (also called randomised controlled trial), conducted with the device under evaluation in its intended purpose, with patients and users that are representative of the target population [1]. It is acknowledged by the regulators that randomized clinical investigations may not always be feasible and/or appropriate, and the use of alternative study designs may provide relevant clinical information of adequate weighting. When rejecting evidence, the evaluators should document the reasons.

4. Analysis and conclusions generated from the clinical data

During the analysis stage (Stage 3), a comprehensive assessment is done to determine if the data found actually meets the clinical safety requirements, clinical performance requirements, and General Safety and Performance Requirements (GSPR). It is important to evaluate if the risk-benefit ratio of the medical device is appropriate based on the intended purpose of the device, or if the device can actually achieve all performance claims made by the manufacturer. Also, if the materials supplied by the manufacturer (labelling/instructions) are adequate to describe the intended purpose and mitigate the risk [7]. All in all, the evaluation is intended to conclude whether the risks of the device are minimal and acceptable according to its purpose. As such, understanding the interaction between the device and body, the number and severity of adverse events, and the current standards of care, are some of the gaps that will need to be taken into account [1].

The data from the literature is often put into Excel tables, which is a convenient way to compare different study details, patient populations, endpoints, adverse events, etc. [7]. This is extremely helpful in noting differences between studies when writing the summary and conclusions. The evaluators should also include aspects such as rare complications, uncertainties regarding medium- and long-term performance, or safety under wide-spread use; and, identify additional clinical investigations, or other measures, that are necessary in order to generate any missing data [1].

5. Informatic tools

This massive task of literature search can be streamlined with the right research informatics solutions. Nowadays, the key is to select a literature database with appropriate medical device coverage, in terms of content and indexing. In a case study reviewing literature about a particular medical device, the top research informatics solutions were Embase and Science Citation Index (SCI) [3]. But also, Medline, and BioMed Central are considered top research informatic solutions for retrieving medical device literature and can be used by anyone.

Using a solution that performs automated searches and notifies the user of relevant new data via email alerts or RSS feeds saves considerable time; and, keeps the evaluator updated until the final stage of submission [3]. If the tool has the appropriate indexing and tagging, this will simplify literature triage; and, more importantly, it dramatically reduces the risk of missing adverse event reports. The combination of a good literature search tool and a trained evaluator can be the best solution to avoid errors and limitations of literature review search.

Also read: Writing a Clinical Evaluation Report: 5 Quick Tips

6. Process flow

Modern regulatory requirements have made biomedical literature research an essential part of the medical device life cycle; as such, a good strategy to find and summarize all the relevant clinical data is a must:

• Identify the question to be answered;
• Decide which database better fits the question,
• Identify the Medical Subject Headings (MeSH), or Embase Subject Headings (EmTree);
• Recognize the correct Boolean terms to use;
• Create and document the Literature Search Protocol;
• Run the Literature search automation tool;
• Appraise and analyse the literature (tabulation),
• Summarize conclusions.

Conclusions

The future looks promising for the dauting task of doing a systematic literature review, since artificial intelligence and natural language processing based-tools with cognitive capabilities provide a near-perfect solution [6]. But, until then, an organized and highly-skilled research-evaluator is essential to execute a dedicated strategy for literature monitoring, triage and analysis.

Need help with literature reviews for medical devices? Hire clinical evaluation experts and literature search specialists on Kolabtree. 

References:

1. European Commission. CLINICAL EVALUATION: A GUIDE FOR MANUFACTURERS AND NOTIFIED BODIES UNDER DIRECTIVES 93/42/EEC and 90/385/EEC. MEDDEV 27/1 revision 4. 2016.
2. FDA. Premarket Approval (PMA). 2019;18th May 2020.
3. Elsevier. BOOSTING THE SUCCESS OF MEDICAL DEVICE DEVELOPMENT WITH SYSTEMATIC LITERATURE REVIEWS. 2014;18th May 2020.
4. FDA. Premarket Notification 510(k). 2020;19th May 2020.
5. OrielStat. Creating an EU CER Literature Review Protocol and Reviewing Medical Device Clinical Data. 2019;18th May 2020.
6. Ashish Indani* SRB, Nadeem Ansari. Literature Search for Scientific Processes in Medical Devices: Challenges, Errors, and Mitigation Strategies. Tata Consultancies. 2017;7.
7. OrielStat. Analyzing Your Medical Device Clinical Datasets and Drawing Conclusions. 2019;18th May 2020.