Doctor: Linda Nyanchoka
Title: Methods for identifying and displaying gaps in health research
Supervisor: Raphaël Porcher et Catrin Tudur Smith
Doctoral school: ED 393 Epidemiology and Biomedical Information Sciences, Université Paris Cité
Date of thesis defense: 07/05/2021
Jury: Jacques Demotes, Nicky Cullum, Corinne Alberti, James Thomas, Raphaël Porcher, Catrin Tudur Smith
Summary:
Statistics on the use of the available health research is troubling: an estimated 85% is wasted and more than 50% never gets to be published. From the published research, 50% is not usable in practice because items are missing. In addition, among the remaining half of the published research, 50% contains design flaws. Planning a study focusing on the wrong research question is a frequent cause of waste in research. Hence, published, unpublished, completed, and on-going research should be used to assess whether research gaps justify new research that can inform the design, conduct and reporting of further research.
The overall topic area of methods to identify and display gaps in health research is still not well established; also, there is no standard definition for the term “research gaps” nor standardized methods to identify research gaps. Furthermore, with a lack of a clear definition, consensus is lacking on what constitutes the best methodological approaches to identify research gaps, determine research priorities and display research gaps or priorities. Therefore, with an aim to improve understanding research gaps as a whole, and specifically defining, identifying and displaying research gaps, I undertook this PhD project. With the specific objectives to 1) identify different definitions reported for the term “research gap” and describe the methods used to identify, prioritize and display gaps in health research; 2) explore key stakeholders’ perspectives and experiences with defining, identifying and displaying gaps in health research; and 3) develop methodological guidance for identifying and displaying gaps in health research.
In the first project, I conducted a scoping review to map reported definitions of research gaps and methods to identify, prioritize and display gaps in health research. The study provided an overview of different definitions and methods used to identify, prioritize, and display gaps or priorities in health research. The most frequent methods in the review aimed at gap identification and involved secondary research, which included evidence synthesis (80/116 articles, 69%), specifically systematic reviews and scoping reviews (58/80 articles, 73%). Among studies aimed at research prioritization, the most frequent methods were combined primary and secondary research, accounting for 24 (49%) articles, followed by secondary methods, 8 (16%) articles. Finally, 37% articles described methods for displaying gaps and/or priorities in health research.
The scoping review revealed a comprehensive list of scientific articles that reported the definition, identification and/or prioritization, and display of research gaps. These articles related to defining research gaps and methods used to identify, prioritize, and display research gaps varied significantly across topic areas in health research. For the studies that reported on terms and definitions used to describe research gaps, I identified 12 different definitions based on the report title, yet with some similarities in the description provided for each. Three similar cross-cutting themes were identified: definitions related to missing information, inadequate information, and insufficient information. This analysis shows that despite the term “research gap” being commonly used in health research, its meaning can differ. Therefore, having a clear term and description supports the clarity in reporting health research and subsequently communication of what research gaps exist.
The methods to identify research gaps were more clearly presented than were the terms related to research gaps and the definitions used to describe the research gaps. The methods identified were grouped in three main categories: primary, secondary, and combined primary and secondary research. The most frequently used method was secondary methods, followed by both primary and secondary, then finally primary methods.
Among the secondary methods, knowledge synthesis was commonly used. Knowledge synthesis is an efficient scientific approach to identify and summarize evidence that allows for generalizability and consistency of the research findings to be assessed and data inconsistencies to be explored. The purpose of knowledge synthesis is to summarize all pertinent studies on a specific question, improve the understanding of inconsistencies in diverse evidence and identify gaps in research evidence to define future research agendas. Moreover, within knowledge synthesis, the scoping review is one of the only methods used to identify research gaps that explicitly included identification of research gaps as part of the purpose of the method.
I then classified the methods used to identify research gaps. The most frequent methods in the review were aimed at gap identification (including both identification and prioritization) and involved secondary research, including knowledge synthesis (80/116 articles, 69%), specifically systematic reviews and scoping reviews (58/80, 73%). Overall, 24/116 (21%) articles described the use of both primary and secondary research and 12/116 (10%) only primary research. I found 7 specific methods for identifying research gaps that are described, along with the purpose of the method and further elaborated in the thesis.
A mix of primary and secondary research was most frequently used to determine research prioritization, namely, priority setting (e.g., James Lind Alliance [JLA]-Priority Setting Partnerships methods). These methods involve the participation of patients, caregivers and healthcare and social-care professionals in identifying research questions, then prioritizing them by using a combination of primary and secondary research. The main method for determining research prioritization with primary research was the Delphi survey, which is a practical and productive approach to obtaining opinions from a wide number of relevant experts for identifying potential priority topic areas for research.
To display research gaps, half of the methods still used traditional ways to present findings (e.g., summary table and bar charts), and the other half used more advanced ways to display information (e.g., tree map charts, radial bar plots and bubble plots). The non-traditional methods used more colors and diagrams to present research gaps or priorities. For example, bubble plots use different shapes, sizes, and colors to display information and can be used to present up to three different variables in one diagram. These characteristics could be further explored to determine the appropriate method to be used along with specified methods to identify research gaps and/or determine research priorities.
The second project was a qualitative study involving semi-structured interviews of key informants to investigate their knowledge, perceptions, and experiences with defining research gaps and characterizing methods/practices used for identifying and displaying gaps in health research. The results provided evidence on what participants reported as gaps in health research: the terms ranged from “lack of information”, “inadequate information”, “insufficient information”, “quality of evidence” and “treatment uncertainty”. The study showed detailed participants’ experiences with and perceptions of different research methodologies used (i.e., primary, secondary, both). The variety of identified methods reflected the state of the field in the sense of the wide array of methods currently used with no clear consensus or guidance. Researcher participants also expressed a difficulty in systematically identifying research gaps. With experiences in displaying research gaps, participants expressed the importance of data visualization and the difficulty in researcher- participants finding the right tool to use to present research findings.
Overall the qualitative study found that various methods to identify gaps can be adopted (i.e., primary, secondary and both primary and secondary). Of all the methods to identify gaps, secondary methods were the most commonly used, specifically systematic reviews, considered the gold standard: these address a highly focused question related to the existing evidence and thus present difficulties for explicitly identifying research gaps in a general area. Hence using both primary and secondary methods (e.g., JLA method) was the most robust because it involves the participation of patients, caregivers and health and social care professionals in identifying research questions, then confirming, and prioritizing them by using a combination of primary and secondary methods. The main reported advantage of the JLA method is that it identified gaps (treatment uncertainties), and involved different stakeholders including patients and the public to confirm and prioritize the gaps. The main disadvantages are that it is labor-intensive (requires a team of different specialists) and expensive (administrative support, meeting rooms, catering, among others) as compared with secondary methods (evidence synthesis) or primary methods (survey). Concerning data visualisation, the study showed different stakeholders, particularly researchers, desire the use of data visualisation when communicating research, although few examples of experiences with developing and using them. The participants also expressed the difficulty in finding the right tool to use to present research findings.
In the third project, I focused on the development of methodological guidance, aimed at systematically identifying and reporting research gaps, to provide a clearer picture of the status of the evidence base. The guidance highlights the importance of clearly reporting research gaps for subsequence clear identification of research gaps. The methodological guidance merged findings from the scoping review and the qualitative study. It demonstrates when identifying and addressing research gaps in a topic area, the most important step is to first define clearly what is meant by a research gap in the text. This can be done by either adopting an existing definition that best describes the research gap or explicitly describing the nature and type of the research gap. Describing the nature of the research gap can involve different items, as follows:
- State the scale of the research gap (i.e., specific or broad)
- Clearly define the research gap
- Specify the cause of research gap
The methodological guidance involves six steps: 1) specify the topic area and/or research question; 2) map and clearly state the existing research gaps ; 3) identify research gap(s); 4) clearly describe the research gaps(s) identified; 5) characterize the research gaps; and 6) present the research gaps. To be able to determine the feasibility and usability of this methodological guidance in practice, the importance, relevance, and applicability of each step proposed needs to be thoroughly discussed among different stakeholders, then implemented and evaluated accordingly. The evaluation of the methodological guidance will help determine its applicability and future implementation and adaption in health research and other fields.
