A literature review of 31 relevant articles published between January 2005 and March 2015 identified 10 variables relevant to user adoption of mobile technology: Perceived usefulness, perceived ease of use, income/ wealth, employment, mobility requirement, education, social resources, etc.“User adoption variables” is the theme
And remember, most university and college libraries also have valuable information on literature reviews. Here is the link to one such website: JIBC Literature Review information [PDF]
There are a few acceptable sources for literature reviews, and I will list these in order from what will be considered most acceptable to less acceptable sources for your literature review assignments, as follows:
The following sections will explain and provide examples of these various sources.
A peer reviewed journal article is a paper that has been submitted to a scholarly journal, accepted, and published. Peer review journal papers go through a rigorous, blind review process of peer review. What this means is that two to three experts in the area of research featured in the paper have reviewed and accepted the paper for publication. The names of the author(s) who are seeking to publish the research have been removed (blind review), so as to minimize any bias towards the authors of the research. Albeit, sometimes a savvy reviewer can discern who has done the research based upon previous publications, etc. This blind review process can be long (often 12 to 18 months) that may involve many back and forth edits on the behalf of the researchers, as they work to address the edits and concerns of the peers who reviewed their paper. Often, reviewers will reject the paper for a variety of reasons, such as unclear or questionable methods, lack of contribution to the field, etc. Because peer reviewed journal articles have gone through a rigorous process of review, they are considered to be the premier source for research. Peer reviewed journal articles should serve as the foundation for your literature review.
The following website will provide more information on peer-reviewed journal articles: Evaluating Information Sources: What Is A Peer-Reviewed Article? Make sure you watch the little video on the upper left-hand side of your screen, in addition to reading the material.
An edited academic book is a collection of scholarly scientific papers written by different authors. The works are original papers, not published elsewhere (Wikipedia, 2018). The papers within the text also go through a process of review; however, the review is often not a blind review because the authors have been invited to contribute to the book. Consequently, edited academic books are fine to use for your literature review, but you also want to ensure that your literature review contains mostly peer reviewed journal papers.
Articles from professional journals should be used with caution, as far as it relates to a source for your literature review. This is because articles in trade journals are not usually peer reviewed, even though they may appear as such. A good way to find out is to read the “About us” section of the professional journal. They should state there if the papers are peer reviewed. You can also google the name of the journal and add peer reviewed to the search and you should be able to find out that way.
Governmental websites can be excellent sources for statistical data. For examples, Statistics Canada collects and publishes data related to the economy, society, and the environment.
Material from other websites can also serve as a source for statistics that you may need for your literature review. As you want to justify the value of the research you are interested in, you might make use of a professional association´s website to learn how many members they have, for example. As a hypothetical example, you might want to demonstrate, as part of the introduction to your literature review, why more research on the topic of PTSD in police officers is important. You could use peer reviewed journal articles to determine the prevalence of PTSD in police officers in Canada in the last ten years and then use the Ontario Police Officers´ Association website to determine the approximate number of police officers employed in the Province of Ontario over the last ten years. This might help you create an approximation of how many police officers could be suffering with PTSD in Ontario. That number could potentially help to justify a research grand down the road. But again, this type of website-based material should be used with caution and sparingly.
The Literature Review Copyright © 2019 by Valerie A. Sheppard is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License , except where otherwise noted.
You have full access to this open access article
30k Accesses
24 Citations
18 Altmetric
Explore all metrics
Researchers and practitioners rely on literature reviews to synthesize large bodies of knowledge. Many types of literature reviews have been developed, each targeting a specific purpose. However, these syntheses are hampered if the review type’s paradigmatic roots, methods, and markers of rigor are only vaguely understood. One literature review type whose methodology has yet to be elucidated is the state-of-the-art (SotA) review. If medical educators are to harness SotA reviews to generate knowledge syntheses, we must understand and articulate the paradigmatic roots of, and methods for, conducting SotA reviews.
We reviewed 940 articles published between 2014–2021 labeled as SotA reviews. We (a) identified all SotA methods-related resources, (b) examined the foundational principles and techniques underpinning the reviews, and (c) combined our findings to inductively analyze and articulate the philosophical foundations, process steps, and markers of rigor.
In the 940 articles reviewed, nearly all manuscripts (98%) lacked citations for how to conduct a SotA review. The term “state of the art” was used in 4 different ways. Analysis revealed that SotA articles are grounded in relativism and subjectivism.
This article provides a 6-step approach for conducting SotA reviews. SotA reviews offer an interpretive synthesis that describes: This is where we are now. This is how we got here. This is where we could be going. This chronologically rooted narrative synthesis provides a methodology for reviewing large bodies of literature to explore why and how our current knowledge has developed and to offer new research directions.
Reviewing the literature, how systematic is systematic.
Avoid common mistakes on your manuscript.
Literature reviews play a foundational role in scientific research; they support knowledge advancement by collecting, describing, analyzing, and integrating large bodies of information and data [ 1 , 2 ]. Indeed, as Snyder [ 3 ] argues, all scientific disciplines require literature reviews grounded in a methodology that is accurate and clearly reported. Many types of literature reviews have been developed, each with a unique purpose, distinct methods, and distinguishing characteristics of quality and rigor [ 4 , 5 ].
Each review type offers valuable insights if rigorously conducted [ 3 , 6 ]. Problematically, this is not consistently the case, and the consequences can be dire. Medical education’s policy makers and institutional leaders rely on knowledge syntheses to inform decision making [ 7 ]. Medical education curricula are shaped by these syntheses. Our accreditation standards are informed by these integrations. Our patient care is guided by these knowledge consolidations [ 8 ]. Clearly, it is important for knowledge syntheses to be held to the highest standards of rigor. And yet, that standard is not always maintained. Sometimes scholars fail to meet the review’s specified standards of rigor; other times the markers of rigor have never been explicitly articulated. While we can do little about the former, we can address the latter. One popular literature review type whose methodology has yet to be fully described, vetted, and justified is the state-of-the-art (SotA) review.
While many types of literature reviews amalgamate bodies of literature, SotA reviews offer something unique. By looking across the historical development of a body of knowledge, SotA reviews delves into questions like: Why did our knowledge evolve in this way? What other directions might our investigations have taken? What turning points in our thinking should we revisit to gain new insights? A SotA review—a form of narrative knowledge synthesis [ 5 , 9 ]—acknowledges that history reflects a series of decisions and then asks what different decisions might have been made.
SotA reviews are frequently used in many fields including the biomedical sciences [ 10 , 11 ], medicine [ 12 , 13 , 14 ], and engineering [ 15 , 16 ]. However, SotA reviews are rarely seen in medical education; indeed, a bibliometrics analysis of literature reviews published in 14 core medical education journals between 1999 and 2019 reported only 5 SotA reviews out of the 963 knowledge syntheses identified [ 17 ]. This is not to say that SotA reviews are absent; we suggest that they are often unlabeled. For instance, Schuwirth and van der Vleuten’s article “A history of assessment in medical education” [ 14 ] offers a temporally organized overview of the field’s evolving thinking about assessment. Similarly, McGaghie et al. published a chronologically structured review of simulation-based medical education research that “reviews and critically evaluates historical and contemporary research on simulation-based medical education” [ 18 , p. 50]. SotA reviews certainly have a place in medical education, even if that place is not explicitly signaled.
This lack of labeling is problematic since it conceals the purpose of, and work involved in, the SotA review synthesis. In a SotA review, the author(s) collects and analyzes the historical development of a field’s knowledge about a phenomenon, deconstructs how that understanding evolved, questions why it unfolded in specific ways, and posits new directions for research. Senior medical education scholars use SotA reviews to share their insights based on decades of work on a topic [ 14 , 18 ]; their junior counterparts use them to critique that history and propose new directions [ 19 ]. And yet, SotA reviews are generally not explicitly signaled in medical education. We suggest that at least two factors contribute to this problem. First, it may be that medical education scholars have yet to fully grasp the unique contributions SotA reviews provide. Second, the methodology and methods of SotA reviews are poorly reported making this form of knowledge synthesis appear to lack rigor. Both factors are rooted in the same foundational problem: insufficient clarity about SotA reviews. In this study, we describe SotA review methodology so that medical educators can explicitly use this form of knowledge synthesis to further advance the field.
We developed a four-step research design to meet this goal, illustrated in Fig. 1 .
Four-step research design process used for developing a State-of-the-Art literature review methodology
To build our initial corpus of articles reporting SotA reviews, we searched PubMed using the strategy (″state of the art review″[ti] OR ″state of the art review*″) and limiting our search to English articles published between 2014 and 2021. We strategically focused on PubMed, which includes MEDLINE, and is considered the National Library of Medicine’s premier database of biomedical literature and indexes health professions education and practice literature [ 20 ]. We limited our search to 2014–2021 to capture modern use of SotA reviews. Of the 960 articles identified, nine were excluded because they were duplicates, erratum, or corrigendum records; full text copies were unavailable for 11 records. All articles identified ( n = 940) constituted the corpus for analysis.
EB, JM, or LV independently reviewed the 940 full-text articles to identify all references to resources that explained, informed, described, or otherwise supported the methods used for conducting the SotA review. Articles that met our criteria were obtained for analysis.
To ensure comprehensive retrieval, we also searched Scopus and Web of Science. Additionally, to find resources not indexed by these academic databases, we searched Google (see Electronic Supplementary Material [ESM] for the search strategies used for each database). EB also reviewed the first 50 items retrieved from each search looking for additional relevant resources. None were identified. Via these strategies, nine articles were identified and added to the collection of methods-related resources for analysis.
In Step 3, we extracted three kinds of information from the 940 articles papers identified in Step 1. First, descriptive data on each article were compiled (i.e., year of publication and the academic domain targeted by the journal). Second, each article was examined and excerpts collected about how the term state-of-the-art review was used (i.e., as a label for a methodology in-and-of itself; as an adjective qualifying another type of literature review; as a term included in the paper’s title only; or in some other way). Finally, we extracted excerpts describing: the purposes and/or aims of the SotA review; the methodology informing and methods processes used to carry out the SotA review; outcomes of analyses; and markers of rigor for the SotA review.
Two researchers (EB and JM) coded 69 articles and an interrater reliability of 94.2% was achieved. Any discrepancies were discussed. Given the high interrater reliability, the two authors split the remaining articles and coded independently.
The methods-related resources identified in Step 2 and the data extractions from Step 3 were inductively analyzed by LV and EB to identify statements and research processes that revealed the ontology (i.e., the nature of reality that was reflected) and the epistemology (i.e., the nature of knowledge) underpinning the descriptions of the reviews. These authors studied these data to determine if the synthesis adhered to an objectivist or a subjectivist orientation, and to synthesize the purposes realized in these papers.
To confirm these interpretations, LV and EB compared their ontology, epistemology, and purpose determinations against two expectations commonly required of objectivist synthesis methods (e.g., systematic reviews): an exhaustive search strategy and an appraisal of the quality of the research data. These expectations were considered indicators of a realist ontology and objectivist epistemology [ 21 ] (i.e., that a single correct understanding of the topic can be sought through objective data collection {e.g., systematic reviews [ 22 ]}). Conversely, the inverse of these expectations were considered indicators of a relativist ontology and subjectivist epistemology [ 21 ] (i.e., that no single correct understanding of the topic is available; there are multiple valid understandings that can be generated and so a subjective interpretation of the literature is sought {e.g., narrative reviews [ 9 ]}).
Once these interpretations were confirmed, LV and EB reviewed and consolidated the methods steps described in these data. Markers of rigor were then developed that aligned with the ontology, epistemology, and methods of SotA reviews.
Of the 940 articles identified in Step 1, 98% ( n = 923) lacked citations or other references to resources that explained, informed, or otherwise supported the SotA review process. Of the 17 articles that included supporting information, 16 cited Grant and Booth’s description [ 4 ] consisting of five sentences describing the overall purpose of SotA reviews, three sentences noting perceived strengths, and four sentences articulating perceived weaknesses. This resource provides no guidance on how to conduct a SotA review methodology nor markers of rigor. The one article not referencing Grant and Booth used “an adapted comparative effectiveness research search strategy that was adapted by a health sciences librarian” [ 23 , p. 381]. One website citation was listed in support of this strategy; however, the page was no longer available in summer 2021. We determined that the corpus was uninformed by a cardinal resource or a publicly available methodology description.
In Step 2 we identified nine resources [ 4 , 5 , 24 , 25 , 26 , 27 , 28 ]; none described the methodology and/or processes of carrying out SotA reviews. Nor did they offer explicit descriptions of the ontology or epistemology underpinning SotA reviews. Instead, these resources provided short overview statements (none longer than one paragraph) about the review type [ 4 , 5 , 24 , 25 , 26 , 27 , 28 ]. Thus, we determined that, to date, there are no available methodology papers describing how to conduct a SotA review.
Step 3 revealed that “state of the art” was used in 4 different ways across the 940 articles (see Fig. 2 for the frequency with which each was used). In 71% ( n = 665 articles), the phrase was used only in the title, abstract, and/or purpose statement of the article; the phrase did not appear elsewhere in the paper and no SotA methodology was discussed. Nine percent ( n = 84) used the phrase as an adjective to qualify another literature review type and so relied entirely on the methodology of a different knowledge synthesis approach (e.g., “a state of the art systematic review [ 29 ]”). In 5% ( n = 52) of the articles, the phrase was not used anywhere within the article; instead, “state of the art” was the type of article within a journal. In the remaining 15% ( n = 139), the phrase denoted a specific methodology (see ESM for all methodology articles). Via Step 4’s inductive analysis, the following foundational principles of SotA reviews were developed: (1) the ontology, (2) epistemology, and (3) purpose of SotA reviews.
Four ways the term “state of the art” is used in the corpus and how frequently each is used
SotA reviews rest on four propositions:
The literature addressing a phenomenon offers multiple perspectives on that topic (i.e., different groups of researchers may hold differing opinions and/or interpretations of data about a phenomenon).
The reality of the phenomenon itself cannot be completely perceived or understood (i.e., due to limitations [e.g., the capabilities of current technologies, a research team’s disciplinary orientation] we can only perceive a limited part of the phenomenon).
The reality of the phenomenon is a subjective and inter-subjective construction (i.e., what we understand about a phenomenon is built by individuals and so their individual subjectivities shape that understanding).
The context in which the review was conducted informs the review (e.g., a SotA review of literature about gender identity and sexual function will be synthesized differently by researchers in the domain of gender studies than by scholars working in sex reassignment surgery).
As these propositions suggest, SotA scholars bring their experiences, expectations, research purposes, and social (including academic) orientations to bear on the synthesis work. In other words, a SotA review synthesizes the literature based on a specific orientation to the topic being addressed. For instance, a SotA review written by senior scholars who are experts in the field of medical education may reflect on the turning points that have shaped the way our field has evolved the modern practices of learner assessment, noting how the nature of the problem of assessment has moved: it was first a measurement problem, then a problem that embraced human judgment but needed assessment expertise, and now a whole system problem that is to be addressed from an integrated—not a reductionist—perspective [ 12 ]. However, if other scholars were to examine this same history from a technological orientation, learner assessment could be framed as historically constricted by the media available through which to conduct assessment, pointing to how artificial intelligence is laying the foundation for the next wave of assessment in medical education [ 30 ].
Given these foundational propositions, SotA reviews are steeped in a relativist ontology—i.e., reality is socially and experientially informed and constructed, and so no single objective truth exists. Researchers’ interpretations reflect their conceptualization of the literature—a conceptualization that could change over time and that could conflict with the understandings of others.
SotA reviews embrace subjectivism. The knowledge generated through the review is value-dependent, growing out of the subjective interpretations of the researcher(s) who conducted the synthesis. The SotA review generates an interpretation of the data that is informed by the expertise, experiences, and social contexts of the researcher(s). Furthermore, the knowledge developed through SotA reviews is shaped by the historical point in time when the review was conducted. SotA reviews are thus steeped in the perspective that knowledge is shaped by individuals and their community, and is a synthesis that will change over time.
SotA reviews create a subjectively informed summary of modern thinking about a topic. As a chronologically ordered synthesis, SotA reviews describe the history of turning points in researchers’ understanding of a phenomenon to contextualize a description of modern scientific thinking on the topic. The review presents an argument about how the literature could be interpreted; it is not a definitive statement about how the literature should or must be interpreted. A SotA review explores: the pivotal points shaping the historical development of a topic, the factors that informed those changes in understanding, and the ways of thinking about and studying the topic that could inform the generation of further insights. In other words, the purpose of SotA reviews is to create a three-part argument: This is where we are now in our understanding of this topic. This is how we got here. This is where we could go next.
Based on study findings and analyses, we constructed a six-stage SotA review methodology. This six-stage approach is summarized and guiding questions are offered in Tab. 1 .
In Stage 1, the researcher(s) creates an initial description of the topic to be summarized and so must determine what field of knowledge (and/or practice) the search will address. Knowledge developed through the SotA review process is shaped by the context informing it; thus, knowing the domain in which the review will be conducted is part of the review’s foundational work.
This stage involves determining the period of time that will be defined as SotA for the topic being summarized. The researcher(s) should engage in a broad-scope overview of the literature, reading across the range of literature available to develop insights into the historical development of knowledge on the topic, including the turning points that shape the current ways of thinking about a topic. Understanding the full body of literature is required to decide the dates or events that demarcate the timeframe of now in the first of the SotA’s three-part argument: where we are now . Stage 2 is complete when the researcher(s) can explicitly justify why a specific year or event is the right moment to mark the beginning of state-of-the-art thinking about the topic being summarized.
Based on the insights developed in Stage 2, the researcher(s) will likely need to revise their initial description of the topic to be summarized. The formal research question(s) framing the SotA review are finalized in Stage 3. The revised description of the topic, the research question(s), and the justification for the timeline start year must be reported in the review article. These are markers of rigor and prerequisites for moving to Stage 4.
In Stage 4, the researcher(s) develops a search strategy to identify the literature that will be included in the SotA review. The researcher(s) needs to determine which literature databases contain articles from the domain of interest. Because the review describes how we got here , the review must include literature that predates the state-of-the-art timeframe, determined in Stage 2, to offer this historical perspective.
Developing the search strategy will be an iterative process of testing and revising the search strategy to enable the researcher(s) to capture the breadth of literature required to meet the SotA review purposes. A librarian should be consulted since their expertise can expedite the search processes and ensure that relevant resources are identified. The search strategy must be reported (e.g., in the manuscript itself or in a supplemental file) so that others may replicate the process if they so choose (e.g., to construct a different SotA review [and possible different interpretations] of the same literature). This too is a marker of rigor for SotA reviews: the search strategies informing the identification of literature must be reported.
The literature analysis undertaken will reflect the subjective insights of the researcher(s); however, the foundational premises of inductive research should inform the analysis process. Therefore, the researcher(s) should begin by reading the articles in the corpus to become familiar with the literature. This familiarization work includes: noting similarities across articles, observing ways-of-thinking that have shaped current understandings of the topic, remarking on assumptions underpinning changes in understandings, identifying important decision points in the evolution of understanding, and taking notice of gaps and assumptions in current knowledge.
The researcher(s) can then generate premises for the state-of-the-art understanding of the history that gave rise to modern thinking, of the current body of knowledge, and of potential future directions for research. In this stage of the analysis, the researcher(s) should document the articles that support or contradict their premises, noting any collections of authors or schools of thinking that have dominated the literature, searching for marginalized points of view, and studying the factors that contributed to the dominance of particular ways of thinking. The researcher(s) should also observe historical decision points that could be revisited. Theory can be incorporated at this stage to help shape insights and understandings. It should be highlighted that not all corpus articles will be used in the SotA review; instead, the researcher(s) will sample across the corpus to construct a timeline that represents the seminal moments of the historical development of knowledge.
Next, the researcher(s) should verify the thoroughness and strength of their interpretations. To do this, the researcher(s) can select different articles included in the corpus and examine if those articles reflect the premises the researcher(s) set out. The researcher(s) may also seek out contradictory interpretations in the literature to be sure their summary refutes these positions. The goal of this verification work is not to engage in a triangulation process to ensure objectivity; instead, this process helps the researcher(s) ensure the interpretations made in the SotA review represent the articles being synthesized and respond to the interpretations offered by others. This is another marker of rigor for SotA reviews: the authors should engage in and report how they considered and accounted for differing interpretations of the literature, and how they verified the thoroughness of their interpretations.
Given the relativist subjectivism of a SotA review, it is important that the manuscript offer insights into the subjectivity of the researcher(s). This reflexivity description should articulate how the subjectivity of the researcher(s) informed interpretations of the data. These reflections will also influence the suggested directions offered in the last part of the SotA three-part argument: where we could go next. This is the last marker of rigor for SotA reviews: researcher reflexivity must be considered and reported.
SotA reviews have much to offer our field since they provide information on the historical progression of medical education’s understanding of a topic, the turning points that guided that understanding, and the potential next directions for future research. Those future directions may question the soundness of turning points and prior decisions, and thereby offer new paths of investigation. Since we were unable to find a description of the SotA review methodology, we inductively developed a description of the methodology—including its paradigmatic roots, the processes to be followed, and the markers of rigor—so that scholars can harness the unique affordances of this type of knowledge synthesis.
Given their chronology- and turning point-based orientation, SotA reviews are inherently different from other types of knowledge synthesis. For example, systematic reviews focus on specific research questions that are narrow in scope [ 32 , 33 ]; in contrast, SotA reviews present a broader historical overview of knowledge development and the decisions that gave rise to our modern understandings. Scoping reviews focus on mapping the present state of knowledge about a phenomenon including, for example, the data that are currently available, the nature of that data, and the gaps in knowledge [ 34 , 35 ]; conversely, SotA reviews offer interpretations of the historical progression of knowledge relating to a phenomenon centered on significant shifts that occurred during that history. SotA reviews focus on the turning points in the history of knowledge development to suggest how different decisions could give rise to new insights. Critical reviews draw on literature outside of the domain of focus to see if external literature can offer new ways of thinking about the phenomenon of interest (e.g., drawing on insights from insects’ swarm intelligence to better understand healthcare team adaptation [ 36 ]). SotA reviews focus on one domain’s body of literature to construct a timeline of knowledge development, demarcating where we are now, demonstrating how this understanding came to be via different turning points, and offering new research directions. Certainly, SotA reviews offer a unique kind of knowledge synthesis.
Our six-stage process for conducting these reviews reflects the subjectivist relativism that underpins the methodology. It aligns with the requirements proposed by others [ 24 , 25 , 26 , 27 ], what has been written about SotA reviews [ 4 , 5 ], and the current body of published SotA reviews. In contrast to existing guidance [ 4 , 5 , 20 , 21 , 22 , 23 ], our description offers a detailed reporting of the ontology, epistemology, and methodology processes for conducting the SotA review.
This explicit methodology description is essential since many academic journals list SotA reviews as an accepted type of literature review. For instance, Educational Research Review [ 24 ], the American Academy of Pediatrics [ 25 ], and Thorax all lists SotA reviews as one of the types of knowledge syntheses they accept [ 27 ]. However, while SotA reviews are valued by academia, guidelines or specific methodology descriptions for researchers to follow when conducting this type of knowledge synthesis are conspicuously absent. If academics in general, and medical education more specifically, are to take advantage of the insights that SotA reviews can offer, we need to rigorously engage in this synthesis work; to do that, we need clear descriptions of the methodology underpinning this review. This article offers such a description. We hope that more medical educators will conduct SotA reviews to generate insights that will contribute to further advancing our field’s research and scholarship.
Cooper HM. Organizing knowledge syntheses: a taxonomy of literature reviews. Knowl Soc. 1988;1:104.
Google Scholar
Badger D, Nursten J, Williams P, Woodward M. Should all literature reviews be systematic? Eval Res Educ. 2000;14:220–30.
Article Google Scholar
Snyder H. Literature review as a research methodology: an overview and guidelines. J Bus Res. 2019;104:333–9.
Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26:91–108.
Sutton A, Clowes M, Preston L, Booth A. Meeting the review family: exploring review types and associated information retrieval requirements. Health Info Libr J. 2019;36:202–22.
Moher D, Liberati A, Tetzlaff J, Altman DG, Prisma Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097.
Tricco AC, Langlois E, Straus SE, World Health Organization, Alliance for Health Policy and Systems Research. Rapid reviews to strengthen health policy and systems: a practical guide. Geneva: World Health Organization; 2017.
Jackson R, Feder G. Guidelines for clinical guidelines: a simple, pragmatic strategy for guideline development. Br Med J. 1998;317:427–8.
Greenhalgh T, Thorne S, Malterud K. Time to challenge the spurious hierarchy of systematic over narrative reviews? Eur J Clin Invest. 2018;48:e12931.
Bach QV, Chen WH. Pyrolysis characteristics and kinetics of microalgae via thermogravimetric analysis (TGA): a state-of-the-art review. Bioresour Technol. 2017;246:88–100.
Garofalo C, Milanović V, Cardinali F, Aquilanti L, Clementi F, Osimani A. Current knowledge on the microbiota of edible insects intended for human consumption: a state-of-the-art review. Food Res Int. 2019;125:108527.
Carbone S, Dixon DL, Buckley LF, Abbate A. Glucose-lowering therapies for cardiovascular risk reduction in type 2 diabetes mellitus: state-of-the-art review. Mayo Clin Proc. 2018;93:1629–47.
Hofkens PJ, Verrijcken A, Merveille K, et al. Common pitfalls and tips and tricks to get the most out of your transpulmonary thermodilution device: results of a survey and state-of-the-art review. Anaesthesiol Intensive Ther. 2015;47:89–116.
Schuwirth LW, van der Vleuten CP. A history of assessment in medical education. Adv Health Sci Educ Theory Pract. 2020;25:1045–56.
Arena A, Prete F, Rambaldi E, et al. Nanostructured zirconia-based ceramics and composites in dentistry: a state-of-the-art review. Nanomaterials. 2019;9:1393.
Bahraminasab M, Farahmand F. State of the art review on design and manufacture of hybrid biomedical materials: hip and knee prostheses. Proc Inst Mech Eng H. 2017;231:785–813.
Maggio LA, Costello JA, Norton C, Driessen EW, Artino AR Jr. Knowledge syntheses in medical education: a bibliometric analysis. Perspect Med Educ. 2021;10:79–87.
McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. A critical review of simulation-based medical education research: 2003–2009. Med Educ. 2010;44:50–63.
Krishnan DG, Keloth AV, Ubedulla S. Pros and cons of simulation in medical education: a review. Education. 2017;3:84–7.
National Library of Medicine. MEDLINE: overview. 2021. https://www.nlm.nih.gov/medline/medline_overview.html . Accessed 17 Dec 2021.
Bergman E, de Feijter J, Frambach J, et al. AM last page: a guide to research paradigms relevant to medical education. Acad Med. 2012;87:545.
Maggio LA, Samuel A, Stellrecht E. Systematic reviews in medical education. J Grad Med Educ. 2022;14:171–5.
Bandari J, Wessel CB, Jacobs BL. Comparative effectiveness in urology: a state of the art review utilizing a systematic approach. Curr Opin Urol. 2017;27:380–94.
Elsevier. A guide for writing scholarly articles or reviews for the educational research review. 2010. https://www.elsevier.com/__data/promis_misc/edurevReviewPaperWriting.pdf . Accessed 3 Mar 2020.
American Academy of Pediatrics. Pediatrics author guidelines. 2020. https://pediatrics.aappublications.org/page/author-guidelines . Accessed 3 Mar 2020.
Journal of the American College of Cardiology. JACC instructions for authors. 2020. https://www.jacc.org/pb-assets/documents/author-instructions-jacc-1598995793940.pdf . Accessed 3 Mar 2020.
Thorax. Authors. 2020. https://thorax.bmj.com/pages/authors/ . Accessed 3 Mar 2020.
Berven S, Carl A. State of the art review. Spine Deform. 2019;7:381.
Ilardi CR, Chieffi S, Iachini T, Iavarone A. Neuropsychology of posteromedial parietal cortex and conversion factors from mild cognitive impairment to Alzheimer’s disease: systematic search and state-of-the-art review. Aging Clin Exp Res. 2022;34:289–307.
Chan KS, Zary N. Applications and challenges of implementing artificial intelligence in medical education: integrative review. JMIR Med Educ. 2019;5:e13930.
World Health Organization. Framework for action on interprofessional education and collaborative practice. 2010. https://www.who.int/publications/i/item/framework-for-action-on-interprofessional-education-collaborative-practice . Accessed July 1 2021.
Hammersley M. On ‘systematic’ reviews of research literatures: a ‘narrative’ response to Evans & Benefield. Br Educ Res J. 2001;27:543–54.
Chen F, Lui AM, Martinelli SM. A systematic review of the effectiveness of flipped classrooms in medical education. Med Educ. 2017;51:585–97.
Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8:19–32.
Matsas B, Goralnick E, Bass M, Barnett E, Nagle B, Sullivan E. Leadership development in US undergraduate medical education: a scoping review of curricular content and competency frameworks. Acad Med. 2022;97:899–908.
Cristancho SM. On collective self-healing and traces: How can swarm intelligence help us think differently about team adaptation? Med Educ. 2021;55:441–7.
Download references
We thank Rhonda Allard for her help with the literature review and compiling all available articles. We also want to thank the PME editors who offered excellent development and refinement suggestions that greatly improved this manuscript.
Authors and affiliations.
Department of Anesthesiology, F. Edward Hébert School of Medicine, Uniformed Services University, Bethesda, MD, USA
Erin S. Barry
School of Health Professions Education (SHE), Maastricht University, Maastricht, The Netherlands
Department of Medicine, F. Edward Hébert School of Medicine, Uniformed Services University, Bethesda, MD, USA
Jerusalem Merkebu & Lara Varpio
You can also search for this author in PubMed Google Scholar
Correspondence to Erin S. Barry .
Conflict of interest.
E.S. Barry, J. Merkebu and L. Varpio declare that they have no competing interests.
The opinions and assertions contained in this article are solely those of the authors and are not to be construed as reflecting the views of the Uniformed Services University of the Health Sciences, the Department of Defense, or the Henry M. Jackson Foundation for the Advancement of Military Medicine.
40037_2022_725_moesm1_esm.docx.
For information regarding the search strategy to develop the corpus and search strategy for confirming capture of any available State of the Art review methodology descriptions. Additionally, a list of the methodology articles found through the search strategy/corpus is included
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .
Reprints and permissions
Barry, E.S., Merkebu, J. & Varpio, L. State-of-the-art literature review methodology: A six-step approach for knowledge synthesis. Perspect Med Educ 11 , 281–288 (2022). https://doi.org/10.1007/s40037-022-00725-9
Download citation
Received : 03 December 2021
Revised : 25 July 2022
Accepted : 27 July 2022
Published : 05 September 2022
Issue Date : October 2022
DOI : https://doi.org/10.1007/s40037-022-00725-9
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative
Literature reviews.
Finding literature on research methodologies, sage research methods online.
Research methodology is the specific strategies, processes, or techniques utilised in the collection of information that is created and analysed.
The methodology section of a research paper, or thesis, enables the reader to critically evaluate the study’s validity and reliability by addressing how the data was collected or generated, and how it was analysed.
There are three main types of research methods which use different designs for data collection.
Qualitative research gathers data about lived experiences, emotions or behaviours, and the meanings individuals attach to them. It assists in enabling researchers to gain a better understanding of complex concepts, social interactions or cultural phenomena. This type of research is useful in the exploration of how or why things have occurred, interpreting events and describing actions.
Examples of qualitative research designs include:
Quantitative research gathers numerical data which can be ranked, measured or categorised through statistical analysis. It assists with uncovering patterns or relationships, and for making generalisations. This type of research is useful for finding out how many, how much, how often, or to what extent.
Examples of quantitative research designs include:
Mixed Methods research integrates both Qualitative research and Quantitative research. It provides a holistic approach combining and analysing the statistical data with deeper contextualised insights. Using Mixed Methods also enables triangulation, or verification, of the data from two or more sources.
Sometimes in your literature review, you might need to discuss and evaluate relevant research methodologies in order to justify your own choice of research methodology.
When searching for literature on research methodologies it is important to search across a range of sources. No single information source will supply all that you need. Selecting appropriate sources will depend upon your research topic.
Developing a robust search strategy will help reduce irrelevant results. It is good practice to plan a strategy before you start to search.
(1) free text keywords.
Free text searching is the use of natural language words to conduct your search. Use selective free text keywords such as: phenomenological, "lived experience", "grounded theory", "life experiences", "focus groups", interview, quantitative, survey, validity, variance, correlation and statistical.
To locate books on your desired methodology, try LibrarySearch . Remember to use refine options such as books, ebooks, subject, and publication date.
Databases categorise their records using subject terms, or a controlled vocabulary (thesaurus). These subject headings may be useful to use, in addition to utilising free text keywords in a database search.
Subject headings will differ across databases, for example, the PubMed database uses 'Qualitative Research' whilst the CINHAL database uses 'Qualitative Studies.'
Databases enable sets of results to be limited or filtered by specific fields, look for options such as Publication Type, Article Type, etc. and apply them to your search.
To find books on research methods browse the Library shelves at call number 001.42
SAGE Research Methods Overview (2:07 min) by SAGE Publishing ( YouTube )
Saul Mcleod, PhD
Editor-in-Chief for Simply Psychology
BSc (Hons) Psychology, MRes, PhD, University of Manchester
Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.
Learn about our Editorial Process
Olivia Guy-Evans, MSc
Associate Editor for Simply Psychology
BSc (Hons) Psychology, MSc Psychology of Education
Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.
On This Page:
A scoping review is a type of research synthesis that maps the existing literature on a broad topic to identify key concepts, research gaps, and types of evidence.
This mapping exercise involves systematically searching for, identifying, and charting relevant literature to understand its characteristics, such as the volume of research, types of studies conducted, key concepts addressed, and prevalent research gaps.
Unlike systematic reviews, which aim to answer specific questions, scoping reviews are exploratory and often used to assess the extent of available evidence and inform future research directions. They involve comprehensive searches and data extraction but do not typically include a detailed synthesis of findings or a critical appraisal of study quality.
Scoping reviews can be used as a preliminary step to a systematic review , helping to identify the types of evidence available, potential research questions, and relevant inclusion criteria.
They can save time and resources by identifying potential challenges or limitations before embarking on a full systematic review.
Scoping reviews can help clarify key concepts/definitions in the literature. If a research area has inconsistent terminology or definitions, a scoping review can map out how different concepts are used and potentially propose a unified understanding. This can help refine the focus and scope of a subsequent systematic review.
Scoping reviews help identify areas needing further research, whereas systematic reviews aim to draw conclusions about intervention effectiveness.
Methodological guidelines aim to improve the consistency and transparency of scoping reviews, enabling researchers to synthesize evidence effectively.
Arksey and O’Malley (2005) | Levac et al. (2010) | Joanna Briggs Institute |
---|---|---|
6 stages, including optional consultation; most flexible approach | 6 stages with more detailed guidance; moderate flexibility | More prescriptive approach with additional elements; most structured |
Broad research question | Clearly articulated research question | Clearly defined research question with concept, population, and context |
Study selection process not specified | Recommends two reviewers for study selection | Provides detailed guidance on study selection process |
Basic data charting | More comprehensive data extraction | Detailed guidance on data extraction with specific tools |
Basic summary of findings | Numeric summary and qualitative thematic analysis | Introduces evidence mapping for analysis |
Quality assessment not included | Quality assessment not emphasized | Introduces potential for quality appraisal |
Optional stakeholder consultation | Recommended stakeholder consultation | Stakeholder consultation as an integral part of the process |
Provides basic framework | Offers enhanced detail on methodology | Provides most detailed guidance on conducting scoping reviews |
A well-defined objective and a set of aligned research questions are crucial for a scoping review’s coherence and direction.
They guide the subsequent steps of the review process, including determining the inclusion and exclusion criteria, developing a search strategy, and guiding data extraction and analysis.
This stage involves a thoughtful and iterative process to ensure that the review’s aims and questions are explicitly stated and closely intertwined.
This step outlines the overarching goals of the scoping review. It explains the rationale behind conducting the review and what the reviewers aim to achieve.
The objective statement should succinctly capture the essence of the review and provide a clear understanding of its purpose.
For instance, a scoping review’s objective might be to map the existing literature on a particular topic and identify knowledge gaps.
“Parents, in particular, greatly influence participation at school, at home and in the community. They undertake many actions to improve their children’s participation in daily life. Understanding the actions of parents and also their challenges and needs will contribute to how society can support these parents and thereby enable the participation of children with physical disabilities. Pediatric rehabilitation, aiming for optimal participation, could benefit from this understanding to improve Family-centered services (FCS)… However, it is unclear what kind of information is available in literature about what parents live through, do, and what kind of problems and needs they have in supporting their child’s participation? For these reasons, a scoping review was conducted in order to systematically map the research done in this area, as well as to identify any existing gaps in knowledge”
Piškur, B., Beurskens, A. J., Jongmans, M. J., Ketelaar, M., Norton, M., Frings, C. A., … & Smeets, R. J. (2012). Parents’ actions, challenges, and needs while enabling participation of children with a physical disability: a scoping review. BMC pediatrics , 12 , 1-13.
The research question(s) stem from the objectives and provide a focused roadmap for the review. These questions should be answerable through the scoping review process. The research question(s) should be clear, concise, and directly relevant to the overall objectives.
Using Frameworks: While not mandatory, frameworks can be helpful tools to guide the development of objectives and research questions. Frameworks like PCC (Population, Concept, Context).
How do cultural beliefs and practices ( C -context) influence the ways in which parents ( P -parents of children with physical disabilities) perceive and address ( C -concept) their children’s physical disabilities?
What are the barriers and facilitators ( C -concept) to mental health service utilization ( C -concept) among veterans ( P -population) experiencing homelessness ( C -context)?
This scoping review aims to summarize what is known in the African scientific literature ( C -context) among cisgender persons ( P ) about a) individual experiences of GBS within health care settings ( C -concept) and b) associations between GBS experiences and health care-related outcomes ( C -concept).
What are the main theoretical and methodological characteristics ( C -concept) of the current literature ( C -context) in the area of stigma and hearing loss and stigma and hearing aids in the elderly population ( P -older adults with acquired hearing impairment), and how should future research proceed in expanding this important field of enquiry?
A research protocol is a detailed plan that outlines the methodology to be employed throughout the review process, detailing steps like documenting results, outlining search strategy, and stating the review’s objective
The protocol should be created a priori (before starting the review) to ensure transparency and reproducibility.
While not mandatory, registering your protocol is highly recommended, e.g. FigShare and Open Science Framework (OSF).
Some journals, such as the Journal of Advanced Nursing , Systematic Reviews , BMC Medical Research Methodology , BMJ Open , and JBI Evidence Synthesis , accept scoping review protocols for publication.
It’s important to note that PROSPERO, the international prospective register of systematic reviews, does not currently accept scoping review protocols for registration.
Registering a scoping review protocol is highly recommended, even if not mandatory, as it promotes transparency, reduces duplication of effort, and helps to prevent publication bias
“Our protocol was drafted using the Preferred Reporting Items for Systematic Reviews and Meta-analysis Protocols (PRISMAP…), which was revised by the research team and members of Health Canada, and was disseminated through our programme’s Twitter account (@KT-Canada) and newsletter to solicit additional feedback. The final protocol was registered prospectively with the Open Science Framework on 6 September 2016 ( https://osf.io/kv9hu/ ).”
Tricco, A. C., Zarin, W., Lillie, E., Pham, B., & Straus, S. E. (2017). Utility of social media and crowd-sourced data for pharmacovigilance: a scoping review protocol. BMJ open , 7 (1), e013474.
“ Our protocol was developed using the scoping review methodological framework proposed by Arksey and O’Malley (2005) [1] and further refined by the Joanna Briggs Institute [3]. The draft protocol was revised upon receiving feedback from the research team, including methodologists and healthcare providers, as well as the peer-review panel of the Canadian Institutes of Health Research. The final version of the protocol is available upon request from the corresponding author. ”
Tricco, A. C., Lillie, E., Zarin, W., O’brien, K., Colquhoun, H., Kastner, M., … & Straus, S. E. (2016). A scoping review on the conduct and reporting of scoping reviews. BMC medical research methodology , 16 , 1-10.
This step involves developing and aligning the inclusion criteria with the objective(s) and question(s).
By providing transparent and well-justified eligibility criteria, researchers can ensure the replicability of their scoping review and allow readers to assess the relevance and appropriateness of the included sources.
When reporting eligibility criteria, emphasize the importance of clarity, justification, and a clear link to the review’s objectives.
By using the PCC framework, researchers can systematically establish boundaries for their scoping review, ensuring that the included sources are relevant to the research question. The framework helps to ensure that the eligibility criteria are comprehensive and well-defined, enabling a more focused and meaningful synthesis of the literature
It is important to note that the absence of an explicitly stated framework (e.g. PCC) does not necessarily mean that the authors did not utilize a systematic approach when developing their eligibility criteria. It is possible that they employed a framework implicitly or that their criteria development was guided by other factors.
The initial set of eligibility criteria outlined in the protocol may be subject to adjustments based on the type and volume of studies identified in the initial searches.
“ Studies that identified the key terms in the title, abstract, article, or MeSH heading were retained for further examination. Studies published as abstracts, conference proceedings or pilot results published in non-peer-reviewed journals were excluded. In addition, books, book chapters, comments on publications, and dissertations were also excluded. No exclusion criteria were established regarding the type of research design. Inclusion criteria were (a) older adults with progressive hearing loss being the population of interest and (b) the outcome measure was clearly focused on (or at least on some aspects of) stigma regarding hearing loss and/or hearing aids. Although given the descriptive aim of the review, no definitions of stigma and/or hearing aids were set a priori, and all articles including these terms were retrieved, the analysis of the data relied on the most common dimensions of the concept of stigma cited in the literarture: the cognitive dimension (i.e., stereotypes), the emotional dimension (i.e., prejudice) and the behavioral dime. ”
“ An extensive search was conducted to locate peer-reviewed articles that addressed questions related to parent involvement in organized youth sport. To guide article retrieval, two inclusion criteria were used. First, articles were required to highlight some form of parent involvement in organized youth sport. In the present study, organized youth sport was operationalized as “adultorganized and controlled athletic programs for young people,” wherein “participants are formally organized [and] attend practices and scheduled competitions under the supervision of an adult leader” (Smoll & Smith, 2002, p. xi). In line with this criterion, we did not include physical activity, exercise, physical education, and free play settings, which comprise a substantial volume of research in sport and exercise psychology. We also excluded research that simply collected data on parents or from parents but did not explicitly assess their involvement in their children’s sport participation. Second, articles were required to have been published in peer-reviewed, Englishlanguage, academic journals. As such, we did not include books, chapters, reviews, conceptual papers, conference proceedings, theses and (Jones, 2004) dissertations, or organizational “white papers” in this scoping review. ”
Dorsch, T. E., Wright, E., Eckardt, V. C., Elliott, S., Thrower, S. N., & Knight, C. J. (2021). A history of parent involvement in organized youth sport: A scoping review. Sport, Exercise, and performance psychology , 10 (4), 536.
“…to be included in the review, papers needed to measure or focus on specific dimensions of treatment burden, developed in the conceptual framework (e.g. financial, medication, administrative, lifestyle, healthcare and time/travel). Peer-reviewed journal papers were included if they were: published between the period of 2000–2016, written in English, involved human participants and described a measure for burden of treatment, e.g. including single measurements, measuring and/or incorporating one or two dimensions of burden of treatment. Quantitative, qualitative and mixed-method studies were included in order to consider different aspects of measuring treatment burden. Papers were excluded if they did not fit into the conceptual framework of the study, focused on a communicable chronic condition, for example human immunodeficiency virus infection and acquired immune deficiency syndrome (HIV/AIDS) or substance abuse. Papers talking about carer burden, in addition to patient burden of treatment, were also included.”
Sav, A., Salehi, A., Mair, F. S., & McMillan, S. S. (2017). Measuring the burden of treatment for chronic disease: implications of a scoping review of the literature. BMC medical research methodology , 17 , 1-14.
Scoping reviews aim to identify a broad range of relevant studies, including both published and unpublished literature, to provide a comprehensive overview of the topic.
The goal is to be inclusive rather than exhaustive, which differentiates scoping reviews from systematic reviews that seek to collate all empirical evidence fitting pre-specified criteria to answer specific research questions.
Information sources for scoping reviews can include a wide range of resources like scholarly databases, unpublished literature, conference papers, books, and even expert consultations.
Report who developed and executed the search strategy, such as an information specialist or librarian. Mention if the search strategy was peer-reviewed using the Peer Review of Electronic Search Strategies (PRESS) checklist.
“To identify potentially relevant documents, the following bibliographic databases were searched from 2004 to June 2015: MEDLINE, EMBASE, LexisNexis Academic, the Legal Scholarship Network, Justis, LegalTrac, QuickLaw, and HeinOnline. The search strategies were drafted by an experienced librarian [name] and further refined through team discussion. The final search strategy for MEDLINE can be found in Additional file 3. The final search results were exported into EndNote, and duplicates were removed by a library technician. The electronic database search was supplemented by searching the Canadian Medical Protective Association website (https://www.cmpa-acpm.ca/en) and scanning relevant reviews.”
Cardoso, R., Zarin, W., Nincic, V., Barber, S. L., Gulmezoglu, A. M., Wilson, C., … & Tricco, A. C. (2017). Evaluative reports on medical malpractice policies in obstetrics: a rapid scoping review. Systematic reviews , 6 , 1-11.
Scoping reviews typically start with a broader, more inclusive search strategy. The initial search is intentionally wide-ranging to capture the breadth of available literature on the topic
To balance breadth and depth in your initial search strategy for a scoping review, consider the following tips based on the gathered search results:
” The planned literature search was developed on June 23, 2022. The inclusion and exclusion criteria were further refined, along with electronic databases to identify psychological and education literature (e.g., ProQuest), programs for storing data (i.e., Covidence, n.d. accessed via https://www.covidence.org/) and key search terms (e.g., resistance and transgender). The key search terms were “transgender/trans/LGBT/gender diverse/gender expansive/nonbinary,” “resistance,” and “faith/economic status/ethnicity/gender.” Daniel Abela used terms such as nonbinary, gender diverse, LGBT, and gender expansive to capture the broad spectrum of language employed in the literature when relating to individuals whose gender identification extends beyond conventional norms associated with their assigned sex at birth. Moreover, the authors wanted a diverse sample through an intersectionality lens; therefore, terms such as faith, economic status, and ethnicity were used. These terms were selected as they were deemed by all authors to be most appropriate to evaluate this study’s research question. A complete list of the final search terms and the entire electronic search strategy for the Ovid database are presented in Table 1. ”
Abela, D., Patlamazoglou, L., & Lea, S. (2024). The resistance of transgender and gender expansive people: A scoping review. Psychology of Sexual Orientation and Gender Diversity .
Search strategy can also be reported in the appendix. For example: Supplementary A: Search strategy for scoping review .
Citation chasing involves reviewing the reference lists of included studies and examining articles that cite those studies to identify additional relevant literature. This process helps ensure that you capture a comprehensive view of the research landscape.
If citation chasing leads to the identification of new keywords or concepts, document these adjustments and how they were incorporated into the overall search strategy.
While articles included in a scoping review are selected systematically, it is important to acknowledge that there is no assumption that the evidence reviewed is exhaustive. This is often due to limitations in the search strategy or difficulty locating specific types of sources.
The goal is to identify relevant studies, with less emphasis on methodological quality. Scoping reviews generally do not appraise the quality of included studies.
Instead, scoping reviews prioritize mapping the existing literature and identifying gaps in research, regardless of the quality of the individual studies.
Two reviewers should independently screen titles and abstracts, removing duplicates and irrelevant studies based on predefined inclusion and exclusion criteria.
“To increase consistency among reviewers, all reviewers screened the same 50 publications, discussed the results and amended the screening and data extraction manual before beginning screening for this review. Nine reviewers working in pairs sequentially evaluated the titles, abstracts and then full text of all publications identified by our searches for potentially relevant publications. . . . We resolved disagreements on study selection and data extraction by consensus and discussion with other reviewers if needed.”
Duffett, M., Choong, K., Hartling, L., Menon, K., Thabane, L., & Cook, D. J. (2013). Randomized controlled trials in pediatric critical care: a scoping review. Critical care , 17 , 1-9.
Charting, also known as data extraction, is a crucial stage in conducting a scoping review.
This process involves systematically collecting relevant information from the sources included in the review using a structured form. It is considered best practice to have at least two reviewers independently extract data from each source
Data charting in scoping reviews differs from data extraction in systematic reviews. While systematic reviews aim to synthesize the results and assess the quality of individual studies, scoping reviews focus on mapping the existing literature and identifying key concepts, themes, and gaps in the research.
Therefore, the data charting process in scoping reviews is typically broader in scope and may involve collecting a wider range of data items compared to the more focused data extraction process used in systematic reviews.
This process goes beyond simply extracting data; it involves characterizing and summarizing research evidence, which ultimately helps identify research gaps.
“Search results for all databases were merged. Duplicates and nonrelated papers were excluded. Titles and abstracts of the remaining papers were assessed against the inclusion and exclusion criteria independently by both authors. The resulting papers were pooled and disagreements were resolved through discussion based on the full text article. Following this stage, a standardized form was used to summarize the information in each article. The variables extracted were: reference/ country, aim of the study, study design, year of publication, and main finding/results.”
“A data-charting form was jointly developed by two reviewers to determine which variables to extract. The two reviewers independently charted the data, discussed the results and continuously updated the data-charting form in an iterative process.”
Lenzen, S. A., Daniëls, R., van Bokhoven, M. A., van der Weijden, T., & Beurskens, A. (2017). Disentangling self-management goal setting and action planning: A scoping review. PloS one , 12 (11), e0188822.
If an article was eligible for inclusion in this study, data related to the patient-centered care framework or model presented in the article was extracted by the lead author and reviewed by a second author (JCM). Data extracted from the reviewed patient-centered care frameworks and models was entered into data extraction records and synthesized in summary format. Data were systematically charted using the data charting form developed in Microsoft Excel. Information on authorship, article type, population, and patientcentered care approach were recorded on this form. A second data charting form was developed to chart data on the communication systematic reviews identified. Information on clinical context, patient-centered care focus, number of studies reviewed and key findings were recorded on this form.
Constand, M. K., MacDermid, J. C., Dal Bello-Haas, V., & Law, M. (2014). Scoping review of patient-centered care approaches in healthcare. BMC health services research , 14 , 1-9.
The final charting form, which clearly defines each item, should be included in the scoping review as an appendix or supplementary file, if possible.
“We abstracted data on article characteristics (e.g., country of origin, funder), engagement characteristics and contextual factors (e.g., type of knowledge user, country income level, type of engagement activity, frequency and intensity of engagement, use of a framework to inform the intervention), barriers and facilitators to engagement, and results of any formal assessment of engagement (e.g., attitudes, beliefs, knowledge, benefits, unintended consequences).”
Tricco, A. C., Zarin, W., Rios, P., Nincic, V., Khan, P. A., Ghassemi, M., … & Langlois, E. V. (2018). Engaging policy-makers, health system managers, and policy analysts in the knowledge synthesis process: a scoping review. Implementation Science , 13 , 1-19.
The key element of a scoping review is the synthesis: that is the process that brings together the findings from the set of included studies in order to draw conclusions based on the body of evidence.
Data synthesis in a scoping review involves collating, combining, and summarizing findings from the included studies.
This process aims to provide a reliable and comprehensive answer to the review question by considering the strength of the evidence, examining the consistency of observed effects, and investigating any inconsistencies.
The data synthesis will be presented in the results section of the scoping review.
Scoping reviews often use a more descriptive approach to synthesis, summarizing the types of evidence available, key findings, and research gaps.
Remember, the goal in a scoping review is not to critically appraise the quality of individual studies or to provide a definitive answer to a narrow research question.
Instead, the synthesis aims to provide a broad overview of the field, mapping out the existing literature and identifying areas for further research.
This descriptive approach allows for a comprehensive understanding of the landscape of a particular research area.
“We grouped the studies by the types of behavior they analyzed, and summarized the type of settings, populations and study designs for each group, along with the measures used and broad findings. Where we identified a systematic review, we counted the number of studies included in the review that potentially met our inclusion criteria and noted how many studies had been missed by our search.”
Hutchinson, J., Prady, S. L., Smith, M. A., White, P. C., & Graham, H. M. (2015). A scoping review of observational studies examining relationships between environmental behaviors and health behaviors. International journal of environmental research and public health , 12 (5), 4833-4858.
The findings should be presented in a clear and logical way that answers the research question(s). This section might include tables, figures, or narrative summaries to illustrate the data.
Write a clear, concise narrative that brings together all of these elements. This should provide readers with a comprehensive overview of the current state of knowledge in the field, highlighting both what is known and what remains to be explored.
The primary goal of a narrative summary is to weave together the information extracted from multiple sources into a cohesive and understandable narrative. This story should focus on why a specific action is necessary, should be discontinued, or lacks sufficient evidence to determine its efficacy
A well-crafted narrative summary often utilizes headings and subheadings to organize the synthesized information logically.
This approach makes it easier for readers to follow the thought process and understand the relationships between different pieces of evidence.
Strategies on how to be sensitive to patient needs were primarily discussed in the qualitative research articles included in this review. Such strategies included acknowledging and adapting to unique patient identifiers [19,24,25]. For example, clinicians are urged to observe and reflect on fluctuating levels of patient alertness, patient comfort levels in the presence or absence of family members, and different communication barriers such as hearing loss, in order to facilitate clinical interactions [15,19,22]. Of the articles reviewed, 58% identified that careful observation of unique patient characteristics is necessary to providing care that will lead to optimal patient receptiveness and positive health outcomes.
While narrative summaries primarily use text, incorporating tables, charts, or diagrams can enhance clarity, particularly when presenting complex data patterns.
However, always accompany these visual aids with a clear textual explanation to ensure comprehensive understanding.
Using a PRISMA flowchart in a scoping review is considered good practice. It promotes transparency and allows for a clear understanding of how sources were selected.
The flowchart illustrates the step-by-step process of screening, filtering, and selecting studies based on predefined inclusion and exclusion criteria.
Petersen, B., Koshy-Chenthittayil, S., DeArmond, M., & Caromile, L. A. (2023). Assessment of diversity-based approaches used by American Universities to increase recruitment and retention of biomedical sciences research faculty members: A scoping review protocol. Plos one , 18 (6), e0276089.
Summarizing the evidence in relation to the purpose of the review, making conclusions and noting any implications of the findings.
It is also essential to remember that scoping reviews, unlike systematic reviews, do not aim to provide concrete recommendations for practice or policy.
Their primary function is to map the existing evidence, identify knowledge gaps, and clarify concepts, rather than synthesize results for direct application in clinical or policy settings
“In this scoping review we identified 88 primary studies addressing dissemination and implementation research across various settings of dementia care published between 1998 and 2015. Our findings indicate a paucity of research focusing specifically on dissemination of knowledge within dementia care and a limited number of studies on implementation in this area. We also found that training and educating professionals, developing stakeholder interrelationships, and using evaluative and iterative strategies are frequently employed to introduce and promote change in practice. However, although important and feasible, these strategies only partly address what is repeatedly highlighted in the evidence base: that organisational factors are reported as the main barrier to implementation of knowledge within dementia care. Moreover, included studies clearly support an increased effort to improve the quality of dementia care provided in residential settings in the last decade.”
Lourida, I., Abbott, R. A., Rogers, M., Lang, I. A., Stein, K., Kent, B., & Thompson Coon, J. (2017). Dissemination and implementation research in dementia care: a systematic scoping review and evidence map. BMC geriatrics , 17 , 1-12.
When considering the limitations of a review process, particularly scoping reviews, it’s essential to acknowledge that the goal is breadth, not depth, of information.
This means that unlike systematic reviews, scoping reviews generally don’t involve a formal appraisal of the methodological quality of included studies, unless specifically required by the review’s aim.
“Our scoping review has some limitations. To make our review more feasible, we were only able to include a random sample of rapid reviews from websites of rapid review producers. Further adding to this issue is that many rapid reviews contain proprietary information and are not publicly available. As such, our results are only likely generalizable to rapid reviews that are publicly available. Furthermore, this scoping review was an enormous undertaking and our results are only up to date as of May 2013.”
Tricco, A. C., Antony, J., Zarin, W., Strifler, L., Ghassemi, M., Ivory, J., … & Straus, S. E. (2015). A scoping review of rapid review methods. BMC medicine , 13 , 1-15.
Discuss implications:.
“The lack of evidence to support physiotherapy interventions for this population appears to pose a challenge to physiotherapists. The aim of this scoping review was to identify gaps in the literature which may guide a future systematic review. However, the lack of evidence found means that undertaking a systematic review is not appropriate or necessary […]. This advocates high quality research being needed to determine what physiotherapy techniques may be of benefit for this population and to help guide physiotherapists as how to deliver this.”
Hall, A. J., Lang, I. A., Endacott, R., Hall, A., & Goodwin, V. A. (2017). Physiotherapy interventions for people with dementia and a hip fracture—a scoping review of the literature. Physiotherapy , 103 (4), 361-368.
You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.
All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .
Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.
Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.
Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.
Original Submission Date Received: .
Find support for a specific problem in the support section of our website.
Please let us know what you think of our products and services.
Visit our dedicated information section to learn more about MDPI.
How can soil quality be accurately and quickly studied a review.
2. soil quality definitions and evaluation, 3. soil quality index, 4. soil quality indicators, 5. approaches to soil quality assessment, 5.1. conventional approaches, 5.2. novel approaches, 5.2.1. gis and rs, 5.2.2. artificial intelligence, machine learning, and deep learning.
7. conclusions, data availability statement, conflicts of interest.
Click here to enlarge figure
Parameters | Conventional Approaches | Novel Approaches |
---|---|---|
Cost | Expensive | Cheap |
Hazards | Need potentially harmful compounds that could ruin the sample | Eco-friendly |
Ability | Demand a mathematical action that is linear and is unable to satisfy the goal function under nonlinear circumstances | Successful in finding solutions to challenging issues in a variety of agricultural research domains, particularly when the underlying relationship is murky. Capable of handling errors, adapting to working with partial facts, and making conclusions even in unpredictable circumstances |
Time consumption | High | Low |
Benefits for community | Provide an overview of soil quality in the studied area | Allow farmers to use data to manage a variety of challenges, including crop and livestock management, agriculture, and the management of water and soil |
Procedure | Need complex calculations | Predicting soil quality has the potential to be extremely accurate and efficient with machine learning (ML) and deep learning (DL). Because machine learning and deep learning can forecast the soil quality index, this makes their application unique. Satellite imagery offers exact coverage of spatial data in a reliable, timely, and economical manner. Specially referenced data can be updated, stored, and manipulated by the GIS. Furthermore, spatially referenced datasets can be integrated via GIS and used in spatial modeling |
The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
El Behairy, R.A.; El Arwash, H.M.; El Baroudy, A.A.; Ibrahim, M.M.; Mohamed, E.S.; Kucher, D.E.; Shokr, M.S. How Can Soil Quality Be Accurately and Quickly Studied? A Review. Agronomy 2024 , 14 , 1682. https://doi.org/10.3390/agronomy14081682
El Behairy RA, El Arwash HM, El Baroudy AA, Ibrahim MM, Mohamed ES, Kucher DE, Shokr MS. How Can Soil Quality Be Accurately and Quickly Studied? A Review. Agronomy . 2024; 14(8):1682. https://doi.org/10.3390/agronomy14081682
El Behairy, Radwa A., Hasnaa M. El Arwash, Ahmed A. El Baroudy, Mahmoud M. Ibrahim, Elsayed Said Mohamed, Dmitry E. Kucher, and Mohamed S. Shokr. 2024. "How Can Soil Quality Be Accurately and Quickly Studied? A Review" Agronomy 14, no. 8: 1682. https://doi.org/10.3390/agronomy14081682
Article access statistics, further information, mdpi initiatives, follow mdpi.
Subscribe to receive issue release notifications and newsletters from MDPI journals
Objectives To identify evidence-practice gaps regarding shoulder injury risk factors in competitive swimmers.
Methods We gathered insights from 27 swimming experts including elite swimmers, coaches, high-performance staff and applied researchers using Concept Mapping. Participants brainstormed, sorted and rated (from 1 (least) to 10 (most) important and modifiable) their ideas of shoulder injury risk factors in competitive swimmers. Proposed risk factors rated above the grand mean for importance (6.2±0.4) or modifiability (6.5±0.5) ratings were considered highly important/modifiable. Expert opinions were then juxtaposed with systematic review findings to identify overlaps or convergences.
Results Brainstorming generated 126 proposed shoulder injury risk factors for competitive swimmers, subsequently refined to 61 unique proposed risk factors by removing duplicates and combining similar responses. The participants sorted the 61 risk factors into seven distinct clusters. Experts perceived 36/61 proposed risk factors as highly important, of which 6 were supported by literature, 6 showed no association with injury, 2 had conflicting evidence and the remaining 22 have not yet been investigated, suggesting an evidence-practice gap. Three proposed risk factors ‘inconsistent training load’, ‘poor stroke technique’ and ‘low posterior shoulder strength-endurance’ exhibited high perceived importance, high perceived modifiability and supporting evidence.
Conclusion An evidence-practice gap was identified for 28 proposed risk factors perceived as highly important by swimming experts despite either (1) no relevant empirical research (n=22), or (2) no association with injury (n=6) from synthesised evidence. Greater collaboration between researchers and practitioners is needed to effectively address shoulder injury risk factors in competitive swimmers.
Data are available upon reasonable request. All collected data is presented in-text. Please email corresponding author for any further information.
https://doi.org/10.1136/bjsports-2023-108023
Request permissions.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
X @a_k_mckenzie, @andrea_hams, @Jon_Headrick, @AlexDonaldson13, @josephcoyne, @duhigs
Contributors This study was part of AKM's (guarantor) research programme supervised by SJD, AH and JH. AD provided expert guidance on concept mapping methodologies. JC provided practical input as an elite swimming strength and conditioning practitioner. RD aided with concept mapping analysis, data synthesis and manuscript drafting.
Funding This project was funded by a $A3000 grant award by Swimming Australia.
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
IMAGES
VIDEO
COMMENTS
What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic.
This paper discusses literature review as a methodology for conducting research and offers an overview of different types of reviews, as well as some guidelines to how to both conduct and evaluate a literature review paper. It also discusses common pitfalls and how to get literature reviews published. 1.
A literature review is a comprehensive analysis of existing research on a topic, identifying trends, gaps, and insights to inform new scholarly contributions. Read this comprehensive article to learn how to write a literature review, with examples.
A literature review is a review and synthesis of existing research on a topic or research question. A literature review is meant to analyze the scholarly literature, make connections across writings and identify strengths, weaknesses, trends, and missing conversations. A literature review should address different aspects of a topic as it ...
A literature or narrative review is a comprehensive review and analysis of the published literature on a specific topic or research question. The literature that is reviewed contains: books, articles, academic articles, conference proceedings, association papers, and dissertations. It contains the most pertinent studies and points to important ...
Narrative review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings ...
A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.
Literature Review is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works. Also, we can define a literature review as the ...
A literature review is a discussion of the literature (aka. the "research" or "scholarship") surrounding a certain topic. A good literature review doesn't simply summarize the existing material, but provides thoughtful synthesis and analysis.
The literature review can serve various functions in the contexts of education and research. It aids in identifying knowledge gaps, informing research methodology, and developing a theoretical framework during the planning stages of a research study or project, as well as reporting of review findings in the context of the existing literature.
This paper draws input from a study that employed a systematic literature review as its main source of data. A systematic review can be explained as a research method and process for identifying ...
Writing a literature review requires a range of skills to gather, sort, evaluate and summarise peer-reviewed published data into a relevant and informative unbiased narrative. Digital access to research papers, academic texts, review articles, reference databases and public data sets are all sources of information that are available to enrich ...
In the field of research, the term method represents the specific approaches and procedures that the researcher systematically utilizes that are manifested in the research design, sampling design, data collec-tion, data analysis, data interpretation, and so forth. The literature review represents a method because the literature reviewer chooses ...
Definition A literature review involves researching, reading, analyzing, evaluating, and summarizing scholarly literature (typically journals and articles) about a specific topic. The results of a literature review may be an entire report or article OR may be part of a article, thesis, dissertation, or grant proposal. A literature review helps the author learn about the history and nature of ...
The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ).
What is a literature review? A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question. That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.
In this article, through a systematic search on the methodology of literature review, we categorize a typology of literature reviews, discuss steps in conducting a systematic literature review, and provide suggestions on how to enhance rigor in literature reviews in planning education and research.
A literature review is a survey of everything that has been written about a particular topic, theory, or research question. The word "literature" means "sources of information". The literature will inform you about the research that has already been conducted on your chosen subject. This is important because we do not want to repeat ...
Abstract Performing a literature review is a critical first step in research to understanding the state-of-the-art and identifying gaps and challenges in the field. A systematic literature review is a method which sets out a series of steps to methodically organize the review. In this paper, we present a guide designed for researchers and in particular early-stage researchers in the computer ...
Introduction Researchers and practitioners rely on literature reviews to synthesize large bodies of knowledge. Many types of literature reviews have been developed, each targeting a specific purpose. However, these syntheses are hampered if the review type's paradigmatic roots, methods, and markers of rigor are only vaguely understood. One literature review type whose methodology has yet to ...
Sometimes in your literature review, you might need to discuss and evaluate relevant research methodologies in order to justify your own choice of research methodology. When searching for literature on research methodologies it is important to search across a range of sources. No single information source will supply all that you need.
A scoping review is a type of research synthesis that aims to map the existing literature on a broad topic to identify key concepts, gaps, and types of evidence.
Evaluating soil quality is crucial for ensuring the sustainable use of agricultural lands. This review examines the definition, evaluation methods, indicator selection, and relevant case studies. The concept of soil quality supplements soil science research by deepening our understanding of soils and aiding in the allocation of resources as agriculture intensifies to meet rising global demand.
Objectives To identify evidence-practice gaps regarding shoulder injury risk factors in competitive swimmers. Methods We gathered insights from 27 swimming experts including elite swimmers, coaches, high-performance staff and applied researchers using Concept Mapping. Participants brainstormed, sorted and rated (from 1 (least) to 10 (most) important and modifiable) their ideas of shoulder ...