3 a systematic literature review is

Systematic Reviews

• What is a Systematic Review?

A systematic review is an evidence synthesis that uses explicit, reproducible methods to perform a comprehensive literature search and critical appraisal of individual studies and that uses appropriate statistical techniques to combine these valid studies.

Key Characteristics of a Systematic Review:

Generally, systematic reviews must have:

• a clearly stated set of objectives with pre-defined eligibility criteria for studies
• an explicit, reproducible methodology
• a systematic search that attempts to identify all studies that would meet the eligibility criteria
• an assessment of the validity of the findings of the included studies, for example through the assessment of the risk of bias
• a systematic presentation, and synthesis, of the characteristics and findings of the included studies.

A meta-analysis is a systematic review that uses quantitative methods to synthesize and summarize the pooled data from included studies.

Additional Information

• How-to Books
• Beyond Health Sciences

• Cochrane Handbook For Systematic Reviews of Interventions Provides guidance to authors for the preparation of Cochrane Intervention reviews. Chapter 6 covers searching for reviews.
• Systematic Reviews: CRD’s Guidance for Undertaking Reviews in Health Care From The University of York Centre for Reviews and Dissemination: Provides practical guidance for undertaking evidence synthesis based on a thorough understanding of systematic review methodology. It presents the core principles of systematic reviewing, and in complementary chapters, highlights issues that are specific to reviews of clinical tests, public health interventions, adverse effects, and economic evaluations.
• Cornell, Sytematic Reviews and Evidence Synthesis Beyond the Health Sciences Video series geared for librarians but very informative about searching outside medicine.
• << Previous: Getting Started
• Next: Levels of Evidence >>
• Getting Started
• Levels of Evidence
• Locating Systematic Reviews
• Searching Systematically
• Developing Answerable Questions
• Identifying Synonyms & Related Terms
• Using Truncation and Wildcards
• Identifying Search Limits/Exclusion Criteria
• Keyword vs. Subject Searching
• Where to Search
• Search Filters
• Sensitivity vs. Precision
• Core Databases
• Other Databases
• Clinical Trial Registries
• Conference Presentations
• Databases Indexing Grey Literature
• Web Searching
• Handsearching
• Citation Indexes
• Documenting the Search Process
• Managing your Review

Research Support

• Last Updated: May 1, 2024 4:09 PM
• URL: https://guides.library.ucdavis.edu/systematic-reviews

Easy guide to conducting a systematic review

Affiliations.

• 1 Discipline of Child and Adolescent Health, University of Sydney, Sydney, New South Wales, Australia.
• 2 Department of Nephrology, The Children's Hospital at Westmead, Sydney, New South Wales, Australia.
• 3 Education Department, The Children's Hospital at Westmead, Sydney, New South Wales, Australia.
• PMID: 32364273
• DOI: 10.1111/jpc.14853

A systematic review is a type of study that synthesises research that has been conducted on a particular topic. Systematic reviews are considered to provide the highest level of evidence on the hierarchy of evidence pyramid. Systematic reviews are conducted following rigorous research methodology. To minimise bias, systematic reviews utilise a predefined search strategy to identify and appraise all available published literature on a specific topic. The meticulous nature of the systematic review research methodology differentiates a systematic review from a narrative review (literature review or authoritative review). This paper provides a brief step by step summary of how to conduct a systematic review, which may be of interest for clinicians and researchers.

Keywords: research; research design; systematic review.

© 2020 Paediatrics and Child Health Division (The Royal Australasian College of Physicians).

Publication types

• Systematic Review
• Research Design*

• Locations and Hours
• UCLA Library
• Research Guides
• Biomedical Library Guides

Systematic Reviews

• Types of Literature Reviews

What Makes a Systematic Review Different from Other Types of Reviews?

• Planning Your Systematic Review
• Database Searching
• Creating the Search
• Search Filters and Hedges
• Grey Literature
• Managing and Appraising Results
• Further Resources

Reproduced from Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91–108. doi:10.1111/j.1471-1842.2009.00848.x

• << Previous: Home
• Next: Planning Your Systematic Review >>
• Last Updated: Apr 17, 2024 2:02 PM
• URL: https://guides.library.ucla.edu/systematicreviews

• Research Process

Systematic Literature Review or Literature Review?

• 3 minute read
• 44.6K views

Table of Contents

As a researcher, you may be required to conduct a literature review. But what kind of review do you need to complete? Is it a systematic literature review or a standard literature review? In this article, we’ll outline the purpose of a systematic literature review, the difference between literature review and systematic review, and other important aspects of systematic literature reviews.

What is a Systematic Literature Review?

The purpose of systematic literature reviews is simple. Essentially, it is to provide a high-level of a particular research question. This question, in and of itself, is highly focused to match the review of the literature related to the topic at hand. For example, a focused question related to medical or clinical outcomes.

The components of a systematic literature review are quite different from the standard literature review research theses that most of us are used to (more on this below). And because of the specificity of the research question, typically a systematic literature review involves more than one primary author. There’s more work related to a systematic literature review, so it makes sense to divide the work among two or three (or even more) researchers.

Your systematic literature review will follow very clear and defined protocols that are decided on prior to any review. This involves extensive planning, and a deliberately designed search strategy that is in tune with the specific research question. Every aspect of a systematic literature review, including the research protocols, which databases are used, and dates of each search, must be transparent so that other researchers can be assured that the systematic literature review is comprehensive and focused.

Most systematic literature reviews originated in the world of medicine science. Now, they also include any evidence-based research questions. In addition to the focus and transparency of these types of reviews, additional aspects of a quality systematic literature review includes:

• Clear and concise review and summary
• Comprehensive coverage of the topic
• Accessibility and equality of the research reviewed

Systematic Review vs Literature Review

The difference between literature review and systematic review comes back to the initial research question. Whereas the systematic review is very specific and focused, the standard literature review is much more general. The components of a literature review, for example, are similar to any other research paper. That is, it includes an introduction, description of the methods used, a discussion and conclusion, as well as a reference list or bibliography.

A systematic review, however, includes entirely different components that reflect the specificity of its research question, and the requirement for transparency and inclusion. For instance, the systematic review will include:

• Eligibility criteria for included research
• A description of the systematic research search strategy
• An assessment of the validity of reviewed research
• Interpretations of the results of research included in the review

As you can see, contrary to the general overview or summary of a topic, the systematic literature review includes much more detail and work to compile than a standard literature review. Indeed, it can take years to conduct and write a systematic literature review. But the information that practitioners and other researchers can glean from a systematic literature review is, by its very nature, exceptionally valuable.

This is not to diminish the value of the standard literature review. The importance of literature reviews in research writing is discussed in this article . It’s just that the two types of research reviews answer different questions, and, therefore, have different purposes and roles in the world of research and evidence-based writing.

Systematic Literature Review vs Meta Analysis

It would be understandable to think that a systematic literature review is similar to a meta analysis. But, whereas a systematic review can include several research studies to answer a specific question, typically a meta analysis includes a comparison of different studies to suss out any inconsistencies or discrepancies. For more about this topic, check out Systematic Review VS Meta-Analysis article.

Language Editing Plus

With Elsevier’s Language Editing Plus services , you can relax with our complete language review of your systematic literature review or literature review, or any other type of manuscript or scientific presentation. Our editors are PhD or PhD candidates, who are native-English speakers. Language Editing Plus includes checking the logic and flow of your manuscript, reference checks, formatting in accordance to your chosen journal and even a custom cover letter. Our most comprehensive editing package, Language Editing Plus also includes any English-editing needs for up to 180 days.

• Publication Recognition

How to Make a PowerPoint Presentation of Your Research Paper

• Manuscript Preparation

What is and How to Write a Good Hypothesis in Research?

You may also like.

Descriptive Research Design and Its Myriad Uses

Five Common Mistakes to Avoid When Writing a Biomedical Research Paper

Making Technical Writing in Environmental Engineering Accessible

To Err is Not Human: The Dangers of AI-assisted Academic Writing

When Data Speak, Listen: Importance of Data Collection and Analysis Methods

Choosing the Right Research Methodology: A Guide for Researchers

Why is data validation important in research?

Writing a good review article

Input your search keywords and press Enter.

• Open access
• Published: 14 August 2018

Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies

• Chris Cooper   ORCID: orcid.org/0000-0003-0864-5607 1 ,
• Andrew Booth 2 ,
• Jo Varley-Campbell 1 ,
• Nicky Britten 3 &
• Ruth Garside 4  

BMC Medical Research Methodology volume  18 , Article number:  85 ( 2018 ) Cite this article

202k Accesses

203 Citations

118 Altmetric

Metrics details

Systematic literature searching is recognised as a critical component of the systematic review process. It involves a systematic search for studies and aims for a transparent report of study identification, leaving readers clear about what was done to identify studies, and how the findings of the review are situated in the relevant evidence.

Information specialists and review teams appear to work from a shared and tacit model of the literature search process. How this tacit model has developed and evolved is unclear, and it has not been explicitly examined before.

The purpose of this review is to determine if a shared model of the literature searching process can be detected across systematic review guidance documents and, if so, how this process is reported in the guidance and supported by published studies.

A literature review.

Two types of literature were reviewed: guidance and published studies. Nine guidance documents were identified, including: The Cochrane and Campbell Handbooks. Published studies were identified through ‘pearl growing’, citation chasing, a search of PubMed using the systematic review methods filter, and the authors’ topic knowledge.

The relevant sections within each guidance document were then read and re-read, with the aim of determining key methodological stages. Methodological stages were identified and defined. This data was reviewed to identify agreements and areas of unique guidance between guidance documents. Consensus across multiple guidance documents was used to inform selection of ‘key stages’ in the process of literature searching.

Eight key stages were determined relating specifically to literature searching in systematic reviews. They were: who should literature search, aims and purpose of literature searching, preparation, the search strategy, searching databases, supplementary searching, managing references and reporting the search process.

Conclusions

Eight key stages to the process of literature searching in systematic reviews were identified. These key stages are consistently reported in the nine guidance documents, suggesting consensus on the key stages of literature searching, and therefore the process of literature searching as a whole, in systematic reviews. Further research to determine the suitability of using the same process of literature searching for all types of systematic review is indicated.

Peer Review reports

Systematic literature searching is recognised as a critical component of the systematic review process. It involves a systematic search for studies and aims for a transparent report of study identification, leaving review stakeholders clear about what was done to identify studies, and how the findings of the review are situated in the relevant evidence.

Information specialists and review teams appear to work from a shared and tacit model of the literature search process. How this tacit model has developed and evolved is unclear, and it has not been explicitly examined before. This is in contrast to the information science literature, which has developed information processing models as an explicit basis for dialogue and empirical testing. Without an explicit model, research in the process of systematic literature searching will remain immature and potentially uneven, and the development of shared information models will be assumed but never articulated.

One way of developing such a conceptual model is by formally examining the implicit “programme theory” as embodied in key methodological texts. The aim of this review is therefore to determine if a shared model of the literature searching process in systematic reviews can be detected across guidance documents and, if so, how this process is reported and supported.

Identifying guidance

Key texts (henceforth referred to as “guidance”) were identified based upon their accessibility to, and prominence within, United Kingdom systematic reviewing practice. The United Kingdom occupies a prominent position in the science of health information retrieval, as quantified by such objective measures as the authorship of papers, the number of Cochrane groups based in the UK, membership and leadership of groups such as the Cochrane Information Retrieval Methods Group, the HTA-I Information Specialists’ Group and historic association with such centres as the UK Cochrane Centre, the NHS Centre for Reviews and Dissemination, the Centre for Evidence Based Medicine and the National Institute for Clinical Excellence (NICE). Coupled with the linguistic dominance of English within medical and health science and the science of systematic reviews more generally, this offers a justification for a purposive sample that favours UK, European and Australian guidance documents.

Nine guidance documents were identified. These documents provide guidance for different types of reviews, namely: reviews of interventions, reviews of health technologies, reviews of qualitative research studies, reviews of social science topics, and reviews to inform guidance.

Whilst these guidance documents occasionally offer additional guidance on other types of systematic reviews, we have focused on the core and stated aims of these documents as they relate to literature searching. Table  1 sets out: the guidance document, the version audited, their core stated focus, and a bibliographical pointer to the main guidance relating to literature searching.

Once a list of key guidance documents was determined, it was checked by six senior information professionals based in the UK for relevance to current literature searching in systematic reviews.

Identifying supporting studies

In addition to identifying guidance, the authors sought to populate an evidence base of supporting studies (henceforth referred to as “studies”) that contribute to existing search practice. Studies were first identified by the authors from their knowledge on this topic area and, subsequently, through systematic citation chasing key studies (‘pearls’ [ 1 ]) located within each key stage of the search process. These studies are identified in Additional file  1 : Appendix Table 1. Citation chasing was conducted by analysing the bibliography of references for each study (backwards citation chasing) and through Google Scholar (forward citation chasing). A search of PubMed using the systematic review methods filter was undertaken in August 2017 (see Additional file 1 ). The search terms used were: (literature search*[Title/Abstract]) AND sysrev_methods[sb] and 586 results were returned. These results were sifted for relevance to the key stages in Fig.  1 by CC.

The key stages of literature search guidance as identified from nine key texts

Extracting the data

To reveal the implicit process of literature searching within each guidance document, the relevant sections (chapters) on literature searching were read and re-read, with the aim of determining key methodological stages. We defined a key methodological stage as a distinct step in the overall process for which specific guidance is reported, and action is taken, that collectively would result in a completed literature search.

The chapter or section sub-heading for each methodological stage was extracted into a table using the exact language as reported in each guidance document. The lead author (CC) then read and re-read these data, and the paragraphs of the document to which the headings referred, summarising section details. This table was then reviewed, using comparison and contrast to identify agreements and areas of unique guidance. Consensus across multiple guidelines was used to inform selection of ‘key stages’ in the process of literature searching.

Having determined the key stages to literature searching, we then read and re-read the sections relating to literature searching again, extracting specific detail relating to the methodological process of literature searching within each key stage. Again, the guidance was then read and re-read, first on a document-by-document-basis and, secondly, across all the documents above, to identify both commonalities and areas of unique guidance.

Results and discussion

Our findings.

We were able to identify consensus across the guidance on literature searching for systematic reviews suggesting a shared implicit model within the information retrieval community. Whilst the structure of the guidance varies between documents, the same key stages are reported, even where the core focus of each document is different. We were able to identify specific areas of unique guidance, where a document reported guidance not summarised in other documents, together with areas of consensus across guidance.

Unique guidance

Only one document provided guidance on the topic of when to stop searching [ 2 ]. This guidance from 2005 anticipates a topic of increasing importance with the current interest in time-limited (i.e. “rapid”) reviews. Quality assurance (or peer review) of literature searches was only covered in two guidance documents [ 3 , 4 ]. This topic has emerged as increasingly important as indicated by the development of the PRESS instrument [ 5 ]. Text mining was discussed in four guidance documents [ 4 , 6 , 7 , 8 ] where the automation of some manual review work may offer efficiencies in literature searching [ 8 ].

Agreement between guidance: Defining the key stages of literature searching

Where there was agreement on the process, we determined that this constituted a key stage in the process of literature searching to inform systematic reviews.

From the guidance, we determined eight key stages that relate specifically to literature searching in systematic reviews. These are summarised at Fig. 1 . The data extraction table to inform Fig. 1 is reported in Table  2 . Table 2 reports the areas of common agreement and it demonstrates that the language used to describe key stages and processes varies significantly between guidance documents.

For each key stage, we set out the specific guidance, followed by discussion on how this guidance is situated within the wider literature.

Key stage one: Deciding who should undertake the literature search

The guidance.

Eight documents provided guidance on who should undertake literature searching in systematic reviews [ 2 , 4 , 6 , 7 , 8 , 9 , 10 , 11 ]. The guidance affirms that people with relevant expertise of literature searching should ‘ideally’ be included within the review team [ 6 ]. Information specialists (or information scientists), librarians or trial search co-ordinators (TSCs) are indicated as appropriate researchers in six guidance documents [ 2 , 7 , 8 , 9 , 10 , 11 ].

How the guidance corresponds to the published studies

The guidance is consistent with studies that call for the involvement of information specialists and librarians in systematic reviews [ 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 ] and which demonstrate how their training as ‘expert searchers’ and ‘analysers and organisers of data’ can be put to good use [ 13 ] in a variety of roles [ 12 , 16 , 20 , 21 , 24 , 25 , 26 ]. These arguments make sense in the context of the aims and purposes of literature searching in systematic reviews, explored below. The need for ‘thorough’ and ‘replicable’ literature searches was fundamental to the guidance and recurs in key stage two. Studies have found poor reporting, and a lack of replicable literature searches, to be a weakness in systematic reviews [ 17 , 18 , 27 , 28 ] and they argue that involvement of information specialists/ librarians would be associated with better reporting and better quality literature searching. Indeed, Meert et al. [ 29 ] demonstrated that involving a librarian as a co-author to a systematic review correlated with a higher score in the literature searching component of a systematic review [ 29 ]. As ‘new styles’ of rapid and scoping reviews emerge, where decisions on how to search are more iterative and creative, a clear role is made here too [ 30 ].

Knowing where to search for studies was noted as important in the guidance, with no agreement as to the appropriate number of databases to be searched [ 2 , 6 ]. Database (and resource selection more broadly) is acknowledged as a relevant key skill of information specialists and librarians [ 12 , 15 , 16 , 31 ].

Whilst arguments for including information specialists and librarians in the process of systematic review might be considered self-evident, Koffel and Rethlefsen [ 31 ] have questioned if the necessary involvement is actually happening [ 31 ].

Key stage two: Determining the aim and purpose of a literature search

The aim: Five of the nine guidance documents use adjectives such as ‘thorough’, ‘comprehensive’, ‘transparent’ and ‘reproducible’ to define the aim of literature searching [ 6 , 7 , 8 , 9 , 10 ]. Analogous phrases were present in a further three guidance documents, namely: ‘to identify the best available evidence’ [ 4 ] or ‘the aim of the literature search is not to retrieve everything. It is to retrieve everything of relevance’ [ 2 ] or ‘A systematic literature search aims to identify all publications relevant to the particular research question’ [ 3 ]. The Joanna Briggs Institute reviewers’ manual was the only guidance document where a clear statement on the aim of literature searching could not be identified. The purpose of literature searching was defined in three guidance documents, namely to minimise bias in the resultant review [ 6 , 8 , 10 ]. Accordingly, eight of nine documents clearly asserted that thorough and comprehensive literature searches are required as a potential mechanism for minimising bias.

The need for thorough and comprehensive literature searches appears as uniform within the eight guidance documents that describe approaches to literature searching in systematic reviews of effectiveness. Reviews of effectiveness (of intervention or cost), accuracy and prognosis, require thorough and comprehensive literature searches to transparently produce a reliable estimate of intervention effect. The belief that all relevant studies have been ‘comprehensively’ identified, and that this process has been ‘transparently’ reported, increases confidence in the estimate of effect and the conclusions that can be drawn [ 32 ]. The supporting literature exploring the need for comprehensive literature searches focuses almost exclusively on reviews of intervention effectiveness and meta-analysis. Different ‘styles’ of review may have different standards however; the alternative, offered by purposive sampling, has been suggested in the specific context of qualitative evidence syntheses [ 33 ].

What is a comprehensive literature search?

Whilst the guidance calls for thorough and comprehensive literature searches, it lacks clarity on what constitutes a thorough and comprehensive literature search, beyond the implication that all of the literature search methods in Table 2 should be used to identify studies. Egger et al. [ 34 ], in an empirical study evaluating the importance of comprehensive literature searches for trials in systematic reviews, defined a comprehensive search for trials as:

a search not restricted to English language;

where Cochrane CENTRAL or at least two other electronic databases had been searched (such as MEDLINE or EMBASE); and

at least one of the following search methods has been used to identify unpublished trials: searches for (I) conference abstracts, (ii) theses, (iii) trials registers; and (iv) contacts with experts in the field [ 34 ].

Tricco et al. (2008) used a similar threshold of bibliographic database searching AND a supplementary search method in a review when examining the risk of bias in systematic reviews. Their criteria were: one database (limited using the Cochrane Highly Sensitive Search Strategy (HSSS)) and handsearching [ 35 ].

Together with the guidance, this would suggest that comprehensive literature searching requires the use of BOTH bibliographic database searching AND supplementary search methods.

Comprehensiveness in literature searching, in the sense of how much searching should be undertaken, remains unclear. Egger et al. recommend that ‘investigators should consider the type of literature search and degree of comprehension that is appropriate for the review in question, taking into account budget and time constraints’ [ 34 ]. This view tallies with the Cochrane Handbook, which stipulates clearly, that study identification should be undertaken ‘within resource limits’ [ 9 ]. This would suggest that the limitations to comprehension are recognised but it raises questions on how this is decided and reported [ 36 ].

What is the point of comprehensive literature searching?

The purpose of thorough and comprehensive literature searches is to avoid missing key studies and to minimize bias [ 6 , 8 , 10 , 34 , 37 , 38 , 39 ] since a systematic review based only on published (or easily accessible) studies may have an exaggerated effect size [ 35 ]. Felson (1992) sets out potential biases that could affect the estimate of effect in a meta-analysis [ 40 ] and Tricco et al. summarize the evidence concerning bias and confounding in systematic reviews [ 35 ]. Egger et al. point to non-publication of studies, publication bias, language bias and MEDLINE bias, as key biases [ 34 , 35 , 40 , 41 , 42 , 43 , 44 , 45 , 46 ]. Comprehensive searches are not the sole factor to mitigate these biases but their contribution is thought to be significant [ 2 , 32 , 34 ]. Fehrmann (2011) suggests that ‘the search process being described in detail’ and that, where standard comprehensive search techniques have been applied, increases confidence in the search results [ 32 ].

Does comprehensive literature searching work?

Egger et al., and other study authors, have demonstrated a change in the estimate of intervention effectiveness where relevant studies were excluded from meta-analysis [ 34 , 47 ]. This would suggest that missing studies in literature searching alters the reliability of effectiveness estimates. This is an argument for comprehensive literature searching. Conversely, Egger et al. found that ‘comprehensive’ searches still missed studies and that comprehensive searches could, in fact, introduce bias into a review rather than preventing it, through the identification of low quality studies then being included in the meta-analysis [ 34 ]. Studies query if identifying and including low quality or grey literature studies changes the estimate of effect [ 43 , 48 ] and question if time is better invested updating systematic reviews rather than searching for unpublished studies [ 49 ], or mapping studies for review as opposed to aiming for high sensitivity in literature searching [ 50 ].

Aim and purpose beyond reviews of effectiveness

The need for comprehensive literature searches is less certain in reviews of qualitative studies, and for reviews where a comprehensive identification of studies is difficult to achieve (for example, in Public health) [ 33 , 51 , 52 , 53 , 54 , 55 ]. Literature searching for qualitative studies, and in public health topics, typically generates a greater number of studies to sift than in reviews of effectiveness [ 39 ] and demonstrating the ‘value’ of studies identified or missed is harder [ 56 ], since the study data do not typically support meta-analysis. Nussbaumer-Streit et al. (2016) have registered a review protocol to assess whether abbreviated literature searches (as opposed to comprehensive literature searches) has an impact on conclusions across multiple bodies of evidence, not only on effect estimates [ 57 ] which may develop this understanding. It may be that decision makers and users of systematic reviews are willing to trade the certainty from a comprehensive literature search and systematic review in exchange for different approaches to evidence synthesis [ 58 ], and that comprehensive literature searches are not necessarily a marker of literature search quality, as previously thought [ 36 ]. Different approaches to literature searching [ 37 , 38 , 59 , 60 , 61 , 62 ] and developing the concept of when to stop searching are important areas for further study [ 36 , 59 ].

The study by Nussbaumer-Streit et al. has been published since the submission of this literature review [ 63 ]. Nussbaumer-Streit et al. (2018) conclude that abbreviated literature searches are viable options for rapid evidence syntheses, if decision-makers are willing to trade the certainty from a comprehensive literature search and systematic review, but that decision-making which demands detailed scrutiny should still be based on comprehensive literature searches [ 63 ].

Key stage three: Preparing for the literature search

Six documents provided guidance on preparing for a literature search [ 2 , 3 , 6 , 7 , 9 , 10 ]. The Cochrane Handbook clearly stated that Cochrane authors (i.e. researchers) should seek advice from a trial search co-ordinator (i.e. a person with specific skills in literature searching) ‘before’ starting a literature search [ 9 ].

Two key tasks were perceptible in preparing for a literature searching [ 2 , 6 , 7 , 10 , 11 ]. First, to determine if there are any existing or on-going reviews, or if a new review is justified [ 6 , 11 ]; and, secondly, to develop an initial literature search strategy to estimate the volume of relevant literature (and quality of a small sample of relevant studies [ 10 ]) and indicate the resources required for literature searching and the review of the studies that follows [ 7 , 10 ].

Three documents summarised guidance on where to search to determine if a new review was justified [ 2 , 6 , 11 ]. These focused on searching databases of systematic reviews (The Cochrane Database of Systematic Reviews (CDSR) and the Database of Abstracts of Reviews of Effects (DARE)), institutional registries (including PROSPERO), and MEDLINE [ 6 , 11 ]. It is worth noting, however, that as of 2015, DARE (and NHS EEDs) are no longer being updated and so the relevance of this (these) resource(s) will diminish over-time [ 64 ]. One guidance document, ‘Systematic reviews in the Social Sciences’, noted, however, that databases are not the only source of information and unpublished reports, conference proceeding and grey literature may also be required, depending on the nature of the review question [ 2 ].

Two documents reported clearly that this preparation (or ‘scoping’) exercise should be undertaken before the actual search strategy is developed [ 7 , 10 ]).

The guidance offers the best available source on preparing the literature search with the published studies not typically reporting how their scoping informed the development of their search strategies nor how their search approaches were developed. Text mining has been proposed as a technique to develop search strategies in the scoping stages of a review although this work is still exploratory [ 65 ]. ‘Clustering documents’ and word frequency analysis have also been tested to identify search terms and studies for review [ 66 , 67 ]. Preparing for literature searches and scoping constitutes an area for future research.

Key stage four: Designing the search strategy

The Population, Intervention, Comparator, Outcome (PICO) structure was the commonly reported structure promoted to design a literature search strategy. Five documents suggested that the eligibility criteria or review question will determine which concepts of PICO will be populated to develop the search strategy [ 1 , 4 , 7 , 8 , 9 ]. The NICE handbook promoted multiple structures, namely PICO, SPICE (Setting, Perspective, Intervention, Comparison, Evaluation) and multi-stranded approaches [ 4 ].

With the exclusion of The Joanna Briggs Institute reviewers’ manual, the guidance offered detail on selecting key search terms, synonyms, Boolean language, selecting database indexing terms and combining search terms. The CEE handbook suggested that ‘search terms may be compiled with the help of the commissioning organisation and stakeholders’ [ 10 ].

The use of limits, such as language or date limits, were discussed in all documents [ 2 , 3 , 4 , 6 , 7 , 8 , 9 , 10 , 11 ].

Search strategy structure

The guidance typically relates to reviews of intervention effectiveness so PICO – with its focus on intervention and comparator - is the dominant model used to structure literature search strategies [ 68 ]. PICOs – where the S denotes study design - is also commonly used in effectiveness reviews [ 6 , 68 ]. As the NICE handbook notes, alternative models to structure literature search strategies have been developed and tested. Booth provides an overview on formulating questions for evidence based practice [ 69 ] and has developed a number of alternatives to the PICO structure, namely: BeHEMoTh (Behaviour of interest; Health context; Exclusions; Models or Theories) for use when systematically identifying theory [ 55 ]; SPICE (Setting, Perspective, Intervention, Comparison, Evaluation) for identification of social science and evaluation studies [ 69 ] and, working with Cooke and colleagues, SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type) [ 70 ]. SPIDER has been compared to PICO and PICOs in a study by Methley et al. [ 68 ].

The NICE handbook also suggests the use of multi-stranded approaches to developing literature search strategies [ 4 ]. Glanville developed this idea in a study by Whitting et al. [ 71 ] and a worked example of this approach is included in the development of a search filter by Cooper et al. [ 72 ].

Writing search strategies: Conceptual and objective approaches

Hausner et al. [ 73 ] provide guidance on writing literature search strategies, delineating between conceptually and objectively derived approaches. The conceptual approach, advocated by and explained in the guidance documents, relies on the expertise of the literature searcher to identify key search terms and then develop key terms to include synonyms and controlled syntax. Hausner and colleagues set out the objective approach [ 73 ] and describe what may be done to validate it [ 74 ].

The use of limits

The guidance documents offer direction on the use of limits within a literature search. Limits can be used to focus literature searching to specific study designs or by other markers (such as by date) which limits the number of studies returned by a literature search. The use of limits should be described and the implications explored [ 34 ] since limiting literature searching can introduce bias (explored above). Craven et al. have suggested the use of a supporting narrative to explain decisions made in the process of developing literature searches and this advice would usefully capture decisions on the use of search limits [ 75 ].

Key stage five: Determining the process of literature searching and deciding where to search (bibliographic database searching)

Table 2 summarises the process of literature searching as reported in each guidance document. Searching bibliographic databases was consistently reported as the ‘first step’ to literature searching in all nine guidance documents.

Three documents reported specific guidance on where to search, in each case specific to the type of review their guidance informed, and as a minimum requirement [ 4 , 9 , 11 ]. Seven of the key guidance documents suggest that the selection of bibliographic databases depends on the topic of review [ 2 , 3 , 4 , 6 , 7 , 8 , 10 ], with two documents noting the absence of an agreed standard on what constitutes an acceptable number of databases searched [ 2 , 6 ].

The guidance documents summarise ‘how to’ search bibliographic databases in detail and this guidance is further contextualised above in terms of developing the search strategy. The documents provide guidance of selecting bibliographic databases, in some cases stating acceptable minima (i.e. The Cochrane Handbook states Cochrane CENTRAL, MEDLINE and EMBASE), and in other cases simply listing bibliographic database available to search. Studies have explored the value in searching specific bibliographic databases, with Wright et al. (2015) noting the contribution of CINAHL in identifying qualitative studies [ 76 ], Beckles et al. (2013) questioning the contribution of CINAHL to identifying clinical studies for guideline development [ 77 ], and Cooper et al. (2015) exploring the role of UK-focused bibliographic databases to identify UK-relevant studies [ 78 ]. The host of the database (e.g. OVID or ProQuest) has been shown to alter the search returns offered. Younger and Boddy [ 79 ] report differing search returns from the same database (AMED) but where the ‘host’ was different [ 79 ].

The average number of bibliographic database searched in systematic reviews has risen in the period 1994–2014 (from 1 to 4) [ 80 ] but there remains (as attested to by the guidance) no consensus on what constitutes an acceptable number of databases searched [ 48 ]. This is perhaps because thinking about the number of databases searched is the wrong question, researchers should be focused on which databases were searched and why, and which databases were not searched and why. The discussion should re-orientate to the differential value of sources but researchers need to think about how to report this in studies to allow findings to be generalised. Bethel (2017) has proposed ‘search summaries’, completed by the literature searcher, to record where included studies were identified, whether from database (and which databases specifically) or supplementary search methods [ 81 ]. Search summaries document both yield and accuracy of searches, which could prospectively inform resource use and decisions to search or not to search specific databases in topic areas. The prospective use of such data presupposes, however, that past searches are a potential predictor of future search performance (i.e. that each topic is to be considered representative and not unique). In offering a body of practice, this data would be of greater practicable use than current studies which are considered as little more than individual case studies [ 82 , 83 , 84 , 85 , 86 , 87 , 88 , 89 , 90 ].

When to database search is another question posed in the literature. Beyer et al. [ 91 ] report that databases can be prioritised for literature searching which, whilst not addressing the question of which databases to search, may at least bring clarity as to which databases to search first [ 91 ]. Paradoxically, this links to studies that suggest PubMed should be searched in addition to MEDLINE (OVID interface) since this improves the currency of systematic reviews [ 92 , 93 ]. Cooper et al. (2017) have tested the idea of database searching not as a primary search method (as suggested in the guidance) but as a supplementary search method in order to manage the volume of studies identified for an environmental effectiveness systematic review. Their case study compared the effectiveness of database searching versus a protocol using supplementary search methods and found that the latter identified more relevant studies for review than searching bibliographic databases [ 94 ].

Key stage six: Determining the process of literature searching and deciding where to search (supplementary search methods)

Table 2 also summaries the process of literature searching which follows bibliographic database searching. As Table 2 sets out, guidance that supplementary literature search methods should be used in systematic reviews recurs across documents, but the order in which these methods are used, and the extent to which they are used, varies. We noted inconsistency in the labelling of supplementary search methods between guidance documents.

Rather than focus on the guidance on how to use the methods (which has been summarised in a recent review [ 95 ]), we focus on the aim or purpose of supplementary search methods.

The Cochrane Handbook reported that ‘efforts’ to identify unpublished studies should be made [ 9 ]. Four guidance documents [ 2 , 3 , 6 , 9 ] acknowledged that searching beyond bibliographic databases was necessary since ‘databases are not the only source of literature’ [ 2 ]. Only one document reported any guidance on determining when to use supplementary methods. The IQWiG handbook reported that the use of handsearching (in their example) could be determined on a ‘case-by-case basis’ which implies that the use of these methods is optional rather than mandatory. This is in contrast to the guidance (above) on bibliographic database searching.

The issue for supplementary search methods is similar in many ways to the issue of searching bibliographic databases: demonstrating value. The purpose and contribution of supplementary search methods in systematic reviews is increasingly acknowledged [ 37 , 61 , 62 , 96 , 97 , 98 , 99 , 100 , 101 ] but understanding the value of the search methods to identify studies and data is unclear. In a recently published review, Cooper et al. (2017) reviewed the literature on supplementary search methods looking to determine the advantages, disadvantages and resource implications of using supplementary search methods [ 95 ]. This review also summarises the key guidance and empirical studies and seeks to address the question on when to use these search methods and when not to [ 95 ]. The guidance is limited in this regard and, as Table 2 demonstrates, offers conflicting advice on the order of searching, and the extent to which these search methods should be used in systematic reviews.

Key stage seven: Managing the references

Five of the documents provided guidance on managing references, for example downloading, de-duplicating and managing the output of literature searches [ 2 , 4 , 6 , 8 , 10 ]. This guidance typically itemised available bibliographic management tools rather than offering guidance on how to use them specifically [ 2 , 4 , 6 , 8 ]. The CEE handbook provided guidance on importing data where no direct export option is available (e.g. web-searching) [ 10 ].

The literature on using bibliographic management tools is not large relative to the number of ‘how to’ videos on platforms such as YouTube (see for example [ 102 ]). These YouTube videos confirm the overall lack of ‘how to’ guidance identified in this study and offer useful instruction on managing references. Bramer et al. set out methods for de-duplicating data and reviewing references in Endnote [ 103 , 104 ] and Gall tests the direct search function within Endnote to access databases such as PubMed, finding a number of limitations [ 105 ]. Coar et al. and Ahmed et al. consider the role of the free-source tool, Zotero [ 106 , 107 ]. Managing references is a key administrative function in the process of review particularly for documenting searches in PRISMA guidance.

Key stage eight: Documenting the search

The Cochrane Handbook was the only guidance document to recommend a specific reporting guideline: Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [ 9 ]. Six documents provided guidance on reporting the process of literature searching with specific criteria to report [ 3 , 4 , 6 , 8 , 9 , 10 ]. There was consensus on reporting: the databases searched (and the host searched by), the search strategies used, and any use of limits (e.g. date, language, search filters (The CRD handbook called for these limits to be justified [ 6 ])). Three guidance documents reported that the number of studies identified should be recorded [ 3 , 6 , 10 ]. The number of duplicates identified [ 10 ], the screening decisions [ 3 ], a comprehensive list of grey literature sources searched (and full detail for other supplementary search methods) [ 8 ], and an annotation of search terms tested but not used [ 4 ] were identified as unique items in four documents.

The Cochrane Handbook was the only guidance document to note that the full search strategies for each database should be included in the Additional file 1 of the review [ 9 ].

All guidance documents should ultimately deliver completed systematic reviews that fulfil the requirements of the PRISMA reporting guidelines [ 108 ]. The guidance broadly requires the reporting of data that corresponds with the requirements of the PRISMA statement although documents typically ask for diverse and additional items [ 108 ]. In 2008, Sampson et al. observed a lack of consensus on reporting search methods in systematic reviews [ 109 ] and this remains the case as of 2017, as evidenced in the guidance documents, and in spite of the publication of the PRISMA guidelines in 2009 [ 110 ]. It is unclear why the collective guidance does not more explicitly endorse adherence to the PRISMA guidance.

Reporting of literature searching is a key area in systematic reviews since it sets out clearly what was done and how the conclusions of the review can be believed [ 52 , 109 ]. Despite strong endorsement in the guidance documents, specifically supported in PRISMA guidance, and other related reporting standards too (such as ENTREQ for qualitative evidence synthesis, STROBE for reviews of observational studies), authors still highlight the prevalence of poor standards of literature search reporting [ 31 , 110 , 111 , 112 , 113 , 114 , 115 , 116 , 117 , 118 , 119 ]. To explore issues experienced by authors in reporting literature searches, and look at uptake of PRISMA, Radar et al. [ 120 ] surveyed over 260 review authors to determine common problems and their work summaries the practical aspects of reporting literature searching [ 120 ]. Atkinson et al. [ 121 ] have also analysed reporting standards for literature searching, summarising recommendations and gaps for reporting search strategies [ 121 ].

One area that is less well covered by the guidance, but nevertheless appears in this literature, is the quality appraisal or peer review of literature search strategies. The PRESS checklist is the most prominent and it aims to develop evidence-based guidelines to peer review of electronic search strategies [ 5 , 122 , 123 ]. A corresponding guideline for documentation of supplementary search methods does not yet exist although this idea is currently being explored.

How the reporting of the literature searching process corresponds to critical appraisal tools is an area for further research. In the survey undertaken by Radar et al. (2014), 86% of survey respondents (153/178) identified a need for further guidance on what aspects of the literature search process to report [ 120 ]. The PRISMA statement offers a brief summary of what to report but little practical guidance on how to report it [ 108 ]. Critical appraisal tools for systematic reviews, such as AMSTAR 2 (Shea et al. [ 124 ]) and ROBIS (Whiting et al. [ 125 ]), can usefully be read alongside PRISMA guidance, since they offer greater detail on how the reporting of the literature search will be appraised and, therefore, they offer a proxy on what to report [ 124 , 125 ]. Further research in the form of a study which undertakes a comparison between PRISMA and quality appraisal checklists for systematic reviews would seem to begin addressing the call, identified by Radar et al., for further guidance on what to report [ 120 ].

Limitations

Other handbooks exist.

A potential limitation of this literature review is the focus on guidance produced in Europe (the UK specifically) and Australia. We justify the decision for our selection of the nine guidance documents reviewed in this literature review in section “ Identifying guidance ”. In brief, these nine guidance documents were selected as the most relevant health care guidance that inform UK systematic reviewing practice, given that the UK occupies a prominent position in the science of health information retrieval. We acknowledge the existence of other guidance documents, such as those from North America (e.g. the Agency for Healthcare Research and Quality (AHRQ) [ 126 ], The Institute of Medicine [ 127 ] and the guidance and resources produced by the Canadian Agency for Drugs and Technologies in Health (CADTH) [ 128 ]). We comment further on this directly below.

The handbooks are potentially linked to one another

What is not clear is the extent to which the guidance documents inter-relate or provide guidance uniquely. The Cochrane Handbook, first published in 1994, is notably a key source of reference in guidance and systematic reviews beyond Cochrane reviews. It is not clear to what extent broadening the sample of guidance handbooks to include North American handbooks, and guidance handbooks from other relevant countries too, would alter the findings of this literature review or develop further support for the process model. Since we cannot be clear, we raise this as a potential limitation of this literature review. On our initial review of a sample of North American, and other, guidance documents (before selecting the guidance documents considered in this review), however, we do not consider that the inclusion of these further handbooks would alter significantly the findings of this literature review.

This is a literature review

A further limitation of this review was that the review of published studies is not a systematic review of the evidence for each key stage. It is possible that other relevant studies could help contribute to the exploration and development of the key stages identified in this review.

This literature review would appear to demonstrate the existence of a shared model of the literature searching process in systematic reviews. We call this model ‘the conventional approach’, since it appears to be common convention in nine different guidance documents.

The findings reported above reveal eight key stages in the process of literature searching for systematic reviews. These key stages are consistently reported in the nine guidance documents which suggests consensus on the key stages of literature searching, and therefore the process of literature searching as a whole, in systematic reviews.

In Table 2 , we demonstrate consensus regarding the application of literature search methods. All guidance documents distinguish between primary and supplementary search methods. Bibliographic database searching is consistently the first method of literature searching referenced in each guidance document. Whilst the guidance uniformly supports the use of supplementary search methods, there is little evidence for a consistent process with diverse guidance across documents. This may reflect differences in the core focus across each document, linked to differences in identifying effectiveness studies or qualitative studies, for instance.

Eight of the nine guidance documents reported on the aims of literature searching. The shared understanding was that literature searching should be thorough and comprehensive in its aim and that this process should be reported transparently so that that it could be reproduced. Whilst only three documents explicitly link this understanding to minimising bias, it is clear that comprehensive literature searching is implicitly linked to ‘not missing relevant studies’ which is approximately the same point.

Defining the key stages in this review helps categorise the scholarship available, and it prioritises areas for development or further study. The supporting studies on preparing for literature searching (key stage three, ‘preparation’) were, for example, comparatively few, and yet this key stage represents a decisive moment in literature searching for systematic reviews. It is where search strategy structure is determined, search terms are chosen or discarded, and the resources to be searched are selected. Information specialists, librarians and researchers, are well placed to develop these and other areas within the key stages we identify.

This review calls for further research to determine the suitability of using the conventional approach. The publication dates of the guidance documents which underpin the conventional approach may raise questions as to whether the process which they each report remains valid for current systematic literature searching. In addition, it may be useful to test whether it is desirable to use the same process model of literature searching for qualitative evidence synthesis as that for reviews of intervention effectiveness, which this literature review demonstrates is presently recommended best practice.

Abbreviations

Behaviour of interest; Health context; Exclusions; Models or Theories

Cochrane Database of Systematic Reviews

The Cochrane Central Register of Controlled Trials

Database of Abstracts of Reviews of Effects

Enhancing transparency in reporting the synthesis of qualitative research

Institute for Quality and Efficiency in Healthcare

National Institute for Clinical Excellence

Population, Intervention, Comparator, Outcome

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Setting, Perspective, Intervention, Comparison, Evaluation

Sample, Phenomenon of Interest, Design, Evaluation, Research type

STrengthening the Reporting of OBservational studies in Epidemiology

Trial Search Co-ordinators

Booth A. Unpacking your literature search toolbox: on search styles and tactics. Health Information & Libraries Journal. 2008;25(4):313–7.

Article   Google Scholar  

Petticrew M, Roberts H. Systematic reviews in the social sciences: a practical guide. Oxford: Blackwell Publishing Ltd; 2006.

Book   Google Scholar  

Institute for Quality and Efficiency in Health Care (IQWiG). IQWiG Methods Resources. 7 Information retrieval 2014 [Available from: https://www.ncbi.nlm.nih.gov/books/NBK385787/ .

NICE: National Institute for Health and Care Excellence. Developing NICE guidelines: the manual 2014. Available from: https://www.nice.org.uk/media/default/about/what-we-do/our-programmes/developing-nice-guidelines-the-manual.pdf .

Sampson M. MJ, Lefebvre C, Moher D, Grimshaw J. Peer Review of Electronic Search Strategies: PRESS; 2008.

Google Scholar  

Centre for Reviews & Dissemination. Systematic reviews – CRD’s guidance for undertaking reviews in healthcare. York: Centre for Reviews and Dissemination, University of York; 2009.

eunetha: European Network for Health Technology Assesment Process of information retrieval for systematic reviews and health technology assessments on clinical effectiveness 2016. Available from: http://www.eunethta.eu/sites/default/files/Guideline_Information_Retrieval_V1-1.pdf .

Kugley SWA, Thomas J, Mahood Q, Jørgensen AMK, Hammerstrøm K, Sathe N. Searching for studies: a guide to information retrieval for Campbell systematic reviews. Oslo: Campbell Collaboration. 2017; Available from: https://www.campbellcollaboration.org/library/searching-for-studies-information-retrieval-guide-campbell-reviews.html

Lefebvre C, Manheimer E, Glanville J. Chapter 6: searching for studies. In: JPT H, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions; 2011.

Collaboration for Environmental Evidence. Guidelines for Systematic Review and Evidence Synthesis in Environmental Management.: Environmental Evidence:; 2013. Available from: http://www.environmentalevidence.org/wp-content/uploads/2017/01/Review-guidelines-version-4.2-final-update.pdf .

The Joanna Briggs Institute. Joanna Briggs institute reviewers’ manual. 2014th ed: the Joanna Briggs institute; 2014. Available from: https://joannabriggs.org/assets/docs/sumari/ReviewersManual-2014.pdf

Beverley CA, Booth A, Bath PA. The role of the information specialist in the systematic review process: a health information case study. Health Inf Libr J. 2003;20(2):65–74.

Article   CAS   Google Scholar  

Harris MR. The librarian's roles in the systematic review process: a case study. Journal of the Medical Library Association. 2005;93(1):81–7.

PubMed   PubMed Central   Google Scholar  

Egger JB. Use of recommended search strategies in systematic reviews and the impact of librarian involvement: a cross-sectional survey of recent authors. PLoS One. 2015;10(5):e0125931.

Li L, Tian J, Tian H, Moher D, Liang F, Jiang T, et al. Network meta-analyses could be improved by searching more sources and by involving a librarian. J Clin Epidemiol. 2014;67(9):1001–7.

Article   PubMed   Google Scholar  

McGowan J, Sampson M. Systematic reviews need systematic searchers. J Med Libr Assoc. 2005;93(1):74–80.

Rethlefsen ML, Farrell AM, Osterhaus Trzasko LC, Brigham TJ. Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. J Clin Epidemiol. 2015;68(6):617–26.

Weller AC. Mounting evidence that librarians are essential for comprehensive literature searches for meta-analyses and Cochrane reports. J Med Libr Assoc. 2004;92(2):163–4.

Swinkels A, Briddon J, Hall J. Two physiotherapists, one librarian and a systematic literature review: collaboration in action. Health Info Libr J. 2006;23(4):248–56.

Foster M. An overview of the role of librarians in systematic reviews: from expert search to project manager. EAHIL. 2015;11(3):3–7.

Lawson L. OPERATING OUTSIDE LIBRARY WALLS 2004.

Vassar M, Yerokhin V, Sinnett PM, Weiher M, Muckelrath H, Carr B, et al. Database selection in systematic reviews: an insight through clinical neurology. Health Inf Libr J. 2017;34(2):156–64.

Townsend WA, Anderson PF, Ginier EC, MacEachern MP, Saylor KM, Shipman BL, et al. A competency framework for librarians involved in systematic reviews. Journal of the Medical Library Association : JMLA. 2017;105(3):268–75.

Cooper ID, Crum JA. New activities and changing roles of health sciences librarians: a systematic review, 1990-2012. Journal of the Medical Library Association : JMLA. 2013;101(4):268–77.

Crum JA, Cooper ID. Emerging roles for biomedical librarians: a survey of current practice, challenges, and changes. Journal of the Medical Library Association : JMLA. 2013;101(4):278–86.

Dudden RF, Protzko SL. The systematic review team: contributions of the health sciences librarian. Med Ref Serv Q. 2011;30(3):301–15.

Golder S, Loke Y, McIntosh HM. Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects. J Clin Epidemiol. 2008;61(5):440–8.

Maggio LA, Tannery NH, Kanter SL. Reproducibility of literature search reporting in medical education reviews. Academic medicine : journal of the Association of American Medical Colleges. 2011;86(8):1049–54.

Meert D, Torabi N, Costella J. Impact of librarians on reporting of the literature searching component of pediatric systematic reviews. Journal of the Medical Library Association : JMLA. 2016;104(4):267–77.

Morris M, Boruff JT, Gore GC. Scoping reviews: establishing the role of the librarian. Journal of the Medical Library Association : JMLA. 2016;104(4):346–54.

Koffel JB, Rethlefsen ML. Reproducibility of search strategies is poor in systematic reviews published in high-impact pediatrics, cardiology and surgery journals: a cross-sectional study. PLoS One. 2016;11(9):e0163309.

Article   PubMed   PubMed Central   CAS   Google Scholar  

Fehrmann P, Thomas J. Comprehensive computer searches and reporting in systematic reviews. Research Synthesis Methods. 2011;2(1):15–32.

Booth A. Searching for qualitative research for inclusion in systematic reviews: a structured methodological review. Systematic Reviews. 2016;5(1):74.

Article   PubMed   PubMed Central   Google Scholar  

Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health technology assessment (Winchester, England). 2003;7(1):1–76.

Tricco AC, Tetzlaff J, Sampson M, Fergusson D, Cogo E, Horsley T, et al. Few systematic reviews exist documenting the extent of bias: a systematic review. J Clin Epidemiol. 2008;61(5):422–34.

Booth A. How much searching is enough? Comprehensive versus optimal retrieval for technology assessments. Int J Technol Assess Health Care. 2010;26(4):431–5.

Papaioannou D, Sutton A, Carroll C, Booth A, Wong R. Literature searching for social science systematic reviews: consideration of a range of search techniques. Health Inf Libr J. 2010;27(2):114–22.

Petticrew M. Time to rethink the systematic review catechism? Moving from ‘what works’ to ‘what happens’. Systematic Reviews. 2015;4(1):36.

Betrán AP, Say L, Gülmezoglu AM, Allen T, Hampson L. Effectiveness of different databases in identifying studies for systematic reviews: experience from the WHO systematic review of maternal morbidity and mortality. BMC Med Res Methodol. 2005;5

Felson DT. Bias in meta-analytic research. J Clin Epidemiol. 1992;45(8):885–92.

Article   PubMed   CAS   Google Scholar  

Franco A, Malhotra N, Simonovits G. Publication bias in the social sciences: unlocking the file drawer. Science. 2014;345(6203):1502–5.

Hartling L, Featherstone R, Nuspl M, Shave K, Dryden DM, Vandermeer B. Grey literature in systematic reviews: a cross-sectional study of the contribution of non-English reports, unpublished studies and dissertations to the results of meta-analyses in child-relevant reviews. BMC Med Res Methodol. 2017;17(1):64.

Schmucker CM, Blümle A, Schell LK, Schwarzer G, Oeller P, Cabrera L, et al. Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research. PLoS One. 2017;12(4):e0176210.

Egger M, Zellweger-Zahner T, Schneider M, Junker C, Lengeler C, Antes G. Language bias in randomised controlled trials published in English and German. Lancet (London, England). 1997;350(9074):326–9.

Moher D, Pham B, Lawson ML, Klassen TP. The inclusion of reports of randomised trials published in languages other than English in systematic reviews. Health technology assessment (Winchester, England). 2003;7(41):1–90.

Pham B, Klassen TP, Lawson ML, Moher D. Language of publication restrictions in systematic reviews gave different results depending on whether the intervention was conventional or complementary. J Clin Epidemiol. 2005;58(8):769–76.

Mills EJ, Kanters S, Thorlund K, Chaimani A, Veroniki A-A, Ioannidis JPA. The effects of excluding treatments from network meta-analyses: survey. BMJ : British Medical Journal. 2013;347

Hartling L, Featherstone R, Nuspl M, Shave K, Dryden DM, Vandermeer B. The contribution of databases to the results of systematic reviews: a cross-sectional study. BMC Med Res Methodol. 2016;16(1):127.

van Driel ML, De Sutter A, De Maeseneer J, Christiaens T. Searching for unpublished trials in Cochrane reviews may not be worth the effort. J Clin Epidemiol. 2009;62(8):838–44.e3.

Buchberger B, Krabbe L, Lux B, Mattivi JT. Evidence mapping for decision making: feasibility versus accuracy - when to abandon high sensitivity in electronic searches. German medical science : GMS e-journal. 2016;14:Doc09.

Lorenc T, Pearson M, Jamal F, Cooper C, Garside R. The role of systematic reviews of qualitative evidence in evaluating interventions: a case study. Research Synthesis Methods. 2012;3(1):1–10.

Gough D. Weight of evidence: a framework for the appraisal of the quality and relevance of evidence. Res Pap Educ. 2007;22(2):213–28.

Barroso J, Gollop CJ, Sandelowski M, Meynell J, Pearce PF, Collins LJ. The challenges of searching for and retrieving qualitative studies. West J Nurs Res. 2003;25(2):153–78.

Britten N, Garside R, Pope C, Frost J, Cooper C. Asking more of qualitative synthesis: a response to Sally Thorne. Qual Health Res. 2017;27(9):1370–6.

Booth A, Carroll C. Systematic searching for theory to inform systematic reviews: is it feasible? Is it desirable? Health Info Libr J. 2015;32(3):220–35.

Kwon Y, Powelson SE, Wong H, Ghali WA, Conly JM. An assessment of the efficacy of searching in biomedical databases beyond MEDLINE in identifying studies for a systematic review on ward closures as an infection control intervention to control outbreaks. Syst Rev. 2014;3:135.

Nussbaumer-Streit B, Klerings I, Wagner G, Titscher V, Gartlehner G. Assessing the validity of abbreviated literature searches for rapid reviews: protocol of a non-inferiority and meta-epidemiologic study. Systematic Reviews. 2016;5:197.

Wagner G, Nussbaumer-Streit B, Greimel J, Ciapponi A, Gartlehner G. Trading certainty for speed - how much uncertainty are decisionmakers and guideline developers willing to accept when using rapid reviews: an international survey. BMC Med Res Methodol. 2017;17(1):121.

Ogilvie D, Hamilton V, Egan M, Petticrew M. Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go? J Epidemiol Community Health. 2005;59(9):804–8.

Royle P, Milne R. Literature searching for randomized controlled trials used in Cochrane reviews: rapid versus exhaustive searches. Int J Technol Assess Health Care. 2003;19(4):591–603.

Pearson M, Moxham T, Ashton K. Effectiveness of search strategies for qualitative research about barriers and facilitators of program delivery. Eval Health Prof. 2011;34(3):297–308.

Levay P, Raynor M, Tuvey D. The Contributions of MEDLINE, Other Bibliographic Databases and Various Search Techniques to NICE Public Health Guidance. 2015. 2015;10(1):19.

Nussbaumer-Streit B, Klerings I, Wagner G, Heise TL, Dobrescu AI, Armijo-Olivo S, et al. Abbreviated literature searches were viable alternatives to comprehensive searches: a meta-epidemiological study. J Clin Epidemiol. 2018;102:1–11.

Briscoe S, Cooper C, Glanville J, Lefebvre C. The loss of the NHS EED and DARE databases and the effect on evidence synthesis and evaluation. Res Synth Methods. 2017;8(3):256–7.

Stansfield C, O'Mara-Eves A, Thomas J. Text mining for search term development in systematic reviewing: A discussion of some methods and challenges. Research Synthesis Methods.n/a-n/a.

Petrova M, Sutcliffe P, Fulford KW, Dale J. Search terms and a validated brief search filter to retrieve publications on health-related values in Medline: a word frequency analysis study. Journal of the American Medical Informatics Association : JAMIA. 2012;19(3):479–88.

Stansfield C, Thomas J, Kavanagh J. 'Clustering' documents automatically to support scoping reviews of research: a case study. Res Synth Methods. 2013;4(3):230–41.

PubMed   Google Scholar  

Methley AM, Campbell S, Chew-Graham C, McNally R, Cheraghi-Sohi S. PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res. 2014;14:579.

Andrew B. Clear and present questions: formulating questions for evidence based practice. Library Hi Tech. 2006;24(3):355–68.

Cooke A, Smith D, Booth A. Beyond PICO: the SPIDER tool for qualitative evidence synthesis. Qual Health Res. 2012;22(10):1435–43.

Whiting P, Westwood M, Bojke L, Palmer S, Richardson G, Cooper J, et al. Clinical effectiveness and cost-effectiveness of tests for the diagnosis and investigation of urinary tract infection in children: a systematic review and economic model. Health technology assessment (Winchester, England). 2006;10(36):iii-iv, xi-xiii, 1–154.

Cooper C, Levay P, Lorenc T, Craig GM. A population search filter for hard-to-reach populations increased search efficiency for a systematic review. J Clin Epidemiol. 2014;67(5):554–9.

Hausner E, Waffenschmidt S, Kaiser T, Simon M. Routine development of objectively derived search strategies. Systematic Reviews. 2012;1(1):19.

Hausner E, Guddat C, Hermanns T, Lampert U, Waffenschmidt S. Prospective comparison of search strategies for systematic reviews: an objective approach yielded higher sensitivity than a conceptual one. J Clin Epidemiol. 2016;77:118–24.

Craven J, Levay P. Recording database searches for systematic reviews - what is the value of adding a narrative to peer-review checklists? A case study of nice interventional procedures guidance. Evid Based Libr Inf Pract. 2011;6(4):72–87.

Wright K, Golder S, Lewis-Light K. What value is the CINAHL database when searching for systematic reviews of qualitative studies? Syst Rev. 2015;4:104.

Beckles Z, Glover S, Ashe J, Stockton S, Boynton J, Lai R, et al. Searching CINAHL did not add value to clinical questions posed in NICE guidelines. J Clin Epidemiol. 2013;66(9):1051–7.

Cooper C, Rogers M, Bethel A, Briscoe S, Lowe J. A mapping review of the literature on UK-focused health and social care databases. Health Inf Libr J. 2015;32(1):5–22.

Younger P, Boddy K. When is a search not a search? A comparison of searching the AMED complementary health database via EBSCOhost, OVID and DIALOG. Health Inf Libr J. 2009;26(2):126–35.

Lam MT, McDiarmid M. Increasing number of databases searched in systematic reviews and meta-analyses between 1994 and 2014. Journal of the Medical Library Association : JMLA. 2016;104(4):284–9.

Bethel A, editor Search summary tables for systematic reviews: results and findings. HLC Conference 2017a.

Aagaard T, Lund H, Juhl C. Optimizing literature search in systematic reviews - are MEDLINE, EMBASE and CENTRAL enough for identifying effect studies within the area of musculoskeletal disorders? BMC Med Res Methodol. 2016;16(1):161.

Adams CE, Frederick K. An investigation of the adequacy of MEDLINE searches for randomized controlled trials (RCTs) of the effects of mental health care. Psychol Med. 1994;24(3):741–8.

Kelly L, St Pierre-Hansen N. So many databases, such little clarity: searching the literature for the topic aboriginal. Canadian family physician Medecin de famille canadien. 2008;54(11):1572–3.

Lawrence DW. What is lost when searching only one literature database for articles relevant to injury prevention and safety promotion? Injury Prevention. 2008;14(6):401–4.

Lemeshow AR, Blum RE, Berlin JA, Stoto MA, Colditz GA. Searching one or two databases was insufficient for meta-analysis of observational studies. J Clin Epidemiol. 2005;58(9):867–73.

Sampson M, Barrowman NJ, Moher D, Klassen TP, Pham B, Platt R, et al. Should meta-analysts search Embase in addition to Medline? J Clin Epidemiol. 2003;56(10):943–55.

Stevinson C, Lawlor DA. Searching multiple databases for systematic reviews: added value or diminishing returns? Complementary Therapies in Medicine. 2004;12(4):228–32.

Suarez-Almazor ME, Belseck E, Homik J, Dorgan M, Ramos-Remus C. Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough. Control Clin Trials. 2000;21(5):476–87.

Taylor B, Wylie E, Dempster M, Donnelly M. Systematically retrieving research: a case study evaluating seven databases. Res Soc Work Pract. 2007;17(6):697–706.

Beyer FR, Wright K. Can we prioritise which databases to search? A case study using a systematic review of frozen shoulder management. Health Info Libr J. 2013;30(1):49–58.

Duffy S, de Kock S, Misso K, Noake C, Ross J, Stirk L. Supplementary searches of PubMed to improve currency of MEDLINE and MEDLINE in-process searches via Ovid. Journal of the Medical Library Association : JMLA. 2016;104(4):309–12.

Katchamart W, Faulkner A, Feldman B, Tomlinson G, Bombardier C. PubMed had a higher sensitivity than Ovid-MEDLINE in the search for systematic reviews. J Clin Epidemiol. 2011;64(7):805–7.

Cooper C, Lovell R, Husk K, Booth A, Garside R. Supplementary search methods were more effective and offered better value than bibliographic database searching: a case study from public health and environmental enhancement (in Press). Research Synthesis Methods. 2017;

Cooper C, Booth, A., Britten, N., Garside, R. A comparison of results of empirical studies of supplementary search techniques and recommendations in review methodology handbooks: A methodological review. (In Press). BMC Systematic Reviews. 2017.

Greenhalgh T, Peacock R. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ (Clinical research ed). 2005;331(7524):1064–5.

Article   PubMed Central   Google Scholar  

Hinde S, Spackman E. Bidirectional citation searching to completion: an exploration of literature searching methods. PharmacoEconomics. 2015;33(1):5–11.

Levay P, Ainsworth N, Kettle R, Morgan A. Identifying evidence for public health guidance: a comparison of citation searching with web of science and Google scholar. Res Synth Methods. 2016;7(1):34–45.

McManus RJ, Wilson S, Delaney BC, Fitzmaurice DA, Hyde CJ, Tobias RS, et al. Review of the usefulness of contacting other experts when conducting a literature search for systematic reviews. BMJ (Clinical research ed). 1998;317(7172):1562–3.

Westphal A, Kriston L, Holzel LP, Harter M, von Wolff A. Efficiency and contribution of strategies for finding randomized controlled trials: a case study from a systematic review on therapeutic interventions of chronic depression. Journal of public health research. 2014;3(2):177.

Matthews EJ, Edwards AG, Barker J, Bloor M, Covey J, Hood K, et al. Efficient literature searching in diffuse topics: lessons from a systematic review of research on communicating risk to patients in primary care. Health Libr Rev. 1999;16(2):112–20.

Bethel A. Endnote Training (YouTube Videos) 2017b [Available from: http://medicine.exeter.ac.uk/esmi/workstreams/informationscience/is_resources,_guidance_&_advice/ .

Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplication of database search results for systematic reviews in EndNote. Journal of the Medical Library Association : JMLA. 2016;104(3):240–3.

Bramer WM, Milic J, Mast F. Reviewing retrieved references for inclusion in systematic reviews using EndNote. Journal of the Medical Library Association : JMLA. 2017;105(1):84–7.

Gall C, Brahmi FA. Retrieval comparison of EndNote to search MEDLINE (Ovid and PubMed) versus searching them directly. Medical reference services quarterly. 2004;23(3):25–32.

Ahmed KK, Al Dhubaib BE. Zotero: a bibliographic assistant to researcher. J Pharmacol Pharmacother. 2011;2(4):303–5.

Coar JT, Sewell JP. Zotero: harnessing the power of a personal bibliographic manager. Nurse Educ. 2010;35(5):205–7.

Moher D, Liberati A, Tetzlaff J, Altman DG, The PG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.

Sampson M, McGowan J, Tetzlaff J, Cogo E, Moher D. No consensus exists on search reporting methods for systematic reviews. J Clin Epidemiol. 2008;61(8):748–54.

Toews LC. Compliance of systematic reviews in veterinary journals with preferred reporting items for systematic reviews and meta-analysis (PRISMA) literature search reporting guidelines. Journal of the Medical Library Association : JMLA. 2017;105(3):233–9.

Booth A. "brimful of STARLITE": toward standards for reporting literature searches. Journal of the Medical Library Association : JMLA. 2006;94(4):421–9. e205

Faggion CM Jr, Wu YC, Tu YK, Wasiak J. Quality of search strategies reported in systematic reviews published in stereotactic radiosurgery. Br J Radiol. 2016;89(1062):20150878.

Mullins MM, DeLuca JB, Crepaz N, Lyles CM. Reporting quality of search methods in systematic reviews of HIV behavioral interventions (2000–2010): are the searches clearly explained, systematic and reproducible? Research Synthesis Methods. 2014;5(2):116–30.

Yoshii A, Plaut DA, McGraw KA, Anderson MJ, Wellik KE. Analysis of the reporting of search strategies in Cochrane systematic reviews. Journal of the Medical Library Association : JMLA. 2009;97(1):21–9.

Bigna JJ, Um LN, Nansseu JR. A comparison of quality of abstracts of systematic reviews including meta-analysis of randomized controlled trials in high-impact general medicine journals before and after the publication of PRISMA extension for abstracts: a systematic review and meta-analysis. Syst Rev. 2016;5(1):174.

Akhigbe T, Zolnourian A, Bulters D. Compliance of systematic reviews articles in brain arteriovenous malformation with PRISMA statement guidelines: review of literature. Journal of clinical neuroscience : official journal of the Neurosurgical Society of Australasia. 2017;39:45–8.

Tao KM, Li XQ, Zhou QH, Moher D, Ling CQ, Yu WF. From QUOROM to PRISMA: a survey of high-impact medical journals' instructions to authors and a review of systematic reviews in anesthesia literature. PLoS One. 2011;6(11):e27611.

Wasiak J, Tyack Z, Ware R. Goodwin N. Jr. Poor methodological quality and reporting standards of systematic reviews in burn care management. International wound journal: Faggion CM; 2016.

Tam WW, Lo KK, Khalechelvam P. Endorsement of PRISMA statement and quality of systematic reviews and meta-analyses published in nursing journals: a cross-sectional study. BMJ Open. 2017;7(2):e013905.

Rader T, Mann M, Stansfield C, Cooper C, Sampson M. Methods for documenting systematic review searches: a discussion of common issues. Res Synth Methods. 2014;5(2):98–115.

Atkinson KM, Koenka AC, Sanchez CE, Moshontz H, Cooper H. Reporting standards for literature searches and report inclusion criteria: making research syntheses more transparent and easy to replicate. Res Synth Methods. 2015;6(1):87–95.

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40–6.

Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol. 2009;62(9):944–52.

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ (Clinical research ed). 2017;358.

Whiting P, Savović J, Higgins JPT, Caldwell DM, Reeves BC, Shea B, et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016;69:225–34.

Relevo R, Balshem H. Finding evidence for comparing medical interventions: AHRQ and the effective health care program. J Clin Epidemiol. 2011;64(11):1168–77.

Medicine Io. Standards for Systematic Reviews 2011 [Available from: http://www.nationalacademies.org/hmd/Reports/2011/Finding-What-Works-in-Health-Care-Standards-for-Systematic-Reviews/Standards.aspx .

CADTH: Resources 2018.

Download references

Acknowledgements

CC acknowledges the supervision offered by Professor Chris Hyde.

This publication forms a part of CC’s PhD. CC’s PhD was funded through the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Programme (Project Number 16/54/11). The open access fee for this publication was paid for by Exeter Medical School.

RG and NB were partially supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care South West Peninsula.

The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

Author information

Authors and affiliations.

Institute of Health Research, University of Exeter Medical School, Exeter, UK

Chris Cooper & Jo Varley-Campbell

HEDS, School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UK

Andrew Booth

Nicky Britten

European Centre for Environment and Human Health, University of Exeter Medical School, Truro, UK

Ruth Garside

You can also search for this author in PubMed   Google Scholar

Contributions

CC conceived the idea for this study and wrote the first draft of the manuscript. CC discussed this publication in PhD supervision with AB and separately with JVC. CC revised the publication with input and comments from AB, JVC, RG and NB. All authors revised the manuscript prior to submission. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Chris Cooper .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:.

Appendix tables and PubMed search strategy. Key studies used for pearl growing per key stage, working data extraction tables and the PubMed search strategy. (DOCX 30 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Cooper, C., Booth, A., Varley-Campbell, J. et al. Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies. BMC Med Res Methodol 18 , 85 (2018). https://doi.org/10.1186/s12874-018-0545-3

Download citation

Received : 20 September 2017

Accepted : 06 August 2018

Published : 14 August 2018

DOI : https://doi.org/10.1186/s12874-018-0545-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Literature Search Process
• Citation Chasing
• Tacit Models
• Unique Guidance
• Information Specialists

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

A systematic literature review on social innovation for sustainable development in China

• Published: 05 May 2024

Cite this article

This study analyses issues faced during the pre-implementation and implementation stages of the social innovation (SI) process, which is considered an effective approach to addressing sustainable development (SD) challenges. Specifically, it presents the results of a rigorous literature review and analysis of 55 English and Chinese-language articles on Chinese SI published through the end of April 2021. After a brief description of the characteristics of SI related to SD in China, it investigates from a systematic perspective the key elements, processes, and outcomes of SI seeking to deliver sustainable values through a comparative analysis of findings from the Western and the Eastern world. The results advance our understanding of how SI emerges and evolves throughout the process of addressing SD problems. The framework and propositions presented in this work advance the development of theory related to the pre-implementation and implementation stages of SI for SD issues. The study also extends SI theories from a systematic perspective to provide analysis and decision support for various players involved in these stages. It explains how key elements and the SI process itself affect the outcome, and how this affects the decisions of initiators and other stakeholders. This work can help practitioners and researchers understand the internal workings of the SI system at different stages, thus allowing them to formulate policies, strategies, and guidelines for pursuing SD goals.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Source Author

Source Adapted from Di Vaio et al., 2022

Availability of data and materials

The datasets used during the current study are available from the author on reasonable request.

Adiguzel, F., BozdoganSert, E., Dinc, Y., Cetin, M., Gungor, S., Yuka, P., SertkayaDogan, O., Kaya, E., Karakaya, K., & Vural, E. (2022). Determining the relationships between climatic elements and thermal comfort and tourism activities using the tourism climate index for urban planning: A case study of Izmir Province. Theoretical and Applied Climatology, 147 , 1105–1120. https://doi.org/10.1007/s00704-021-03874-9

Article   Google Scholar  

Agostini, M. R., Vieira, L. M., Tondolo, R. D. R. P., & Tondolo, V. A. G. (2017). An overview on social innovation research: Guiding future studies. Brazilian Business Review, 14 , 385–402.

Aksoy, T., Dabanli, A., Cetin, M., Anil SenyelKurkcuoglu, M., EmreCengiz, M., NihanCabuk, S., Agacsapan, B., & Cabuk, A. (2022). Evaluation of comparing urban area land use change with Urban Atlas and CORINE data. Environmental Science and Pollution Research, 29 , 28995–29015. https://doi.org/10.1007/s11356-021-17766-y

Angelidou, M., & Psaltoglou, A. (2017). An empirical investigation of social innovation initiatives for sustainable urban development. Sustainable Cities and Society, 33 , 113–125.

Angelini, L., Carrino, S., Abou Khaled, O., Riva-Mossman, S., & Mugellini, E. (2016). Senior living lab: An ecological approach to foster social innovation in an ageing society. Future Internet, 8 (4), 50.

Avelino, F., Wittmayer, J. M., Kemp, R., & Haxeltine, A. (2017). Game-changers and transformative social innovation. Ecology and Society , 22(4).

Bandara, W., Furtmueller, E., Gorbacheva, E., Miskon, S., & Beekhuyzen, J. (2015). Achieving rigor in literature reviews: Insights from qualitative data analysis and tool-support. Communications of the Association for Information Systems, 37 (1), 8.

Google Scholar  

Bao, Y. J. (2007). Mechanism of “Service According to Circumstances” & Systematic Innovation. Journal of Chengdu University, 01 , 28–33.

Bellandi, M., Donati, L., & Cataneo, A. (2021). Social innovation governance and the role of universities: Cases of quadruple helix partnerships in Italy. Technological Forecasting and Social Change, 164 , 120518.

Bertalanffy, L. V. (1968). General system theory: Foundations, development, applications . G. Braziller.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3 (2), 77–101.

Cajaiba-Santana, G. (2014). Social innovation: Moving the field forward. A conceptual framework. Technological Forecasting and Social Change, 82 , 42–51.

Cao, H. (2020). Research on promoting social enterprises to participate in sheltered employment . Nanjing Normal University.

Cetin, M. (2015a). Using GIS analysis to assess urban green space in terms of accessibility: Case study in Kutahya. International Journal of Sustainable Development & World Ecology, 22 (5), 420–424.

Cetin, M. (2015b). Evaluation of the sustainable tourism potential of a protected area for landscape planning: A case study of the ancient city of Pompeipolis in Kastamonu. International Journal of Sustainable Development & World Ecology, 22 (6), 490–495.

Cetin, M. (2016). Sustainability of urban coastal area management: A case study on Cide. Journal of Sustainable Forestry, 35 (7), 527–541.

Cetin, M. (2019). The effect of urban planning on urban formations determining bioclimatic comfort area’s effect using satellitia imagines on air quality: A case study of Bursa city. Air Quality, Atmosphere & Health, 12 (10), 1237–1249.

Article   CAS   Google Scholar  

Cetin, M. (2020a). Climate comfort depending on different altitudes and land use in the urban areas in Kahramanmaras City. Air Quality, Atmosphere & Health, 13 (8), 991–999.

Cetin, M. (2020b). The changing of important factors in the landscape planning occur due to global climate change in temperature, Rain and climate types: A case study of Mersin City. Turkish Journal of Agriculture-Food Science and Technology, 8 (12), 2695–2701.

Cetin, M., Onac, A. K., Sevik, H., Canturk, U., & Akpinar, H. (2018a). Chronicles and geoheritage of the ancient Roman city of Pompeiopolis: A landscape plan. Arabian Journal of Geosciences, 11 , 1–12.

Cetin, M., & Sevik, H. (2016). Assessing potential areas of ecotourism through a case study in Ilgaz Mountain National Park. Tourism-from Empirical Research towards Practical Application, 190 , 81–110.

Cetin, M., Zeren, I., Sevik, H., Cakir, C., & Akpinar, H. (2018b). A study on the determination of the natural park’s sustainable tourism potential. Environmental Monitoring and Assessment, 190 , 1–8.

Chan, C. H., Chui, C. H. K., Chan, K. S. T., & Yip, P. S. F. (2019). The role of the social innovation and entrepreneurship development fund in fostering social entrepreneurship in Hong Kong: A study on public policy innovation. Social Policy and Administration, 53 (6), 903–919.

Chin, T., Yang, Y., Zhang, P., Yu, X. F., & Cao, L. Y. (2019). Co-creation of social innovation: corporate universities as innovative strategies for Chinese firms to engage with society. Sustainability, 11 (5), 1438.

China National Bureau of Statistics. (2021). China Statistical Yearbook 2020 . Retrieved May 13, 2021, from http://www.stats.gov.cn/tjsj/ndsj/2020/indexeh.htm

Chow, J. C. C., Ren, C., Mathias, B., & Liu, J. (2019). InterBoxes: A social innovation in education in rural China. Children and Youth Services Review, 101 , 217–224.

Cui, L. L., Wang, L. J., & Wang, J. Q. (2014). An empirical analysis of social innovation factors promoting the development of e-commerce in “Taobao Village”——Taking Lishui, Zhejiang Province as an example. Chinese Rural Economy, 12 , 50–60.

Cui, M., Pan, S. L., Newell, S., & Cui, L. (2017). Strategy, resource orchestration and e-commerce enabled social innovation in Rural China. The Journal of Strategic Information Systems, 26 (1), 3–21.

Degerli, B., & Cetin, M. (2022a). Evaluation from rural to urban scale for the effect of NDVI-NDBI indices on land surface temperature, in Samsun, Türkiye. Turkish Journal of Agriculture–food Science and Technology, 10 (12), 2446–2452. https://doi.org/10.24925/turjaf.v10i12.2446-2452.5535

Degerli, B., & Cetin, M. (2022b). Using the remote sensing method to simulate the land change in the year 2030. Turkish Journal of Agriculture–food Science and Technology, 10 (12), 2453–2466. https://doi.org/10.24925/turjaf.v10i12.2453-2466.5555

Deng, H., Huang, J., Xu, Z., & Rozelle, S. (2010). Policy support and emerging farmer professional cooperatives in rural China. China Economic Review, 21 (4), 495–507.

Di Vaio, A., Hassan, R., Chhabra, M., Arrigo, E., & Palladino, R. (2022). Sustainable entrepreneurship impact and entrepreneurial venture life cycle: A systematic literature review. Journal of Cleaner Production, 378 , 134469.

Diedrich, A., Upham, P., Levidow, L., & van den Hove, S. (2011). Framing environmental sustainability challenges for research and innovation in European policy agendas. Environmental Science & Policy, 14 (8), 935–939.

Dong, S. (2017). Himalayan grasslands: Indigenous knowledge and institutions for social innovation. In S. Dong, J. Bandyopadhyay, & S. Chaturvedi (Eds.), Environmental Sustainability from the Himalayas to the Oceans (pp. 99–126). Springer.

Chapter   Google Scholar  

Down, S. (2006). Narratives of enterprise: Crafting entrepreneurial self-identity in a small firm . Edward Elgar Publishing.

Book   Google Scholar  

Eichler, G. M., & Schwarz, E. J. (2019). What sustainable development goals do social innovations address? A systematic review and content analysis of social innovation literature. Sustainability, 11 (2), 522.

Feng, L.L. (2017). An exploratory study of social enterprises in thefield of Agriculture---Taking the practice of Y organization as an example . Qinghai Normal University.

Fischer, D., Reinermann, J. L., Mandujano, G. G., DesRoches, C. T., Diddi, S., & Vergragt, P. J. (2021). Sustainable consumption communication: A review of an emerging field of research. Journal of Cleaner Production, 300 , 126880.

Gerometta, J., Haussermann, H., & Longo, G. (2005). Social innovation and civil society in urban governance: Strategies for an inclusive city. Urban Studies, 42 (11), 2007–2021.

Gou, T. L., Bi, Y. Z., & Hu, X. P. (2012). Limitations encountered in social innovation process. Chinese Public Administration, 09 , 75–78.

Haskell, L., Bonnedahl, K. J., & Stål, H. I. (2021). Social innovation related to ecological crises: A systematic literature review and a research agenda for strong sustainability. Journal of Cleaner Production, 325 , 129316.

Hewitt, R. J., Bradley, N., Baggio Compagnucci, A., Barlagne, C., Ceglarz, A., Cremades, R., McKeen, M., Otto, I. M., & Slee, B. (2019). Social innovation in community energy in Europe: A review of the evidence. Frontiers in Energy Research, 7 , 31.

Holtgrewe, U., & Millard, J. (2018). Social innovation addressing societal needs and challenges. In J. Howaldt, C. Kaletka, A. Schröder, & M. Zirngiebl (Eds.), Atlas of Social Innovation—New Practices for a Better Future (pp. 70–73). TU Dortmund University.

Huang, C. C., Jin, H., Zhang, J., Zheng, Q., Chen, Y., Cheung, S., & Liu, C. (2020). The effects of an innovative e-commerce poverty alleviation platform on Chinese rural laborer skills development and family well-being. Children and Youth Services Review, 116 , 105189.

Jaeger-Erben, M., Rückert-John, J., & Schäfer, M. (2015). Sustainable consumption through social innovation: A typology of innovations for sustainable consumption practices. Journal of Cleaner Production, 108 , 784–798.

Jia, X. (2020). Priming the pump of impact entrepreneurship and social finance in China. Agriculture and Human Values, 37 (4), 1293–1311.

Jia, X., & Desa, G. (2020). Social entrepreneurship and impact investment in rural–urban transformation: An orientation to systemic social innovation and symposium findings. Agriculture and Human Values, 37 (4), 1217–1239.

Jing, Y. J., & Gong, T. (2015). Managed social innovation: The case of government-sponsored venture philanthropy in Shanghai. Public Administration and Policy Review, 71 (2), 233–245.

Kako, M., Ranse, J., Yamamoto, A., & Arbon, P. (2014). What was the role of nurses during the 2011 Great East Earthquake of Japan? An integrative review of the Japanese literature. Prehospital and Disaster Medicine, 29 (3), 275–279.

Kerlin, J. A. (2012). Considering context: Social innovation in comparative perspective. In A. Nicholls & A. Murdock (Eds.), Social innovation (pp. 66–88). Palgrave Macmillan.

Kong, F. Y., Wang, H., & Rao, Y. (2013). The innovation and development of social organizations must be based on the national conditions-a case of the “Microfinance poverty alleviation and women’s development” project in Chifeng City Inner Mongolia. Administration Reform, 04 , 37–40.

Kong, L., & Liu, B. R. (2018). A research on corporate social innovation model based on the sustainable development strategy—take China resources group for example. Journal of Nanjing University of Aeronautics and Astronautics (social Sciences), 20 (03), 27–32.

Lan, J., Ma, Y. G., Zhu, D. J., Mangalagiu, D., & Thornton, T. F. (2017). Enabling value co-creation in the sharing economy: The case of Mobike. Sustainability, 9 (9), 1504.

Lee, E. W., & Restrepo, J. M. (2015). Institutional embeddedness and the scaling-up of collaboration and social innovation: The case of a Hong Kong-based international NGO. Policy and Politics, 43 (3), 459–471.

Li, J.J. (2013). Social enterprises: the new strength of rural povery alleviation and development. Shandong University.

Li, S.Y. (2018). Social innovation process based on stakeholder network: a multi-case study . Zhejiang Gongshang University.

Li, X. F., Lin, T. Y., & Ma, R. M. (2015). Exploration of social innovation by private social work organizations in the view of community governance: The case study of One Foundation. Social Construction, 2 (03), 35–47.

CAS   Google Scholar  

Li, Y. X., & Liu, R. (2016). Characteristics, motivation and performance of social innovation in China: A multiple case text analysis of the China social innovation award program. Journal of Public Administration, 9 (04), 147–170+209.

Li, Y., Xu, Y. H., & Lan, L. H. (2014). Measuring the dimensions of urban social quality and community innovation: Based on an empirical analysis of Shenzhen. Social Sciences in China, 03 , 142–164+207.

Lin, W. (2016). Community service contracting for older people in urban China: A case study in Guangdong Province. Australian Journal of Primary Health, 22 (1), 55–62.

Liu, H. (2016). An empirical investigation of the relationship between the competency and social capital of village cadres and rural development in Western China. Revista De Cercetare Şi Intervenţie Socială, 52 , 145–172.

Liu, L., & Chen, S. (2017). Social impact investment: A tool for social innovation. China Third Sector Research, 14 (02), 21–41+181.

Liu, L., & Zhang, J. F. (2019). Empirical research and suggestions on the diffusion of social innovation. Social Governance Review, 01 , 50–57.

Liu, R.X. (2018). Research on the governance of stakeholders’ relationships in corporate social innovation: a case study of residencial care for the elderly . Zhejiang Gongshang University.

Liu, Z. Y., Li, B., & Chen, H. W. (2018). Social entrepreneurship and rural revitalization. Academic Monthly, 50 (11), 77–88.

Luo, J. L., Guo, H. D., & Jia, F. (2019). Technology acquisition mode, technology spillovers and innovation performance: A case study in farmers co-operatives. Science Research Management, 40 (05), 120–133.

Ma, Y. G., Thornton, T. F., Mangalagiu, D., Lan, J., Hestad, D., Cappello, E. A., & Van Der Leeuw, S. (2020). Co-creation, co-evolution and co-governance: Understanding green businesses and urban transformations. Climatic Change, 160 (4), 621–636.

Ma, Z.R., Shi, Y, Wang, X, Li, X.C., & He, C.M. (2018). Research on community renewal methods based on the concept of co-construction and sharing: Taking Zhuhai special economic zone jinwan social innovation valley as an example , 2018 China Urban Plan. Annual Conference.

Manzini, E. (2014). Making things happen: Social innovation and design. Design Issues, 30 (1), 57–66.

Martin, R. L., & Osberg, S. (2007). Social entrepreneurship: The case for definition. Stanford Social Innovation Review (pp 29–39). Spring

Maton, K. I. (2000). Making a difference: The social ecology of social transformation. American Journal of Community Psychology, 28 (1), 25–57.

Mehmood, A., Marsden, T., Taherzadeh, A., Axinte, L. F., & Rebelo, C. (2020). Transformative roles of people and places: Learning, experiencing, and regenerative action through social innovation. Sustainability Science, 15 (2), 455–466.

Mert-Cakal, T., & Miele, M. (2020). ‘Workable utopias’ for social change through inclusion and empowerment? Community supported agriculture (CSA) in Wales as social innovation. Agriculture and Human Values, 37 (4), 1241–1260.

Mikkonen, I., Gynther, L., Matschoss, K., Koukoufikis, G., Murauskaite-Bull, I. , & Uihlein, A. (2020). Social innovations for the energy transition . Luxembourg.

Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., PRISMA-P Group. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4 (1), 1–9.

Morris-Suzuki, T., & Soh, E. J. (2017). Social Innovation in Asia: Trends and Characteristics in China, Korea, India, Japan and Thailand. In T. Morris-Suzuki & E. J. Soh (Eds.), New Worlds from Below: Informal Life Politics and Grassroots Action in Twenty-First-Century Northeast Asia. ANU Press.

Moulaert, F., Martinelli, F., Swyngedouw, E., & Gonzalez, S. (2005). Towards alternative model(s) of local innovation. Urban Studies, 42 (11), 1969–1990.

Mulgan, G., Tucker, S., Ali, R., & Sanders, B. (2007). Social Innovation: what it is, why it matters, how it can be accelerated .

Mumford, M. D. (2002). Social innovation: Ten cases from Benjamin Franklin. Creativity Research Journal, 14 (2), 253–266.

Murphy, K. (2012). The social pillar of sustainable development: A literature review and framework for policy analysis. Sustainability: Science, Practice and Policy, 8 (1), 15–29.

Nicholls, A., & Murdock, A. (2012). Social Innovation: Blurring Boundaries to Reconfigure Markets . Palgrave Macmillan.

Nicholls, A., Simon, J., & Gabriel, M. (2015). New Frontiers in Social Innovation Research . Springer Nature.

Onitsuka, K. (2019). How social media can foster social innovation in disadvantaged rural communities. Sustainability, 11 (9), 2697.

Ou, J. Y., Zhang, H. N., & Ji, L. (2020). Service design strategy of rural culture rejuvenation under the guidance of social innovation vision. Packaging Engineering, 41 (22), 161–167.

Pan, S., & Sarantou, M. (2019). Ageing communities as co-designers of social innovation. China Journal of Social Work, 12 (3), 273–286.

Pel, B., Haxeltine, A., Avelino, F., Dumitru, A., Kemp, R., Bauler, T., Kunze, I., Dorland, J., Wittmayer, J., & Jørgensen, M. S. (2020). Towards a theory of transformative social innovation: A relational framework and 12 propositions. Research Policy, 49 (8), 104080.

Peters, M. A., & Heraud, R. (2015). Toward a political theory of social innovation: Collective intelligence and the co-creation of social goods. Journal of Self-Governance & Management Economics , 3(3)

Phillips, W., Lee, H., Ghobadian, A., O’regan, N., & James, P. (2015). Social innovation and social entrepreneurship: A systematic review. Group & Organization Management, 40 (3), 428–461.

Phills, J. A., Deiglmeier, K., & Miller, D. T. (2008). Rediscovering social innovation. Stanford Social Innovation Review, 6 (4), 34–43.

Pol, E., & Ville, S. (2009). Social innovation: Buzz word or enduring term? The Journal of Socio-Economics, 38 (6), 878–885.

Post, C., Sarala, R., Gatrell, C., & Prescott, J. E. (2020). Advancing theory with review articles. Journal of Management Studies, 57 (2), 351–376.

Repo, P., & Matschoss, K. (2019). Social innovation for sustainability challenges. Sustainability, 12 (1), 319.

Rose, R., & Mackenzie, W. J. M. (1991). Comparing forms of comparative analysis. Political Studies, 39 (3), 446–462.

Rui, Y. (2020). The impact of corporate social innovation on economic performance: The perspective of value co-creation . Harbin University of Commerce.

Schäfer, M., de Figueiredo, M. D., Iran, S., Jaeger-Erben, M., Silva, M. E., Lazaro, J. C., & Meissner, M. (2021). Imitation, adaptation, or local emergency?–A cross-country comparison of social innovations for sustainable consumption in Brazil, Germany, and Iran. Journal of Cleaner Production, 284 , 124740.

Schleicher, D. J., Baumann, H. M., Sullivan, D. W., Levy, P. E., Hargrove, D. C., & Barros-Rivera, B. A. (2018). Putting the system into performance management systems: A review and agenda for performance management research. Journal of Management, 44 (6), 2209–2245.

Secco, L., Pisani, E., Da Re, R., Rogelja, T., Burlando, C., Vicentini, K., Pettenella, D., Masiero, M., Miller, D., & Nijnjk, M. (2019). Towards a method of evaluating social innovation in forest-dependent rural communities: First suggestions from a science-stakeholder collaboration. Forest Policy and Economics, 104 , 9–22.

Seyfang, G., & Smith, A. (2007). Grassroots innovations for sustainable development: Towards a new research and policy agenda. Environmental Politics, 16 (4), 584–603.

Sheng, Y., & Chen, J. X. (2017). Organization mode of subjective cooperation in the enterprise social innovation. Journal of Zhejiang University of Technology (social Science), 16 (02), 197–201.

Sheng, Y., & Jiang, X. H. (2020). Corporate social innovation process from the perspective of stakeholder network —A case study. Science Research Management, 41 (08), 160–170.

Sheng, Y., & Wang, F. (2017). Business model of enterprise social innovation: Case of Hangzhou Xiezhi college students hostel. Journal of Zhejiang Shuren University, 17 (03), 46–53.

Signori, S., & Forno, F. (2019). Consumer groups as grassroots social innovation niches. British Food Journal, 121 (3), 803–814.

Sinclair, S., & Baglioni, S. (2014). Social innovation and social policy–promises and risks. Social Policy and Society, 13 (3), 469–476.

Snyder, H. (2019). Literature review as a research methodology: An overview and guidelines. Journal of Business Research, 104 , 333–339.

Srinivas, M. L., Yang, E. J., Shrestha, P., Wu, D., Peeling, R. W., & Tucker, J. D. (2020). Social innovation in diagnostics: Three case studies. Infectious Diseases of Poverty, 9 (1), 1–7.

Tan, F.T.C., & Pan, S.L. (2016). IT-enabled social innovation in China’s taobao villages: The role of netrepreneurs , ICIS 2016.

Tan, F.T.C., Pan, S. L., & Cui, L. (2014). An information processing perspective of digital social innovation: Insights from China’s Taobao Villages . ICIS 2014.

Tang, W., Han, L., Best, J., Zhang, Y., Mollan, K., Kim, J., Liu, F. Y., Hudgens, M., Bayus, B., TerrisPrestholt, F., Galler, S., Yang, L. G., Peeling, R., Volberding, P., Ma, B. L., Xu, H. F., Yang, B., Huang, S. J., Fenton, K., & Tucker, J. D. (2016). Crowdsourcing HIV test promotion videos: A noninferiority randomized controlled trial in China. Clinical Infectious Diseases, 62 (11), 1436–1442.

Teets, J. C. (2012). Reforming service delivery in China: The emergence of a social innovation model. Journal of Chinese Political Science, 17 (1), 15–32.

Van Wijk, J., Zietsma, C., Dorado, S., De Bakker, F. G., & Marti, I. (2019). Social innovation: Integrating micro, meso, and macro level insights from institutional theory. Business & Society, 58 (5), 887–918.

Wang, Y. (2016). Rural middle key influence factors of e-commerce—Based on the stakeholders and social innovation . South China Agricultural University.

Xie, K. S., & Wang, Y. (2021). Self-organized collaborative innovation model of service-oriented open source community. Industrial and Engineering Design, 3 (01), 39–46.

Ye, Q., Zhou, R., Anwar, M. A., Siddiquei, A. N., & Asmi, F. (2020). Entrepreneurs and environmental sustainability in the digital era: Regional and institutional perspectives. International Journal of Environmental Research and Public Health, 17 (4), 1355.

Yu, X. (2016). Social entrepreneurship in China’s non-profit sector. The case of innovative participation of civil society in post-disaster reconstruction. China Perspectives, 2016 (2016/3), 53–61.

Yu, X. M., & Li, N. (2017). The innovation model of social enterprise-like online charity shops: A case study of Shantaowang. Comparative Economic & Social Systems, 05 , 136–145.

Yu, X., & Shi, Y. F. (2020). Practical exploration of social organizations participating in social innovation: Taking Ningbo used clothes recycling box as an example. Journal of Zhejiang Wanli University, 33 (01), 54–59.

Zahra, S. A., Gedajlovic, E., Neubaum, D. O., & Shulman, J. M. (2009). A typology of social entrepreneurs: Motives, search processes and ethical challenges. Journal of Business Venturing, 24 (5), 519–532.

Zhang, L. N. (2019). The cooperation of government, enterprises and social organizations in urban community governance——Taking Suzhou Zheng Rong academy project as an example. Journal of Shaanxi Academy of Governance, 33 (01), 12–16.

Zhang, L. H., & Yang, G. M. (2014). Cross-culture thinking in sustainable service design of social innovation: Case comparison between Milan, IT and Wuxi, China. Creation and Design, 03 , 66–70.

Zhang, M. H., Zeng, W. J., Tse, Y. K., Wang, Y. C., & Smart, P. (2021). Examining the antecedents and consequences of green product innovation. Industrial Marketing Management, 93 , 413–427.

Zhang, Q., Hu, Y. M., & Lu, Q. B. (2013). The emergence of social innovation in China: A synergistic, “government–market–society”model. Comparative Economic & Social Systems, 04 , 125–136.

Zhao, W. P. (2020). Theoretical logic and innovative orientation of finance with priority to social purpose: A case study of network mutual aid. Journal of Shanghai University (social Sciences Edition), 37 (04), 131–140.

Zheng, G. L., & Lan, Y. X. (2021). Gradual embedding: The strategic choices of social organizations participating in rural revitalization from the perspective of uncertainty—A case study on S foundation. Journal of Public Management, 18 (01), 126–136+174.

Zhou, H. Y. (2014). Social innovation sin China: An empirical study based on data from two sessions. Comparative Economic & Social Systems, 04 , 170–183.

Zhou, R. T., & Xie, X. Y. (2015). Social innovation mode research based on the bee enterprises. Science & Technology Progress and Policy, 32 (18), 93–97.

Zhu, N. P., Zhu, L., Kong, Y. S., & Shen, Y. (2014). Study on interaction of technological innovation investment and corporate social responsibility on corporate financial performance. Accounting Research, 02 , 57–63+95.

Ziegler, R. (2010). Innovations in doing and being: Capability innovations at the intersection of Schumpeterian political economy and human development. Journal of Social Entrepreneurship, 1 (2), 255–272.

Zimmerman, M. A. (2000). Empowerment theory: Psychological, organizational and community levels of analysis. Handbook of Community Psychology (pp. 43–63)

Download references

Acknowledgements

I am extremely grateful to the editors of Environment, Development and Sustainability and the anonymous reviewers who provided insightful comments.

This work was supported by Sichuan Normal University (Grant numbers 22XW044).

Author information

Authors and affiliations.

School of Business, Sichuan Normal University, Chengdu, People’s Republic of China

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Yao Lu .

Ethics declarations

Conflict of interest.

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Glossary of concepts translated from Chinese to English.

Entries reviewed., rights and permissions.

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Lu, Y. A systematic literature review on social innovation for sustainable development in China. Environ Dev Sustain (2024). https://doi.org/10.1007/s10668-024-04928-4

Download citation

Received : 29 January 2022

Accepted : 15 April 2024

Published : 05 May 2024

DOI : https://doi.org/10.1007/s10668-024-04928-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Sustainable development
• Social innovation
• Systematic literature review
• Find a journal
• Publish with us
• Track your research

• Open access
• Published: 29 April 2024

What is context in knowledge translation? Results of a systematic scoping review

• Tugce Schmitt   ORCID: orcid.org/0000-0001-6893-6428 1 ,
• Katarzyna Czabanowska 1 &
• Peter Schröder-Bäck 1  

Health Research Policy and Systems volume  22 , Article number:  52 ( 2024 ) Cite this article

210 Accesses

4 Altmetric

Metrics details

Knowledge Translation (KT) aims to convey novel ideas to relevant stakeholders, motivating their response or action to improve people’s health. Initially, the KT literature focused on evidence-based medicine, applying findings from laboratory and clinical research to disease diagnosis and treatment. Since the early 2000s, the scope of KT has expanded to include decision-making with health policy implications.

This systematic scoping review aims to assess the evolving knowledge-to-policy concepts, that is, macro-level KT theories, models and frameworks (KT TMFs). While significant attention has been devoted to transferring knowledge to healthcare settings (i.e. implementing health policies, programmes or measures at the meso-level), the definition of 'context' in the realm of health policymaking at the macro-level remains underexplored in the KT literature. This study aims to close the gap.

A total of 32 macro-level KT TMFs were identified, with only a limited subset of them offering detailed insights into contextual factors that matter in health policymaking. Notably, the majority of these studies prompt policy changes in low- and middle-income countries and received support from international organisations, the European Union, development agencies or philanthropic entities.

Peer Review reports

Few concepts are used by health researchers as vaguely and yet as widely as Knowledge Translation (KT), a catch-all term that accommodates a broad spectrum of ambitions. Arguably, to truly understand the role of context in KT, we first need to clarify what KT means. The World Health Organization (WHO) defines KT as ‘the synthesis, exchange and application of knowledge by relevant stakeholders to accelerate the benefits of global and local innovation in strengthening health systems and improving people’s health’ [ 1 ]. Here, particular attention should be paid to ‘innovation’, given that without unpacking this term, the meaning of KT would still remain ambiguous. Rogers’ seminal work ‘Diffusion of Innovations’ [ 2 ] defines innovation as an idea, practice or object that is perceived as novel by individuals or groups adopting it. In this context, he argues that the objective novelty of an idea in terms of the amount of time passed after its discovery holds little significance [ 2 ]. Rather, it is the subjective perception of newness by the individual that shapes their response [ 2 ]. In other words, if an idea seems novel to individuals, and thereby relevant stakeholders according to the aforementioned WHO definition, it qualifies as an innovation. From this perspective, it can be stated that a fundamental activity of KT is to communicate ideas that could be perceived as original to the targeted stakeholders, with the aim of motivating their response to improve health outcomes. This leaves us with the question of who exactly these stakeholders might be and what kind of actions would be required from them.

The scope of stakeholders in KT has evolved over time, along with their prompted responses. Initially, during the early phases of KT, the focus primarily revolved around healthcare providers and their clinical decisions, emphasising evidence-based medicine. Nearly 50 years ago, the first scientific article on KT was published, introducing Tier 1 KT, which concentrated on applying laboratory discoveries to disease diagnosis or treatment, also known as bench-to-bedside KT [ 3 ]. The primary motivation behind this initial conceptualisation of KT was to engage healthcare providers as the end-users of specific forms of knowledge, primarily related to randomised controlled trials of pharmaceuticals and evidence-based medicine [ 4 ]. In the early 2000s, the second phase of KT (Tier 2) emerged under the term ‘campus-to-clinic KT’ [ 3 ]. This facet, also known as translational research, was concerned with using evidence from health services research in healthcare provision, both in practice and policy [ 4 ]. Consequently, by including decision-makers as relevant end-users, KT scholars expanded the realm of research-to-action from the clinical environment to policy-relevant decision-making [ 5 ]. Following this trajectory, additional KT schemes (Tier 3–Tier 5) have been introduced into academic discourse, encompassing the dissemination, implementation and broader integration of knowledge into public policies [ 6 , 7 ]. Notably, the latest scheme (Tier 5) is becoming increasingly popular and represents the broadest approach, which describes the translation of knowledge to global communities and aims to involve fundamental, universal change in attitudes, policies and social systems [ 7 ].

In other words, a noticeable shift in KT has occurred with time towards macro-level interventions, named initially as evidence- based policymaking and later corrected to evidence- informed policymaking. In parallel with these significant developments, various alternative terms to KT have emerged, including ‘implementation science’, ‘knowledge transfer’, and ‘dissemination and research use’, often with considerable overlap [ 8 ]. Arguably, among the plethora of alternative terms proposed, implementation science stands out prominently. While initially centred on evidence-based medicine at the meso-level (e.g. implementing medical guidelines), it has since broadened its focus to ‘encompass all aspects of research relevant to the scientific study of methods to promote the uptake of research findings into routine settings in clinical, community and policy contexts’ [ 9 ], closely mirroring the definition to KT. Thus, KT, along with activities under different names that share the same objective, has evolved into an umbrella term over the years, encompassing a wide range of strategies aimed at enhancing the impact of research not only on clinical practice but also on public policies [ 10 ]. Following the adoption of such a comprehensive definition of KT, some researchers have asserted that using evidence in public policies is not merely commendable but essential [ 11 ].

In alignment with the evolution of KT from (bio-)medical sciences to public policies, an increasing number of scholars have offered explanations on how health policies should be developed [ 12 ], indicating a growing focus on exploring the mechanisms of health policymaking in the KT literature. However, unlike in the earlier phases of KT, which aimed to transfer knowledge from the laboratory to healthcare provision, decisions made for public policies may be less technical and more complex than those in clinical settings [ 3 , 13 , 14 ]. Indeed, social scientists point out that scholarly works on evidence use in health policies exhibit theoretical shortcomings as they lack engagement with political science and public administration theories and concepts [ 15 , 16 , 17 , 18 ]; only a few of these works employ policy theories and political concepts to guide data collection and make sense of their findings [ 19 ]. Similarly, contemporary literature that conceptualises KT as an umbrella term for both clinical and public policy decision-making, with calls for a generic ‘research-to-action’ [ 20 ], may fail to recognise the different types of actions required to change clinical practices and influence health policies. In many respects, such calls can even lead to a misconception that evidence-informed policymaking is simply a scaled-up version of evidence-based medicine [ 21 ].

In this study, we systematically review knowledge translation theories, models and frameworks (also known as KT TMFs) that were developed for health policies. Essentially, KT TMFs can be depicted as bridges that connect findings across diverse studies, as they establish a common language and standardise the measurement and assessment of desired policy changes [ 22 ]. This makes them essential for generalising implementation efforts and research findings [ 23 ]. While distinctions between a theory, a model or a framework are not always crystal-clear [ 24 ], the following definitions shed light on how they are interpreted in the context of KT. To start with, theory can be described as a set of analytical principles or statements crafted to structure our observations, enhance our understanding and explain the world [ 24 ]. Within implementation science, theories are encapsulated as either generalised models or frameworks. In other words, they are integrated into broader concepts, allowing researchers to form assumptions that help clarify phenomena and create hypotheses for testing [ 25 ].

Whereas theories in the KT literature are explanatory as well as descriptive, KT models are only descriptive with a more narrowly defined scope of explanation [ 24 ]; hence they have a more specific focus than theories [ 25 ]. KT models are created to facilitate the formulation of specific assumptions regarding a set of parameters or variables, which can subsequently be tested against outcomes using predetermined methods [ 25 ]. By offering simplified representations of complex situations, KT models can describe programme elements expected to produce desired results, or theoretical constructs believed to influence or moderate observed outcomes. In this way, they encompass theories related to change or explanation [ 22 ].

Lastly, frameworks in the KT language define a set of variables and the relations among them in a broad sense [ 25 ]. Frameworks, without the aim of providing explanations, solely describe empirical phenomena, representing a structure, overview, outline, system or plan consisting of various descriptive categories and the relations between them that are presumed to account for a phenomenon [ 24 ]. They portray loosely-structured constellations of theoretical constructs, without necessarily specifying their relationships; they can also offer practical methods for achieving implementation objectives [ 22 ]. Some scholars suggest sub-classifications and categorise a framework as ‘actionable’ if it has the potential to facilitate macro-level policy changes [ 11 ].

Context, which encompasses the entire environment in which policy decisions are made, is not peripheral but central to policymaking, playing a crucial role in its conceptualisation [ 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 ]. In the KT literature, the term ‘context’ is frequently employed, albeit often with a lack of precision [ 35 ]. It tends to serve as a broad term including various elements within a situation that are relevant to KT in some way but have not been explicitly identified [36]. However, there is a growing interest in delving deeper into what context refers to, as evidenced by increasing research attention [ 31 , 32 , 37 , 38 , 39 , 40 , 41 ]. While the definition of context in the transfer of knowledge to healthcare settings (i.e. implementing health policies, programmes or measures at the meso-level) has been systematically studied [ 36 , 37 , 42 , 43 ], the question of how KT scholars detail context in health policymaking remains unanswered. With our systematic scoping review, we aim to close this gap.

While KT TMFs, emerged from evidence-based medicine, have historically depicted the use of evidence from laboratories or healthcare organisations as the gold standard, we aimed to assess in this study whether and to what extent the evolving face of KT, addressing health policies, succeeded in foregrounding ‘context’. Our objective was thus not to evaluate the quality of these KT TMFs but rather to explore how scholars have incorporated contextual influences into their reasoning. We conducted a systematic scoping review to explore KT TMFs that are relevant to agenda-setting, policy formulation or policy adoption, in line with the aim of this study. Therefore, publications related to policy implementation in healthcare organisations or at the provincial level, as well as those addressing policy evaluation, did not meet our inclusion criteria. Consequently, given our focus on macro-level interventions, we excluded all articles that concentrate on translating clinical research into practice (meso-level interventions) and health knowledge to patients or citizens (micro-level interventions).

Prior systematic scoping reviews in the area of KT TMFs serve as a valuable foundation upon which to build further studies [ 44 , 45 ]. Using established methodologies may ensure a validated approach, allowing for a more nuanced understanding of KT TMFs in the context of existing scholarly work. Our review methodology employed a similar approach to that followed by Strifler et al. in 2018, who conducted a systematic scoping review of KT TMFs in the field of cancer prevention and management, as well as other chronic diseases [ 44 ]. Their search strategy was preferred over others for two primary reasons. First, Strifler et al. investigated KT TMFs altogether, systematically and comprehensively. Second, unlike many other review studies on KT, they focused on macro-level KT and included all relevant keywords useful for the purpose of our study in their Ovid/MEDLINE search query [ 44 ]. For our scoping review, we adapted their search query with the assistance of a specialist librarian. This process involved eliminating terms associated with cancer and chronic diseases, removing time limitation on the published papers, and including an additional language other than English due to authors’ proficiency in German. We included articles published in peer-reviewed journals until November 2022, excluding opinion papers, conference abstracts and study protocols, without any restriction on publication date or place. Our search query is presented in Table  1 .

Following a screening methodology similar to that employed by Votruba et al. [ 11 ], the first author conducted an initial screening of the titles and abstracts of 2918 unique citations. Full texts were selected and scrutinised if they appeared relevant to the topics of agenda-setting, policy formulation or policy adoption. Among these papers, the first author also identified those that conceptualised a KT TMF. Simultaneously, the last author independently screened 2918 titles and abstracts, randomly selecting 20% of them to identify studies related to macro-level KT. Regarding papers that conceptualised a KT TMF, all those initially selected by the first author underwent a thorough examination by the last author as well. In the papers reviewed by these two authors of this study, KT TMFs were typically presented as either Tables or Figures. In cases where these visual representations did not contain sufficient information about ‘context’, the main body of the study was carefully scrutinised by both reviewers to ensure no relevant information was missed. Any unclear cases were discussed and resolved to achieve 100% inter-rater agreement between the first and second reviewers. This strategy resulted in the inclusion of 32 relevant studies. The flow chart outlining our review process is provided in Fig.  1 .

Flow chart of the review process

According to the results of our systematic scoping review (Table  2 ), the first KT TMF developed for health policies dates back to 2003, confirming the emergence of a trend that expanded the meaning of the term Knowledge Translation to include policymakers as end-users of evidence during approximately the same period. In their study, Jacobson et al. [ 46 ] present a framework derived from a literature review to enhance understanding of user groups by organising existing knowledge, identifying gaps and emphasising the importance of learning about new contexts. However, despite acknowledging the significance of the user group context, the paper lacks a thorough explanation of the authors’ understanding of this term. The second study in our scoping review provides some details. Recognising a shift from evidence-based medicine to evidence-based health policymaking in the KT literature, the article by Dobrow et al. from 2004 [ 30 ] emphasises the importance of considering contextual factors. They present a conceptual framework for evidence-based decision-making, highlighting the influence of context in KT. Illustrated through examples from colorectal cancer screening policy development, their conceptual framework emphasises the significance of context in the introduction, interpretation and application of evidence. Third, Lehoux et al. [ 47 ] examine the field of Health Technology Assessment (HTA) and its role in informing decision and policymaking in Canada. By developing a conceptual framework for HTA dissemination and use, they touch on the institutional environment and briefly describe contextual factors.

Notably, the first three publications in our scoping review are authored by scholars affiliated with Canada, which is less of a coincidence, given the role of Canadian Institutes of Health Research (CIHR), the federal funding agency for health research: The CIHR Act (Bill C-13) mandates CIHR to ensure that the translation of health knowledge permeates every aspect of its work [ 48 ]. Moreover, it was CIHR that coined the term Knowledge Translation, defining KT as ‘a dynamic and iterative process that includes the synthesis, dissemination, exchange and ethically sound application of knowledge to improve health, provide more effective health services and products, and strengthen the health care system’ [ 49 ] . This comprehensive definition has since been adapted by international organisations (IOs), including WHO. The first document published by WHO that utilised KT to influence health policies dates back to 2005, entitled ‘Bridging the “know-do” gap: Meeting on knowledge translation in global health’, an initiative that was supported by the Canadian Coalition for Global Health Research, the Canadian International Development Agency, the German Agency for Technical Cooperation and the WHO Special Programme on Research and Training in Tropical Diseases [ 1 ]. Following this official recognition by WHO, studies in our scoping review after 2005 indicate a noticeable expansion of KT, encompassing a wider geographical area than Canada.

The article of Ashford et al. from 2006 [ 50 ] discusses the challenge of policy decisions in Kenya in the health field being disconnected from scientific evidence and presents a model for translating knowledge into policy actions through agenda-setting, coalition building and policy learning. However, the framework lacks explicit incorporation of contextual factors influencing health policies. Bauman et al. [ 51 ] propose a six-step framework for successful dissemination of physical activity evidence, illustrated through four case studies from three countries (Canada, USA and Brazil) and a global perspective. They interpret contextual factors as barriers and facilitators to physical activity and public health innovations. Focusing on the USA, Gold [ 52 ] explains factors, processes and actors that shape pathways between research and its use in a summary diagram, including a reference to ‘other influences in process’ for context. Green et al. [ 4 ] examine the gap between health research and its application in public health without focusing on a specific geographical area. Their study comprehensively reviews various concepts of diffusion, dissemination and implementation in public health, proposing ways to blend diffusion theory with other theories. Their ‘utilization-focused surveillance framework’ interprets context as social determinants as structures, economics, politics and culture.

Further, the article by Dhonukshe-Rutten et al. from 2010 [ 53 ] presents a general framework that outlines the process of translating nutritional requirements into policy applications from a European perspective. The framework incorporates scientific evidence, stakeholder interests and the socio-political context. The description of this socio-political context is rather brief, encompassing political and social priorities, legal context, ethical issues and economic implications. Ir et al. [ 54 ] analyse the use of knowledge in shaping policy on health equity funds in Cambodia, with the objective of understanding how KT contributes to the development of health policies that promote equity. Yet no information on context is available in the framework that they suggest. A notable exception among these early KT TMFs until 2010 is the conceptual framework for analysing integration of targeted health interventions into health systems by Atun et al. [ 55 ], in which the authors provide details about the factors that have an influence on the process of bringing evidence to health policies. Focusing on the adoption, diffusion and assimilation of health interventions, their conceptual framework provides a systematic approach for evaluating and informing policies in this field. Compared to the previous studies discussed above, their definition of context for this framework is comprehensive (Table  2 ). Overall, most of the studies containing macro-level KT TMFs published until 2010 either do not fully acknowledge contextual factors or provide generic terms such as cultural, political and economic for brief description (9 out of 10; 90%).

Studies published after 2010 demonstrate a notable geographical shift, with a greater emphasis on low- and middle-income countries (LMICs). By taking the adoption of the directly observed treatment, short-course (DOTS) strategy for tuberculosis control in Mexico as a case study, Bissell et al. [ 56 ] examine policy transfer to Mexico and its relevance to operational research efforts and suggest a model for analysis of health policy transfer. The model interprets context as health system, including political, economic, social, cultural and technological features. Focusing on HIV/AIDS in India, Tran et al. [ 57 ] explore KT by considering various forms of evidence beyond scientific evidence, such as best practices derived from programme experience and disseminated through personal communication. Their proposed framework aims to offer an analytical tool for understanding how evidence-based influence is exerted. In their framework, no information is available on context. Next, Bertone et al. [ 58 ] report on the effectiveness of Communities of Practice (CoPs) in African countries and present a conceptual framework for analysing and assessing transnational CoPs in health policy. The framework organises the key elements of CoPs, linking available resources, knowledge management activities, policy and practice changes, and improvements in health outcomes. Context is only briefly included in this framework.

Some other studies include both European and global perspectives. The publication from Timotijevic et al. from 2013 [ 59 ] introduces an epistemological framework that examines the considerations influencing the policy-making process, with a specific focus on micronutrient requirements in Europe. They present case studies from several European countries, highlighting the relevance of the framework in understanding the policy context related to micronutrients. Context is interpreted in this framework as global trends, data, media, broader consumer beliefs, ethical considerations, and wider social, legal, political, and economic environment. Next, funded by the European Union, the study by Onwujekwe et al. [ 60 ] examines the role of different types of evidence in health policy development in Nigeria. Although they cover the factors related to policy actors in their framework for assessing the role of evidence in policy development, they provide no information on context. Moreover, Redman et al. [ 61 ] present the SPIRIT Action Framework, which aims to enhance the use of research in policymaking. Context is interpreted in this framework as policy influences, i.e. public opinion, media, economic climate, legislative/policy infrastructure, political ideology and priorities, stakeholder interests, expert advice, and resources. From a global perspective, Spicer et al. [ 62 ] explore the contextual factors that influenced the scale-up of donor-funded maternal and newborn health innovations in Ethiopia, India and Nigeria, highlighting the importance of context in assessing and adapting innovations. Their suggested contextual factors influencing government decisions to accept, adopt and finance innovations at scale are relatively comprehensive (Table  2 ).

In terms of publication frequency, the pinnacle of reviewed KT studies was in 2017. Among six studies published in 2017, four lack details about context in their KT conceptualisations and one study touches on context very briefly. Bragge et al. [ 5 ] brought for their study an international terminology working group together to develop a simplified framework of interventions to integrate evidence into health practices, systems, and policies, named as the Aims, Ingredients, Mechanism, Delivery framework, albeit without providing details on contextual factors. Second, Mulvale et al. [ 63 ] present a conceptual framework that explores the impact of policy dialogues on policy development, illustrating how these dialogues can influence different stages of the policy cycle. Similar to the previous one, this study too, lacks information on context. In a systematic review, Sarkies et al. [ 64 ] evaluate the effectiveness of research implementation strategies in promoting evidence-informed policy decisions in healthcare. The study explores the factors associated with effective strategies and their inter-relationship, yet without further information on context. Fourth, Houngbo et al. [ 65 ] focus on the development of a strategy to implement a good governance model for health technology management in the public health sector, drawing from their experience in Benin. They outline a six-phase model that includes preparatory analysis, stakeholder identification and problem analysis, shared analysis and visioning, development of policy instruments for pilot testing, policy development and validation, and policy implementation and evaluation. They provide no information about context in their model. Fifth, Mwendera et al. [ 66 ] present a framework for improving the use of malaria research in policy development in Malawi, which was developed based on case studies exploring the policymaking process, the use of local malaria research, and assessing facilitators and barriers to research utilisation. Contextual setting is considered as Ministry of Health (MoH) with political set up, leadership system within the MoH, government policies and cultural set up. In contrast to these five studies, Ellen et al. [ 67 ] present a relatively comprehensive framework to support evidence-informed policymaking in ageing and health. The framework includes thought-provoking questions to discover contextual factors (Table  2 ).

Continuing the trend, studies published after 2017 focus increasingly on LMICs. In their embedded case study, Ongolo-Zogo et al. [ 68 ] examine the influence of two Knowledge Translation Platforms (KTPs) on policy decisions to achieve the health millennium development goals in Cameroon and Uganda. It explores how these KTPs influenced policy through interactions within policy issue networks, engagement with interest groups, and the promotion of evidence-supported ideas, ultimately shaping the overall policy climate for evidence-informed health system policymaking. Contextual factors are thereby interpreted as institutions (structures, legacies, policy networks), interests, ideas (values, research evidence) and external factors (reports, commitments). Focusing on the ‘Global South’, Plamondon et al. [ 69 ] suggest blending integrated knowledge translation with global health governance as an approach for strengthening leadership for health equity action. In terms of contextual factors, they include some information such as adapting knowledge to local context, consideration of the composition of non-traditional actors, such as civil society and private sector, in governance bodies and guidance for meaningful engagement between actors, particularly in shared governance models. Further, Vincenten et al. [ 70 ] propose a conceptual model to enhance understanding of interlinking factors that influence the evidence implementation process. Their evidence implementation model for public health systems refers to ‘context setting’, albeit without providing further detail.

Similarly, the study by Motani et al. from 2019 [ 71 ] assesses the outcomes and lessons learned from the EVIDENT partnership that focused on knowledge management for evidence-informed decision-making in nutrition and health in Africa. Although they mention ‘contextualising evidence’ in their conceptual framework, information about context is lacking. Focusing on Latin America and the Caribbean, Varallyay et al. [ 72 ] introduce a conceptual framework for evaluating embedded implementation research in various contexts. The framework outlines key stages of evidence-informed decision-making and provides guidance on assessing embeddedness and critical contextual factors. Compared to others, their conceptual framework provides a relatively comprehensive elaboration on contextual factors. In addition, among all the studies reviewed, Leonard et al. [ 73 ] present an exceptionally comprehensive analysis, where they identify the facilitators and barriers to the sustainable implementation of evidence-based health innovations in LMICs. Through a systematic literature review, they scrutinise 79 studies and categorise the identified barriers and facilitators into seven groups: context, innovation, relations and networks, institutions, knowledge, actors, and resources. The first one, context, contains rich information that could be seen in Table  2 .

Continuing from LMICs, Votruba et al. [ 74 ] present in their study the EVITA (EVIdence To Agenda setting) conceptual framework for mental health research-policy interrelationships in LMICs with some information about context, detailed as external influences and political context. In a follow-up study, they offer an updated framework for understanding evidence-based mental health policy agenda-setting [ 75 ]. In their revised framework, context is interpreted as external context and policy sphere, encompassing policy agenda, window of opportunity, political will and key individuals. Lastly, to develop a comprehensive monitoring and evaluation framework for evidence-to-policy networks, Kuchenmüller et al. [ 76 ] present the EVIPNet Europe Theory of Change and interpret contextual factors for evidence-informed policymaking as political, economic, logistic and administrative. Overall, it can be concluded that studies presenting macro-level KT TMFs from 2011 until 2022 focus mainly on LMICs (15 out of 22; close to 70%) and the majority of them were funded by international (development) organisations, the European Commission and global health donor agencies. An overwhelming number of studies among them (19 out of 22; close to 90%) provide either no information on contextual details or these were included only partly with some generic terms in KT TMFs.

Our systematic scoping review suggests that the approach of KT, which has evolved from evidence-based medicine to evidence-informed policymaking, tends to remain closely tied to its clinical origins when developing TMFs. In other words, macro-level KT TMFs place greater emphasis on the (public) health issue at hand rather than considering the broader decision-making context, a viewpoint shared by other scholars as well [ 30 ]. One reason could be that in the early stages of KT TMFs, the emphasis primarily focused on implementing evidence-based practices within clinical settings. At that time, the spotlight was mostly on content, including aspects like clinical studies, checklists and guidelines serving as the evidence base. In those meso-level KT TMFs, a detailed description of context, i.e. the overall environment in which these practices should be implemented, might have been deemed less necessary, given that healthcare organisations, such as hospitals to implement medical guidelines or surgical safety checklists, show similar characteristics globally.

However, as the scope of KT TMFs continues to expand to include the influence on health policies, a deeper understanding of context-specific factors within different jurisdictions and the dynamics of the policy process is becoming increasingly crucial. This is even more important for KT scholars aiming to conceptualise large-scale changes, as described in KT Tier 5, which necessitate a thorough understanding of targeted behaviours within societies. As the complexity of interventions increases due to the growing number of stakeholders either affecting or being affected by them, the interventions are surrounded by a more intricate web of attitudes, incentives, relationships, rules of engagement and spheres of influence [ 7 ]. The persisting emphasis on content over context in the evolving field of KT may oversimplify the complex process of using evidence in policymaking and understanding the society [ 77 ]. Some scholars argue that this common observation in public health can be attributed to the dominance of experts primarily from medical sciences [ 78 , 79 , 80 ]. Our study confirms the potential limitation of not incorporating insights from political science and public policy studies, which can lead to what is often termed a ‘naïve’ conceptualisation of evidence-to-policy schemes [ 15 , 16 , 17 ]. It is therefore strongly encouraged that the emerging macro-level KT concepts draw on political science and public administration if KT scholars intend to effectively communicate new ideas to policymakers, with the aim of prompting their action or response. We summarised our findings into three points.

Firstly, KT scholars may want to identify and pinpoint exactly where a change should occur within the policy process. The main confusion that we observed in the KT literature arises from a lack of understanding of how public policies are made. Notably, the term ‘evidence-informed policymaking’ can refer to any stage of the policy cycle, spanning from agenda-setting to policy formulation, adoption, implementation and evaluation. Understanding these steps will allow researchers to refine their language when advocating for policy changes across various jurisdictions; for instance, the word ‘implementation’ is often inappropriately used in KT literature. As commonly known, at the macro-level, public policies take the form of legislation, law-making and regulation, thereby shaping the practices or policies to be implemented at the meso- and micro-levels [ 81 ]. In other words, the process of using specific knowledge to influence health policies, however evidence-based it might be, falls mostly under the responsibility and jurisdiction of sovereign states. For this reason, macro-level KT TMFs should reflect the importance of understanding the policy context and the complexities associated with policymaking, rather than suggesting flawed or unrealistic top-down ‘implementation’ strategies in countries by foregrounding the content, or the (public) health issue at hand.

Our second observation from this systematic scoping review points towards a selective perception among researchers when reporting on policy interventions. Research on KT does not solely exist due to the perceived gap between scientific evidence and policy but also because of the pressures the organisations or researchers face in being accountable to their funding sources, ensuring the continuity of financial support for their activities and claiming output legitimacy to change public policies [ 8 ]. This situation indirectly compels researchers working to influence health policies in the field to provide ‘evidence-based’ feedback on the success of their projects to donors [ 82 ]. In doing so, researchers may overly emphasise the content of the policy intervention in their reporting to secure further funding, while they underemphasis the contextual factors. These factors, often perceived as a given, might actually be the primary facilitators of their success. Such a lack of transparency regarding the definition of context is particularly visible in the field of global health, where LMICs often rely on external donors. It is important to note that this statement is not intended as a negative critique of their missions or an evaluation of health outcomes in countries following such missions. Rather, it seeks to explain the underlying reason why researchers, particularly those reliant on donors in LMICs, prioritise promoting the concept of KT from a technical standpoint, giving less attention to contextual factors in their reasoning.

Lastly, and connected to the previous point, it is our observation that the majority of macro-level KT TMFs fail to give adequate consideration to both power dynamics in countries (internal vs. external influences) and the actual role that government plays in public policies. Notably, although good policymaking entails an honest effort to use the best available evidence, the belief that this will completely negate the role of power and politics in decision-making is a technocratic illusion [ 83 ]. Among the studies reviewed, the framework put forth by Leonard et al. [ 73 ] offers the most comprehensive understanding of context and includes a broad range of factors (such as political, social, and economic) discovered also in other reviewed studies. Moreover, the framework, developed through an extensive systematic review, offers a more in-depth exploration of these contextual factors than merely listing them as a set of keywords. Indeed, within the domains of political science and public policy, such factors shaping health policies have received considerable scholarly attention for decades. To define what context entails, Walt refers in her book ‘Health Policy: An Introduction to Process and Power’ [ 84 ] to the work of Leichter from 1979 [ 85 ], who provides a scheme for analysing public policy. This includes i) situational factors, which are transient, impermanent, or idiosyncratic; ii) structural factors, which are relatively unchanging elements of the society and polity; iii) cultural factors, which are value commitments of groups; and iv) environmental factors, which are events, structures and values that exist outside the boundaries of a political system and influence decisions within it. His detailed sub-categories for context can be found in Table  3 . This flexible public policy framework may offer KT researchers a valuable approach to understanding contextual factors and provide some guidance to define the keywords to focus on. Scholars can adapt this framework to suit a wide range of KT topics, creating more context-sensitive and comprehensive KT TMFs.

Admittedly, our study has certain limitations. Despite choosing one of the most comprehensive bibliographic databases for our systematic scoping review, which includes materials from biomedicine, allied health fields, biological and physical sciences, humanities, and information science in relation to medicine and healthcare, we acknowledge that we may have missed relevant articles indexed in other databases. Hence, exclusively using Ovid/MEDLINE due to resource constraints may have narrowed the scope and diversity of scholarly literature examined in this study. Second, our review was limited to peer-reviewed publications in English and German. Future studies could extend our findings by examining the extent to which contextual factors are detailed in macro-level KT TMFs published in grey literature and in different languages. Given the abundance of KT reports, working papers or policy briefs published by IOs and development agencies, such an endeavour could enrich our findings and either support or challenge our conclusions. Nonetheless, to our knowledge, this study represents the first systematic review and critical appraisal of emerging knowledge-to-policy concepts, also known as macro-level KT TMFs. It successfully blends insights from both biomedical and public policy disciplines, and could serve as a roadmap for future research.

The translation of knowledge to policymakers involves more than technical skills commonly associated with (bio-)medical sciences, such as creating evidence-based guidelines or clinical checklists. Instead, evidence-informed policymaking reflects an ambition to engage in the political dimensions of states. Therefore, the evolving KT concepts addressing health policies should be seen as a political decision-making process, rather than a purely analytical one, as is the case with evidence-based medicine. To better understand the influence of power dynamics and governance structures in policymaking, we suggest that future macro-level KT TMFs draw on insights from political science and public administration. Collaborative, interdisciplinary research initiatives could be undertaken to bridge the gap between these fields. Technocratic KT TMFs that overlook contextual factors risk propagating misconceptions in academic circles about how health policies are made, as they become increasingly influential over time. Research, the systematic pursuit of knowledge, is neither inherently good nor bad; it can be sought after, used or misused, like any other tool in policymaking. What is needed in the KT discourse is not another generic call for ‘research-to-action’ but rather an understanding of the dividing line between research-to- clinical -action and research-to- political -action.

Availability of data and materials

Available upon reasonable request.

WHO. Bridging the ‘Know-Do’ Gap: Meeting on Knowledge Translation in Global Health : 10–12 October 2005 World Health Organization Geneva, Switzerland [Internet]. 2005. https://www.measureevaluation.org/resources/training/capacity-building-resources/high-impact-research-training-curricula/bridging-the-know-do-gap.pdf

Rogers EM. Diffusion of innovations. 3rd ed. New York: Free Press; 1983.

Google Scholar  

Greenhalgh T, Wieringa S. Is it time to drop the ‘knowledge translation’ metaphor? A critical literature review. J R Soc Med. 2011;104(12):501–9.

Article   PubMed   PubMed Central   Google Scholar  

Green LW, Ottoson JM, García C, Hiatt RA. Diffusion theory and knowledge dissemination, utilization, and integration in public health. Annu Rev Public Health. 2009;30(1):151–74.

Article   PubMed   Google Scholar  

Bragge P, Grimshaw JM, Lokker C, Colquhoun H, Albrecht L, Baron J, et al. AIMD—a validated, simplified framework of interventions to promote and integrate evidence into health practices, systems, and policies. BMC Med Res Methodol. 2017;17(1):38.

Zarbin M. What Constitutes Translational Research? Implications for the Scope of Translational Vision Science and Technology. Transl Vis Sci Technol 2020;9(8).

Hassmiller Lich K, Frerichs L, Fishbein D, Bobashev G, Pentz MA. Translating research into prevention of high-risk behaviors in the presence of complex systems: definitions and systems frameworks. Transl Behav Med. 2016;6(1):17–31.

Tetroe JM, Graham ID, Foy R, Robinson N, Eccles MP, Wensing M, et al. Health research funding agencies’ support and promotion of knowledge translation: an international study. Milbank Q. 2008;86(1):125–55.

Eccles MP, Mittman BS. Welcome to Implementation Science. Implement Sci. 2006;1(1):1.

Article   PubMed Central   Google Scholar  

Rychetnik L, Bauman A, Laws R, King L, Rissel C, Nutbeam D, et al. Translating research for evidence-based public health: key concepts and future directions. J Epidemiol Community Health. 2012;66(12):1187–92.

Votruba N, Ziemann A, Grant J, Thornicroft G. A systematic review of frameworks for the interrelationships of mental health evidence and policy in low- and middle-income countries. Health Res Policy Syst. 2018;16(1):85.

Delnord M, Tille F, Abboud LA, Ivankovic D, Van Oyen H. How can we monitor the impact of national health information systems? Results from a scoping review. Eur J Public Health. 2020;30(4):648–59.

Malterud K, Bjelland AK, Elvbakken KT. Evidence-based medicine—an appropriate tool for evidence-based health policy? A case study from Norway. Health Res Policy Syst. 2016;14(1):15.

Borst RAJ, Kok MO, O’Shea AJ, Pokhrel S, Jones TH, Boaz A. Envisioning and shaping translation of knowledge into action: a comparative case-study of stakeholder engagement in the development of a European tobacco control tool. Health Policy. 2019;123(10):917–23.

Liverani M, Hawkins B, Parkhurst JO. Political and institutional influences on the use of evidence in public health policy: a systematic review. PLoS ONE. 2013;8(10): e77404.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Cairney P. The politics of evidence-based policy making, 1st ed. London: Palgrave Macmillan UK: Imprint: Palgrave Pivot, Palgrave Macmillan; 2016.

Parkhurst J. The Politics of Evidence: From evidence-based policy to the good governance of evidence [Internet]. Routledge; 2016. https://www.taylorfrancis.com/books/9781315675008

Cairney P, Oliver K. Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy? Health Res Policy Syst. 2017;15(1):35.

Verboom B, Baumann A. Mapping the Qualitative Evidence Base on the Use of Research Evidence in Health Policy-Making: A Systematic Review. Int J Health Policy Manag. 2020;16.

Ward V, House A, Hamer S. Developing a framework for transferring knowledge into action: a thematic analysis of the literature. J Health Serv Res Policy. 2009;14(3):156–64.

Swinburn B, Gill T, Kumanyika S. Obesity prevention: a proposed framework for translating evidence into action. Obes Rev. 2005;6(1):23–33.

Article   CAS   PubMed   Google Scholar  

Damschroder LJ. Clarity out of chaos: Use of theory in implementation research. Psychiatry Res. 2020;283: 112461.

Birken SA, Rohweder CL, Powell BJ, Shea CM, Scott J, Leeman J, et al. T-CaST: an implementation theory comparison and selection tool. Implement Sci. 2018;13(1):143.

Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10(1):53.

Rapport F, Clay-Williams R, Churruca K, Shih P, Hogden A, Braithwaite J. The struggle of translating science into action: foundational concepts of implementation science. J Eval Clin Pract. 2018;24(1):117–26.

Hagenaars LL, Jeurissen PPT, Klazinga NS. The taxation of unhealthy energy-dense foods (EDFs) and sugar-sweetened beverages (SSBs): An overview of patterns observed in the policy content and policy context of 13 case studies. Health Policy. 2017;121(8):887–94.

Sheikh K, Gilson L, Agyepong IA, Hanson K, Ssengooba F, Bennett S. Building the field of health policy and systems research: framing the questions. PLOS Med. 2011;8(8): e1001073.

Tran NT, Hyder AA, Kulanthayan S, Singh S, Umar RSR. Engaging policy makers in road safety research in Malaysia: a theoretical and contextual analysis. Health Policy. 2009;90(1):58–65.

Walt G, Gilson L. Reforming the health sector in developing countries: the central role of policy analysis. Health Policy Plan. 1994;9(4):353–70.

Dobrow MJ, Goel V, Upshur REG. Evidence-based health policy: context and utilisation. Soc Sci Med. 2004;58(1):207–17.

Barnfield A, Savolainen N, Lounamaa A. Health Promotion Interventions: Lessons from the Transfer of Good Practices in CHRODIS-PLUS. Int J Environ Res Public Health. 2020;17(4).

van de Goor I, Hämäläinen RM, Syed A, Juel Lau C, Sandu P, Spitters H, et al. Determinants of evidence use in public health policy making: results from a study across six EU countries. Health Policy Amst Neth. 2017;121(3):273–81.

Article   Google Scholar  

Ornstein JT, Hammond RA, Padek M, Mazzucca S, Brownson RC. Rugged landscapes: complexity and implementation science. Implement Sci. 2020;15(1):85.

Seward N, Hanlon C, Hinrichs-Kraples S, Lund C, Murdoch J, Taylor Salisbury T, et al. A guide to systems-level, participatory, theory-informed implementation research in global health. BMJ Glob Health. 2021;6(12): e005365.

Pfadenhauer LM, Gerhardus A, Mozygemba K, Lysdahl KB, Booth A, Hofmann B, et al. Making sense of complexity in context and implementation: the Context and Implementation of Complex Interventions (CICI) framework. Implement Sci. 2017;12(1):21.

Rogers L, De Brún A, McAuliffe E. Defining and assessing context in healthcare implementation studies: a systematic review. BMC Health Serv Res. 2020;20(1):591.

Nilsen P, Bernhardsson S. Context matters in implementation science: a scoping review of determinant frameworks that describe contextual determinants for implementation outcomes. BMC Health Serv Res. 2019;19(1):189.

Arksey H, O’Malley L, Baldwin S, Harris J, Mason A, Golder S. Literature review report: services to support carers of people with mental health problems. 2002;182.

Tabak RG, Khoong EC, Chambers D, Brownson RC. Bridging research and practice. Am J Prev Med. 2012;43(3):337–50.

O’Donovan MA, McCallion P, McCarron M, Lynch L, Mannan H, Byrne E. A narrative synthesis scoping review of life course domains within health service utilisation frameworks. HRB Open Res. 2019.

Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A, et al. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005;14(1):26–33.

Bate P, Robert G, Fulop N, Øvretviet J, Dixon-Woods M. Perspectives on context: a collection of essays considering the role of context in successful quality improvement [Internet]. 2014. https://www.health.org.uk/sites/default/files/PerspectivesOnContext_fullversion.pdf

Ziemann A, Brown L, Sadler E, Ocloo J, Boaz A, Sandall J. Influence of external contextual factors on the implementation of health and social care interventions into practice within or across countries—a protocol for a ‘best fit’ framework synthesis. Syst Rev. 2019. https://doi.org/10.1186/s13643-019-1180-8 .

Strifler L, Cardoso R, McGowan J, Cogo E, Nincic V, Khan PA, et al. Scoping review identifies significant number of knowledge translation theories, models, and frameworks with limited use. J Clin Epidemiol. 2018;100:92–102.

Esmail R, Hanson HM, Holroyd-Leduc J, Brown S, Strifler L, Straus SE, et al. A scoping review of full-spectrum knowledge translation theories, models, and frameworks. Implement Sci. 2020;15(1):11.

Jacobson N, Butterill D, Goering P. Development of a framework for knowledge translation: understanding user context. J Health Serv Res Policy. 2003;8(2):94–9.

Lehoux P, Denis JL, Tailliez S, Hivon M. Dissemination of health technology assessments: identifying the visions guiding an evolving policy innovation in Canada. J Health Polit Policy Law. 2005;30(4):603–42.

Parliament of Canada. Government Bill (House of Commons) C-13 (36–2) - Royal Assent - Canadian Institutes of Health Research Act [Internet]. https://parl.ca/DocumentViewer/en/36-2/bill/C-13/royal-assent/page-31 . Accessed 1 Apr 2023.

Straus SE, Tetroe J, Graham I. Defining knowledge translation. CMAJ Can Med Assoc J. 2009;181(3–4):165–8.

Ashford L. Creating windows of opportunity for policy change: incorporating evidence into decentralized planning in Kenya. Bull World Health Organ. 2006;84(8):669–72.

Bauman AE, Nelson DE, Pratt M, Matsudo V, Schoeppe S. Dissemination of physical activity evidence, programs, policies, and surveillance in the international public health arena. Am J Prev Med. 2006;31(4):57–65.

Gold M. Pathways to the use of health services research in policy. Health Serv Res. 2009;44(4):1111–36.

Dhonukshe-Rutten RAM, Timotijevic L, Cavelaars AEJM, Raats MM, de Wit LS, Doets EL, et al. European micronutrient recommendations aligned: a general framework developed by EURRECA. Eur J Clin Nutr. 2010;64(2):S2-10.

Ir P, Bigdeli M, Meessen B, Van Damme W. Translating knowledge into policy and action to promote health equity: The Health Equity Fund policy process in Cambodia 2000–2008. Health Policy. 2010;96(3):200–9.

Atun R, de Jongh T, Secci F, Ohiri K, Adeyi O. Integration of targeted health interventions into health systems: a conceptual framework for analysis. Health Policy Plan. 2010;25(2):104–11.

Bissell K, Lee K, Freeman R. Analysing policy transfer: perspectives for operational research. Int J Tuberc Lung Dis Off J Int Union Tuberc Lung Dis. 2011;15(9).

Tran NT, Bennett SC, Bishnu R, Singh S. Analyzing the sources and nature of influence: how the Avahan program used evidence to influence HIV/AIDS prevention policy in India. Implement Sci. 2013;8(1):44.

Bertone MP, Meessen B, Clarysse G, Hercot D, Kelley A, Kafando Y, et al. Assessing communities of practice in health policy: a conceptual framework as a first step towards empirical research. Health Res Policy Syst. 2013;11(1):39.

Timotijevic L, Brown KA, Lähteenmäki L, de Wit L, Sonne AM, Ruprich J, et al. EURRECA—a framework for considering evidence in public health nutrition policy development. Crit Rev Food Sci Nutr. 2013;53(10):1124–34.

Onwujekwe O, Uguru N, Russo G, Etiaba E, Mbachu C, Mirzoev T, et al. Role and use of evidence in policymaking: an analysis of case studies from the health sector in Nigeria. Health Res Policy Syst. 2015;13(1):46.

Redman S, Turner T, Davies H, Williamson A, Haynes A, Brennan S, et al. The SPIRIT action framework: a structured approach to selecting and testing strategies to increase the use of research in policy. Soc Sci Med. 2015;136–137:147–55.

Spicer N, Berhanu D, Bhattacharya D, Tilley-Gyado RD, Gautham M, Schellenberg J, et al. ‘The stars seem aligned’: a qualitative study to understand the effects of context on scale-up of maternal and newborn health innovations in Ethiopia, India and Nigeria. Glob Health. 2016;12(1):75.

Mulvale G, McRae SA, Milicic S. Teasing apart “the tangled web” of influence of policy dialogues: lessons from a case study of dialogues about healthcare reform options for Canada. Implement Sci IS. 2017;12.

Sarkies MN, Bowles KA, Skinner EH, Haas R, Lane H, Haines TP. The effectiveness of research implementation strategies for promoting evidence-informed policy and management decisions in healthcare: a systematic review. Implement Sci. 2017;12(1):132.

Houngbo PTh, Coleman HLS, Zweekhorst M, De Cock Buning TJ, Medenou D, Bunders JFG. A Model for Good Governance of Healthcare Technology Management in the Public Sector: Learning from Evidence-Informed Policy Development and Implementation in Benin. PLoS ONE. 2017;12(1):e0168842.

Mwendera C, de Jager C, Longwe H, Hongoro C, Phiri K, Mutero CM. Development of a framework to improve the utilisation of malaria research for policy development in Malawi. Health Res Policy Syst. 2017;15(1):97.

Ellen ME, Panisset U, de AraujoCarvalho I, Goodwin J, Beard J. A knowledge translation framework on ageing and health. Health Policy. 2017;121(3):282–91.

Ongolo-Zogo P, Lavis JN, Tomson G, Sewankambo NK. Assessing the influence of knowledge translation platforms on health system policy processes to achieve the health millennium development goals in Cameroon and Uganda: a comparative case study. Health Policy Plan. 2018;33(4):539–54.

Plamondon KM, Pemberton J. Blending integrated knowledge translation with global health governance: an approach for advancing action on a wicked problem. Health Res Policy Syst. 2019;17(1):24.

Vincenten J, MacKay JM, Schröder-Bäck P, Schloemer T, Brand H. Factors influencing implementation of evidence-based interventions in public health systems—a model. Cent Eur J Public Health. 2019;27(3):198–203.

Motani P, Van de Walle A, Aryeetey R, Verstraeten R. Lessons learned from Evidence-Informed Decision-Making in Nutrition & Health (EVIDENT) in Africa: a project evaluation. Health Res Policy Syst. 2019;17(1):12.

Varallyay NI, Langlois EV, Tran N, Elias V, Reveiz L. Health system decision-makers at the helm of implementation research: development of a framework to evaluate the processes and effectiveness of embedded approaches. Health Res Policy Syst. 2020;18(1):64.

Leonard E, de Kock I, Bam W. Barriers and facilitators to implementing evidence-based health innovations in low- and middle-income countries: a systematic literature review. Eval Program Plann. 2020;82: 101832.

Votruba N, Grant J, Thornicroft G. The EVITA framework for evidence-based mental health policy agenda setting in low- and middle-income countries. Health Policy Plan. 2020;35(4):424–39.

Votruba N, Grant J, Thornicroft G. EVITA 2.0, an updated framework for understanding evidence-based mental health policy agenda-setting: tested and informed by key informant interviews in a multilevel comparative case study. Health Res Policy Syst. 2021;19(1):35.

Kuchenmüller T, Chapman E, Takahashi R, Lester L, Reinap M, Ellen M, et al. A comprehensive monitoring and evaluation framework for evidence to policy networks. Eval Program Plann. 2022;91: 102053.

Ettelt S. The politics of evidence use in health policy making in Germany—the case of regulating hospital minimum volumes. J Health Polit Policy Law. 2017;42(3):513–38.

Greer SL, Bekker M, de Leeuw E, Wismar M, Helderman JK, Ribeiro S, et al. Policy, politics and public health. Eur J Public Health. 2017;27(suppl 4):40–3.

Fafard P, Cassola A. Public health and political science: challenges and opportunities for a productive partnership. Public Health. 2020;186:107–9.

Löblová O. Epistemic communities and experts in health policy-making. Eur J Public Health. 2018;28(suppl 3):7–10.

Maddalena V. Evidence-Based Decision-Making 8: Health Policy, a Primer for Researchers. In: Parfrey PS, Barrett BJ, editors. Clinical Epidemiology: Practice and Methods. New York, NY: Springer; 2015. (Methods in Molecular Biology).

Louis M, Maertens L. Why international organizations hate politics - Depoliticizing the world [Internet]. London and New York: Routledge; 2021. (Global Institutions). https://library.oapen.org/bitstream/handle/20.500.12657/47578/1/9780429883279.pdf

Hassel A, Wegrich K. How to do public policy. 1st ed. Oxford: Oxford University Press; 2022.

Book   Google Scholar  

Walt G. Health policy: an introduction to process and power. 7th ed. Johannesburg: Witwatersrand University Press; 2004.

Leichter HM. A comparative approach to policy analysis: health care policy in four nations. Cambridge: Cambridge University Press; 1979.

Download references

Acknowledgements

Not applicable.

Author information

Authors and affiliations.

Department of International Health, Care and Public Health Research Institute – CAPHRI, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands

Tugce Schmitt, Katarzyna Czabanowska & Peter Schröder-Bäck

You can also search for this author in PubMed   Google Scholar

Contributions

TS: Conceptualization, Methodology, Formal analysis, Investigation, Writing—Original Draft; KC: Writing—Review & Editing; PSB: Validation, Formal analysis, Writing—Review & Editing, Supervision.

Corresponding author

Correspondence to Tugce Schmitt .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests, additional information, publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Schmitt, T., Czabanowska, K. & Schröder-Bäck, P. What is context in knowledge translation? Results of a systematic scoping review. Health Res Policy Sys 22 , 52 (2024). https://doi.org/10.1186/s12961-024-01143-5

Download citation

Received : 26 June 2023

Accepted : 11 April 2024

Published : 29 April 2024

DOI : https://doi.org/10.1186/s12961-024-01143-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Knowledge Translation
• Evidence-informed policymaking
• Health systems

Health Research Policy and Systems

ISSN: 1478-4505

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Open access
• Published: 26 April 2024

Risk factors for low back pain in the Chinese population: a systematic review and meta-analysis

• Qiang Li 1 ,
• Leyun Peng 1 ,
• Yiding Wang 2 ,
• Yonghui Yang 1 , 3 &
• Zongbao Wang 1  

BMC Public Health volume  24 , Article number:  1181 ( 2024 ) Cite this article

305 Accesses

Metrics details

In China, the world’s largest developing country, low back pain (LBP) is a common public health issue affecting workability. This meta-analysis aimed to systematically assess the risk factors of LBP in the Chinese population.

Four English language and four Chinese databases were searched, and cross-sectional studies on the risk factors for LBP in Chinese populations were identified and collected. The search timeframe covered the period from the establishment of the database to November 2023. Two researchers independently reviewed the literature, extracted the data, and evaluated the risk of bias. Begg’s and Egger’s tests were used to evaluate publication bias.

Fifteen cross-sectional studies involving 86,575 people were included. Seven risk factors for LBP were identified. Six risk factors were statistically significant: Cigarette smoking (odds ratio [OR] = 1.55; 95% confidence interval [CI]: 1.15, 2.08, P  = 0.004, I 2  = 72%), body mass index (BMI) ≥ 28 kg/m² (OR = 4.51; 95% CI: 3.36, 6.07, P  < 0.00001, I 2  = 8%), female sex (OR = 1.54; 95% CI: 1.25, 1.90, P  < 0.0001, I 2  = 63%), vibration exposure at work (OR = 1.65; 95% CI: 1.16, 2.34, P  = 0.006, I 2  = 84%), working overtime (OR = 2.57; 95% CI: 1.12, 5.91, P  = 0.03, I 2  = 85%), and lack of exercise (OR = 2.48; 95% CI: 1.62, 3.78, P  < 0.0001, I 2  = 0%). One risk factor that was not statistically significant was standing for long periods (OR = 1.02; 95% CI: 0.82, 1.26, P  = 0.88, I 2  = 73%).

Conclusions

This study found that smoking, a BMI ≥ 28 kg/m², female sex, vibration exposure at work, working overtime, and lack of exercise may be risk factors for LBP in the Chinese population. Because the included studies were cross-sectional and the certainty of the evidence was very low, the results need to be interpreted cautiously. Multicentre, high-quality studies should be conducted in the future. To reduce the prevalence of LBP, the Chinese government and hospitals must develop early screening programs and implement effective preventive and interventional measures.

Trial registration

This study is registered in the PROSPERO database (No. CRD42023447857).

Peer Review reports

Low back pain (LBP) is a common symptom of many known and unknown pathologies or diseases [ 1 ]. Pain is usually located at the lower edge of the ribs, in the lumbosacral region, or accompanied by unilateral and bilateral lower limb nervous system symptoms [ 2 ]. LBP is a health problem that can occur in all age groups. According to literature reports, approximately 90% of people worldwide experience LBP at some stage of their lives [ 3 ]. LBP ranks sixth among all disease burdens in developed and developing countries [ 4 , 5 ]. Hartvigsen et al. reported [ 1 ] that the global economic burden of LBP is likely to increase further.

In China, LBP is also a major public health problem. According to Chen et al. [ 6 ], the annual prevalence of LBP in Chinese adults is 20.88–29.88%. This is closely related to China’s social and economic conditions. China is a large agricultural country with a population of more than 1.4 billion, and many people perform manual labour work. Furthermore, the level of agricultural mechanisation in China is significantly lower than that in developed countries [ 7 ], and the degree of industrial intelligence is also relatively low. As a rapidly emerging developing country, many adults in China work long hours to earn more income for their household expenses in response to escalating costs of living [ 8 ]. During their leisure time, the participation rate of Chinese people in sports activities was only 13.8%, which was significantly lower than that in Western countries [ 9 ]. The vast differences between Chinese and Western cultures and the lack of medical knowledge in the Chinese population may aggravate the occurrence of chronic pain in the Chinese population [ 10 ]. One study found that the prevalence of chronic pain in China was 31.54%, and most patients with chronic pain were in the northern and southern coastal areas [ 10 ]. Among the patients with chronic pain, 24.06% did not visit the hospital, and 36.788% did not receive any treatment [ 10 ]. Compared with developed Western countries, Chinese people have an insufficient understanding of chronic pain; the medical treatment rate of the affected population is low, and the number of patients undergoing standard treatment is also low [ 10 ]. Additionally, there are 1.14 billion smokers worldwide and an estimated 306 million adult smokers in China, the country with the largest number of smokers worldwide [ 11 , 12 ]. China’s economic development, popularisation of medical knowledge, and lifestyle habits show unique characteristics. These factors jointly affect LBP prevalence and risk in the Chinese population.

The risk factors for LBP reported in the literature can be divided into three aspects: (1) lifestyle factors such as obesity, smoking, and lack of exercise; (2) social factors such as working overtime, physical labour, education, and job satisfaction; and (3) psychological factors such as depression and anxiety [ 1 , 13 , 14 ]. If people have an unhealthy lifestyle, choose to eat junk food, smoke, and are unwilling to exercise, there is an increased risk of LBP. In highly competitive metropolitan cities, employees often work overtime and engage in heavy physical labour, resulting in low job satisfaction and the possibility of LBP. However, psychological factors are often overlooked. A cross-sectional study in South Korea found that the severity of depressive symptoms was associated with an increased risk of LBP [ 15 ]. A retrospective study conducted in the United States revealed that a healthy emotional state and a positive lifestyle can aid recovery from LBP [ 16 ]. In addition, regarding certain factors such as prolonged standing and education level, the research findings are inconsistent. Research conducted by Li et al. [ 17 ] and Jia et al. [ 18 ] indicate that prolonged standing may be a protective factor against LBP, whereas research conducted by Peng et al. [ 19 ] and Yue et al. [ 20 ] shows otherwise. Conflicting research is also seen in education. Xu et al. [ 21 ], Liu [ 22 ], and Jia et al. [ 18 ] reported an association between education and LBP, whereas Ye et al. [ 23 ] reported the opposite. The inconsistency in these research results may be related to the significant differences in the industries in which the research participants were located.

Although existing studies have provided valuable knowledge about LBP, few studies on the risk factors of LBP in the Chinese population exist. Investigations and research are limited by the sample size and study population. After reviewing Chinese and English language databases, we could not locate any evidence-based studies on the risk factors for LBP in the Chinese population. This finding suggests a systematic and quantitative approach is required to address this issue. As an evidence-based research method, meta-analysis can comprehensively analyse the results of multiple studies, improve the accuracy of result estimation, and identify and quantify potential risk factors. Therefore, the purpose of this study was to systematically review and integrate existing studies on the risk factors of LBP in the Chinese population through a meta-analysis. The aims were to enhance statistical precision, address inconsistencies among studies, and ensure robust conclusions, thereby providing an evidence-based foundation for formulating effective prevention and intervention strategies.

This evidence-based medical study is registered in The International Prospective Register of Systematic Reviews (PROSPERO) database (No. CRD42023447857). The study was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [ 24 , 25 ].

Search strategy

PubMed, Web of Science, Embase, Cochrane Library, CNKI, WanFang Date, VIP, and CBM databases were searched to collect cross-sectional studies on the risk factors for LBP in the Chinese population. From the time the databases were created until 1 November 2023, eight databases were searched using MeSH Terms, index terms, and keywords. Chinese search terms: ‘low back pain’ or ‘lower back pain’, ‘risk factors’ or ‘influencing factors’ or ‘related factors’. The English language search strategy is detailed in Additional File 1 .

Inclusion criteria

Research individuals: People with LBP (pain arising from the lower edge of the ribs in the lumbosacral and sacroiliac areas, with or without radiating pain in the lower limbs [ 2 ]) in China.

Exposure factors: Risk factors that may lead to the onset of LBP.

Outcome measure: Clinical diagnosis confirming the presence of LBP.

Study type: Cross-sectional studies.

Studies with original clear odds ratios (ORs) and 95% confidence intervals (95% CIs).

Language: Chinese and English.

Exclusion criteria

Republished studies.

Studies with different research participants and methods.

Studies on individuals with lumbar stenosis or other comorbidities.

Dissertations and conference papers.

Research lacking data or not including analytical factors.

Data extraction and quality assessment

Two researchers (Qiang Li and Yiding Wang) with extensive retrieval experience read the literature and extracted the information. The contents of the records included the first author, publication year, sample size, risk factors, and OR (95% CI). The third researcher (Leyun Peng) made the final decision whenever problems were encountered regarding data extraction; quality was evaluated. Two reviewers evaluated cross-sectional studies using the Agency for Healthcare Research and Quality methods [ 26 ]. With this method [ 27 ], 1–3 points mean low quality, 4–7 points mean medium, and 8–11 points mean high quality. In the Newcastle-Ottawa Scale, 0–4 points mean low quality, 5–6 points mean medium, and 7–9 points mean high quality.

The Grading of Recommendations, Assessment, Development, and Evaluation system [ 28 ] was employed to assess the evidence quality for the risk factors. This is based on factors such as study design, risk of bias, imprecision, inconsistency, indirectness, publication bias, and effect size [ 29 , 30 , 31 , 32 , 33 , 34 ]. The level of evidence was categorised as high, medium, low, or very low.

Statistical analyses

To guarantee the accuracy of the pooled effect estimates, we conducted a meta-analysis based solely on risk factors evaluated in a minimum of four studies. We used Revman 5.3 software for statistical analysis. OR (95% CI) was used as the effect analysis statistic. I² was used to test heterogeneity using P -values. Low, moderate, and high degrees of heterogeneity were indicated by the I 2 statistical values of 25, 50, and 75%, respectively [ 35 ]. When I 2 was > 50%, sensitivity analyses were performed to exclude each study individually to determine the stability of the findings. Funnel plot analysis was performed when at least 10 studies were included. Begg’s and Egger’s tests were used to evaluate the publication bias.

Study selection

We initially searched the database and obtained 1,984 records. In total, 836 duplicate records were deleted using EndNote X9. We then reviewed the titles and abstracts of the literature and excluded 1,051 records. After reading the full texts, 82 studies were excluded. Finally, 15 studies were included for appraisal and analysis. A flow chart of the article screening is shown in Fig.  1 . The studies excluded in the final step are presented in Additional File 2 .

Flow chart of the search process for the studies

Study characteristics and methodologic quality

The basic characteristics and quality assessment results of the included studies are summarised in Table  1 . For details of the quality assessment of the literature, please refer to Additional File 3 . Fifteen papers [ 18 , 19 , 20 , 22 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 ] were published between 2006 and 2022. The sample size ranged from 96 to 57,501, with a total of 86,575 individuals. The quality assessment of the 15 cross-sectional studies yielded scores ranging from 8 to 10.

Risk factors for LBP in the Chinese population

Seven risk factors were identified: cigarette smoking, body mass index (BMI) ≥ 28 kg/m², female sex, vibration exposure at work, working overtime, lack of exercise, and standing for long periods.

The certainty of the evidence for these seven risk factors was very low. Some risk factors, such as smoking, vibration exposure at work, and standing for a long time, might have had publication bias. The comprehensive results and the certainty of evidence assessment are presented in Table  2 . A detailed rating of the certainty of evidence assessment is provided in Additional File 4 . Begg’s and Egger’s tests are presented in Additional File 5 .

Cigarette smoking

Six studies assessing the correlation between smoking and LBP were included in this meta-analysis [ 18 , 36 , 37 , 38 , 45 , 46 ]. Comprehensive findings suggest that individuals in China who smoke are at a heightened risk of experiencing LBP compared with those who do not smoke (OR = 1.55; 95% CI: 1.15, 2.08, P  = 0.004, I 2  = 72%) (Fig.  2 ). Sensitivity analysis indicated significant heterogeneity regardless of which study was excluded.

Forest plot of cigarette smoking

BMI ≥ 28 kg/m²

Five studies evaluating the relationship between BMI ≥ 28 kg/m² and LBP were included [ 36 , 37 , 38 , 39 , 41 ]. The comprehensive findings suggest that individuals in the Chinese population with a BMI of 28 kg/m² or higher are at a heightened risk of experiencing LBP compared with those with a BMI below 28 kg/m² (OR = 4.51; 95% CI: 3.36, 6.07, P  < 0.00001, I 2  = 8%) (Fig.  3 ).

Forest plot of BMI ≥ 28 kg/m²

Five studies assessing the association between women and LBP were included [ 19 , 20 , 22 , 41 , 46 ]. The comprehensive findings suggest that within the Chinese population, women are more susceptible to LBP than men (OR = 1.54; 95% CI: 1.25, 1.90, P  < 0.0001, I 2  = 63%) (Fig.  4 ). Sensitivity analysis indicated that eliminating Liu et al.’s study could reduce I 2 to 0% (OR = 1.67; 95% CI: 1.44, 1.93, P  < 0.00001).

Forest plot of female sex

Vibration exposure at work

Five studies evaluating the relationship between work-related exposure to vibration sources and LBP were included in the meta-analysis [ 18 , 22 , 39 , 40 , 43 ]. The findings indicated that individuals in China who were exposed to vibration sources were at a higher risk of experiencing LBP than those not exposed to vibration sources (OR = 1.65; 95% CI: 1.16, 2.34, P  = 0.006, I 2  = 84%) (Fig.  5 ). Sensitivity analysis indicated that eliminating Jia et al.’s study could reduce I 2 to 8% (OR = 1.70; 95% CI: 1.42, 2.03, P  < 0.00001). This may be related to Jia’s definition of risk factors, which differs from those of other studies that only mention work involving vibrations. Jia’s study mentioned the use of vibration tools.

Forest plot of vibration exposure at work

Working overtime

Four studies assessing the association between extended working hours and LBP were included in the meta-analysis [ 36 , 38 , 42 , 44 ]. These findings suggest that individuals in China who work extended hours are more susceptible to LBP than those who work standard hours (OR = 2.57; 95% CI: 1.12, 5.91, P  < 0.03, I 2  = 85%) (Fig.  6 ). Sensitivity analysis indicated that eliminating the study by Xu et al. could reduce the I 2 to 0% (OR = 3.55; 95% CI: 2.24, 5.63, P  < 0.00001).

Forest plot of working overtime

Lack of exercise

Four studies assessing the association between exercise and LBP were included in the meta-analysis [ 37 , 38 , 39 , 40 ]. The comprehensive findings suggest that individuals in China who insufficiently engage in physical activity are at a greater risk of developing LBP than those who exercise regularly (OR = 2.48; 95% CI: 1.62, 3.78, P  < 0.0001, I 2  = 0%) (Fig.  7 ).

Forest plot of lack of exercise

Standing for long periods

Four studies assessing the association between prolonged standing and LBP were included [ 18 , 19 , 20 , 44 ]. The comprehensive findings indicate that prolonged standing is not a risk factor for LBP in the Chinese population (OR = 1.02; 95% CI: 0.82, 1.26, P  = 0.88, I 2  = 73%) (Fig.  8 ). Sensitivity analysis indicated that eliminating Jia’s study could reduce the I 2 to 0% (OR = 1.10; 95% CI: 0.94, 1.30, P  = 0.24).

Forest plot of standing for long periods

LBP is a common clinical problem in the Chinese population. Limited studies on LBP risk factors exist in China, and their results vary widely. Most studies have only explored the risk factors for LBP in specific occupational and regional populations. For instance, the study participants of Yue et al. [ 20 ] and Yao et al. [ 53 ] focused on Chinese teachers and adolescents, while Barrero et al. [ 43 ] only focused on individuals from rural areas. The risk factors they present have inherent limitations and do not apply to the entire Chinese population. Therefore, we conducted a systematic review to identify the risk factors that may cause LBP in the Chinese population. Seven risk factors were identified: smoking, BMI ≥ 28 kg/m², female sex, vibration exposure at work, working overtime, lack of exercise, and prolonged standing. Among them, prolonged standing was not related to the occurrence of LBP, which may be owing to different occupations and research studies conducted. Wong et al. found that female sex, obesity, and smoking were risk factors for LBP [ 54 ]. A LBP study among African school teachers found that lack of exercise and female sex are risk factors for LBP [ 55 ]. Jia et al.’s study found that vibration exposure at work and working overtime were risk factors for LBP [ 18 ]. These results are consistent with those of the present study.

Smoking and LBP are global public health concerns. Numerous studies [ 56 , 57 , 58 ] indicate that smoking increases the risk of LBP. Using a passive smoking model in rats, Nemoto et al. [ 56 ] found cracks, tears, and dislocations in the intervertebral discs. A recent study [ 58 ] has also shown that smoking and tetramer tryptase accelerate intervertebral disc degeneration by inducing methyltransferase 14-mediated dishevelled-axin domain-containing 1 m6 modifications. Smokers experience significantly greater levels of lumbar pain and dysfunction than non-smokers [ 57 ]. The greater the daily tobacco intake, the more pronounced the LBP. Therefore, to prevent the onset of LBP, Chinese individuals are advised to quit smoking or reduce the frequency of smoking.

BMI ≥ 28 kg/m 2 is a risk factor for LBP in the Chinese population. Adipose tissue due to obesity can release several adipokines (including leptin, resistin, and adiponectin) as well as proinflammatory cytokines such as interleukin-1β, interleukin-6, and tumour necrosis factor-α [ 59 ]. Leptin can accelerate the calcification of hyaline cartilage in the cartilaginous endplate, interfering with the transport of nutrients to disc cells [ 60 ]. Resistin expression in normal intervertebral discs is low but increases during intervertebral disc degeneration. In intervertebral disc tissues, resistin can activate nuclear factor-κB and p38 mitogen-activated protein kinase, followed by resistin binding to toll-like receptor 4 and increasing chemokine ligand 4 expression. This promotes the infiltration of macrophages [ 61 ]. However, the correlation between adiponectin level and LBP remains unclear. All these molecules may cause low-grade inflammation and have been shown to exert detrimental effects on nucleus pulposus and annulus fibrosus cells in vitro [ 62 ]. Fortunately, obesity can be modified through exercise and dietary management. Individuals with obesity should consider appropriate weight management strategies. Moreover, there are variations in the risk of LBP across sex groups. Women are at a higher risk of developing LBP than men, and patients with LBP exhibit poor dynamic postural control [ 63 ]. This may be attributed to the additional housework that women perform and their relatively low muscle strength. Women, as a group susceptible to LBP, should prioritise enhancing self-management awareness and mitigating LBP stemming from intrinsic factors.

A previous study showed that whole-body vibration can lead to compression, stretching, rotation, and spine flexion, all of which involve back muscles and cause fatigue [ 64 ]. Studies of the erector spinae found that electromyography signals increased during whole-body vibration, resulting in a decrease in the signal frequency of muscle fatigue, mainly at the resonance frequency of 5 Hz [ 65 , 66 , 67 ]. Kim et al. and Burström et al. concluded that irregular whole-body vibrations are closely associated with musculoskeletal disorders, particularly LBP onset [ 68 , 69 ]. Moreover, our previous research [ 70 ] indicated that vibration therapy remains a viable treatment for LBP, which may be closely related to vibration frequency and amplitude.

The reality many people face is the long working hours they must endure. Individuals are susceptible to musculoskeletal disorders in prolonged and high-intensity work environments, leading to an increased risk of LBP. People who frequently work overtime should be vigilant in preventing and managing this condition. In addition, extended working hours can lead to sedentary behaviour, a fixed posture at work, and prolonged neck forward tilt among professionals working in offices and drivers. A positive correlation exists between sedentary time and two common acute phase reactants, C-reactive protein and fibrinogen [ 71 , 72 ]. Sedentary behaviour also has a detrimental impact on systemic inflammation [ 71 , 72 ]. Compared with men, the level of inflammation in women is more influenced by sedentary behaviour [ 71 , 72 ]. Sedentary behaviour may also reduce bone density, decrease flexibility, and promote weight gain [ 59 ]. Furthermore, all participants in this study were adults, and no consideration was given to children. An Iranian meta-analysis [ 73 ] suggests that various forms of sedentary behaviour (such as prolonged TV viewing or electronic device use) may also contribute to LBP in children.

This study suggests that a lack of exercise may be a risk factor for LBP. As work and life pressures grow in China, ensuring regular physical activity for industry workers, such as factory employees, office staff, and healthcare professionals, has become challenging. A meta-analysis in China also suggested that Taijiquan, Pilates, sling and core stability exercises can improve LBP symptoms [ 74 ]. Taijiquan, a traditional Chinese sport, can enhance joint flexibility and mobility, boost muscle strength and endurance, increase the tensile strength of ligaments and bursae, improve cardiopulmonary function, and alleviate stress and anxiety [ 75 ]. In addition, Taijiquan can decrease the expression of serum B-type linalool peptides, enhance blood circulation, increase the metabolism of the lumbar spine, improve the absorption of calcium and other minerals by bone cells, and increase bone density in the lumbar region [ 76 ]. We recommend engaging in physical activity as a treatment or preventive measure for LBP.

Strengths and limitations

To our knowledge, this is the first meta-analysis to explore the risk factors for LBP in the Chinese population. We quantitatively assessed multiple studies on the risk factors for LBP in the Chinese population. With enhanced statistical precision and the resolution of inconsistencies found in previous studies, we arrived at more robust conclusions.

However, this study had inherent limitations. First, the included participants had various occupations and were from different regions of China, leading to heterogeneity in the results. Second, the included studies were cross-sectional, making it impossible to establish a causal relationship between the exposure factors and outcomes. Recall bias was also highly probable. Third, the limited literature included in this study on certain risk factors might have introduced bias into the research outcomes. Lastly, only Chinese and English language literature were included, with no gray literature retrieved, leading to a limited number of articles in the final analysis.

This study employed evidence-based medical research methods to analyse and explore the risk factors for LBP in the Chinese population. We found that smoking, BMI ≥ 28 kg/m², female sex, vibration exposure at work, working overtime, and lack of exercise may be risk factors for LBP in this population. However, because of the limited number of included studies and their cross-sectional design, causality could not be determined. The certainty of the evidence is also very low. Therefore, these findings should be interpreted with caution. Multicentre, high-quality studies should be conducted in the future. To reduce the prevalence of LBP, the Chinese government and hospitals must develop early screening programs and implement effective preventive and interventional measures.

Data availability

The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

• Low back pain

Body mass index

95% Confidence interval

Hartvigsen J, Hancock MJ, Kongsted A, Louw Q, Ferreira ML, Genevay S, et al. What low back pain is and why we need to pay attention. Lancet. 2018;391(10137):2356–67.

Article   PubMed   Google Scholar  

Dionne CE, Dunn KM, Croft PR, Nachemson AL, Buchbinder R, Walker BF, et al. A consensus approach toward the standardization of back pain definitions for use in prevalence studies. Spine. 2008;33:95–103.

Freburger JK, Holmes GM, Agans RP, Jackman AM, Darter JD, Wallace AS, et al. The rising prevalence of chronic low back pain. Arch Intern Med. 2009;169(3):251–8.

Article   PubMed   PubMed Central   Google Scholar  

Hoy D, March L, Brooks P, Blyth F, Woolf A, Bain C, et al. The global burden of low back pain: estimates from the global burden of Disease 2010 study. Ann Rheum Dis. 2014;73(6):968–74.

Global Burden of Disease Study 2013 Collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 301 acute and chronic diseases and injuries in 188 countries, 1990–2013: a systematic analysis for the global burden of Disease Study 2013. Lancet. 2015;386(9995):743–800.

Article   PubMed Central   Google Scholar  

Chen D, Chen CH, Hu ZC, Shao ZX, Lin JL, Wu AM. Prevalence of low back pain in adult population in China: a systematic review. Chin J Evid Based Med. 2019;19(6):651–5.

CAS   Google Scholar  

Shang SQ, Wang JS, Wang DW. Development of agricultural mechanization in China and its current Strategic Focus. Agricultural Mach Technol. 2019. https://doi.org/10.22314/2073-7599-2019-13-3-4-7 . 13.4-7.

Article   Google Scholar  

Liang Y, Li Z, Wang X, Liu P, Ma L, Wang X. Association between overtime and depressive symptoms among Chinese employees. Front Public Health. 2023;11:1241994.

Zou Q, Wang H, Du W, Su C, Ouyang Y, Wang Z, et al. Trends in leisure-time physical activity among Chinese adults - China, 2000–2015. China CDC Wkly. 2020;2(9):135–9.

Zheng YJ, Zhang TJ, Yang XQ, Feng ZY, Qiu F, Xin GK, et al. A survey of chronic pain in China. Libyan J Med. 2020;15(1):1730550.

GBD 2019 Tobacco Collaborators. Spatial, temporal, and demographic patterns in prevalence of smoking tobacco use and attributable disease burden in 204 countries and territories, 1990–2019: a systematic analysis from the global burden of Disease Study 2019. Lancet. 2021;397(10292):2337–60.

Chinese Center for Disease Control and Prevention. The results of Chinese adult Tobacco Survey in 2018 [R]. Beijing: Chinese Center for Disease Control and Prevention; 2020.

Google Scholar  

Fatoye F, Gebrye T, Odeyemi I. Real-world incidence and prevalence of low back pain using routinely collected data. Rheumatol Int. 2019;39(4):619–26.

Liu Y, Tang G, Li J. Causations between obesity, diabetes, lifestyle factors and the risk of low back pain. Eur Spine J. 2023. https://doi.org/10.1007/s00586-023-08069-6 .

Park SM, Kim HJ, Jang S, Kim H, Chang BS, Lee CK, et al. Depression is closely Associated with Chronic Low Back Pain in patients over 50 years of age: a cross-sectional study using the Sixth Korea National Health and Nutrition Examination Survey (KNHANES VI-2). Spine (Phila Pa 1976). 2018;43(18):1281–8.

Roberts KE, Beckenkamp PR, Ferreira ML, Duncan GE, Calais-Ferreira L, Gatt JM, et al. Positive lifestyle behaviours and emotional health factors are associated with low back pain resilience. Eur Spine J. 2022;31(12):3616–26.

Article   CAS   PubMed   Google Scholar  

Li JY, Wang S, He LH, Wu SS, Yang L, Yu SF, et al. Risk factors of low back pain among the Chinese occupational population: a case-control study. Biomed Environ Sci. 2012;25(4):421–9.

PubMed   Google Scholar  

Jia N, Zhang MB, Zhang HD, Ling RJ, Liu YM, Li G et al. Prevalence and risk factors analysis for low back pain among occupational groups in key industries of China. BMC Public Health, 2022(22):1493.

Peng BL, Wu JB, Qi C, Zhao CH, Lin RJ, Yang L. Prevalence and influence factors of low back pain among workers in an automobile industry corporation. Chin J Public Health. 2017;33(4):663–7.

Yue PY, Liu FY, Li LP. Neck/shoulder pain and low back pain among school teachers in China, prevalence and risk factors. BMC Public Health,2012; (12):789.

Xu XY, Qiu SJ, An SL, Jin AM, Min SX. Analysis of risk factors of nonspecific low back pain in a community population: a case-control study. J South Med Univ. 2014;34(12):1794–8.

Liu F. Analysis of the related factors of chronic nonspecific low back pain in community population. J Cervic Lumb. 2020;41(05):568–71.

Ye SY, Jing QL, Chen W, Lu J. Risk factors of non-specific neck pain and low back pain in computer- using office workers in China: a cross- sectional study. BMJ Open, 2017(7):e014914.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hofmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.

Page MJ, Moher D, Bossuyt PM, Boutron I, Hofmann TC, Mulrow CD, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ. 2021;372:n160.

Agency for Healthcare Research and Quality (US). AHRQ methods for effective health care. Methods guide for effectiveness and comparative effectiveness reviews. Rockville: 2008.

Rostom A, Dubé C, Cranney A, Celiac, Disease. Rockville (MD): Agency for Healthcare Research and Quality (US); 2004 Sep. (Evidence Reports/Technology Assessments, No. 104.) Appendix D. Quality Assessment Forms. http://www.ncbi.nlm.nih.gov/books/NBK35156/ .

Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1.Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64:383–94.

Balshem H, Helfand M, Schunemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64:401–6.

Guyatt GH, Oxman AD, Vist G, Kunz R, Brozek J, Alonso-Coello P, et al. GRADE guidelines: 4. Rating the quality of evidencedstudy limitations (risk of bias). J Clin Epidemiol. 2011;64:407–15.

Guyatt GH, Oxman AD, Kunz R, Brozek J, Alonso-Coello P, Rind D, et al. GRADE guidelines 6. Rating the quality of evidencedimprecision. J Clin Epidemiol. 2011;64:1283–93.

Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: 7. Rating the quality of evidencedinconsistency. J Clin Epidemiol. 2011;64:1294–302.

Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: 8. Rating the quality of evidencedindirectness. J Clin Epidemiol. 2011;64:1303–10.

Guyatt GH, Oxman AD, Montori V, Vist G, Kunz R, Brozek J, et al. GRADE guidelines: 5. Rating the quality of evidence-publication bias. J Clin Epidemiol. 2011;64:1277–82.

Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557–60.

Zhang Y, Liu J, Chen G, Liu CJ. Risk factors of nonspecific low back pain in cadre health care population. J North Sichuan Med Coll. 2021;36(11):1424–7.

Wu G. Analysis of disease situation and risk factors of non-specific low back pain in taxi drivers in a city. J Cervic Lumb. 2019;40(6):770–2.

Yang QC. Investigation and analysis of risk factors of nonspecific low back pain in teachers from a high school. J Cervic Lumb. 2020;41(4):452–4.

Wang X, Liu YQ, Tian XC, Li XH, Wu T, Xu FZ, et al. Analysis of the influence factors of non-specific low back pain in urban population. J Cervic Lumb. 2019;40(6):740–2.

Wang JQ, Chen T, Zhang X. Analysis of risk factors of non- specific low back Pain in Young People. J Med Res. 2022;51(3):80–2.

Zhang Q, Dong HY, Zhu CJ, Liu JZ. Low back pain in emergency ambulance workers in tertiary hospitals in China and its risk factors among ambulance nurses: a cross-sectional study. BMJ Open. 2019;9(9):e029264.

Wang M, Yu J, Liu N, Liu Z, Wei X, Yan F, et al. Low back pain among taxi drivers: a cross-sectional study. Occup Med. 2017;67:290–5.

Article   CAS   Google Scholar  

Barrero LH, Hsu YH, Terwedow H, Perry MJ, Dennerlein JT, Brain JD, et al. Prevalence and physical determinants of low back pain in a rural Chinese population. Spine. 2006;31(23):2728–34.

Xu G, Pang D, Liu F, Pei D, Wang S, Li L. Prevalence of low back pain and associated occupational factors among Chinese coal miners. BMC Public Health. 2012;12:149.

Wei G, Li H, Wang B, Wu J, Wu F, Lin Z. A retrospective cross-sectional survey of non-specific lower back pain among a cohort of Chinese army soldiers. Int J Surg. 2018;56:288–93.

Liu X, Wang L, Stallones L, Wheeler KK, Zhao W, Smith GA, et al. Back pain among farmers in a northern area of China. Spine (Phila Pa 1976). 2012;37(6):508–14.

Expert Group of the Spinal Cord Professional Committee of the Chinese Rehabilitation Medicine Association. Expert consensus on acute/chronic nonspecific low back pain in China. Chin J Spine Spinal Cord. 2016;26(12):1134–8.

Salvendy. Handbook of Human Factors and Ergonomics. 4th ed; 2012.

Kuorinka I, Jonsson B, Kilbom A, Vinterberg H, Biering-Sørensen F, Andersson G, et al. Standardised nordic questionnaires for the analysis of musculoskeletal symptoms. Appl Ergon. 1987;18:233–7.

Hildebrandt VH, Bongers PM, Van Dijk FJ, Kemper HC, Dul J. Dutch musculoskeletal questionnaire: description and basic qualities. Ergonomics. 2001;44:1038–55.

Li J, Yang W, Liu P, Xu Z, Cho SI. Psychometric evaluation of the Chinese(mainland) version of job content questionnaire: a study in university hospitals. Ind Health. 2004;42:260–7.

de Barros EN, Alexandre NM. Cross-cultural adaptation of the nordic Musculoskeletal Questionnaire. Int Nurs Rev. 2003;50:101–8.

Yao W, Luo C, Ai F, Chen Q. Risk factors for nonspecific low-back pain in Chinese adolescents: a case-control study. Pain Med. 2012;13(5):658–64.

Wong CK, Mak RY, Kwok TS, Tsang JS, Leung MY, Funabashi M, et al. Prevalence, incidence, and factors Associated with non-specific chronic low back Pain in Community-Dwelling older adults aged 60 years and older: a systematic review and Meta-analysis. J Pain. 2022;23(4):509–34.

Tesfaye AH, Abere G, Mekonnen TH, Jara AG, Aragaw FM. A systematic review and meta-analysis of low back pain and its associated factors among school teachers in Africa. BMC Musculoskelet Disord. 2023;24(1):499.

Nemoto Y, Matsuzaki H, Tokuhasi Y, Okawa A, Uematu Y, Nishimura T, et al. Histological changes in intervertebral discs after smoking cessation: experimental study using a rat passive smoking model. J Orthop Sci. 2006;11(2):191–7.

Yang QH, Zhang YH, Du SH, Wang YC, Wang XQ. Association between smoking and pain, functional disability, anxiety and depression in patients with chronic low back pain. Int J Public Health. 2023;68:1605583.

Tu J, Li W, Hansbro PM, Yan Q, Bai X, Donovan C, et al. Smoking and tetramer tryptase accelerate intervertebral disc degeneration by inducing METTL14-mediated DIXDC1 m6 modification. Mol Ther. 2023;31(8):2524–42.

Ambrosio L, Mazzuca G, Maguolo A, Russo F, Cannata F, Vadalà G, et al. The burden of low back pain in children and adolescents with overweight and obesity: from pathophysiology to prevention and treatment strategies. Ther Adv Musculoskelet Dis. 2023;15:1759720X231188831.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Han YC, Ma B, Guo S, Yang M, Li LJ, Wang SJ, et al. Leptin regulates disc cartilage endplate degeneration and ossification through activation of the MAPK-ERK signalling pathway in vivo and in vitro. J Cell Mol Med. 2018;22:2098–109.

Li Z, Wang X, Pan H, Yang H, Li X, Zhang K, et al. Resistin promotes CCL4 expression through toll-like receptor-4 and activation of the p38-MAPK and NF-κB signaling pathways: implications for intervertebral disc degeneration. Osteoarthr Cartil. 2017;25:341–50.

Molinos M, Almeida CR, Caldeira J, Cunha C, Gonçalves RM, Barbosa MA. Inflammation in intervertebral disc degeneration and regeneration. J R Soc Interface. 2015;12:20141191.

Wu XH, Yu H, Yang XJ, Wu YM, Yang P. Gender differences in postural control in patients with nonspecific chronic low back pain. J Cervic Lumb. 2020;41(2):135–9.

Patterson F, Miralami R, Tansey KE, Prabhu RK, Priddy LB. Deleterious effects of whole-body vibration on the spine: a review of in vivo, ex vivo, and in vitro models. Anim Model Exp Med. 2021;4(2):77–86.

Pope MH, Wilder DG, Magnusson M. Possible mechanisms of low back pain due to whole-body vibration. J Sound Vib. 1998;215(4):687–97.

Blüthner R, Hinz B, Menzel G, Seidel H. Back muscle response to transient whole-body vibration. Int J Ind Ergon. 1993;12(1–2):49–59.

Hansson T, Magnusson M, Broman H. Back muscle fatigue and seated whole body vibrations: an experimental study in man. Clin Biomech Elsevier Ltd. 1991;6:173–8.

Burström L, Nilsson T, Wahlström J. Whole-body vibration and the risk of low back pain and sciatica: a systematic review and meta-analysis. Int arch Occup Environ Health. 2015;88(4):403–18.

Kim JH, Zigman M, Aulck LS, Ibbotson JA, Dennerlein JT, Johnson PW. Whole body vibration exposures and health status among professional truck drivers: a cross-sectional analysis. Ann Occup Hyg. 2016;60(8):936–48.

Li Q, Liu P, Wang ZB, Li X. Vibration therapy to improve pain and function in patients with chronic low back pain: a systematic review and meta-analysis. J Orthop Surg Res. 2023;18(1):1–12.

Howard BJ, Balkau B, Thorp AA, Magliano DJ, Shaw JE, Owen N, et al. Associations of overall sitting time and TV viewing time with fibrinogen and C reactive protein: the AusDiab study. Br J Sports Med. 2015;49(4):255–8.

Yates T, Khunti K, Wilmot EG, Brady E, Webb D, Srinivasan B, et al. Self-reported sitting time and markers of inflammation, insulin resistance, and adiposity. Am J Prev Med. 2012;42(1):1–7.

Baradaran Mahdavi S, Riahi R, Vahdatpour B, Kelishadi R. Association between sedentary behavior and low back pain; a systematic review and meta-analysis. Health Promot Perspect. 2021;11(4):393–410.

Li Y, Yan L, Hou L, Zhang X, Zhao H, Yan C, et al. Exercise intervention for patients with chronic low back pain: a systematic review and network meta-analysis. Front Public Health. 2023;11:1155225.

Kong LJ, Lauche R, Klose P, Bu JH, Yang XC, Guo CQ, et al. Tai Chi for Chronic Pain conditions: a systematic review and Meta-analysis of Randomized controlled trials. Sci Rep. 2016;6:25325.

Yeh GY, Wood MJ, Lorell BH, Stevenson LW, Eisenberg DM, Wayne PM, et al. Effects of Tai Chi mind-body movement therapy on functional status and exercise capacity in patients with chronic heart failure: a randomized controlled trial. Am J Med. 2004;117(8):541–8.

Download references

Acknowledgements

Not applicable.

1. Yang Yonghui Famous Traditional Chinese Medicine Studio of Anhui Province; 2. Anhui provincial-level regional Chinese medicine rehabilitation center construction project (Anhui Chinese medicine service secretary [2022] 34); 3. The fourth batch of national regional medical center construction project (Development and Reform Office of Society [2022] 887).

Author information

Authors and affiliations.

Shuguang-Anhui Hospital Affiliated to Shanghai University of Traditional Chinese Medicine, Anhui University of Chinese Medicine, No. 45 Shihe Road, Shushan District, 230000, Hefei, Anhui, People’s Republic of China

Qiang Li, Leyun Peng, Yonghui Yang & Zongbao Wang

School of Medicine, Shandong Xiandai University, No. 20288 Jingshi East Road, Licheng District, 250104, Jinan, Shandong, People’s Republic of China

Yiding Wang

The Second Affiliated Hospital of Anhui University of Chinese Medicine, No. 300 Shouchun Road, Luyang District, 230000, Hefei, Anhui, People’s Republic of China

Yonghui Yang

You can also search for this author in PubMed   Google Scholar

Contributions

QL and LYP conceived and designed the study. QL and YDW searched and selected relevant studies. QL and YDW extracted and interpreted data. QL and YDW analyzed the data. QL and LYP wrote the paper. QL, YHY, and ZBW revised the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Zongbao Wang .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, supplementary material 4, supplementary material 5, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Li, Q., Peng, L., Wang, Y. et al. Risk factors for low back pain in the Chinese population: a systematic review and meta-analysis. BMC Public Health 24 , 1181 (2024). https://doi.org/10.1186/s12889-024-18510-0

Download citation

Received : 29 November 2023

Accepted : 03 April 2024

Published : 26 April 2024

DOI : https://doi.org/10.1186/s12889-024-18510-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Chinese population
• Risk factors
• Meta-analysis

BMC Public Health

ISSN: 1471-2458

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 06 May 2024

Chronic atrophic gastritis and risk of incident upper gastrointestinal cancers: a systematic review and meta-analysis

• Junqiu Li   ORCID: orcid.org/0000-0003-3563-4831 1   na1 ,
• Jielu Pan 1   na1 ,
• Dinghong Xiao 1 ,
• Nan Shen 1 ,
• Ruiqing Wang 1 ,
• Hongyv Miao 1 ,
• Peimin Pu 1 ,
• Haiyan Zhang 1 ,
• Xiao Yv 1 &
• Lianjun Xing 1  

Journal of Translational Medicine volume  22 , Article number:  429 ( 2024 ) Cite this article

Metrics details

Previous literature has explored the relationship between chronic atrophic gastritis (CAG) and isolated cancers within the upper gastrointestinal cancers; However, an integrative synthesis across the totality of upper gastrointestinal cancers was conspicuously absent. The research objective was to assess the relationship between CAG and the risk of incident upper gastrointestinal cancers, specifically including gastric cancer, oesophageal cancer, and oesophagogastric junction cancer.

Rigorous systematic searches were conducted across three major databases, namely PubMed, Embase and Web of Science, encompassing the timeline from database inception until August 10, 2023. We extracted the necessary odds ratio (OR) and their corresponding 95% confidence interval (CI) for subsequent meta-analysis. Statistical analyses were conducted using Stata 17.0 software.

This meta-analysis included a total of 23 articles encompassing 5858 patients diagnosed with upper gastrointestinal cancers. CAG resulted in a statistically significant 4.12-fold elevated risk of incident gastric cancer (OR = 4.12, 95% CI 3.20–5.30). Likewise, CAG was linked to a 2.08-fold increased risk of incident oesophageal cancer (OR = 2.08, 95%CI 1.60–2.72). Intriguingly, a specific correlation was found between CAG and the risk of incident oesophageal squamous cell carcinoma (OR = 2.29, 95%CI 1.77–2.95), while no significant association was detected for oesophageal adenocarcinoma (OR = 0.62, 95%CI 0.17–2.26). Moreover, CAG was correlated with a 2.77-fold heightened risk of oesophagogastric junction cancer (OR = 2.77, 95%CI 2.21–3.46). Notably, for the same type of upper gastrointestinal cancer, it was observed that diagnosing CAG through histological methods was linked to a 33–77% higher risk of developing cancer compared to diagnosing CAG through serological methods.

This meta-analysis indicated a two- to fourfold increased risk of gastric cancer, oesophageal cancer, and oesophagogastric junction cancer in patients with CAG. Importantly, for the same upper gastrointestinal cancer, the risk of incident cancer was higher when CAG was diagnosed histologically compared to serological diagnosis. Further rigorous study designs are required to explore the impact of CAG diagnosed through both diagnostic methods on the risk of upper gastrointestinal cancers.

Introduction

Within the global healthcare arena, cancer plays a dual role, being both a disease of significant global interest and a principal factor in clinical mortality. It is characterized by a protracted disease course, a predisposition for deterioration, low survival rates, and a significant economic burden. With the ageing of the population and an increase in cancer risk factors, the incidence and mortality of cancer have also risen.

Upper gastrointestinal cancers, comprising gastric cancer (GC), oesophagogastric junction cancer (OJC), and oesophageal cancer (OC); Oesophageal cancer is mainly classified into two subtypes: oesophageal adenocarcinoma (OAC) and oesophageal squamous cell carcinoma (OSCC). In 2019, there were approximately 23.6 million new cancer cases reported worldwide, with upper gastrointestinal cancers representing about 7.6% of these cases; Meanwhile, the worldwide cancer-related mortality rate reached an estimated 10.0 million, and upper gastrointestinal cancers were responsible for roughly 14.6% of these deaths [ 1 ]. The etiopathogenesis and progression of upper gastrointestinal cancers are closely linked to numerous factors, including diet, lifestyle, and others [ 2 , 3 ]. Notably, Chronic atrophic gastritis (CAG) has captured the attention of researchers as a potential risk factor. This association is consistent with the involvement of chronic inflammation in cancer development [ 4 , 5 ].

CAG is a chronic inflammatory disease characterized by the reduction or loss of gastric mucosal glands, with or without metaplasia of the intestinal epithelium or pyloric glands. A primary etiological factor in the development of this disease is the infection of H. pylori [ 6 , 7 , 8 ]. Upon infection, the gastric mucosa undergoes an intense inflammatory response, resulting in tissue damage and an increased risk of cancer [ 9 ]. Subsequently, some researchers initiated studies of the associations between CAG and upper gastrointestinal cancers. Over the last 15 years, the majority of studies have primarily centred around meta-analyses examining the relationship between CAG and GC [ 10 , 11 ]. However, there has been relatively limited research concerning the relationship between CAG and OC or OJC. Notably, it was not until 2010 that a meta-analysis was published, reporting on the risk of gastric atrophy in the development of OAC and OSCC [ 12 ]. At present, no exhaustive meta-analysis offers a comprehensive assessment of the risk of upper gastrointestinal cancers in relation to CAG. Meanwhile, with advances in medical science and technology, the primary diagnostic modality for CAG has shifted towards histological methods, whereas previous studies mainly used serologic diagnostic modalities. However, Whether the risk relationship between CAG diagnosed using these two diagnostic methods and upper gastrointestinal cancers is consistent remains unclear and has not been clearly reported.

Consequently, we conducted this systematic review and meta-analysis to comprehensively and accurately assess the magnitude and nature of the relationship between CAG and the incidence risk of upper gastrointestinal cancers. Furthermore, we aimed to report the extent of risk associated with the diagnosis of CAG through histological and serological methods for the development of upper gastrointestinal cancers.

Materials and methods

This study was conducted in accordance with the recommendations of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [ 13 ] and was registered with the PROSPERO (CRD42023455940).

Search strategy

We systematically searched databases (PubMed, Embase, Web of Science) using a combination of search terms and free phrases to assess the risk association between CAG and upper gastrointestinal cancers. The search included articles published from the creation of the database through August 10, 2023. The search strategies used for each database are displayed in Additional file 1 : File 1 Search strategy.

Inclusion and exclusion criteria

The inclusion criteria were as follows: (1) Case–control studies, nested case–control studies, or cohort studies; (2) To investigate the risk relationship between CAG and upper gastrointestinal cancers; (3) The diagnosis of CAG is based on endoscopic histology or serological methods. (4) The study involved human participants, with no restrictions on race or gender, and all individuals were aged 18 years or older. (5) The main outcome was the incidence risk of upper gastrointestinal cancers, which was measured using odds ratio (OR).

The exclusion criteria were as follows: (1) Case reports, reviews, commentaries, animal and cell studies, as well as cross-sectional research; (2) Duplicate publications; (3) Literature with missing research data and inability to extract the required data; (4) Non-English literature; (5) Newcastle–Ottawa Scale score (NOS) < 7.

Data extraction and quality assessment

According to the inclusion and exclusion criteria, two researchers (JQL and XY) independently screened titles and abstracts that met the requirements. Subsequently, they obtained and read the full texts, selecting articles that met the specified criteria. According to the data extraction guidelines for systematic reviews and meta-analyses [ 14 ], two researchers independently extracted the following information: study design, study’s author and year of publication, country, sample size, outcomes, study period, sex, diagnosis of CAG, assessment of cancer, adjustment for covariates, participant source and NOS score.

If a study did not clearly give a standard definition of gastric atrophy, we defined it as atrophic gastritis based on histological evidence of gastric mucosal atrophy and intestinal metaplasia. This was based on an expert review of atrophic gastritis updated by the American Gastrointestinal Association [ 15 ]. Hence, when independently assessing the risks associated with gastric mucosal atrophy and intestinal metaplasia in the literature, we regarded them as separate studies. Similarly, when conducting separate risk assessments for histology and serology, we also treated them as separate studies. If the literature independently assessed the risks of non-cardia cancer and cardia cancer, we extracted relevant data on non-cardia cancer for the study of gastric cancer incidence risk. In accordance with the classification of oesophagogastric junction adenocarcinoma [ 16 ], we included cardia cancer-related data in the study of oesophagogastric junction cancer incidence risk.

We undertook a qualitative evaluation of the included literature utilizing the Newcastle–Ottawa Scale (NOS). This assessment was carried out independently by two researchers (JQL and JLP). The NOS scale comprises three aspects of evaluation, with scores ranging from 0 to 9. In this study, the quality assessment scores for all screened literature were 7 or higher. Therefore, the literature screened in this study was considered to be of high quality [ 17 ].

Any disagreements encountered during the processes of data extraction and quality assessment were addressed through discussions with the senior author (LJX).

Statistical methods

The meta-analysis was conducted by comparing the risk of upper gastrointestinal cancers between patients with and without CAG. We extracted OR, hazard ratio (HR), and relative risk (RR) from the eligible literature. Given the relatively low risk of upper gastrointestinal cancers, during the data analysis, the extracted HR and RR were approximated to be equal to the OR [ 18 ]. We used the OR and its corresponding 95% confidence interval (CI) for statistical analysis.

The statistical analyses were performed using Stata 17.0 software. To assess heterogeneity, we used the Q-test and the I 2 value. When I 2  > 50% or P < 0.1, we considered significant heterogeneity among the studies, allowing for the adoption of a random-effects model. Otherwise, a fixed-effects model was used; Additionally, in order to explore the sources of heterogeneity, we conducted subgroup analyses based on the diagnosis of CAG, study type, participant source, region, year of publication, and NOS score. Sensitivity analysis was conducted to evaluate the robustness and reliability of the results. Funnel plots and Egger's test were used to analyze publication bias.

Search results

Initially, we retrieved a total of 16,039 articles, which included 3422 from PubMed, 5489 from Embase, and 7128 from Web of Science. Among these, 6691 duplicates were identified and subsequently excluded, followed by the exclusion of 9,283 irrelevant articles after a screening of titles and abstracts. We conducted a comprehensive search of the full text of 65 articles, excluding one article that was unavailable. Following a detailed examination of the full texts, we excluded 41 studies for various reasons, including 21 studies lacking relevant outcomes, 11 studies with unrelated outcomes, 2 studies were letters, 1 study was review, 1 study in a non-English language, 1 study was conference abstract, and 4 studies with a NOS < 7.

Finally, a total of 23 articles involving 5858 patients diagnosed with upper gastrointestinal cancers were incorporated into this study. The flowchart of the study screening is shown in Fig.  1 (Page 30).

Flow chart of literature screening

Characteristics of included studies

We have identified 23 articles, containing 36 studies, to assess the relationship between CAG and the risk of incident upper gastrointestinal cancers [ 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 ]. Among these, 13 studies analyzed gastric cancer, 15 studies examined oesophageal cancer, and 8 studies explored oesophagogastric junction cancer. CAG was diagnosed using endoscopic histological methods (found in a total of 7 articles) and serological methods (found in a total of 18 articles), with 2 articles conducting research on both of these diagnostic approaches.

In the included literature, there were 10 case–control studies, 6 nested case–control studies, and 7 cohort studies. Detailed characteristics of these incorporated studies can be found in Table 1 (Pages 31–35). These studies were published between 1995 to 2023 and collectively involved 5858 patients diagnosed with upper gastrointestinal cancers, with two articles exclusively focused on male populations. For the source of study participants, there were 5 articles based on hospital-based research, 17 articles based on population-based research, and 1 article based on a combination of population and clinic-based research. In terms of the regional distribution of the study, 10 articles were performed in Europe, 11 articles in Asia (including 8 articles in Japan), and 2 articles in the Americas. According to the NOS scale score, 6 studies received a score of 9, 18 studies achieved a score of 8, and 12 studies were rated with a score of 7.

Risk of gastric cancer

Thirteen studies were included to assess the relationship between CAG and the incidence of GC. The heterogeneity test (I 2  = 73.6%, P < 0.1) indicated significant heterogeneity in this study. The pooled results were shown in Fig.  2 (Page 39): CAG was associated with a 4.12-fold increase in the risk of GC (pooled random effect OR = 4.12, 95%CI 3.20 ~ 5.30); The risk of incident GC, when diagnosed through histological methods for CAG, was higher (OR = 4.23, 95% CI 2.47–7.25) compared to the risk associated with diagnosing CAG through serological methods (OR = 3.88, 95% CI 3.00–5.00).

Forest plot to assess the relationship between CAG and gastric cancer. CAG chronic atrophic gastritis, OR odds ratio, CI confidence interval, IM intestinal metaplasia, AG atrophic gastritis

Significant heterogeneity was observed in this study. In order to delve the origins of heterogeneity, we conducted subgroup analyses based on the diagnosis method of CAG, study type, participant source, region, year of publication, and NOS score. The relevant results are presented in Table 2 (Page 36). There was no significant heterogeneity in nested case–control studies, European studies, or studies published between 1995 and 2010. However, significant heterogeneity was detected in all other subgroup analyses. In all subgroups, patients with CAG had a significantly increased risk of incident GC. Particularly, studies conducted in the United States showed the highest relative risk of GC incidence among patients with CAG (OR = 14.30, 95% CI 4.83–42.30). This study was both a case–control and a hospital-based study, and it showed that patients with CAG had a relatively low risk of developing GC (OR = 2.66, 95% CI 1.98–3.57). In other subgroup analyses, the risk of incident GC was very similar to the overall pooled risk.

Risk of oesophageal cancer

We included 15 studies that explored the association between CAG and the incidence of OC. The heterogeneity analysis indicated significant heterogeneity within this research (I 2  = 66.0%, P < 0.1). The pooled results were shown in Fig.  3 (Page 39).: CAG was associated with a 2.08-fold increase in the risk of incident OC (pooled random-effect OR = 2.08, 95% CI 1.60–2.72). The risk of incident OC was markedly higher with the diagnosis of CAG through histologic methods (OR = 2.26, 95% CI 1.58–3.23) compared to the risk associated with diagnosing CAG through serologic methods (OR = 1.93, 95%CI 1.22–3.07). Meanwhile, we assessed the relationship between CAG and the risk of incident OSCC and OAC (Figs. 4 , 5 ; Page 40). Our findings indicated that CAG was linked to a 2.29-fold increase in the risk of incident OSCC (pooled random-effects OR = 2.29, 95% CI 1.77–2.95, I 2  = 60.7%, P = 0.002). Nevertheless, there was no significant association between CAG and the risk of incident OAC (pooled random effect OR = 0.62, 95% CI 0.17–2.26, I 2  = 67.0%, P = 0.082).

Forest plot to assess the relationship between CAG and osophageal cancer. CAG chronic atrophic gastritis, OR odds ratio, CI confidence interval, FA fundic atrophy, FIM fundic intestinal metaplasia, FGA fundic gastric atrophy, OSCC oesophageal squamous cell carcinoma, OAC oesophageal adenocarcinoma

Forest plot to assess the relationship between CAG and oesophageal squamous cell carcinoma. CAG chronic atrophic gastritis, OR odds ratio, CI confidence interval, FA fundic atrophy, FIM fundic intestinal metaplasia, FGA fundic gastric atrophy

Forest plot to assess the relationship between CAG and oesophageal adenocarcinoma. CAG chronic atrophic gastritis, OR odds ratio, CI confidence interval

In order to explore the sources of heterogeneity, we performed subgroup analyses based on the diagnosis method of CAG, study type, participant source, region, year of publication, and NOS score. As shown in Table 3 (Pages 37, 38), there was no significant heterogeneity in studies where CAG diagnosis was based on histologic methods and those published from 2011 to 2023. However, significant heterogeneity was observed in all other subgroup analyses. Additionally, studies with an NOS score < 8 did not reveal a significant association between CAG and the risk of OC, whereas all other subgroup analyses indicated a significant correlation. In the nested case–control study (OR = 4.58, 95%CI 2.00–10.48) and the study conducted in the Americas (OR 5.33, 95%CL 1.55–18.30), patients with CAG had a relatively higher risk of incident OC. In studies published from 2011 to 2023, the risk of OC in patients with CAG was relatively lower (OR = 1.66, 95%CI 1.35–2.04). In all other subgroup analyses, the risk of incident cancer was similar to the overall pooled risk.

Risk of oesophagogastric junction cancer

We included 8 studies to examine the association between CAG and the risk of incident OJC. The heterogeneity test indicated a low level of heterogeneity in this study (I 2  = 18.2%, P = 0.286 > 0.1). The pooled results were displayed in Fig.  6 (Page 40): CAG was associated with a 2.77-fold increased risk of OJC (pooled fixed effect OR = 2.77, 95% CI 2.21–3.46). The risk of OJC was significantly higher when CAG was diagnosed through histological methods (OR = 3.40, 95%CI 2.04–5.67) compared to serological methods (OR = 2.63, 95%CI 2.05–3.38), and neither of these groups of studies displayed significant heterogeneity.

Forest plot to assess the relationship between CAG and oesophagogastric junction cancer. CAG chronic atrophic gastritis, OR odds ratio, CI confidence interval

Sensitivity analyses and publication bias

For GC, OC and OJC, we performed sensitivity analyses using a study-by-study exclusion approach, and our findings demonstrated the stability and reliability of the pooled results (Additional file 2 : Fig. S1a–c). To evaluate publication bias in GC and OC, we used funnel plots and Egger's tests. Visual inspection of the funnel plots (Fig.  7 ) (Page 41) showed that the distributions were generally symmetrical, indicating that there was no significant publication bias. The results of the Egger's test (Fig.  8 ) (Pages 41, 42) indicated that, in the analysis of GC (P = 0.283) and OC (P = 0.433), no significant publication bias was observed in the studies.

Publication bias. A Funnel plot of studies assessing the relationship between CAG and risk of gastric cancer. B Funnel plot of studies assessing the relationship between CAG and risk of oesophageal cancer

Publication bias. A Egger’ s test evaluating for the relationship between CAG and risk of gastric cancer. B Egger’ s test evaluating for the relationship between CAG and risk of oesophageal cancer

In this meta-analysis, we included a total of 23 studies involving 5858 patients diagnosed with upper gastrointestinal cancers. Our objective was to analyze the connection between CAG and the incidence of upper gastrointestinal cancers. The results clearly pointed to a significant 4.12-fold elevation in the risk of GC, a 2.77-fold increase in the risk of OJC, and a 2.08-fold rise in the risk of OC among patients with CAG. Furthermore, our findings indicated a 2.29-fold increased risk of OSCC in CAG patients. However, no significant association was detected with the risk of OAC. Intriguingly, when considering the same upper gastrointestinal cancer, the risk of developing cancer was higher with CAG diagnosed through histologic methods rather than serologic methods.

This study represents the first comprehensive assessment of the association between CAG and the incidence of upper gastrointestinal cancers. Our findings reveal a substantial increase in the risk of upper gastrointestinal cancers linked to CAG, which is both consistent and inconsistent with previously published meta-analyses in different regards. Previously, gastric atrophy has been correlated with a 2.89-fold elevated risk of Cardia Cancer when diagnosed through serologic methods [ 12 ]. In contrast, our current study explores the risk of CAG and the incidence of OJC, employing both histologic and serologic diagnostic methods. Additionally, Gastric Atrophy has previously exhibited associations with the risk of OSCC and OAC, with a 1.94-fold heightened risk of OSCC incidence but a reduced risk of OAC development [ 12 ]. In this study, we have not only reported the link between CAG and an elevated risk for developing OSCC and OAC but have also indicated its relevance to the risk of developing EC. Worth noting is that patients with Intestinal Metaplasia (IM) have been previously reported to exhibit a significant 3.58-fold increase in the risk of GC, particularly when IM develops in the gastric body or presents as incomplete IM [ 42 ]; A systematic review and meta-analysis conducted by Sui [ 43 ] indicated that there was a significant 2.91-fold increase in the risk of GC associated with gastric atrophy; These results correspond with the trends observed in our own study. Furthermore, Sui's study further reported that the risk of developing GC was higher with CAG diagnosed through serum pepsinogen levels rather than endoscopy [ 43 ]. Interestingly, this result contradicts the findings of our study.

We hypothesize several potential mechanisms underlying the association between CAG and upper gastrointestinal cancers. First and foremost, CAG is predominantly attributed to H. pylori infection. H. pylori can generate a multitude of virulence factors that target gastric mucosal tissues, disrupting intracellular signalling pathways and lowering the threshold for tumour transformation. Notably, the primary virulence factors of H. pylori include cytotoxin-associated gene A (CagA) and its pathogenicity island (Cag PAI), as well as vacuolating cytotoxin A (VacA). The Wnt signalling pathway, known for its role in cancer development, is implicated in GC through the upregulation of Wnt10A in gastric mucosa-associated cells, further activating the Wnt-β-catenin-Tcf signalling pathway, significantly contributing to GC development [ 44 ]. Similarly, the upregulation of Wnt10A may also be a factor in OC development [ 45 ], as subsequent studies have indicated its enhancement of invasion and migration in OSCC [ 46 ]. Secondly, CAG induced by H. pylori infection is characterized by reduced or complete abstention of gastric acid secretion, leading to the creation of a hypochlorhydric microenvironment. Such a microenvironment fosters the proliferation of oncogenic microorganisms within the stomach and augments the production of N-nitroso compounds, which significantly increases the risk of GC and OSCC [ 47 , 48 , 49 ]. The incidence risk of OAC is positively linked to gastroesophageal reflux symptoms [ 50 ] but negatively associated with H. pylori infection [ 41 ]. Consequently, CAG triggered by H. pylori infection would seem to either reduce the occurrence of OAC or have no discernible impact. The risk of OJC may exhibit two potential scenarios: some cases, similar to OAC, OJC could have a negative correlation or no association with CAG. meanwhile, others, resembling GC, may show a positive correlation with CAG. Lastly, chronic inflammation is one of the potential mechanisms contributing to cancer development, and this applies equally to upper gastrointestinal cancers. When tissue damage occurs, inflammatory cells gather and release inflammatory cytokines, thereby promoting the generation of reactive oxygen species (ROS). These ROS induce cellular proliferation, causing oxidative damage to DNA and, in turn, amplifying the risk of cancer development [ 4 ]. CAG remains in a chronic inflammatory state, particularly following H. pylori infection, which triggers the upregulation of multiple pro-inflammatory factors, including interleukin-8 (IL-8), nuclear factor-κB (NF-κB), tumour necrosis factor-α (TNF-α), and interleukin-6 (IL-6) and others. The upregulation of IL-8 and activation of NF-κB in gastric epithelial cells play pivotal roles in the mechanisms underlying chronic inflammation and the development of GC [ 51 ]. Additionally, NF-κB is closely associated with metastasis, inflammation, and poor prognosis in OC patients [ 52 ].

This meta-analysis reveals a certain degree of heterogeneity. To ensure the robustness of our findings, we conducted a sensitivity analysis, which confirmed the stability of the pooled results. In our assessment of GC and OC studies, both funnel plots and Egger's tests were employed, and the results consistently showed no clear evidence of publication bias.

In the subgroup analyses, notable findings emerged. In the GC study, all subgroup analyses consistently indicated a significant increase in GC risk among patients with CAG. It is worth highlighting that the studies conducted in the United States (US) reported the highest incidence of GC, even though GC is relatively uncommon in the US. This discovery emphasizes the potential relevance of serum pepsin as a predictive marker for GC in the US [ 32 ]. Among Asian research, Japan remains the sole contributor to relevant studies, emphasizing the necessity for broader participation from other Asian regions in future observational research. Notably, there was a study that served as both a case–control study and hospital-based research, revealing a comparatively lower risk level, possibly associated with the control group's population selection process. In the field of OC research, studies with NOS scores below 8 did not reveal any significant risk association. Nevertheless, all other subgroup analyses consistently pointed to a marked increase in OC incidence risk associated with CAG. Studies conducted in Latin America indicated the highest risk of OC, whereas research published between 2011 and 2023 showed a comparatively lower OC risk. Hospital-based studies showed a comparatively higher OC risk than population-based studies, possibly due to the inclusion of more severe cases from hospital settings.

Another crucial aspect of our study is the exploration of the association between CAG diagnosed through two different diagnostic methods and the risk of upper gastrointestinal cancers. In studies assessing GC risk in relation to CAG, histological confirmation of CAG was linked to a 35% increased risk of GC compared to serological diagnosis. Similarly, regarding the relationship between CAG and OC risk, histologic diagnosis of CAG was connected to a 33% higher risk of OC compared to serologic diagnosis. Moreover, in the investigation of the risk of OJC associated with CAG, histological confirmation of CAG was associated with a 77% elevated risk of OJC compared to serological diagnosis. In general, for GC, OC and OJC, the risk of cancer development was linked to a 33%-77% higher when CAG is diagnosed histologically compared to serologically. It's important to note that the guidelines for managing precancerous gastric epithelial lesions and other lesions recommend serum pepsinogen level as the best noninvasive test for detecting atrophic gastritis. However, in cases of low serum pepsinogen levels, reliance on gastroscopy is necessary [ 53 ]. The accuracy of endoscopic biopsy results can be influenced by a range of factors, including the quality of biopsy samples, specimen handling, and the expertise of pathologists [ 54 ]. Furthermore, the use of different analytical methods and threshold values with serum pepsinogen diagnosis can result in varying levels of specificity and sensitivity [ 55 ]. Therefore, further research and investigation are essential to comprehensively assess the risk associated with CAG diagnosis for upper gastrointestinal cancers using these two methods.

Our meta-analysis has also additional important strengths. As previously mentioned, our study aims to provide the most comprehensive evaluation of the connection between CAG and the risk of upper gastrointestinal cancers to date. We have included data from Asia, Europe, and the Americas, ensuring that the selected studies adhere to a high standard of quality.

However, there are certain limitations to our study. Firstly, the pooled results are constrained by the scarcity of studies focusing on the association between CAG and the risk of incident OAC. Future large-scale observational studies are imperative to delve deeper into the relationship between CAG and the incidence of OAC. Furthermore, the studies we have incorporated into our analysis display differences in adjusted factors and involve differing study designs, potentially introducing additional bias into the pooled findings. The data in our study was primarily sourced from medical records and cancer registries, which might introduce a degree of selection bias into the dataset. Finally, the primary diagnostic modalities for CAG include endoscopic histology and serum pepsinogen levels. While these two methods are widely used in current medical practice, they still exhibit certain limitations. The former can be influenced by factors such as the endoscopist's skill, specimen handling protocols, and diagnostic interpretation by the pathologist, whereas the results of the latter may fluctuate based on specimen analysis techniques and the selection of critical values.

This meta-analysis showed a two- to fourfold increased risk of GC, EC and EJC in patients with CAG. Importantly, for the same upper gastrointestinal cancer, the risk of incident cancer was higher when CAG was diagnosed through histological methods compared to serological methods. Further rigorous study designs are required to explore the impact of CAG diagnosed through both diagnostic methods on the risk of upper gastrointestinal cancers.

Availability of data and materials

All data generated or analyzed during this study are included in this published article [and its Additional files].

Abbreviations

• Chronic atrophic gastritis

Confidence interval

• Gastric cancer
• Oesophagogastric junction cancer
• Oesophageal cancer

Oesophageal adenocarcinoma

Oesophageal squamous cell carcinoma

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Newcastle–Ottawa Scale score

Hazard ratio

Relative risk

Intestinal metaplasia

Atrophic gastritis

Fundic atrophy

Fundic intestinal metaplasia

Fundic gastric atrophy

Interleukin-8

Nuclear factor-κB

Tumour necrosis factor-α

Interleukin-6

The United States

Kocarnik JM, Compton K, Dean FE, Fu W, Gaw BL, Harvey JD, Henrikson HJ, Lu D, Pennini A, Xu R, et al. Cancer incidence, mortality, years of life lost, years lived with disability, and disability-adjusted life years for 29 cancer groups from 2010 to 2019: a systematic analysis for the global burden of disease study 2019. JAMA Oncol. 2022;8:420–44.

Article   PubMed   Google Scholar  

Yang H, Wang X-K, Wang J-B, Zhao F-H, Fan J-H, Qiao Y-L, Taylor PR, Abnet CC. Combined risk factors and risk of upper gastrointestinal cancer mortality in the Linxian general population. Int J Cancer. 2022;151:1462–73.

Article   CAS   PubMed   Google Scholar  

Abnet CC, Corley DA, Freedman ND, Kamangar F. Diet and upper gastrointestinal malignancies. Gastroenterology. 2015;148:1234.

Coussens LM, Werb Z. Inflammation and cancer. Nature. 2002;420:860–7.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Mantovani A, Allavena P, Sica A, Balkwill F. Cancer-related inflammation. Nature. 2008;454:436–44.

González CA, Figueiredo C, Lic CB, Ferreira RM, Pardo ML, Ruiz Liso JM, Alonso P, Sala N, Capella G, Sanz-Anquela JM. Helicobacter pylori cagA and vacA genotypes as predictors of progression of gastric preneoplastic lesions: a long-term follow-up in a high-risk area in Spain. Am J Gastroenterol. 2011;106:867–74.

Ohata H, Kitauchi S, Yoshimura N, Mugitani K, Iwane M, Nakamura H, Yoshikawa A, Yanaoka K, Arii K, Tamai H, et al. Progression of chronic atrophic gastritis associated with Helicobacter pylori infection increases risk of gastric cancer. Int J Cancer. 2004;109:138–43.

Sharma PK, Suri TM, Venigalla PM, Garg SK, Mohammad G, Das P, Sood S, Saraya A, Ahuja V. Atrophic gastritis with high prevalence of Helicobacter pylori is a predominant feature in patients with dyspepsia in a high altitude area. Trop Gastroenterol. 2014;35:246–51.

Plummer M, van Doorn L-J, Franceschi S, Kleter B, Canzian F, Vivas J, Lopez G, Colin D, Muñoz N, Kato I. Helicobacter pylori cytotoxin-associated genotype and gastric precancerous lesions. J Natl Cancer Inst. 2007;99:1328–34.

Akbari M, Tabrizi R, Kardeh S, Lankarani KB. Gastric cancer in patients with gastric atrophy and intestinal metaplasia: a systematic review and meta-analysis. PLoS ONE. 2019;14: e0219865.

Spence AD, Cardwell CR, McMenamin ÚC, Hicks BM, Johnston BT, Murray LJ, Coleman HG. Adenocarcinoma risk in gastric atrophy and intestinal metaplasia: a systematic review. BMC Gastroenterol. 2017;17:157.

Article   PubMed   PubMed Central   Google Scholar  

Islami F, Sheikhattari P, Ren JS, Kamangar F. Gastric atrophy and risk of oesophageal cancer and gastric cardia adenocarcinoma—a systematic review and meta-analysis. Ann Oncol. 2011;22:754–60.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ (Clin Res Ed). 2021;372: n71.

Google Scholar  

Taylor KS, Mahtani KR, Aronson JK. Summarising good practice guidelines for data extraction for systematic reviews and meta-analysis. BMJ Evid Based Med. 2021;26:88–90.

Shah SC, Piazuelo MB, Kuipers EJ, Li D. AGA clinical practice update on the diagnosis and management of atrophic gastritis: expert review. Gastroenterology. 2021;161:1325.

Sjs HJ. Classification of adenocarcinoma of the oesophagogastric junction. Br J Surg. 1998;85:1457–9.

Lo CK-L, Mertz D, Loeb M. Newcastle–Ottawa Scale: comparing reviewers’ to authors’ assessments. BMC Med Res Methodol. 2014;14:45.

Localio AR, Margolis DJ, Berlin JA. Relative risks and confidence intervals were easily computed indirectly from multivariable logistic regression. J Clin Epidemiol. 2007;60:874–82.

Akiyama T, Inamori M, Iida H, Endo H, Hosono K, Yoneda K, Fujita K, Yoneda M, Takahashi H, Goto A, et al. Macroscopic extent of gastric mucosal atrophy: increased risk factor for esophageal squamous cell carcinoma in Japan. BMC Gastroenterol. 2009;9:34.

Almodova EdC, De Oliveira WK, Machado LFA, Grejo JR, Da Cunha TR, Colaiacovo W, Ortolan EVP. Atrophic gastritis: risk factor for esophageal squamous cell carcinoma in a Latin-American population. World J Gastroenterol. 2013;19:2060–4.

Anderson LA, Murphy SJ, Johnston BT, Watson RGP, Ferguson HR, Bamford KB, Ghazy A, McCarron P, McGuigan J, Reynolds JV, et al. Relationship between Helicobacter pylori infection and gastric atrophy and the stages of the oesophageal inflammation, metaplasia, adenocarcinoma sequence: results from the FINBAR case–control study. Gut. 2008;57:734–9.

Aromaa A, Kosunen TU, Knekt P, Maatela J, Teppo L, Heinonen OP, Härkönen M, Hakama MK. Circulating anti-Helicobacter pylori immunoglobulin A antibodies and low serum pepsinogen I level are associated with increased risk of gastric cancer. Am J Epidemiol. 1996;144:142–9.

Cook MB, Dawsey SM, Diaw L, Blaser MJ, Perez-Perez GI, Abnet CC, Taylor PR, Albanes D, Virtamo J, Kamangar F. Serum pepsinogens and Helicobacter pylori in relation to the risk of esophageal squamous cell carcinoma in the alpha-tocopherol, beta-carotene cancer prevention study. Cancer Epidemiol Biomarkers Prev. 2010;19:1966–75.

de Vries AC, Capelle LG, Looman CWN, van Blankenstein M, van Grieken NCT, Casparie MK, Meijer GA, Kuipers EJ. Increased risk of esophageal squamous cell carcinoma in patients with gastric atrophy: independent of the severity of atrophic changes. Int J Cancer. 2009;124:2135–8.

Ekheden I, Yang X, Chen H, Chen X, Yuan Z, Jin L, Lu M, Ye W. Associations between gastric atrophy and its interaction with poor oral health and the risk for esophageal squamous cell carcinoma in a high-risk region of China: a population-based case–control study. Am J Epidemiol. 2020;189:931–41.

Fukuda H, Saito D, Hayashi S, Hisai H, Ono H, Yoshida S, Oguro Y, Noda T, Sato T, Katoh M. Helicobacter pylori infection, serum pepsinogen level and gastric cancer: a case–control study in Japan. Jpn J Cancer Res Gann. 1995;86:64–71.

Gao P, Cai N, Yang X, Yuan Z, Zhang T, Lu M, Jin L, Ye W, Suo C, Chen X. Association of Helicobacter pylori and gastric atrophy with adenocarcinoma of the esophagogastric junction in Taixing, China. Int J Cancer. 2022;150:243–52.

Hansen S, Vollset SE, Derakhshan MH, Fyfe V, Melby KK, Aase S, Jellum E, McColl KEL. Two distinct aetiologies of cardia cancer; evidence from premorbid serological markers of gastric atrophy and Helicobacter pylori status. Gut. 2007;56:918–25.

Holleczek B, Schöttker B, Brenner H. Helicobacter pylori infection, chronic atrophic gastritis and risk of stomach and esophagus cancer: results from the prospective population-based ESTHER cohort study. Int J Cancer. 2020;146:2773–83.

Iijima K, Koike T, Abe Y, Inomata Y, Sekine H, Imatani A, Nakaya N, Ohara S, Shimosegawa T. Extensive gastric atrophy: an increased risk factor for superficial esophageal squamous cell carcinoma in Japan. Am J Gastroenterol. 2007;102:1603–9.

Ikeda F, Shikata K, Hata J, Fukuhara M, Hirakawa Y, Ohara T, Mukai N, Nagata M, Yoshida D, Yonemoto K, et al. Combination of Helicobacter pylori antibody and serum pepsinogen as a good predictive tool of gastric cancer incidence: 20-year prospective data from the hisayama study. J Epidemiol. 2016;26:629–36.

In H, Sarkar S, Ward J, Friedmann P, Parides M, Yang J, Epplein M. Serum pepsinogen as a biomarker for gastric cancer in the united states: a nested case-control study using the PLCO cancer screening trial data. Cancer Epidemiol Biomark Prev. 2022;31:1426–32.

Article   CAS   Google Scholar  

Inoue M, Sawada N, Goto A, Shimazu T, Yamaji T, Iwasaki M, Tsugane S. High-negative anti-helicobacter pylori igg antibody titers and long-term risk of gastric cancer: results from a large-scale population-based cohort study in Japan. Cancer Epidemiol Biomark Prev. 2020;29:420–6.

Inoue M, Tajima K, Matsuura A, Suzuki T, Nakamura T, Ohashi K, Nakamura S, Tominaga S. Severity of chronic atrophic gastritis and subsequent gastric cancer occurrence: a 10-year prospective cohort study in Japan. Cancer Lett. 2000;161:105–12.

Nasrollahzadeh D, Malekzadeh R, Aghcheli K, Sotoudeh M, Merat S, Islami F, Kamangar F, Abnet CC, Shakeri R, Pourshams A, et al. Gastric atrophy and oesophageal squamous cell carcinoma: possible interaction with dental health and oral hygiene habit. Br J Cancer. 2012;107:888–94.

Sasazuki S, Inoue M, Iwasaki M, Otani T, Yamamoto S, Ikeda S, Hanaoka T, Tsugane S. Effect of Helicobacter pylori infection combined with CagA and pepsinogen status on gastric cancer development among Japanese men and women: a nested case–control study. Cancer Epidemiol Biomark Prev. 2006;15:1341–7.

Song H, Ekheden IG, Zheng Z, Ericsson J, Nyrén O, Ye W. Incidence of gastric cancer among patients with gastric precancerous lesions: observational cohort study in a low risk Western population. BMJ (Clin Res Ed). 2015;351:h3867.

Venerito M, Kohrs S, Wex T, Adolf D, Kuester D, Schubert D, Peitz U, Mönkemüller K, Malfertheiner P. Helicobacter pylori infection and fundic gastric atrophy are not associated with esophageal squamous cell carcinoma: a case–control study. Eur J Gastroenterol Hepatol. 2011;23:859–64.

Vohlonen I, Pukkala E, Malila N, Härkönen M, Hakama M, Koistinen V, Sipponen P. Risk of gastric cancer in Helicobacter pylori infection in a 15-year follow-up. Scand J Gastroenterol. 2016;51:1159–64.

Watanabe Y, Kurata JH, Mizuno S, Mukai M, Inokuchi H, Miki K, Ozasa K, Kawai K. Helicobacter pylori infection and gastric cancer. A nested case–control study in a rural area of Japan. Digest Dis Sci. 1997;42:1383–7.

Ye W, Held M, Lagergren J, Engstrand L, Blot WJ, McLaughlin JK, Nyrén O. Helicobacter pylori infection and gastric atrophy: risk of adenocarcinoma and squamous-cell carcinoma of the esophagus and adenocarcinoma of the gastric cardia. J Natl Cancer Inst. 2004;96:388–96.

Shao L, Li P, Ye J, Chen J, Han Y, Cai J, Lu X. Risk of gastric cancer among patients with gastric intestinal metaplasia. Int J Cancer. 2018;143:1671–7.

Sui Z, Chen J, Li P, Shao L, Ye J, Lu X, Cai J. Risk for gastric cancer in patients with gastric atrophy: a systematic review and meta-analysis. Transl Cancer Res. 2020;9:1618–24.

Kirikoshi H, Sekihara H, Katoh M. Up-regulation of WNT10A by tumor necrosis factor alpha and Helicobacter pylori in gastric cancer. Int J Oncol. 2001;19:533–6.

CAS   PubMed   Google Scholar  

Kirikoshi H, Inoue S, Sekihara H, Katoh M. Expression of WNT10A in human cancer. Int J Oncol. 2001;19:997.

Long A, Giroux V, Whelan KA, Hamilton KE, Tétreault M-P, Tanaka K, Lee J-S, Klein-Szanto AJ, Nakagawa H, Rustgi AK. WNT10A promotes an invasive and self-renewing phenotype in esophageal squamous cell carcinoma. Carcinogenesis. 2015;36:598–606.

Craddock VM. Aetiology of oesophageal cancer: some operative factors. Eur J Cancer Prev. 1992;1:89.

Derakhshan MH, El-Omar E, Oien K, Gillen D, Fyfe V, Crabtree JE, McColl KEL. Gastric histology, serological markers and age as predictors of gastric acid secretion in patients infected with Helicobacter pylori. J Clin Pathol. 2006;59:1293–9.

Husebye E. The pathogenesis of gastrointestinal bacterial overgrowth. Chemotherapy. 2005;51(Suppl):1.

Lagergren J, Bergström R, Lindgren A, Nyrén O. Symptomatic gastroesophageal reflux as a risk factor for esophageal adenocarcinoma. N Engl J Med. 1999;340:825–31.

Brandt S, Kwok T, Hartig R, König W, Backert S. NF-kappaB activation and potentiation of proinflammatory responses by the Helicobacter pylori CagA protein. Proc Natl Acad Sci USA. 2005;102:9300–5.

Sharma T, Gupta A, Chauhan R, Bhat AA, Nisar S, Hashem S, Akhtar S, Ahmad A, Haris M, Singh M, Uddin S. Cross-talk between the microbiome and chronic inflammation in esophageal cancer: potential driver of oncogenesis. Cancer Metastasis Rev. 2022;41:281–99.

Pimentel-Nunes P, Libânio D, Marcos-Pinto R, Areia M, Leja M, Esposito G, Garrido M, Kikuste I, Megraud F, Matysiak-Budnik T, et al. Management of epithelial precancerous conditions and lesions in the stomach (MAPS II): European Society of Gastrointestinal Endoscopy (ESGE), European Helicobacter and Microbiota Study Group (EHMSG), European Society of Pathology (ESP), and Sociedade Portuguesa de Endoscopia Digestiva (SPED) guideline update 2019. Endoscopy. 2019;51:365–88.

Iijima K, Abe Y, Kikuchi R, Koike T, Ohara S, Sipponen P, Shimosegawa T. Serum biomarker tests are useful in delineating between patients with gastric atrophy and normal, healthy stomach. World J Gastroenterol. 2009;15:853–9.

Huang Y-K, Yu J-C, Kang W-M, Ma Z-Q, Ye X, Tian S-B, Yan C. Significance of serum pepsinogens as a biomarker for gastric cancer and atrophic gastritis screening: a systematic review and meta-analysis. PLoS ONE. 2015;10:e0142080.

Download references

Acknowledgements

This work was supported by the support from the National Natural Science Foundation of China (82174240); the Shanghai Pudong New Area Science and Technology Development Fund Project (NO: PKJ2021-Y69); and the Shanghai Pudong Famous Chinese Medicine Training Program (NO: PWRzm2020-03).

Author information

Junqiu Li and Jielu Pan have contributed equally to this work.

Authors and Affiliations

Department II of Digestive Diseases, Longhua Hospital Affiliated to Shanghai University of Traditional Chinese Medicine, Shanghai, 200032, China

Junqiu Li, Jielu Pan, Dinghong Xiao, Nan Shen, Ruiqing Wang, Hongyv Miao, Peimin Pu, Haiyan Zhang, Xiao Yv & Lianjun Xing

You can also search for this author in PubMed   Google Scholar

Contributions

JQL, JLP, XY and LJX contributed research ideas; JQL and XY contributed to the literature screening and data extraction; JQL and JLP contributed to the study quality assessment; DHX, NS and RQW and contributed software applications and statistical analyses; JQL, HYM, PMP and HYZ contributed writing and preparation of the original manuscript; LJX contributed research supervision; All authors contributed to the original writing and review of the manuscript. All authors read and reviewed the final version of the manuscript.

Corresponding authors

Correspondence to Xiao Yv or Lianjun Xing .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: file 1.

Search strategy.

Additional file 2: Fig. S1.

Sensitivity analysis was performed for gastric cancer (GC), oesophageal cancer (OC), and oesophagogastric junction cancer (OJC) using a study-by-study exclusion approach.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Li, J., Pan, J., Xiao, D. et al. Chronic atrophic gastritis and risk of incident upper gastrointestinal cancers: a systematic review and meta-analysis. J Transl Med 22 , 429 (2024). https://doi.org/10.1186/s12967-023-04736-w

Download citation

Received : 01 October 2023

Accepted : 15 November 2023

Published : 06 May 2024

DOI : https://doi.org/10.1186/s12967-023-04736-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Upper gastrointestinal cancers
• Meta-analysis

Journal of Translational Medicine

ISSN: 1479-5876

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Instructions for Authors
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 14, Issue e1
• Gypsy, Traveller and Roma experiences, views and needs in palliative and end of life care: a systematic literature review and narrative synthesis
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0002-6565-6057 Kathryn Charlotte Dixon 1 ,
• Rebecca Ferris 2 ,
• Isla Kuhn 3 ,
• Anna Spathis 1 and
• http://orcid.org/0000-0002-4505-7743 Stephen Barclay 1
• 1 Department of Public Health and Primary Care , University of Cambridge , Cambridge , Cambridgeshire , UK
• 2 Cambridge Postgraduate Medical Centre , Addenbrooke's Hospital , Cambridge , Cambridgeshire , UK
• 3 Medical Library , University of Cambridge , Cambridge , Cambridgeshire , UK
• Correspondence to Kathryn Charlotte Dixon, University of Cambridge Department of Public Health and Primary Care, Cambridge, UK; kcd23{at}medschl.cam.ac.uk

Background Gypsy, Traveller and Roma communities are known to experience health inequalities. There has been little focus on palliative care in these communities despite the well-recognised inequalities of access to palliative care in other minority ethnic groups.

Methods Systematic review and thematic analysis of the current evidence concerning palliative care experiences, views and needs of Gypsy, Traveller and Roma communities. Medline, Embase, Emcare, CINAHL, PsycINFO, Web of Science, Scopus, AMED, Global Health, Psychological and Behavioural Sciences Collection and BNI were searched up to November 2020, alongside author and citation searching. NHS England, Hospice UK, National Audit Office and OpenGrey were searched as grey literature sources. Gough’s ‘Weight of Evidence’ framework was used for quality appraisal.

Results Thirteen papers from eight studies were included in the synthesis. Although there was variation between communities, three overarching and inter-related themes were identified. (1) Strong family and community values include a preference for healthcare to be provided from within the community, duty to demonstrate respect by attending the bedside and illness as a community problem with decision-making extending beyond the patient. (2) Distinct health beliefs regarding superstitions around illness, personal care, death rituals and bereavement. (3) Practical barriers to non-community healthcare provision include communication difficulties, limited awareness of and access to services, tensions between patients and healthcare professionals and lack of training in delivering culturally appropriate care.

Conclusion A wide range of factors influence Gypsy, Traveller and Roma community access to palliative care. Community diversity requires sensitive and highly individualised approaches to patient care.

PROSPERO registration number 42019147905.

• cultural issues
• methodological research

https://doi.org/10.1136/bmjspcare-2020-002676

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Gypsy, Traveller and Roma communities are Europe’s largest minority ethnic group. 1 Although these communities are culturally diverse, they are frequently grouped together within healthcare policy and also by the communities themselves. 1–3 The Council of Europe uses the term ‘Roma’ to refer to itinerant groups within Europe. 2 Although Irish Travellers are included within the Council of Europe’s definition of ‘Roma’, they are genetically distinct from European Roma, 4 with Irish ancestry as opposed to Indian. 2

Despite a traditional nomadic lifestyle, up to 85% of travelling communities in Europe today are settled in one place with those maintaining a travelling way of life found mainly in France, the Benelux countries, Switzerland, Ireland and the UK. 2 When settled, they may live in permanent bricks-and-mortar housing or in mobile homes or caravans and some groups are seminomadic, travelling within the summer months. 5

Gypsy, Traveller and Roma communities experience significant health inequalities in multiple areas of health and social care. 6 7 These include reduced life expectancy, ranging from 10 to 20 years lower than the general population, higher maternal and infant mortality, lower uptake of immunisations and therefore higher rates of vaccine-preventable diseases, higher prevalence of anxiety and depression and higher prevalence of chronic diseases such as diabetes, asthma, cardiovascular disease and hypertension. 5 7–10 There is recognised reduced access to and engagement with services across the healthcare setting and a multitude of barriers to healthcare including illiteracy, discrimination and reduced awareness of services available. 11

Inequality of access to palliative care is a global health problem and the WHO is spearheading efforts to reduce this. 12 13 Providers of palliative care are increasingly focusing on ‘hard-to-reach groups’ in an attempt to enhance access. 10 14 It is already known that Gypsy, Traveller and Roma communities have poor access to, and engagement across, healthcare services generally, 11 and it is likely that this extends to palliative and end of life care services. Good end of life care should be an integral part of all health systems. 13

Given the known inequalities in healthcare for patients from minority ethnic groups, and specifically for Gypsy, Traveller and Roma patients, and the importance of providing culturally and socially appropriate palliative care, a systematic review of palliative and end of life care for patients in these communities is highly topical. Previous systematic reviews have evaluated Gypsy, Traveller and Roma access to and engagement with health services in general 11 and palliative care within black and Asian minority ethnic groups in the UK. 15 This is the first systematic review to address the palliative and end of life care experiences, views and needs within travelling communities.

Search strategy

The search strategy was developed with a specialist information technologist (IK). Search terms to identify articles about Gypsy, Traveller and Roma peoples were adapted from those used by McFadden et al . 11 The search strategy was tailored for each database searched (Medline, Embase, Emcare, CINAHL, PsycINFO, Web of Science, Scopus, AMED, Global Health, Psychological and Behavioural Sciences Collection and BNI). Full search strategies are available in online supplemental material 1 . No restrictions were applied to the searches. All databases were searched from inception to September 2019, and updated in November 2020. NHS England, Hospice UK, National Audit Office and OpenGrey were searched for ‘gypsy, gipsy, traveller, roma and rroma’. Citation and author searching was undertaken for all relevant full-text articles.

Supplemental material

Study selection.

Studies, including reports and PhD theses, fitted the criteria if they reported original research on the palliative and end of life care experiences, views and needs of adult Traveller, Gypsy or Roma people, whether from the perspective of members of these communities or healthcare professionals. Book chapters, studies focusing on children (under 18 years of age) or articles where the full text was not available in English were excluded.

After exclusion of duplicates using EndNote, two reviewers (KCD and RF) independently screened titles and abstracts and then full-texts, with discrepancy between reviewers resolved by discussion to reach consensus. Title and abstract screening was facilitated by the use of the web app Rayyan. 16 One reviewer (KCD) conducted the grey literature search, citation and author searching and articles chosen for inclusion were agreed by the second reviewer (RF).

The study identification process is summarised in figure 1 . Database and grey literature searches identified 1296 records after removal of duplicates. Seventeen full-text articles were assessed for eligibility with seven of these being eligible for inclusion. Each of these seven articles underwent citation and author searching, identifying a further three papers, one PhD thesis and two reports. Citation and author searching was then conducted for these records. One further unpublished study of relevance was identified but the authors did not respond to requests for access to the article. 17 Of the thirteen final articles selected for inclusion, five research papers reported on the same set of interviews and were therefore treated as one study. In addition, one report and one research paper reported on the same results and were also treated as one study. A total of thirteen papers relating to eight studies were therefore included in the qualitative synthesis.

• Download figure
• Open in new tab
• Download powerpoint

PRISMA Flow.

Quality appraisal

Two reviewers (KCD and RF) independently appraised the quality and relevance of the included studies using Gough’s ‘Weight of Evidence’ (WoE) framework. 18 Disagreements in quality assessment decisions were discussed and consensus achieved. The WoE framework provides an overall judgement on the quality and relevance of a study in answering the review question (WoE D). The adaptation of Gough’s WoE for this review is shown in table 1 .

• View inline

Review-specific Gough’s ‘Weight of Evidence’ high-quality criteria (adapted from Gough 2007) 18

Data extraction and synthesis

Information regarding publication details, study aims, participants, recruitment and research methods were extracted on to a review-specific data extraction form by KCD and checked by RF. The form was piloted on three studies before continuing with its use. Results data were extracted line by line in to an Excel spreadsheet by KCD prior to coding (see below). RF checked 10% of data extracted.

Data synthesis used a thematic analysis approach, adapted from that suggested by Thomas and Harden, 19 involving three main stages after data extraction:

Coding of the data: KCD coded all individual data points in Excel, developing a coding framework whereby different data points shared the same code. By using an inductive framework rather than developing a priori codes, potential for bias was reduced and significance of original data was not lost. RF checked a representative proportion of codes from each study to ensure consistency in code assigning across studies.

Generation of descriptive themes: KCD grouped the codes into descriptive themes on paper, which were agreed by discussion with RF. Codes from studies assessed to be of low quality under WoE D were only used in generation of descriptive themes if they supported codes from a study of medium or high quality.

Generation of analytical themes: KCD and RF independently, and then by discussion, organised the findings into overarching themes.

Eight studies, described in thirteen articles, were synthesised in this review. Studies reported on practice in the UK (n=4), 20–23 Ireland (n=1), 24 25 Romania (n=1) 26–30 and Spain (n=2). 31 32 Studies focused on members of different travelling communities: Roma (n=3), 26–32 Irish Travellers (n=1) 24 25 and English Gypsies (n=1). 20 Two studies did not identify participants beyond ‘Gypsy and Traveller’, 22 23 and one study involved both Irish Travellers and English Gypsies. 21 The majority of participants were female with only two studies incorporating male participants. 26–30 32 Three studies reported the views of healthcare professionals: one in detail via interviews and questionnaires, 24 25 one via the involvement of two participants 32 and one via focus groups with 30 health mediators (although there was limited analysis of this in the article). 30 Methods included focus group discussion (n=4), 21 23–25 31 qualitative interviews with members of travelling communities (n=3), 20 22 26–30 qualitative interviews with healthcare professionals (n=1), 24 25 questionnaire (n=2), 21 24 25 observational fieldwork (n=1) 20 and communicative daily life stories (n=1). 32 The details of each study and their weighting on the WoE framework are summarised in table 2 . One study was weighted as high quality, 21 five as medium 20 22–25 31 and two had a low weighting. 26–30 32 One study was mainly focused on grief and bereavement after the loss of a child and therefore only information relevant to the review question was extracted. 21

Summary of included studies

Three overarching analytical themes were found: (1) strong family and community values, (2) distinct health beliefs and (3) practical barriers to non-community healthcare provision.

Strong family and community values

The importance of family and community was echoed by all studies. Family are considered to have a duty to care for a dying relative and there is a reciprocal desire by the dying person to be cared for by their family. 22–25 28 29 31 Consistent throughout all studies was the overriding desire to die at home, surrounded by family and the community. Attending the bedside of a dying community member is seen as a marker of respect for that person and their family. 20 21 26 32 Large groups gather around the bedside with as many as 50–200 people paying their respects at and around the time of death. 20–25 27–29 32 Financial, as well as emotional, community support is expected. 28 31 32

A consistent theme was that healthcare from within the community is preferred to external care. Although respectful of healthcare staff, it was considered that family members are most informed and best placed to judge a patient’s needs. 24 25 31 Decision-making around death and dying extended beyond the individual patient, with multiple family members requesting personal updates from doctors 24 25 and some families expressing the wish to withhold diagnoses from patients. 20 27

Despite the overwhelming message of the significance of family and the wider community for patients with terminal conditions, three studies described the concept of shame in being terminally ill or seeking support, with a need to hide the illness from the wider community due to the fear of social isolation. 23 27 31 In one study, participants were ashamed to accept care from a non-Traveller or non-Roma person. 32

Distinct health beliefs

Distinct health beliefs were described in relation to general health and hygiene, and specific beliefs around cancer and death. Hygiene rituals, including beliefs about impurity were described by three studies, 20 27 30 with the suggestion that care in a hospital or hospice environment could cause distress if normal hygiene practices could not be followed. 20

Superstitions were described around cancer. This included fear of discussing or saying the word ‘cancer’ with a perception that this could lead to disease progression or bring misfortune on family members. 24 25 27 There was also a fear that non-contagious conditions could spread and that cancer treatments might actually cause disease growth. 20 29 Similarly, death and dying were seen as taboo topics, socially unacceptable to discuss; 21 23–25 27 there were descriptions of research participants blessing themselves during focus groups and interviews in one study. 24 25 Connected to this was a wish not to know the exact details of an illness 24 25 and a belief that a patient knowing their own diagnosis could cause further harm. 20 26 27 Several studies highlighted the importance of ‘hope’ with loss of hope itself causing harm or hastening death. 21 24 25 27 31 Due to this belief, hospice care was described as a last resort by two studies as hospices were seen as places to die. 24 25 31

Grief is considered a private matter and several studies raised the issue that grief is not discussed or shared within these communities, which can lead to prolonged grief or unresolved bereavement. 20–22 24 25 Gender roles were presented in that women may be expected to grieve openly whereas men can cry only at the graveside. 21 The duty to care for unwell relatives resided with women rather than men. 31 In addition, when accessing healthcare or obtaining health-related advice, women only speak to women and men to men. 20

Death rituals were important 23–25 27 30 and although in some cases linked to religion, cultural traditions tended to supersede religious practice. 21 24 25 32 Examples include a ‘rite of light’ (the lighting of a candle near the time of death to enable passage of the soul to ‘the other world’), 24 25 27 30 the need for large numbers of family and community to say goodbye personally and immediately after death, 23–25 29 32 often via a vigil 32 and the need for appropriate respect of a person by dressing them ‘properly’ before death. 30 Death rituals and traditions were described as changing over time, with associated intergenerational differences. 21 24 25 For example, one tradition involves burning the place of death, yet many participants remarked on the change in this belief over time, given the expense and move away from wooden trailers; families report preferring to have the trailer cleaned and blessed by a priest, or selling it. 21 24 25

Some studies evaluated views on life-sustaining treatment and advanced decisions about care. 24 25 27 31 Generally, written documents were not thought useful. Although in part due to variable literacy, this appeared mainly to be due to a perception that the family already know how to care appropriately for the patient, without need for written plans. 24 25 31

Practical barriers to non-community healthcare provision

Barriers to non-community healthcare in general and palliative care specifically were identified. Illiteracy and a misunderstanding of medical terminology were cited as common initial barriers to all non-community healthcare. 20 23–26 32 Three studies raised concerns regarding reduced access to a General Practitioner. 20 22 24 25 Late presentation of disease was then attributed to either fear of the diagnosis, 24 25 lack of registration with a General Practitioner 20 22 or reluctance to see a doctor. 20 24 25 Participants reported experiences of staff basing their ideas on stereotypes or showing a lack of respect for patients and their families from travelling communities. 20 22–25 30 32 Several studies raised the idea of ‘conflict’ between staff and patients with patients feeling that staff did not understand their culture and had limited experience in looking after them. 22–25 30 32 Likewise, healthcare professionals agreed that they had limited knowledge in caring for these patients and would appreciate training, which was felt to not be available. 21 24 25

There were mutual concerns from both the community and healthcare professionals regarding factors leading to reduced access to palliative care specifically. Travellers reported limited knowledge of what palliative care is and what services are available 20 23–25 but also raised concerns that the services available may not be appropriate or targeted towards people from their community. 21 22 Similarly, staff expressed views that the service may not be appropriate for travellers and lack of knowledge as to whether external non-community healthcare was acceptable. 24 25 Healthcare professionals suggested that the living situation and lifestyle for their travelling patients would impact the provision of appropriate care and follow-up. 24–26 They also raised concerns about the safety of other staff when attending to patients alone on a caravan site. 24 25

Solutions were suggested by both parties; travellers raised the issue of a lack of targeted support being available and the idea of trained travellers providing bereavement care or advice to patients and families or acting as a liaison between the community and healthcare professionals. 20–25 32 There were examples where allowances for cultural needs had enabled positive experiences, such as catering for large groups to attend the bedside. 21–23

Theme integration

There were commonalities, but also findings of significant diversity between communities in relation to all three themes. It was also clearly acknowledged across several studies by Irish travellers, English Romany Gypsies and Roma that views differed within and between communities with not all travelling communities sharing the same values or practices. 20 24–28

The three themes do not exist in isolation, nor are they static entities. This relationship is conceptualised in figure 2 . Each pair of the three themes connect, for example the wish to die at home could be seen as a barrier to non-community healthcare if there is limited awareness of services that can help facilitate this, such as hospice at home. Health beliefs and family values cannot exist without the other; the importance of caring for or being with a dying relative is inherently linked to a cultural belief system of right and wrong behaviours. Death rituals and the community view of a hospice as a last resort could prevent a patient and their family from accessing hospice support. The encompassing square of the wider society represents the influence of external factors on the community culture, such as the change in tradition of ritualistic burning over time, or the influence that the healthcare setting or professional has on future engagement of a travelling community with non-community healthcare.

Theme Integration.

This review identified three interconnected themes underpinning palliative and end of life care within travelling communities. Travelling communities have strong family and community values, often preferring to die at home, with care being provided by extended family. Distinct health beliefs include not discussing cancer or death and dying and the importance placed on hope. Illiteracy, lack of knowledge of medical terminology and communication issues also present barriers to non-community healthcare. Healthcare professionals’ knowledge of, or inappropriate professional attitude towards travellers, can be a further barrier. These beliefs, values and barriers were dynamic entities, influenced by varied and distinct community cultures.

There is known inequity in the provision of palliative care in the UK with barriers to accessing palliative care frequently identified. 33 The findings of this review echo those found by Evans et al 15 in black and minority ethnic groups in the UK, with common themes including families preferring to care for their relative at home and community negativity if this does not occur, hospital care being associated with hope, reduced awareness of palliative care services, communication problems, conflicts regarding diagnosis disclosure and healthcare professional concern regarding a need for training. 15 Evans et al acknowledge that their review did not identify any articles regarding white minority ethnic groups in the UK and suggested that this echoes a genuine lack of research in the field.

Other than the studies analysed in this review, there is little evidence available regarding the palliative and end of life care experiences, views and needs of travelling communities. Okely’s book ‘The Traveller-Gypsies’ published in 1983 and based on an ethnographic study, presented a chapter on end of life in the Gypsy community. 34 As inclusion criteria for this review did not include books, this was not included in thematic analysis. There are some similarities between Okely’s book and the findings of this review, but also some key differences such as the preferred place of death, which Okely stated was not at home. This could represent the change in values and practices overtime, with the fieldwork for Okely’s study carried out in the 1970s, compared with 2003 onwards for studies included in this review.

All three themes point to reduced access to palliative care services in travelling communities. This finding is consistent with the candidacy framework which describes access to healthcare by vulnerable groups. 35 Candidacy is a construct which describes how a vulnerable person’s eligibility for healthcare is negotiated between individuals and health services; it is subject to influences from the individual’s social context and from broader factors such as allocation of resources, policy and the healthcare setting. In the context of palliative care in travelling communities, the development of local resources to address the needs of travellers and the building of trusted relationships with primary care professionals, such as General Practitioners, is likely to be vital in facilitating appropriate awareness of and access to specialist services.

Strengths and limitations of the review

The main strength of this review is that it is broad and comprehensive, including extensive evaluation of grey literature, given the limited publication of relevant literature in the conventional electronic databases. Data extraction and analysis were systematic, with an inductive coding framework and second author checking at each stage, reducing author bias in assignment of codes and themes. Thematic analysis allowed for trends across all studies to be identified and only using codes from studies with a low WoE D grade where they supported findings from other studies, further improved the validity of the findings. Although the search strategy was not restricted by language, studies were excluded if an English full text was unavailable. One Spanish study was excluded for this reason. 36

Generalisability of the findings was limited by study heterogeneity. Only two studies involved male participants, both providing only a low weight of evidence on Gough’s scale. 26–30 32 The lack of male participants was stated as being due to either failure to recruit male participants where their involvement had been planned or due to local advisory groups suggesting that male participants would be unlikely to engage in conversation with female researchers. 20 21 24 25

Implications for clinical practice and policy

Although this review identified similarities in palliative and end of life care experiences, views and needs within Gypsy, Traveller and Roma communities, it also identified key differences between and within them. Policies must therefore avoid blanket statements about healthcare for Gypsy, Traveller and Roma communities, given the disparate needs of each community. Healthcare professionals reported inadequate training in how to look after patients from travelling communities. 24 25 However, it is not practical to give training to healthcare professionals for every minority ethnic group, culture or religion. Instead, the most important aspect is that healthcare professionals recognise that there are a multitude of factors affecting patient preferences for palliative and end of life care and only individual discussion with the patient and their family can lead to patient-centred, personalised care.

The need for there to be a good relationship between the community and professionals and the importance of the involvement of a known and trusted, respectful professional to enable a positive experience of non-community healthcare was clear. 21–25 30 Travellers and healthcare professionals need to work together to design culturally sensitive health services. Simple steps may be highly valued, for example providing an external door to a hospice room for extended family members to use. 22 Specific liaison roles could facilitate suitable healthcare provision. It has been suggested that travellers may find it more appropriate to discuss concerns such as bereavement, with another traveller who has been given appropriate training. 21 24 25 Local advocates for travelling communities, whether travellers themselves or healthcare professionals with a specialist interest, could provide significant benefit by acting as a liaison between patients and healthcare professionals. Roma Health Mediators, who are members of the Romani community, in countries such as Romania, Bulgaria and Slovakia have, for example, contributed to improving access to healthcare services by facilitating interaction between Romani patients and medical professionals. 37

Involvement of primary care services is likely to be valuable because of existing trusted relationships. Generalist palliative care may preclude the need for specialist services. A trusted primary care professional, already known to the community and able to provide continuity of care is likely to be invaluable in supporting patients from travelling communities at the end of life. This is in keeping with WHO recommendations that, as many patients prefer to remain at home for end of life care, community palliative care should be integrated into primary care services worldwide. 38

Implications for research

A series of qualitative studies are needed to gain insight into the exact and varied health needs of each travelling community. Research involving cocreation of acceptable services, with key stakeholders within these communities, will be vital. Such services could then be evaluated using outcomes of relevance to each community. Caution would be needed to avoid inappropriate extrapolation of results across different communities. Service design research could include investigating how to optimise selection and training of people in liaison roles. Understanding healthcare professionals’ experiences of caring for people from travelling communities could also provide insights into where service changes could be made.

Given the propensity of female participants, it is vital to determine the experiences, views and needs of men. Considering the cultural sensitivities, carefully designed research with extensive stakeholder consultation will be a prerequisite. Only studies from the UK, Ireland, Spain and Romania were identified. Further investigation of palliative and end of life care within itinerant groups outside of Europe is warranted.

This review has identified a number of shared palliative and end of life care experiences, views and needs across Gypsy, Traveller and Roma communities. Many values, healthcare beliefs and practical factors act to hinder access to palliative care that meets the needs of these communities. Furthermore, the communities are diverse, requiring highly personalised care that addresses individuals’ specific cultural needs.

Successful engagement between health services and patients from these communities requires respectful relationships and trust between the two groups, which can be facilitated by individuals providing mutual liaison. Primary care staff, who are able to build a rapport with their local travelling community, may be ideally placed to coordinate and deliver culturally appropriate palliative care that meets the unique needs of these communities.

Ethics statements

Patient consent for publication.

Not required.

• European Commission
• Council of Europe
• Friends Families and Travellers
• Gilbert E ,
• Ennis S , et al
• European Union Agency for Fundamental Rights
• Spencer S ,
• Greenfield M ,
• Burnett S , et al
• Aspinall PJ
• NHS England
• McFadden A ,
• Siebelt L ,
• Gavine A , et al
• World Health Organization
• Farmer PE ,
• Krakauer EL , et al
• Care Quality Commission
• Andrew EVW , et al
• Ouzzani M ,
• Hammady H ,
• Fedorowicz Z , et al
• McDonagh C ,
• McCormack M
• Griffiths F ,
• McQuillan R ,
• Van Doorslaer O
• Van Doorslaer O ,
• McQuillan R
• Pârvu A , et al
• Enache A , et al
• Peinado-Gorlat P ,
• Castro-Martínez FJ ,
• Arriba-Marcos B , et al
• Garcia-Espinel T ,
• Redondo-Sama G , et al
• Matosevic T ,
• Dixon-Woods M ,
• Agarwal S , et al
• Valera MAB ,
• Lezcano CM ,
• Aguarrón MJ
• Naydenova V

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

• Data supplement 1
• Data supplement 2

Correction notice This article has been updated since it was first published. The article type has been changed to Systematic review.

Contributors Search strategy was developed by KCD and IK. KCD and RF independently screened titles and abstracts and appraised the quality of included studies. KCD coded the data and developed descriptive themes. Coding was checked by RF and themes agreed by discussion with RF and AS. KCD wrote the final manuscript with supervision, comments and review from AS and SB.

Funding KCD is an Academic Clinical Fellow funded by Health Education East of England. SB is supported by the National Institute for Health Research (NIHR) Applied Research Collaboration East of England (ARC EoE) programme. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF:

• Open access
• Published: 05 May 2024

Exoskeleton-based exercises for overground gait and balance rehabilitation in spinal cord injury: a systematic review of dose and dosage parameters

• Patrik Nepomuceno 1 , 2 , 5 ,
• Wagner H. Souza 1 ,
• Maureen Pakosh 1 ,
• Kristin E. Musselman 1 , 3 , 6 &
• B. Catharine Craven 1 , 4 , 5 , 6  

Journal of NeuroEngineering and Rehabilitation volume  21 , Article number:  73 ( 2024 ) Cite this article

Metrics details

Exoskeletons are increasingly applied during overground gait and balance rehabilitation following neurological impairment, although optimal parameters for specific indications are yet to be established.

This systematic review aimed to identify dose and dosage of exoskeleton-based therapy protocols for overground locomotor training in spinal cord injury/disease.

A systematic review was conducted in accordance with the Preferred Reporting Items Systematic Reviews and Meta-Analyses guidelines. A literature search was performed using the CINAHL Complete, Embase, Emcare Nursing, Medline ALL, and Web of Science databases. Studies in adults with subacute and/or chronic spinal cord injury/disease were included if they reported (1) dose (e.g., single session duration and total number of sessions) and dosage (e.g., frequency of sessions/week and total duration of intervention) parameters, and (2) at least one gait and/or balance outcome measure.

Of 2,108 studies identified, after removing duplicates and filtering for inclusion, 19 were selected and dose, dosage and efficacy were abstracted. Data revealed a great heterogeneity in dose, dosage, and indications, with overall recommendation of 60-min sessions delivered 3 times a week, for 9 weeks in 27 sessions. Specific protocols were also identified for functional restoration (60-min, 3 times a week, for 8 weeks/24 sessions) and cardiorespiratory rehabilitation (60-min, 3 times a week, for 12 weeks/36 sessions).

This review provides evidence-based best practice recommendations for overground exoskeleton training among individuals with spinal cord injury/disease based on individual therapeutic goals – functional restoration or cardiorespiratory rehabilitation. There is a need for structured exoskeleton clinical translation studies based on standardized methods and common therapeutic outcomes.

Introduction

Over the past decade, lower limb robotic technologies have been increasingly applied in neurorehabilitation [ 1 , 2 ]. Essentially anthropomorphic in concept, these powered mechanical devices are used for locomotor training [ 3 ] and are classified as end-effectors or exoskeletons [ 4 ]. The first one generates movements from the distal segment through a haptic interface [ 5 ], while the latter encompass independent robot joints guided in a pre-programmed trajectory which is further classified as unilateral or bilateral [ 4 ]. Among such technologies, exoskeletons are reportedly useful to promote mobility in individuals with locomotor dysfunction, including those with complete lower extremity paralysis [ 6 ]. Exoskeletons often do so through motorized actuators that assist hip, knee, and ankle motion in dynamic orthoses capable of supporting, stabilizing and reciprocally progressing the lower limbs [ 4 ]. Newer generation devices offer training modes which allow therapists to manually trigger and control steps, in addition to adaptive and variable assistive features for individuals with incomplete injuries and a fair prognosis for voluntary active movement and functional recovery.

More recently, an alternative robotic exoskeleton classification was suggested based on four categories: end-effectors (e.g., Haptic Walker), grounded exoskeletons (e.g., Lokomat), wearable exoskeletons (e.g., Ekso and ReWalk) and soft exoskeletons (e.g., Myosuit) [ 7 ]. These devices seem especially promising as strategies to improve balance and walking abilities [ 8 , 9 ], two of the most frequent goals following subacute or chronic spinal cord injury/disease (SCI/D) [ 7 , 10 ]. The first, characterized by physiologic responses at a cellular level (e.g., glial scars), occurs within a few weeks after the injury [ 11 , 12 , 13 ]. Conversely, the latter is achieved as of 6 months after the injury. In traumatic SCI, the interval between the acute (< 30 days) and chronic (> 6 months) phases has been labelled the intermediate phase [ 14 ].

In terms of motor support, exoskeletons offer different types of assistance including active (equipment performs the movement, partially or totally, through powered assistance to the user); passive (device does not offer powered assistance to the movement, users execute by themselves); active-assisted (offers powered assistance to complete movements initiated by the user); resistive (offers resistance to movements initiated by the user); and interactive (uses feedback to correct movements based on interactions between actuators and control strategies) [ 4 , 7 , 15 ]. Understanding these different levels of assistance is important to account for the variable forms of haptic feedback involved in robotic motor training which can either enhance or degrade motor performance depending on the patient’s impairments and abilities (e.g., novice learners vs. advanced learners, subacute vs. chronic patients, those with autonomic or sensory function, presence or absence of spasticity, etc.) [ 16 ]. Prior publications with these variable assistive devices have shown that gait and balance training with exoskeletons contribute to increased energy expenditure, muscle activation/recruitment and weight bearing [ 17 , 18 , 19 , 20 ], in addition to improved independence and health-related quality of life [ 21 ]. These outcomes are often achieved in response to neurorecovery fostered by functional restoration programs[ 22 ]. Functional Restoration interventions focus on the refinement of sensorimotor function in daily living. That ability is associated with the stimulation of remaining neural connections that even in SCI/D re-enable sensorimotor function following repeated exposure to directed stimuli, hence yielding [ 23 ] greater motor and autonomic recovery [ 23 , 24 ].

Specific to SCI/D, a recent study of exoskeleton-based rehabilitation among individuals with subacute injury reported that exposure to sixteen 30-min sessions of robotic-assisted gait training led to a significant improvement in gait as measured by the Walking Index SCI II (WISCI-II), which translates to more functional gait and activities of daily living [ 25 ]. Moreover, Tamburella et al. [ 26 ] reported that individuals living with SCI/D could walk significantly faster, with longer steps and reduced gait cycles after rehabilitation with a powered exoskeleton. Similarly, Okawara et al. [ 27 ] reported gains in the 10-Meter Walk Test (10MWT), Time Up and Go (TUG) and Berg Balance Scale (BBS) after twenty 60-min sessions of body weight supported treadmill training (BWSTT) with a hybrid-assisted limb system. These results, however, were only observed in SCI/D patients with prior high walking ability as measured by the WISCI-II. In a similar population, Baunsgaard et al. [ 28 ] performed twenty-four 60-min sessions of robotic exoskeleton gait training, which resulted in improvements in the 10MWT, TUG and BBS, however with no treadmill or body weight support. The aforementioned results suggest that individuals living with subacute spinal cord lesions (< 1 year) are most likely to experience therapeutic benefits. However, individuals living with chronic SCI/D may also benefit from these interventions. While neuroplasticity is primarily expected at earlier phases after SCI/D, improvements are still attainable at later stages, specifically in response to coordinated, repeated motor stimuli as fostered by exoskeletons [ 4 , 17 , 28 ].

In response to the growing interest in exoskeletons to enhance the outcomes of neurorehabilitation, particularly in SCI/D, a significant body of literature has been published on associated topics and therapeutic benefits such as cardiovascular function [ 19 ], gait performance and training [ 17 , 19 ], spasticity and pain [ 18 ], device characteristics [ 29 ], cardiorespiratory function and fatigue [ 30 ]. Although the aforementioned evidence is based on structured rehabilitation protocols, little emphasis has been given to discussing dose and dosage parameters of the exercises used in the respective therapeutic protocols beyond feasibility, safety and the specific outcomes observed. Additionally, interventions using powered exoskeleton-based rehabilitation for gait and balance were reportedly delivered under widely variable designs [ 31 , 32 , 33 , 34 ]. Although dose and dosage parameters were reported by previous systematic review authors in adults with SCI/D who underwent lower limb powered exoskeleton rehabilitation for overground gait and balance, most did not discuss these training parameters. Instead, most authors acknowledged the absence of best practice recommendations in the field and endorsed the need to further understand rehabilitation designs aimed to restore or maintain locomotion with powered exoskeletons [ 7 , 15 , 21 , 35 ].

This systematic review addresses two main questions: (1) To what extent are dose (e.g., single session duration, and total number of sessions) and dosage (e.g., frequency of sessions per week, and total duration of the intervention) of exoskeleton-based exercises reported in the literature on overground gait and balance rehabilitation for adults with SCI/D (subacute or chronic, complete or incomplete)?; and, (2) Which outcome measures are used to inform changes in gait and balance following exoskeleton-based rehabilitation in SCI/D? We hypothesized that the investigation of dose and dosage parameters of exoskeleton-based exercises reported from interventions for overground gait and balance rehabilitation interventions among individuals with SCI/D would contribute to: (1) the identification of consistent dose and dosage parameters to inform best practice recommendations related to locomotor rehabilitation strategies; and, (2) informing the development of innovative, clinically robust protocols evaluating exoskeletons for SCI/D rehabilitation; and, (3) to driving implementation of exoskeleton based training programs within tertiary SCI/D rehabilitation settings.

This systematic review was conducted in accordance with the Preferred Reporting Items Systematic Reviews and Meta-Analyses (PRISMA) guidelines [ 36 ] and registered in the International Prospective Register of Systematic Reviews (PROSPERO) under the number CRD42022319271.

Search strategy and data sources

The search strategy was co-developed by the authors in collaboration with a local Medical Librarian and Information Specialist (MP) using the concepts contained in the PICO framework encompassing P opulation, I ntervention, C omparisons, and O utcomes. Valid subject headings for each database were utilized as appropriate, as were free text terms pertinent to each topic or concept (e.g., Spinal Cord Injuries; Paraplegia; Quadriplegia; Exoskeleton Device; Gait; Postural Balance). The search was performed from inception to 31 March 2022 using five electronic databases: CINAHL Complete (EBSCOhost), Embase (Ovid), Emcare Nursing (Ovid), Medline ALL (Ovid; includes PubMed non-Medline records), and the Web of Science Core Collection. Each concept searched was kept as broad as possible to ensure all relevant materials were identified. The Population encompassed adults with Spinal Cord Injuries. The Intervention was the use of Exoskeletons. The Outcomes included any biomechanical and/or clinical measures related to Gait or Balance. No date or language limits were applied. The full Medline search strategy is shown in Additional file 1 .

Study selection criteria

Studies were included according to the following criteria:

Participants: adults regardless of sex/gender identity (≥ 16 years of age) with subacute/chronic (≥ 30 days post injury onset) complete or incomplete SCI/D of traumatic or non-traumatic etiology; and any neurological level of injury (C1-L4 ASIA Impairment Scale A-D).

Intervention/Exposure: overground gait and balance rehabilitation with a lower limb powered exoskeleton – an anthropomorphic device worn by the participants for orthostatic passive or active (facilitated) motor training [ 3 ].

Comparison: no specific rehabilitation strategy was specified for comparison.

Outcomes: studies which included at least 3 of 4 parameters of dose (e.g., single session duration, and total number of sessions) and dosage (e.g., frequency of sessions per week, and total duration of the intervention) of exoskeleton-based exercises; and at least one measure of gait and/or balance (e.g., Mini-Balance Evaluation Systems Test, Community Balance & Mobility Scale, ABC Scale, 6-min walk test (6MWT), 10MWT or other measure of gait speed, BBS, TUG).

Publication type: Experimental studies with more than five participants in randomized clinical trials, quasi-randomized clinical trials, prospective controlled trials, pre-post studies, cross-sectional, crossover and quasi-experimental studies. Studies with mixed populations (e.g., children and adults) or mixed impairments (e. g., SCI/D, stroke, multiple sclerosis), were included when outcome separation was possible. Only peer-reviewed articles were included. Reasons for exclusion included: literature reviews, qualitative studies, case series (n < 5), grey literature (i.e., letters, editorial, white papers), studies with end-effector or grounded systems, equipment design and development studies, and with gait training carried over specialized surfaces (e.g., treadmill). The inclusion and exclusion criteria are listed in the Table  1 .

Screening criteria and study selection

After the initial search, duplicate manuscripts were excluded, and remaining references were imported into the Covidence Systematic Review Manager (Veritas Health Innovation Ltd, Australia). Articles eligible for title and abstract screening were assessed by PN and WHS independently (a third author, KEM, was assigned to resolve eventual conflicts). Prior to working independently, an initial fidelity agreement regarding the article inclusion/exclusion process was established based on the first 10 studies with a 100% agreement between raters. If titles and abstracts did not report enough information to determine article inclusion or exclusion, the full text was screened. Following the title and abstract screening, remaining citations were independently read in full by the same two authors to verify articles met inclusion criteria. Again, disagreements were resolved by the same third author.

Data charting and analysis

The authors created individual versions of a data extraction form. Their forms were compared and merged into a combined form used to abstract data from the included manuscripts. The data extraction form was pilot tested by two authors (PN and WHS), who independently extracted data from two of the included manuscripts. Following a comparison of the outcomes obtained, minor revisions were implemented towards a final, revised version of the abstraction form.

Data were extracted from the selected papers about authors; year of publication; institution and country of the study; participant demographics (age, number of participants, etiology and level of lesion,); dose (e.g., total number of sessions, and duration of each session, in minutes) and dosage (e.g., frequency of sessions per week, and duration of the complete intervention, in weeks); gait and balance outcomes measures (e.g., Mini-Balance Evaluation Systems Test, Community Balance & Mobility Scale, ABC Scale, 6MWT, 10MWT, BBS, TUG, gait speed). The data were synthesised by the authors and reported in tables and graphics. Narrative syntheses were applied.

In the case of articles with missing data (e.g., total duration of intervention), the corresponding author was contacted by e-mail. For some studies included, dose and dosage parameters were not explicitly stated, but could be estimated using available training parameters in the published article. For instance, sessions per week multiplied by the number of intervention weeks informed the total number of sessions; total number of sessions divided by weeks informed weekly frequency; and total number of sessions divided by sessions per week informed the duration of the intervention. For parameters indicated as best practice recommendations, only studies that reported statistically significant improvements (p < 0.05) and/or improvements equal or greater than the minimal clinically important difference (MCID) were considered. The MCID was observed for the 6MWT, 10MWT and TUG, with the following thresholds: 36 m [ 37 ] or 0.1 m/s [ 38 ], 0.13 m/s [ 39 ], and 10.8 s [ 40 ], respectively. For the cardiorespiratory outcomes, no MCID was set, and only statistically significant improvements (p < 0.05) were considered. Conversely, studies with dramatically large variability within the reported protocol (e.g., participants exposed to a different total number of sessions from 12 to 102, duration of intervention from 4 to 34 weeks) were excluded from the average calculation. As for studies with small variability within the protocol, the mean of the total range (e.g., weekly frequency from 4 to 5, was considered as 4.5; duration of each session from 60 to 90-min, was considered 75-min) were computed. Data regarding dose and dosage parameters were reported as mean and standard deviation (normal distribution) or median and interquartile range (non-normal distribution), to determine distribution the Shapiro–Wilk Test was used considering p < 0.05 as non-normal distribution.

The initial electronic database search identified 2,108 references. After removing the duplicates, 977 references were screened for titles and abstracts. At full text screening, 69 articles were revised (Fig.  1 ). Nineteen (n = 19) full text articles were included in the review with a total of 288 participants (214 male) who underwent exoskeleton gait and/or balance training. Five (n = 5) studies had control/comparison groups treated with conventional physical therapy (n = 2) [ 41 , 42 ], Lokomat gait training (n = 1) [ 43 ], BWSTT or no intervention (n = 1) [ 44 ] and BWSTT with overground gait training with functional electrical stimulation (FES) (n = 1) [ 45 ]. One (n = 1) study had a comparison group of individuals with acute SCI/D who underwent the same exoskeleton protocol [ 46 ]. As for the geographical distribution of study sites, five (n = 5) were developed in the United States [ 6 , 33 , 44 , 47 , 48 ], four (n = 4) in Italy [ 43 , 45 , 49 , 50 ], two (n = 2) in Canada [ 51 , 52 ], two (n = 2) in China [ 41 , 53 ] and two (n = 2) in Korea [ 54 , 55 ], one (n = 1) in France [ 56 ], one (n = 1) in Japan [ 46 ], one (n = 1) in South Africa [ 42 ], and one (n = 1) from a 7 site (Denmark, Germany, the Netherlands, Norway, Spain, Sweden and Switzerland) multicenter study in Europe [ 28 ], Fig.  2 displays the countries of origin for 18 studies, except for the multicenter study in Europe, which is the most active region investigating overground exoskeletons training for gait and balance rehabilitation among individuals with SCI/D. Six (n = 6) studies were partially or totally supported by the industry manufacturer, including equipment loan [ 6 , 57 ], trial funding [ 28 , 44 , 56 ] and employees collaborating in manuscript production [ 54 ].

PRISMA flow diagram

Frequency of study per country. Figure represents the country of origin of 18 of the 19 studies included because 1 study was a multicenter study across Europe

The refined dataset included articles describing participants with subacute (1 to 5 months post-injury) or chronic (> 6- or 12-months post-injury) SCI/D. Thirteen (n = 13) studies investigated chronic SCI/D (> 6 months [ 46 , 51 ], > 12 months [ 6 , 42 , 43 , 44 , 48 , 52 , 54 , 55 , 56 , 57 ], stated it is chronic but did not report time since injury[ 50 ]), one study investigated subacute participants (from 1 to 11 months) [ 41 ] and five studies investigated both subacute and chronic participants [ 28 , 45 , 47 , 49 , 53 ]. The participant’s age ranged from 16 to 78 years, although one study included one participant that was 15 years old, however this paper was not included in our best practice recommendation because the authors did not find significant changes. That study, however, had a mean participant age of 41.3 years [ 53 ]. Regarding the etiology of the injury, nine (n = 9) studies included individuals with SCI/D of traumatic and non-traumatic etiology (four chronic [ 6 , 43 , 44 , 51 ], four chronic and subacute [ 28 , 45 , 53 , 57 ], one subacute only [ 41 ]). Five (n = 5) studies only included individuals with traumatic lesions (four chronic [ 42 , 48 , 52 , 56 ] and one subacute and chronic [ 49 ]). One (n = 1) study focused on chronic non-traumatic participants [ 46 ]. Four (n = 4) studies did not report the etiology (three chronic [ 50 , 54 , 55 ], one chronic and one subacute [ 47 ]). As for the extent of injury, twelve (n = 12) studies were conducted in individuals with complete or incomplete SCI/D [ 28 , 41 , 43 , 45 , 47 , 48 , 50 , 51 , 52 , 53 , 54 , 57 ], four (n = 4) studies in complete SCI/D only [ 6 , 49 , 55 , 56 ] and two (n = 2) studies in incomplete SCI/D [ 42 , 44 ]. One study did not report the extent of participant injury [ 46 ]. Relative to the level of injury, one (n = 1) study included individuals with cervical lesions [ 42 ], four (n = 4) included individuals with thoracic lesions [ 6 , 48 , 55 , 56 ], four (n = 4) included individuals with cervical or thoracic lesions [ 43 , 51 , 52 , 57 ], two (n = 2) included cervical, thoracic, or lumbar [ 44 , 54 ], and six (n = 6) studies included thoracic or lumbar injuries [ 41 , 45 , 47 , 49 , 50 , 53 ]. Two (n = 2) studies did not report the level of injury [ 28 , 46 ]. A summary of participants’ characteristics and the exoskeleton device with their respective study protocols are shown in Table  2 .

Exoskeleton training dose, dosage, and outcome measures

The 19 studies included devices from seven different exoskeleton manufacturers. Seven (n = 7) studies used Ekso devices [ 28 , 42 , 43 , 44 , 45 , 50 , 51 ], six (n = 6) used ReWalk [ 6 , 48 , 49 , 52 , 55 , 57 ], two (n = 2) used AIDER [ 41 , 53 ], one (n = 1) used Indego [ 47 ], one (n = 1) H-MEX[ 54 ], one (n = 1) Hybrid Assistive Limb (HAL) [ 46 ] and one (n = 1) Atalante [ 56 ]. In 15 studies, the rehabilitation protocol included only exoskeleton gait and/or balance training [ 6 , 28 , 41 , 42 , 43 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 56 , 57 ]. Four studies included exoskeleton training associated with overground walking without body weight supported (BWS) [ 44 ], FES cycling [ 45 ], BWS [ 46 ], or knee-ankle–foot orthosis (KAFO) gait training [ 55 ]. In respect to the dose and dosage parameters, the total number of sessions reported ranged from 10 to 102 sessions. The number of sessions per week varied from 2 to 5 sessions. The duration of the total intervention ranged from 2 to 34 weeks. The duration of each gait and balance exoskeleton gait training varied from 30 to 90-min (one paper did not report [ 28 ]). The most frequent dose and dosage parameters were: 60-min sessions [ 42 , 43 , 46 , 48 , 49 , 51 , 52 , 54 , 56 ], 3 sessions a week [ 6 , 28 , 42 , 44 , 45 , 47 , 48 , 49 , 51 , 54 , 57 ], over 8 to 12 weeks [ 6 , 28 , 43 , 44 , 47 , 49 , 52 , 54 ], for a total of 20–40 sessions [ 6 , 28 , 44 , 45 , 47 , 49 , 50 , 54 , 55 ].

Overall, considering the dose and dosage parameter averages across all studies included in this review, regardless of clinically relevant change, a protocol with 60-min individual sessions, 3 times a week, for 9 weeks is suggested for a total of 27 sessions. The mean and standard deviation, or median and interquartile range for overall interventions and for protocols focused on specific therapeutic intent (e.g., functional restoration or cardiorespiratory rehabilitation) are described in Table  3 . As for the total number of sessions and the duration of interventions recommended, most studies showed variability within a range of (24–36 sessions) and (8–12 weeks), respectively [ 6 , 28 , 44 , 47 , 49 , 51 , 52 , 54 ]. Also, the duration of each session (60-min) and weekly frequency (3 times a week) were mostly consistent across the reviewed dataset, including studies with clinically relevant changes [ 6 , 28 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 51 , 52 , 54 , 56 ].

The gait and balance outcome measures used include: the 6MWT [ 6 , 41 , 42 , 44 , 45 , 47 , 48 , 49 , 50 , 52 , 53 , 54 , 55 , 57 ], 10MWT [ 6 , 28 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 56 , 57 ], TUG [ 28 , 44 , 45 , 47 , 50 , 57 ], WISCI-II [ 28 , 44 , 45 , 53 ], gait speed [ 43 , 46 , 50 , 52 ], steps taken [ 46 , 51 , 52 ], BBS [ 28 ], step length [ 46 ], stride length [ 50 ], Hoffer Walking Ability [ 53 ], and one paper adapted the 6MWT to 30-min walk test to evaluate gait function during 30-min [ 55 ], the frequency of the gait and balance outcomes across the studies is indicated in the Fig.  3 . Other non-gait related measures reported as main outcomes across different studies were categorized as either cardiorespiratory or physiologic outcomes and are listed in Fig.  3 .

Frequency of clinical outcomes reported. Heat map presenting the frequency of clinical outcomes measures reported, per studies by manufacturer. %HRR percentage of heart rate reserve, CO2 carbon dioxide, FEF forced expiratory flow, FEV1 forced expiratory volume in 1 s; LEMS Lower Extremities Motor Score, MVV maximum voluntary ventilation, NASA-TLX NASA Task Load Index, NBD neurogenic bladder dysfunction, PCI Physiological Cost Index, PEF peak expiratory flow, PGI-I Patient Global Impression of Improvement; Resp. respiratory, RPE rating of perceived exertion, SCATS Spinal Cord Assessment Tool for Spastic Reflexes, SCIM-II Spinal Cord Independence Measure II, UEMS Upper Extremities Motor Score, VAS Visual Analogue Scale, WISCI-II Walking Index for Spinal Cord Injury II

Protocol therapeutic intent

The studies included in this systematic review of overground exoskeleton training dose and dosage were classified in two groups according to the inferred therapeutic intent based on the described study design which addressed: functional restoration [ 6 , 44 , 45 , 46 , 47 , 49 , 50 , 51 , 52 , 53 , 56 , 57 , 58 ] or cardiorespiratory rehabilitation [ 41 , 42 , 43 , 48 , 54 , 55 ]. The therapeutic intent was determined based on each study’s primary research question, aim and main outcome measures in reference to motor (gait or balance) or cardiorespiratory performance, respectively. Although in recent years changes in body composition (e.g., muscle and bone mineral density) have been increasingly associated with exoskeleton training [ 31 , 59 , 60 , 61 ], none of the studies included in this review focused on anatomical adaptations in response to overground exoskeleton training.

Functional restoration

Thirteen (n = 13) studies focused on functional restoration [ 6 , 44 , 45 , 46 , 47 , 49 , 50 , 51 , 52 , 53 , 56 , 57 , 58 ]. Of those, eleven reported statistically significant improvements and/or showed improvements equal or higher than the MCID for gait and/or balance outcome measures [ 6 , 28 , 44 , 45 , 46 , 47 , 49 , 50 , 51 , 52 , 56 ]. Table 4 summarizes the individual studies’ aims and main results. The functional restoration protocols ranged from 10 to 51.5 sessions, 2 to 5 sessions a week, 3 to 12 weeks of duration for 45- to 90-min. Considering the studies with significant motor improvement (n = 11), it is suggested that a protocol aimed towards functional restoration would encompass 60-min individual sessions carried 3 times a week, over 8 weeks for a total of 24 sessions (Table  3 ).

Functional restoration interventions were shorter than cardiorespiratory interventions. They included subacute or chronic SCI/D patients, mostly with complete or incomplete thoracolumbar lesions. In this group analysis, two manuscripts did not report improvements [ 53 , 57 ]. The first one [ 53 ] reported the effects of a new robotic exoskeleton based on ten 30-min sessions over 2 weeks, that is shorter than the period suggested by our recommendation based on studies with significant functional restoration gains. The second study [ 57 ] focused on describing the protocol performed in a rehabilitation research institute, including the process of participant recruitment, fitting, donning, standing, standing balance, walking, mobility training, sitting and doffing. The functional outcomes, however, were measured only after the intervention. Additionally, among the respective study participants, individuals underwent 12 to 102 sessions over 4 to 34 weeks in remarkably variable study designs.

Regarding the therapeutic content, studies on functional restoration mainly focused on sit to stand, and stand to sit transitions, standing balance and walking training for significant changes or improvements above the MCID as per functional restoration outcome measures. The frequency of training, total number of trainings, therapy content (exercise training) and studies with significant changes are shown in Table  5 .A.

Cardiorespiratory rehabilitation

The six studies (n = 6) focused on cardiorespiratory rehabilitation [ 41 , 42 , 43 , 48 , 54 , 55 ] showed significant improvement of cardiorespiratory function. Table 6 summarizes the cardiorespiratory studies’ aims and main results. Cardiorespiratory-centered interventions ranged from 16 to 72 sessions, 2 to 5 sessions weekly, for 4 to 24 weeks. Individual sessions lasted between 55 to 90 min. Because the six protocols yielded significant improvement in cardiorespiratory function, it is suggested that interventions to that end are likely to succeed when based on 60-min sessions carried 3 times a week for 12 weeks in a total of 36 sessions (Table  3 ).

Unexpectedly, protocols focusing on cardiorespiratory outcomes were longer in average than protocols for functional restoration. Conversely, four of the referred articles also reported significant improvements in gait and balance measures [ 42 , 48 , 54 , 55 ], while two manuscripts reported improvements in cardiorespiratory outcomes alone [ 41 , 43 ]. The latter studies were based on 16 sessions over 4 weeks [ 41 ] and 17 sessions over 9 weeks [ 43 ], indicating that shorter interventions could be enough to improve cardiorespiratory function alone, that is uncoupled from significant functional restoration. In this case, the cardiorespiratory recommendation would include 60-min sessions carried out 3 times a week for 6 weeks for a total of 18 sessions.

Regarding the therapeutic content, studies on cardiorespiratory rehabilitation mainly focused on walking training and sit to stand and/or stand to sit transitions for significant improvements in cardiorespiratory outcome measures. The frequency of training, total number of trainings, therapy content and studies with significant changes in cardiorespiratory parameters are shown in Table  5 .B.

This review aimed to identify the dose and dosage parameters of exoskeleton-based exercises for overground gait and balance training in individuals with SCI/D. Although previous studies have discussed this topic in different neurological populations [ 7 , 62 ], to the best of our knowledge, this is the first review to prioritize the investigation and discussion of dose and dosage of overground exoskeleton therapy among individuals with SCI/D – a need repeatedly acknowledged in recent literature [ 7 , 15 , 21 , 35 ] yet widely overlooked as a primary research topic. We have summarized evidence from 19 manuscripts to determine current training parameters for specific therapeutic indications to inform best practice recommendations in exoskeleton-based SCI/D rehabilitation. Of 19 manuscripts, seventeen [ 6 , 28 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 54 , 55 , 56 ] reported statistically significant improvements and/or gains above the MCID in the gait, balance, cardiorespiratory and/or related physiological outcomes they assessed. The evidence gathered supports the assumption that exoskeletons are a promising therapeutic tool in SCI/D, particularly for functional restoration [ 6 , 28 , 44 , 45 , 46 , 47 , 49 , 50 , 51 , 52 , 56 ] and/or cardiorespiratory improvement [ 41 , 42 , 43 , 48 , 54 , 55 ].

Protocol design

Based on strict adherence to the systematic review inclusion criteria, several manuscripts initially screened did not fully report dose (total number of sessions, and duration of the session) and dosage (frequency per week, and duration of the intervention) parameters and were excluded. Consistent with previous reviews [ 35 , 62 ] on exoskeleton-based gait rehabilitation the lack of dose and dosage parameters ultimately limits the replication and generalizability of the outcomes reported. The absence of dose and dosage information also limits the translation of findings to evidence-based clinical practice, whereas the requirement for routine universal reports of dosing parameters in future studies would foster knowledge dissemination and implementation of precision rehabilitation approaches in the field. To support the development of future studies with structured information for better clinical translation, a checklist for reporting exoskeleton therapy is proposed in Table  7 .

We observed considerable inconsistency of protocols for exoskeleton-based gait and balance training, with substantial variability in dose and dosage parameters used. Indeed, the protocols ranged from 10 [ 46 ] to 102 [ 57 ] sessions over 2 to 34 weeks, two [ 53 ] to five [ 55 ] times a week, with individual sessions lasting from 30 [ 53 ] to 90 min [ 47 ]. Further, the systematic review results indicated that the weekly frequency and session duration are the most consistent parameters, with most protocols reporting 3 sessions a week [ 6 , 28 , 42 , 44 , 45 , 47 , 48 , 49 , 51 , 54 , 57 ] at an average of 60 min per session [ 42 , 43 , 46 , 48 , 49 , 51 , 52 , 54 , 56 ]. Another important variable across studies was the device used and the exoskeleton manufacturer, with Ekso (n = 7) [ 28 , 42 , 43 , 44 , 45 , 50 , 51 ] and ReWalk (n = 6) [ 6 , 48 , 49 , 52 , 55 , 57 ] being the most used devices.

Protocol effectiveness

Training effectiveness (e.g., changes in assessment values at or above the MCID) is shaped by multiple factors beyond dose and dosage, including but not limited to device parameters and the extent or intensity of training. Relative to exoskeleton-based gait and balance rehabilitation such factors include device assistance and resistance levels, different walking patterns (e.g., step and stride length, width of base of support, gait speed and step cadence) as well as exercise intensity. The latter is associated with a lack of specific consensus-based measures and definitions universally adopted by experts in the field of neurorehabilitation [ 63 ]. This is particularly true in the SCI/D populations among whom there is substantial heterogeneity in neurological impairment, and associated variability in prognosis and responsiveness to exoskeleton interventions. As a result, variability in prognosis and responsiveness are commonly observed and personalized prescriptions are provided in the absence of consensus-based terminology and practices [ 64 , 65 ]. In this context, the best practice recommendations derived from this systematic review are valid given the reporting of whether the participants achieved a clinically meaningful change in function/assessment parameter based on the dose and dosage reported despite the lack of data specifying exercise intensity.

Despite protocol variability, including that of device choice, therapeutic intent, and training intensity, it is possible that the significant changes reported are associated with the repeated exposure to active standing time versus non-active sitting time [ 65 ]. However, most studies included similar functional therapeutic activities (e.g., sit to stand transitions, standing and balance training and walking training). In fact, exercise intensity in robotic rehabilitation, although not standardized, is often associated with the number of repetitions (e.g., step count), step frequency and total walking distance. The modulation of intensity on a case-by-case basis likely favored the observed performance improvements across the multiple protocol designs reported in this review. Future studies reporting the therapeutic benefits of exoskeleton therapy should include the therapeutic indication, device choice and parameters, exercise intensity, and the dose and dosage parameters as means to improve precision rehabilitation – particularly among people living with a spinal cord impairment and multimorbidity [ 66 ].

Injury characteristics

In addition to exercise parameters, the influence of injury characteristics on exoskeleton-based SCI/D rehabilitation is very likely, yet controversial. Benson et al. [ 67 ] reported that individuals with complete injuries showed greater improvement in walking speed than incomplete injured pairs. That may be because participants with incomplete lesions were functional walkers before the beginning of their training, benefiting mostly from the ability to walk longer distances with exoskeletons as opposed to participants with complete injuries to whom exoskeletons allowed not only orthostatism, but gait initiation and speed improvements. In agreement with those findings, Xiang et al. [ 53 ] reported that individuals with higher spinal lesions and motor complete injuries showed greater improvement in gait and functional outcomes (gait speed and 6MWT) while using exoskeletons compared to people who were functional walkers with lower and or incomplete lesions. Conversely, it has been reported that adults living with lower neurological level of injury (complete versus incomplete) can achieve significantly faster walking speeds following exoskeleton training [ 6 , 68 , 69 ]. The explanation of these findings may be linked to the fact that people with complete SCI/D obtain more remarkable gains with training (e.g., from no standing to walking), although they still walk slower than individuals with incomplete lesions [ 53 , 67 ]. Differences in gait speed is possibly associated with the remaining neural pathways in individuals with incomplete lesions, which foster better neurorecovery in response to functional restoration strategies [ 2 ]. This assumption agrees with Louie et al.’s [ 17 ] report that walking speed with exoskeletons is positively correlated with the level of spinal injury (coded from 0 (cervical) to 17 (lumbar)) and training duration. Thus, lower injuries and longer training could, favor greater locomotor gains for individuals with SCI/D. Nevertheless, Sale et al. [ 50 ] reported that exoskeleton rehabilitation is safe and feasible across a heterogeneous sample of persons with SCI/D provided it is tailored to their personal needs. Further, it is plausible that there may be additional therapeutic benefits of longitudinal training not addressed in this review.

Exoskeleton-based therapeutic intent and physiological considerations

Upon review of the nineteen manuscripts included, consistent similarities across some of the protocols in terms of their therapeutic goals led us to classify the studies in two categories of therapeutic intent (e.g., functional restoration and cardiorespiratory rehabilitation). While the clinical purpose of individual studies seemed distinguishable enough for us to categorize them, that was not explicitly disclosed by the authors.

The current knowledge of the physiological mechanisms involved in exoskeleton-based therapies remains limited. A prior review reported that neurophysiological responses in exoskeleton recovery are linked to the exploitation of neuroplasticity, sensory stimulation, and coordination of limb and muscle activation during the training. The authors purport that functional restoration and neurorecovery are much like a relearning process where preserved sensorimotor and neural circuits are engaged to promote recovery [ 2 ]. For cardiorespiratory function, exoskeleton gait training’s rationale for the observed improvements in function associated with stimulation of the cardiorespiratory system and activation of the lower limbs is due to an increase in metabolic rate indicating this is an effective way of increasing energy expenditure with consequent improvements of cardiorespiratory fitness. Moreover, exoskeleton training contributes to the augmentation of end-systolic and end-diastolic volume, cardiac output, ventricular mass and reduces heart rate following cardiovascular conditioning [ 42 , 54 , 70 ].

Our findings suggest that different exercise exposures are needed to achieve MCID as per therapeutic intent in SCI/D rehabilitation, with cardiorespiratory changes demanding longer protocols compared to functional restoration. Nevertheless, we hypothesized that shorter interventions would be warranted for cardiorespiratory gains due to faster cardiovascular adaptation to structured exercises compared to neurological responses [ 71 , 72 ]. This unexpected outcome may be related to two cardiorespiratory-focused manuscripts in which participants underwent longer interventions (72 [ 42 ] and 60 [ 48 ] sessions) to evaluate changes over the time (early, mid and late changes), justifying the longer experimental designs. Additionally, of the six studies [ 41 , 42 , 43 , 48 , 54 , 55 ] included in cardiorespiratory rehabilitation, two [ 41 , 43 ] had significant improvement in cardiorespiratory function but not in gait, which was achieved with shorter interventions, in line with our initial hypothesis. Supporting our hypothesis, Faulkner et al. [ 73 ] reported that exoskeleton gait training associated with conventional physiotherapy in 5 sessions over a single week improved cardiovascular health, by reducing the augmentation index and mean arterial pressure. Further, Evans et al. [ 42 ] reported statistically significant increases in cardiovascular efficiency as early as 6 weeks after exoskeleton gait training. Interestingly, despite protocol duration variability, the six articles focused on cardiorespiratory training reported significant improvements in cardiorespiratory health as per increased oxygen consumption, heart rate and metabolic equivalent, in addition to reduced perception of effort and oxygen cost [ 41 , 42 , 43 , 48 , 54 , 55 ]. A prior systematic review reported that exoskeleton gait training elevates the energy expenditure, while allowing participants to exercise at moderate intensity, further indicating exoskeletons are beneficial for cardiorespiratory training [ 19 ].

In SCI/D, reduced lower-limb weight bearing and other health complications contribute to the loss of muscle mass and bone mineral density (BMD), specially below the level of injury [ 74 ]. This leads to an increased risk of fragility fractures, which should be accounted for when performing exoskeleton-based gait training. That is important due to previous reports of lower limb fragility fracture after exoskeleton use, mainly induced by the effect of gravity and pressure points created by the resistance of the equipment against the user’s body [ 75 , 76 ]. Thus, people living with SCI/D should be advised of their fracture risk, prior to using wearable exoskeletons for increased safety, regional improvements in bone strength and BMD [ 59 ]. To prevent fragility fractures, Bass et al. [ 59 ] developed a volume and progression algorithm based on BMD thresholds. Accordingly, individuals with osteoporotic profile (T-score ≤ -2.5) should be exposed to a slow-progression program, individuals with osteopenic profile (-2.5 < T-Score < -1.0) should start with moderate-progression and individuals with preserved BMD profile (T-Score ≥ -1.0) should be enrolled in a fast-progression walking program. It is worth noting that as per the position statement 4 in the International Society for Clinical Densitometry, there is no established threshold BMD value below which weight-bearing activities are absolutely contra-indicated, and that BMD and clinical risk factors should be used together on a case-to-case basis to assess risk exposure [ 74 ]. Furthermore, people living with SCI/D are in a higher risk of developing skin abrasions and tissue injury [ 77 ]. Many studies have reported skin abrasions after the use of exoskeleton in SCI/D population [ 44 , 53 , 56 , 58 ]. The reduction of physical activity levels, immobilization, changes in circulation and microcirculation, sensory loss, skin compression due to positioning and impaired venous return are aspects of injury that preclude individuals to lower extremity abrasions [ 77 , 78 ]. Also, participants with sensory impairment are at greater risk of developing skin lesions [ 79 ], and hence warrant ongoing screening for skin integrity. That is particularly true at points of higher pressure caused the interface between the skin and the exoskeleton [ 44 , 58 , 79 ].

Considerations for translation to practice

Recommendations from systematic reviews are extremely helpful at informing new research designs and guiding the translation of optimal evidence-based findings to clinical practice. However, it is also true that best practice recommendations, as identified by this review cannot always be implemented, particularly considering contextual disparities, including different countries (e.g., North America, Europe and Asia, Fig.  2 ), devices and therapeutic intent. Should a clinician find the implementation of the suggested best practice recommendations infeasible, reproducing the observed dose and dosage of therapy with a specific device can be limited to the shortest study with reported clinical effectiveness above the MCID for the outcome of interest (see the reduced dose and dosage but observed MCID with specific interventions on Table  5 ). For instance, ten 60-min sessions at a frequency of 5 sessions per week over two weeks yielded significant improvements in functional restoration [ 46 ]. Alternatively, sixteen 50–60-min sessions at a frequency of 4 times a week over four weeks yielded significant improvements in cardiorespiratory function [ 41 ]. We also suggest that patients be supported to work incrementally with healthcare providers to further implement best practice dose and dosage recommendations.

Study limitations

This study has limitations that include the relatively scarce literature available, which did not allow us to analyze the results according to the participant’s characteristics (sex, ASIA Impairment Scale, neurologic level of injury, etc.). However, the population described in this review are similar to those described in prior reviews among individuals living with SCI/D [ 17 , 62 ]. Also, it is important to state that the implementation of exoskeleton-based interventions is still limited due to the cost, availability of the equipment, equipment specifications and limitations, and the lack of highly trained staff to support exoskeleton-based therapy [ 80 , 81 , 82 ]. As for the limited study sample size, our search was broadened to identify manuscripts applying overground exoskeletons in SCI/D, but many of the identified references did not fully report dose and dosage – that is at least 3 parameters – and were excluded in a strategy that reduced the already restricted sample, but guaranteed data consistency. Additionally, the references included in this systematic review were classified according to their clinical intent by the review authors, which may not reflect the original authors’ intent. Furthermore, the study quality and risk of bias were not assessed as our search aimed to perform a comprehensive overview of dose and dosage in exoskeleton gait and balance training in SCI/D. Nevertheless, this systematic review is consistent with prior reports in the literature that did not report risk of bias in studies involving exoskeleton rehabilitation [ 7 , 21 , 35 , 62 ]. The exoskeleton device donning and doffing times were inconsistently reported across the reviewed studies, with only two of them [ 6 , 44 ] indicating that donning and doffing times were not part of the reported session duration and a single study [ 43 ] indicating that the session duration included donning and doffing. While we believe that some of the other sixteen studies included donning and doffing times in the session duration, we presume that most studies reported the time dedicated to standing/walking training apart from donning and doffing. Altogether, we encourage readers to implement the enclosed practice recommendations and to report device donning and doffing times, device parameters and therapeutic intensity in future reports. We also encourage clinicians and investigators to describe barriers and facilitators to implementation of best practices in different contexts.

Conclusions

In summary, this systematic review advances the understanding of overground exoskeleton-based gait and balance training in SCI/D and its role in facilitating functional recovery and or cardiorespiratory fitness. The review results provide evidence-based clinical practice recommendations, which are tailored to the therapeutic intent of the intervention. However, problems with inconsistent reporting of exoskeleton training dose and dosage and the heterogeneity of study designs among adults with SCI/D preclude fulsome dissemination of data and are acknowledged as important limitations. To advance the field of exoskeleton rehabilitation in SCI/D and increase research quality, there is an urgent need to standardize clinical practice recommendations and guidelines through well-structured studies with clear indications of their therapeutic intent. Finally, we highlight the need for multicentre studies, which could validate the therapeutic effectiveness of specific dose and dosage parameters for optimal gait and balance rehabilitation among adults with SCI/D based on poling of data from multiple sites and contexts.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

10-Meter Walk Test

6-Minute walk test

Berg Balance Scale

Bone mineral density

Body weight supported

Body weight supported treadmill training

Functional electrical stimulation

Hybrid Assistive Limb

Knee-ankle–foot orthosis

Minimal clinically important difference

Preferred Reporting Items Systematic Reviews and Meta-Analyses

International Prospective Register of Systematic Reviews

Spinal cord injury/disease

Time Up and Go

Walking Index SCI II

Reinkensmeyer DJ, Dietz V. Neurorehabilitation technology. New York: Springer; 2016.

Book   Google Scholar  

Gassert R, Dietz V. Rehabilitation robots for the treatment of sensorimotor deficits: a neurophysiological perspective. J Neuroeng Rehabil. 2018;15(1):46.

Article   PubMed   PubMed Central   Google Scholar  

Geonea ID, Tarnita D. Design and evaluation of a new exoskeleton for gait rehabilitation. Mech Sci. 2017;8(2):307–21.

Article   Google Scholar  

Molteni F, Gasperini G, Cannaviello G, Guanziroli E. Exoskeleton and end-effector robots for upper and lower limbs rehabilitation: narrative review. PM&R. 2018;10(9):S174–88.

Google Scholar  

Hesse S, Uhlenbrock D. A mechanized gait trainer for restoration of gait. J Rehabil Res Dev. 2000;37(6):701–8.

CAS   PubMed   Google Scholar  

Esquenazi A, Talaty M, Packel A, Saulino M. The rewalk powered exoskeleton to restore ambulatory function to individuals with thoracic-level motor-complete spinal cord injury. Am J Phys Med Rehabil. 2012;91(11):911–21.

Article   PubMed   Google Scholar  

Rodríguez-Fernández A, Lobo-Prat J, Font-Llagunes JM. Systematic review on wearable lower-limb exoskeletons for gait training in neuromuscular impairments. J Neuroeng Rehabil. 2021;18(1):22.

Russo M, Maggio MG, Naro A, Portaro S, Porcari B, Balletta T, et al. Can powered exoskeletons improve gait and balance in multiple sclerosis? A retrospective study. Int J Rehab Res. 2021;44(2):126–30.

Nam KY, Kim HJ, Kwon BS, Park J, Lee HJ, Yoo A. Robot-assisted gait training (Lokomat) improves walking function and activity in people with spinal cord injury: a systematic review. J Neuroeng Rehabil. 2017;14(1):24.

Scivoletto G, Tamburella F, Laurenza L, Torre M, Molinari M. Who is going to walk? A review of the factors influencing walking recovery after spinal cord injury. Front Human Neurosci. 2014;8:141.

Fehlings MG, Tetreault LA, Wilson JR, Kwon BK, Burns AS, Martin AR, et al. A clinical practice guideline for the management of acute spinal cord injury: introduction, rationale, and scope. Global Spine J. 2017;7(3 Suppl):84S-94S.

Hachem LD, Fehlings MG. Pathophysiology of spinal cord injury. Neurosurg Clin N Am. 2021;32(3):305–13.

Sandean D. Management of acute spinal cord injury: a summary of the evidence pertaining to the acute management, operative and non-operative management. World J Orthop. 2020;11(12):573–83.

Ahuja CS, Wilson JR, Nori S, Kotter M, Druschel C, Curt A, et al. Traumatic spinal cord injury. Nat Rev Dis Primers. 2017;3:17018.

Gandolfi M, Valè N, Posteraro F, Morone G, Dell’orco A, Botticelli A, et al. State of the art and challenges for the classification of studies on electromechanical and robotic devices in neurorehabilitation: a scoping review. Eur J Phys Rehabil Med. 2021;57(5):831–40.

Basalp E, Wolf P, Marchal-Crespo L. Haptic training: which types facilitate (re) learning of which motor task and for whom? Answers by a review. IEEE Trans Haptics. 2021;14(4):722–39.

Louie DR, Eng JJ, Lam T. Gait speed using powered robotic exoskeletons after spinal cord injury: a systematic review and correlational study. J Neuroeng Rehabil. 2015;12:82.

Fang C, Tsai J, Li G, Lien AS, Chang Y. Effects of robot-assisted gait training in individuals with spinal cord injury: a meta-analysis. Biomed Res Int. 2020;2020:2102785.

Duddy D, Doherty R, Connolly J, McNally S, Loughrey J, Faulkner M. The effects of powered exoskeleton gait training on cardiovascular function and gait performance: A systematic review. Sensors. 2021;21(9):3207.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Cheung EY, Ng TK, Kevin KK, Kwan RL, Cheing GL. Robot-assisted training for people with spinal cord injury: a meta-analysis. Arch Phys Med Rehabil. 2017;98(11):2320-2331.e12.

Mekki M, Delgado AD, Fry A, Putrino D, Huang V. Robotic rehabilitation and spinal cord injury: a narrative review. Neurotherapeutics. 2018;15(3):604–17.

Jacobson PB, Goody R, Lawrence M, Mueller BK, Zhang X, Hooker BA, et al. Elezanumab, a human anti-RGMa monoclonal antibody, promotes neuroprotection, neuroplasticity, and neurorecovery following a thoracic hemicompression spinal cord injury in non-human primates. Neurobiol Dis. 2021;155: 105385.

Article   CAS   PubMed   Google Scholar  

Lin A, Shaaya E, Calvert JS, Parker SR, Borton DA, Fridley JS. A review of functional restoration from spinal cord stimulation in patients with spinal cord injury. Neurospine. 2022;19(3):703–34.

James ND, McMahon SB, Field-Fote EC, Bradbury EJ. Neuromodulation in the restoration of function after spinal cord injury. Lancet Neurol. 2018;17(10):905–17.

Yildirim MA, Öneş K, Gökşenoğlu G. Early term effects of robotic assisted gait training on ambulation and functional capacity in patients with spinal cord injury. Turk J Med Sci. 2019;49(3):838–48.

Tamburella F, Tagliamonte NL, Masciullo M, Pisotta I, Arquilla M, van Asseldonk E, et al. Gait training with Achilles ankle exoskeleton in chronic incomplete spinal cord injury subjects. J Biol Regul Homeost Agents. 2020;34(5 Suppl. 3):147–64.

Okawara H, Sawada T, Matsubayashi K, Sugai K, Tsuji O, Nagoshi N, et al. Gait ability required to achieve therapeutic effect in gait and balance function with the voluntary driven exoskeleton in patients with chronic spinal cord injury: a clinical study. Spinal Cord. 2020;58(5):520–7.

Bach Baunsgaard C, Vig Nissen U, Katrin Brust A, Frotzler A, Ribeill C, Kalke Y, et al. Gait training after spinal cord injury: safety, feasibility and gait function following 8 weeks of training with the exoskeletons from Ekso Bionics. Spinal Cord. 2018;56(2):106–16.

Holanda LJ, Silva PM, Amorim TC, Lacerda MO, Simão CR, Morya E. Robotic assisted gait as a tool for rehabilitation of individuals with spinal cord injury: a systematic review. J Neuroeng Rehabil. 2017;14(1):126.

Lefeber N, Swinnen E, Kerckhofs E. The immediate effects of robot-assistance on energy consumption and cardiorespiratory load during walking compared to walking without robot-assistance: a systematic review. Disab Rehabil Assist Technol. 2017;12(7):657–71.

Karelis AD, Carvalho LP, Castillo MJ, Gagnon DH, Aubertin-Leheudre M. Effect on body composition and bone mineral density of walking with a robotic exoskeleton in adults with chronic spinal cord injury. J Rehabil Med. 2017;49(1):84–7.

Choi H, Kim G, Chai JH, Ko C. Effect of gait training program with mechanical exoskeleton on body composition of paraplegics. J Multidiscip Healthc. 2020;13:1879.

Asselin P, Cirnigliaro CM, Kornfeld S, Knezevic S, Lackow R, Elliott M, et al. Effect of Exoskeletal-Assisted Walking on Soft Tissue Body Composition in Persons With Spinal Cord Injury. Arch Phys Med Rehabil. 2021;102(2):196–202.

Gorgey AS, Poarch H, Harnish C, Miller JM, Dolbow D, Gater DR. Acute effects of locomotor training on neuromuscular and metabolic profile after incomplete spinal cord injury. NeuroRehabilitation. 2011;29(1):79–83.

Tan K, Koyama S, Sakurai H, Teranishi T, Kanada Y, Tanabe S. Wearable robotic exoskeleton for gait reconstruction in patients with spinal cord injury: A literature review. J Orthop Translat. 2021;28:55–64.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int J Surg. 2021;88: 105906.

Academy of Neurologic Physical Therapy. 6 minute walk test (6MWT). 2019. https://www.neuropt.org/docs/default-source/cpgs/core-outcome-measures/6mwt-pocket-guide-proof9.pdf?sfvrsn=9ee25043_0 . Accessed 12 Mar 2023.

Ryan S. 6 minute walk test. 2013. https://www.sralab.org/rehabilitation-measures/6-minute-walk-test . Accessed 12 Mar 2023.

Physiopedia. 10 metre walk test. 2022. https://www.physio-pedia.com/10_Metre_Walk_Test . Accessed 12 Mar 2023.

Ryan S. Time up and go. 2013. https://www.sralab.org/rehabilitation-measures/timed-and-go . Accessed 12 Mar 2023.

Xiang X, Zong H, Ou Y, Yu X, Cheng H, Du C, et al. Exoskeleton-assisted walking improves pulmonary function and walking parameters among individuals with spinal cord injury: a randomized controlled pilot study. J Neuroeng Rehabil. 2021;18(1):86.

Evans RW, Shackleton CL, West S, Derman W, Laurie Rauch H, Baalbergen E, et al. Robotic locomotor training leads to cardiovascular changes in individuals with incomplete spinal cord injury over a 24-week rehabilitation period: a randomized controlled pilot study. Arch Phys Med Rehabil. 2021;102(8):1447–56.

Corbianco S, Cavallini G, Dini M, Franzoni F, D’Avino C, Gerini A, et al. Energy cost and psychological impact of robotic-assisted gait training in people with spinal cord injury: effect of two different types of devices. Neurol Sci. 2021;42(8):3357–66.

Edwards DJ, Forrest G, Cortes M, Weightman MM, Sadowsky C, Chang S, et al. Walking improvement in chronic incomplete spinal cord injury with exoskeleton robotic training (WISE): a randomized controlled trial. Spinal Cord. 2022;60(6):522–32.

Stampacchia G, Olivieri M, Rustici A, Avino C, Gerini A, Mazzoleni S. Gait rehabilitation in persons with spinal cord injury using innovative technologies: an observational study. Spinal Cord. 2020;58(9):988–97.

Puentes S, Kadone H, Kubota S, Abe T, Shimizu Y, Marushima A, et al. Reshaping of gait coordination by robotic intervention in myelopathy patients after surgery. Front Neurosci. 2018;12:99.

Tefertiller C, Hays K, Jones J, Jayaraman A, Hartigan C, Bushnik T, et al. Initial outcomes from a multicenter study utilizing the indego powered exoskeleton in spinal cord injury. Top Spinal Cord Inj Rehabil. 2018;24(1):78–85.

Knezevic S, Asselin PK, Cirnigliaro CM, Kornfeld S, Emmons RR, Spungen AM. Oxygen uptake during exoskeletal-assisted walking in persons with paraplegia. Arch Phys Med Rehabil. 2021;102(2):185–95.

Guanziroli E, Cazzaniga M, Colombo L, Basilico S, Legnani G, Molteni F. Assistive powered exoskeleton for complete spinal cord injury: correlations between walking ability and exoskeleton control. Eur J Phys Rehabil Med. 2019;55(2):209–16.

Sale P, Russo EF, Scarton A, Calabro RS, Masiero S, Filoni S. Training for mobility with exoskeleton robot in spinal cord injury patients: a pilot study. Eur J Phys Rehabil Med. 2018;54(5):745–51.

Gagnon DH, Escalona MJ, Vermette M, Carvalho LP, Karelis AD, Duclos C, et al. Locomotor training using an overground robotic exoskeleton in long-term manual wheelchair users with a chronic spinal cord injury living in the community: Lessons learned from a feasibility study in terms of recruitment, attendance, learnability, performance and safety. J Neuroeng Rehabil. 2018;15(1):12.

Khan AS, Livingstone DC, Hurd CL, Duchcherer J, Misiaszek JE, Gorassini MA, et al. Retraining walking over ground in a powered exoskeleton after spinal cord injury: a prospective cohort study to examine functional gains and neuroplasticity. J Neuroeng Rehabil. 2019;16:145.

Xiang X, Ding M, Zong H, Liu Y, Cheng H, He C, et al. The safety and feasibility of a new rehabilitation robotic exoskeleton for assisting individuals with lower extremity motor complete lesions following spinal cord injury (SCI): an observational study. Spinal Cord. 2020;58(7):787–94.

Park JH, Kim HS, Jang SH, Hyun DJ, Park SI, Yoon J, et al. Cardiorespiratory responses to 10 weeks of exoskeleton-assisted overground walking training in chronic nonambulatory patients with spinal cord injury. Sensors (Basel, Switzerland). 2021;21(15):5022.

Kwon SH, Lee BS, Lee HJ, Kim EJ, Lee JA, Yang SP, et al. Energy efficiency and patient satisfaction of gait with knee-ankle-foot orthosis and robot (ReWalk)-assisted gait in patients with spinal cord injury. Ann Rehabil Med. 2020;44(2):131–41.

Kerdraon J, Previnaire JG, Tucker M, Coignard P, Allegre W, Knappen E, et al. Evaluation of safety and performance of the self balancing walking system Atalante in patients with complete motor spinal cord injury. Spinal Cord Ser Cases. 2021;7(1):71.

Asselin PK, Avedissian M, Knezevic S, Kornfeld S, Spungen AM. Training persons with spinal cord injury to ambulate using a powered exoskeleton. J Vis Exp. 2016;112:54071.

Baunsgaard CB, Nissen UV, Brust AK, Frotzler A, Ribeill C, Kalke Y, et al. Exoskeleton gait training after spinal cord injury: an exploratory study on secondary health conditions. J Rehabil Med. 2018;50(9):806–13.

Bass A, Aubertin-Leheudre M, Morin SN, Gagnon DH. Preliminary training volume and progression algorithm to tackle fragility fracture risk during exoskeleton-assisted overground walking in individuals with a chronic spinal cord injury. Spinal Cord Ser Cases. 2022;8(1):29.

Bass A, Morin S, Guidea M, Lam J, Hammad R, Aubertin-Leheudre M, et al. Effects of an exoskeleton-assisted walking program on bone strength in wheelchair individuals with spinal cord injury: a preliminary study using imaging and serum biomarkers. Physiology. 2023;38(S1):5796080.

Chen S, Wang Z, Li Y, Tang J, Wang X, Huang L, et al. Safety and feasibility of a novel exoskeleton for locomotor rehabilitation of subjects with spinal cord injury: a prospective, multi-center, and cross-over clinical trial. Front Neurorobot. 2022;16: 848443.

Contreras-Vidal JL, Bhagat NA, Brantley J, Cruz-Garza JG, He Y, Manley Q, et al. Powered exoskeletons for bipedal locomotion after spinal cord injury. J Neural Eng. 2016;13(3): 031001.

Goikoetxea-Sotelo G, van Hedel HJ. Defining, quantifying, and reporting intensity, dose, and dosage of neurorehabilitative interventions focusing on motor outcomes. Front Rehabil Sci. 2023;4:1139251.

Hutchinson MJ, Goosey-Tolfrey VL. Rethinking aerobic exercise intensity prescription in adults with spinal cord injury: time to end the use of “moderate to vigorous” intensity? Spinal Cord. 2022;60(6):484–90.

Craven BC, Souza WH, Jaglal S, Gibbs J, Wiest MJ, Sweet SN et al. Reducing endocrine metabolic disease risk in adults with chronic spinal cord injury: strategic activities conducted by the Ontario-Quebec RIISC team. Disabil Rehabil. 2023:1–13. Ahead of print.

Bierman AS, Tinetti ME. Precision medicine to precision care: managing multimorbidity. Lancet. 2016;388(10061):2721–3.

Benson I, Hart K, Tussler D, van Middendorp JJ. Lower-limb exoskeletons for individuals with chronic spinal cord injury: findings from a feasibility study. Clin Rehabil. 2016;30(1):73–84.

Zeilig G, Weingarden H, Zwecker M, Dudkiewicz I, Bloch A, Esquenazi A. Safety and tolerance of the ReWalk™ exoskeleton suit for ambulation by people with complete spinal cord injury: a pilot study. J Spinal Cord Med. 2012;35(2):96–101.

Hartigan C, Kandilakis C, Dalley S, Clausen M, Wilson E, Morrison S, et al. Mobility outcomes following five training sessions with a powered exoskeleton. Top Spinal Cord Inj Rehab. 2015;21(2):93–9.

Escalona MJ, Brosseau R, Vermette M, Comtois AS, Duclos C, Aubertin-Leheudre M, et al. Cardiorespiratory demand and rate of perceived exertion during overground walking with a robotic exoskeleton in long-term manual wheelchair users with chronic spinal cord injury: a cross-sectional study. Ann Phys Rehabil Med. 2018;61(4):215–23.

Hellsten Y, Nyberg M. Cardiovascular adaptations to exercise training. 2015;6 1:1-32.

Škarabot J, Brownstein CG, Casolo A, Del Vecchio A, Ansdell P. The knowns and unknowns of neural adaptations to resistance training. Eur J Appl Physiol. 2021;121(3):675–85.

Faulkner J, Martinelli L, Cook K, Stoner L, Ryan-Stewart H, Paine E, et al. Effects of robotic-assisted gait training on the central vascular health of individuals with spinal cord injury: a pilot study. J Spinal Cord Med. 2021;44(2):299–305.

Morse LR, Biering-Soerensen F, Carbone LD, Cervinka T, Cirnigliaro CM, Johnston TE, et al. Bone mineral density testing in spinal cord injury: 2019 ISCD official position. J Clin Densitom. 2019;22(4):554–66.

Bass A, Morin SN, Vermette M, Aubertin-Leheudre M, Gagnon DH. Incidental bilateral calcaneal fractures following overground walking with a wearable robotic exoskeleton in a wheelchair user with a chronic spinal cord injury: is zero risk possible? Osteoporosis Int. 2020;31(5):1007–11.

Article   CAS   Google Scholar  

Van Herpen F, Van Dijsseldonk RB, Rijken H, Keijsers N, Louwerens J, van Nes I. Case report: description of two fractures during the use of a powered exoskeleton. Spinal Cord Ser Cases. 2019;5(1):99.

Sunn G. Spinal cord injury pressure ulcer treatment: an experience-based approach. Phys Med Rehabil Clin. 2014;25(3):671–80.

Huiming G, Yuming W, Mingliang Y, Changbin L, Qiuchen H, Jianjun L. Study on the characteristics of microcirculation in the site of pressure ulcer in patients with spinal cord injury. Sci Prog. 2021;104(3):00368504211028726.

Ozdemir RA, Perez MA. Afferent input and sensory function after human spinal cord injury. J Neurophysiol J Neurophysiol. 2018;119(1):134–44.

Hill D, Holloway CS, Ramirez DZM, Smitham P, Pappas Y. What are user perspectives of exoskeleton technology? A literature review. Int J Technol Assess Health Care. 2017;33(2):160–7.

Van Dijsseldonk RB, Van Nes IJ, Geurts AC, Keijsers NL. Exoskeleton home and community use in people with complete spinal cord injury. Sci Rep. 2020;10(1):15600.

Mertz L. The next generation of exoskeletons: lighter, cheaper devices are in the works. IEEE Pulse. 2012;3(4):56–61.

Download references

Acknowledgements

Patrik Nepomuceno acknowledges receipt of a scholarship through the Emerging Leaders in the Americas Program with the support of Global Affairs Canada and Government of Canada. Dr. Wagner H. Souza acknowledges fellowship support from the University Health Network (UHN) Foundation and Spinal Cord Injury Ontario, and inspiration from Mr. Robert MacDonald. Dr. Craven acknowledges support from the UHN Foundation for her UHN/University of Toronto Chair in SCI Rehabilitation.

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Author information

Authors and affiliations.

KITE Research Institute, University Health Network, Toronto, ON, Canada

Patrik Nepomuceno, Wagner H. Souza, Maureen Pakosh, Kristin E. Musselman & B. Catharine Craven

Graduate Program in Health Promotion, Department of Health Sciences, University of Santa Cruz do Sul, Santa Cruz do Sul, RS, Brazil

Patrik Nepomuceno

Department of Physical Therapy, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada

Kristin E. Musselman

Department of Medicine, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada

B. Catharine Craven

Institute of Health Policy Management and Evaluation, University of Toronto, Toronto, Canada

Patrik Nepomuceno & B. Catharine Craven

Rehabilitation Sciences Institute, Temerty Faculty of Medicine, University of Toronto, Toronto, Canada

Kristin E. Musselman & B. Catharine Craven

You can also search for this author in PubMed   Google Scholar

Contributions

PN, WHS, KEM and BCC contributed to conceptualization, methodology, design, and development of the study. PN and WHS share first authorship and equally contributed to data curation, acquisition, formal analysis and to the writing of the manuscript. MP contributed to data curation, acquisition and methodology. KEM and BCC contributed to supervision, review and editing the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to B. Catharine Craven .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1:.

Medline Search Strategy. Contain the Medline Search Strategy.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Nepomuceno, P., Souza, W.H., Pakosh, M. et al. Exoskeleton-based exercises for overground gait and balance rehabilitation in spinal cord injury: a systematic review of dose and dosage parameters. J NeuroEngineering Rehabil 21 , 73 (2024). https://doi.org/10.1186/s12984-024-01365-2

Download citation

Received : 28 November 2023

Accepted : 22 April 2024

Published : 05 May 2024

DOI : https://doi.org/10.1186/s12984-024-01365-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Exoskeleton
• Neurorehabilitation
• Overground training
• Spinal cord injury

Journal of NeuroEngineering and Rehabilitation

ISSN: 1743-0003

• Submission enquiries: [email protected]

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• For authors
• New editors
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Online First
• Best practices for the dissemination and implementation of neuromuscular training injury prevention warm-ups in youth team sport: a systematic review
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0009-0005-0529-0398 Destiny Lutz 1 ,
• http://orcid.org/0000-0001-6429-4333 Carla van den Berg 1 ,
• http://orcid.org/0000-0003-3056-8169 Anu M Räisänen 1 , 2 ,
• Isla J Shill 1 , 3 ,
• Jemma Kim 4 , 5 ,
• Kenzie Vaandering 1 ,
• Alix Hayden 6 ,
• http://orcid.org/0000-0002-0427-2877 Kati Pasanen 1 , 7 , 8 , 9 ,
• http://orcid.org/0000-0002-5951-5899 Kathryn J Schneider 1 , 3 , 8 , 9 , 10 ,
• http://orcid.org/0000-0002-9499-6691 Carolyn A Emery 1 , 3 , 8 , 9 , 11 , 12 , 13 ,
• http://orcid.org/0000-0002-5984-9821 Oluwatoyosi B A Owoeye 1 , 4
• 1 Sport Injury Prevention Research Centre, Faculty of Kinesiology , University of Calgary , Calgary , Alberta , Canada
• 2 Department of Physical Therapy Education - Oregon , Western University of Health Sciences College of Health Sciences - Northwest , Lebanon , Oregon , USA
• 3 Hotchkiss Brain Institute , University of Calgary , Calgary , Alberta , Canada
• 4 Department of Physical Therapy & Athletic Training , Doisy College of Health Sciences, Saint Louis University , Saint Louis , Missouri , USA
• 5 Interdisciplinary Program in Biomechanics and Movement Science , University of Delaware College of Health Sciences , Newark , Delaware , USA
• 6 Libraries and Cultural Resources , University of Calgary , Calgary , Alberta , Canada
• 7 Tampere Research Center for Sports Medicine , Ukk Instituutti , Tampere , Finland
• 8 McCaig Institute for Bone and Joint Health , University of Calgary , Calgary , Alberta , Canada
• 9 Alberta Chilrden's Hopsital Research Institute , University of Calgary , Calgary , Alberta , Canada
• 10 Sport Medicine Centre , University of Calgary , Calgary , Alberta , Canada
• 11 O'Brien Institute for Public Health , University of Calgary , Calgary , Alberta , Canada
• 12 Department of Community Health Sciences , Cumming School of Medicine, University of Calgary , Calgary , Alberta , Canada
• 13 Department of Paediatrics , Cumming School of Medicine, University of Calgary , Calgary , Alberta , Canada
• Correspondence to Ms Destiny Lutz, Sport Injury Prevention Research Centre, Faculty of Kinesiology, University of Calgary, Calgary, Alberta, Canada; destiny.lutz{at}ucalgary.ca

Objective To evaluate best practices for neuromuscular training (NMT) injury prevention warm-up programme dissemination and implementation (D&I) in youth team sports, including characteristics, contextual predictors and D&I strategy effectiveness.

Design Systematic review.

Data sources Seven databases were searched.

Eligibility The literature search followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Inclusion criteria: participation in a team sport, ≥70% youth participants (<19 years), D&I outcomes with/without NMT-related D&I strategies. The risk of bias was assessed using the Downs & Black checklist.

Results Of 8334 identified papers, 68 were included. Sport participants included boys, girls and coaches. Top sports were soccer, basketball and rugby. Study designs included randomised controlled trials (RCTs) (29.4%), cross-sectional (23.5%) and quasi-experimental studies (13.2%). The median Downs & Black score was 14/33. Injury prevention effectiveness (vs efficacy) was rarely (8.3%) prioritised across the RCTs evaluating NMT programmes. Two RCTs (2.9%) used Type 2/3 hybrid approaches to investigate D&I strategies. 19 studies (31.6%) used D&I frameworks/models. Top barriers were time restrictions, lack of buy-in/support and limited benefit awareness. Top facilitators were comprehensive workshops and resource accessibility. Common D&I strategies included Workshops with supplementary Resources (WR; n=24) and Workshops with Resources plus in-season Personnel support (WRP; n=14). WR (70%) and WRP (64%) were similar in potential D&I effect. WR and WRP had similar injury reduction (36–72%) with higher adherence showing greater effectiveness.

Conclusions Workshops including supplementary resources supported the success of NMT programme implementation, however, few studies examined effectiveness. High-quality D&I studies are needed to optimise the translation of NMT programmes into routine practice in youth sport.

Data availability statement

Data are available in a public, open access repository. Not Applicable.

https://doi.org/10.1136/bjsports-2023-106906

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THE TOPIC

Neuromuscular training (NMT) injury prevention warm-up programmes are effective at preventing injury rates in youth sports. However, for proper dissemination and implementation (D&I) by multiple stakeholders, barriers such as low adoption, adherence and lack of time must be addressed.

WHAT THIS STUDY ADDS

There are limited high-quality research studies to facilitate the widespread adoption of, and improved adherence to, NMT programmes. Few studies used D&I theories, frameworks or models. Programme flexibility is a common barrier to implementation; adaptation of NMT programmes to fit local contexts is imperative. Comprehensive workshops and supplementary resources currently support the success of NMT programme implementation.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

Promotion of NMT programmes as the standard of practice is essential to increase practical D&I of these programmes, and thus reduce the burden of youth sport injuries. This work provides some directions for stakeholders, including researchers, implementation support practitioners and youth sport policymakers, on current best practices for the delivery of NMT programmes in local youth sport settings. This work also provides the evidence base for more translational research efforts in youth sport injury prevention, a much-needed next step to optimise NMT programmes into youth sport practice.

Introduction

Youth (<19 years) sport participation provides numerous benefits, positively impacting physical and mental health. 1 Youth sport participation rates are high, with up to 90% of youth participating in sport globally. 2–5 However, with increased sport participation comes increased injury risk. One-in-three youth sustain a sport-related injury each year, leading to a significant public health burden with high healthcare costs. 3 6–8 Sport-related injuries may also result in long-term health consequences (eg, poor mental health, reduced physical activity, post-traumatic osteoarthritis). 7–9 Implementing injury prevention strategies is critical to mitigate the injury risk associated with youth sport participation.

Neuromuscular training (NMT) injury prevention warm-up programmes in youth team sport are effective in reducing injury rates by up to 60% and decreasing costs associated with injury based on randomised controlled trials (RCT) and systematic reviews. 10–21 NMT programmes include exercises that can be categorised across aerobic, balance, strength and agility components 22 23 and typically take 10–15 min. 24 25 Originally implemented with the intention of reducing non-contact lower extremity injury risk, 26–28 the effectiveness of NMT programmes has since been evaluated across numerous sports, age groups and levels of play and are associated with lower extremity and overall injury rates compared with standard of practice warm-ups. 12 20 21 25 In youth team sports, a protective effect has been demonstrated in soccer, handball, basketball, netball, rugby and floorball. 11 16 29–31 The International Olympic Committee Consensus Statement on Youth Athletic Development recommends multifaceted NMT warm-up programmes in youth sport. 32

Despite being a primary injury prevention strategy across youth sports, NMT programme adoption remains low. 33–38 For evidence-informed interventions to be successful and have a practical impact, pragmatic approaches derived from dissemination and implementation (D&I) science are necessary across multiple socioecological levels including organisation, coach and player. 36 Dissemination is defined as ‘the active process of spreading evidence-based interventions to a target population through determined channels and using planned strategies’. Implementation is ‘the active process of using strategies across multiple levels of change to translate evidence-based interventions into practice and prompt corresponding behaviour change in a target population’. 36

The aim of this systematic review was to evaluate current best practices for the D&I of NMT programmes in a youth team sport. The specific objectives of this systematic review were to: (1) describe the characteristics of identified D&I-related studies (studies with at least one D&I outcome directly or indirectly assessed as primary, secondary or tertiary outcome); (2) evaluate factors associated with the D&I of NMT warm-up programme across socioecological levels, including barriers and facilitators; (3) examine the effect of D&I strategies in delivering NMT warm-ups across multiple socioecological levels; and (4) examine the influence of D&I strategies on injury rates. Our protocol was registered in PROSPERO (CRD42021271734), and the review is reported in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines ( supplemental appendix S1 ).

Supplemental material

Search strategy and data sources.

A comprehensive search was developed with a librarian (KAH) in MEDLINE, incorporating four main concepts: child/youth, injury prevention, implementation/compliance/adherence and sports. The author team reviewed the final search strategy which was then piloted against the known key studies to ensure that the search was capturing relevant studies. Finally, the MEDLINE search was translated to the other databases. Searches were conducted 25 August 2021 (updated 16–18 August 2022; 5 September 2023). Search strategies are available in Supplemental Appendix S2 . Studies were identified by searching seven databases: MEDLINE(R) and EPUB Ahead of Print, In-Process & Other Non-Indexed Citations and Daily, Embase, Cochrane Central Register of Controlled Trials, Cochrane database of Systematic Reviews (all Ovid); CINAHL Plus with Full Text, SPORTDiscus with Full Text (EBSCO) and ProQuest Dissertations & Thesis Global.

Study selection and eligibility

All database search results were uploaded and duplicates were removed in Covidence (Veritas Health Innovation, Melbourne, Australia). Records were independently reviewed by authors in pairs (DL/IJS, CV/JK, KV/DL), starting with a screening of 50 randomly selected citations to assess inter-rater agreement with a threshold set at 90%. Each pair of reviewers performed title/abstract screening and full-text screening independently, providing reasons for exclusion at full-text stage ( figure 1 ). Any disagreements for exclusion, where a consensus could not be reached within pairs, were resolved by a senior author (OBAO). A secondary evaluation of included manuscripts was performed by senior authors (OBAO and CAE) to ensure appropriate inclusion. Study inclusion criteria were: (1) Participation in a team sport (male and female); (2) a minimum of 70% of participants as a youth (<19 years) or coaches of these youth teams; (3) reported dissemination and/or implementation outcomes (eg, self-efficacy, adherence, intention); (4) reported D&I strategies related to NMT warm-up programmes (ie, NMT delivery strategies, where applicable eg, in RCTs). Exclusion criteria were: (1) Studies evaluating rehabilitation programmes, non-team-based or physical education programmes; (2) non-peer-reviewed; (3) not English. The screening process was reported using the PRISMA flow diagram. 39

• Download figure
• Open in new tab
• Download powerpoint

Study identification Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram.

Risk of bias

To assess the risk of bias, three sets of paired reviewers independently used the Downs & Black (D&B) quality assessment tool. 40 The tool consists of a 27-item checklist (total score/33). A third senior reviewer (OBAO or CAE or AMR) resolved any disagreements. The rating of evidence and strength of recommendations were assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidelines. 41–43

Frameworks/models

The proportion of studies that used D&I research theories/frameworks/models, including behaviour change frameworks/models, was examined to identify commonly used frameworks/models.

Efficacy-effectiveness orientation in RCTs

We assessed the components of 12 RCTs using the Rating Included Trials on the Efficacy-Effectiveness Spectrum (RITES) tool, as adapted by Maddox et al 44 RITES scores RCTs in systematic reviews based on a continuum of efficacy-effectiveness across four domains: Participant characteristics, trial setting, flexibility of intervention(s) and clinical relevance of experimental and comparison intervention(s) ( online supplemental table S1 ). 45 We modified the Likert grading system to classify studies depending on whether their emphasis was more on efficacy or effectiveness or balanced for both. Given that different aspects of each trial may fall in different places along the efficacy-effectiveness continuum, each RITES domain is scored independently and a composite score is not applicable. To minimise subjectivity, the RITES evaluation for included RCTs was completed by two reviewers (AMR and OBAO). Any disagreements were resolved through discussion to reach a consensus.

Study typologies and assessment of study relevance to D&I

The level of relevance of individual studies (RCTs and quasi-experiments) to D&I was determined based on the implementation-effectiveness hybrid taxonomy: Type 1 (primarily focused on clinical/intervention outcomes), Type 2 (balance focused on both clinical/intervention outcomes and D&I outcomes) and Type 3 (primarily or ‘fully’ (our adaptation) focused on D&I outcomes) studies. 36 46 For ease of interpretation of results, studies were rated considering three broad traditional research design categories (ie, hierarchy of evidence): RCTs, quasi-experimental and observational studies, including cohort, cross-sectional, pre-experimental, qualitative, mixed-methods and ecological studies. Observational studies were categorised as ‘fully focused’ observational-implementation (if only D&I outcomes were evaluated) or ‘partially focused’ observational-implementation (if a combination of clinical and D&I outcomes were evaluated’ D&I studies. 47 RCTs and quasi-experimental studies with Type 2 or Type 3 hybrid approaches were indicated as ‘highly relevant’ towards informing D&I best practices. Furthermore, observational-implementation studies that are fully focused on D&I were also indicated as ‘highly relevant’.

Data extraction

The extracted data included: study design, author, journal, year, population (eg, 13–17 years old female soccer players), participant demographics, D&I intervention strategies (eg, workshops, supplementary resources), D&I framework/model, control group strategies, D&I outcomes (eg, adoption, adherence, intention, fidelity, self-efficacy) and injury outcomes. Study design classification was completed based on data extracted and the process taken by authors, 48 which may have differed from the original classification. Furthermore, prospective, and retrospective cohort studies were consolidated into ‘cohort’ to improve ease of readability. D&I outcomes indicated as compliance were included in the appropriate adherence category as defined in Owoeye et al and described as ‘adherence-related’ outcomes, to maintain unified language across results; the full list is provided in online supplemental table S2 . 36 49–51 Based on the dose-response thresholds reported for NMT programmes within current literature, measures of adherence were used to indicate potential D&I effect ( online supplemental table S5 ). 24 36 52–54 Studies with cumulative utilisation (sessions completed/total possible) of ≥70%, utilisation frequency of ≥1.5 sessions/week or a significant association between D&I exposures and outcomes were defined as moderate-to-highly relevant and identified as having a potential D&I effect (ie, yes). Studies presenting cumulative utilisation <70%, utilisation frequency <1.5 sessions/week or no association between D&I exposures and outcomes were defined as low-to-no relevance (ie, no). Studies reporting both frequency utilisation and cumulative utilisation must both reach the established dose-response thresholds to be considered as having a potential D&I effect. D&I barriers and facilitators, factors influencing injury prevention implementation success and the identification of any frameworks used were also extracted and categorised into themes. Measures of potential effect for these results were summarised using OR, proportions and mean differences in D&I outcomes (eg, adoption, adherence). Injury-specific results were reported as incidence rate ratios, risk ratios, ORs or prevalence. D&I strategies were classified into various categories, including workshops, supplementary resources, personnel support, supervision and combinations of these strategies.

Equity, diversity and inclusion statement

Our author team is comprised of student and senior researchers across various disciplines with representation from low-to-middle-income countries. A variety of demographic, socioeconomic and cultural backgrounds were included in our study populations.

PRISMA flow, characteristics of included studies and risk of bias assessment

A total of 68 relevant studies were included from our initial and updated search yield of 9021 studies ( figure 1 ). Across included studies, 13 included only male youth participants, 13 included only female youth participants, 26 included both and 16 reported coach-focused findings. Sports represented were soccer (n=33), rugby (n=8), basketball (n=7), multisport (n=7), handball (n=5), floorball (n=3), field hockey (n=3), volleyball (n=1) and futsal (n=1).

Details of study characteristics and risk of bias are presented in online supplemental table S3 . D&B scores ranged from 4/33 to 24/33 (median=14/33) from a variety of study designs, including 20 RCTs, 16 cross-sectional, 9 quasi-experimental, 8 cohort, 6 qualitative, 3 ecological, 3 mixed-methods and 3 pre-experimental. The D&B scores for the two top D&I-related relevant studies—an RCT Type 2 study (n=1) was 21/33 and an RCT Type 3 hybrid study (n=1) was 17/33. Using the GRADE guidelines for the process of rating the quality of evidence available and interpreting the quality assessment, the strength of recommendations was ‘low’ given the multiplicity of designs. 42 43

Characteristics of current D&I-related studies

23 studies (33.8%) reported using a D&I /behaviour change framework/model. D&I frameworks included Reach, Effectiveness, Adoption, Implementation and Maintentance (RE-AIM) Framework (n=7), Consolidation Framework for Implementation Research (n=1), Precede-Proceed Model (n=1), Translating Research into Injury Prevention Practice (n=1) and Promoting Action on Research Implementation of Health Sciences (n=1) and the Adherence Optimisation Framework (n=1). Behaviour change models included the Health Action Process Approach (HAPA) (n=8), Theory of Planned Behaviour (n=1) and the Health Belief Model (n=1).

Assessment of study relevance to D&I

Two RCTs of 68 included D&I-related studies (2.9%) were identified as highly relevant to D&I best practices (ie, Type 2 or 3 hybrid approach). 55 56 18 (27.9%) RCTs reported a secondary analysis of D&I strategies 12 16 19 30 53 57–69 ; classified as Type 1 hybrids. Five (8.3%) quasi-experimental studies used Type 2 or Type 3 hybrid approach 22 70–73 ; the remaining studies (n=4; 5%) were classified as quasi-experimental Type 1 hybrids. 74–77 Many observational studies (n=17; 26.7%), 78–94 were highly relevant based on being fully-focused observational-implementation studies; 5 (6.7%) were partially-focused observational-implementation studies. 52 95–98 The remaining observational studies (n=17; 23.3%) were observational-implementation studies, 35 99–114 reporting D&I outcomes from a qualitative lens using interviews and surveys.

The RITES scores for the 14 D&I-related RCTs that examined injuries as primary outcome and D&I outcomes as secondary (Type 1 hybrid approach) are presented in table 1 . Almost all (13 of 14; 92.9%) of the RCTs focused mainly on intervention efficacy (as opposed to effectiveness) regarding the flexibility of NMT warm-up programmes. Cumulatively, effectiveness was rarely (7.1%) prioritised as a primary focus across all the 56 possible ratings of the RITES domains for all 14 studies. 50% of the domain ratings demonstrated efficacy as a priority and 42.9% of the ratings were indicated for a balance between efficacy and effectiveness.

• View inline

RITES domain scores for included trials

Contextual predictors of NMT warm-up programme implementation

50 (73.5% of total) studies identified ≥1 barrier or facilitator within their findings, with 10 (14.7% of total) specifically examining barriers/facilitators as their main objectives. Full list is provided in online supplemental table S4 . The most common barriers identified were time restrictions (n=30), 30 35 59 62 69 70 73 74 78 79 81 82 84 87–91 93 96 98 101 102 105 107 108 112 114 reduced buy-in/support (n=8) 62 75 84 87 105 110–112 and limited awareness of preventative effects of programmes (n=8). 74 84 103 104 107 109 113 Facilitators included comprehensive workshops from trained instructors (n=11), 53 71 78–80 84 90 96 99 100 112 accessibility of supplementary resources (n=10) 82 84 87 89 90 105 114 and uptake/support from multiple stakeholders (n=7). 56 67 84 101 103 105 112 Moreover, suggestions from multiple socioecological levels indicated that increasing programme education and support, increased sport-specific activities and improved awareness of preventive effects, influence NMT implementation success. 36 88 89 115 116 Figure 2 , adapted from Basow et al 117 illustrates the contextual factors reported in the literature. 117 This evidence-informed model shows the important barriers and facilitators that influence the end-user implementation of NMT warm-up programmes across the three key socioecological levels of change.

Contextual predictors of NMT implementation across multiple socioecological levels. (Adapted from Basow et al (2021)).(116). Notes. SE, self-efficacy. NMT, neuromuscular training. Bold represents top barrier(s)/facilitator(s).

51 (75%) studies used implementation strategies for NMT warm-up programmes. The most frequently used strategies were Workshops with supplementary Resources (WR; n=24), followed by Workshops with supplementary Resources, plus in-season Personnel support (WRP; n=14). Three studies employed both WR and WRP strategies. Other methods for implementation included only workshops (n=9), only supplementary resources (n=4), supplementary resources and personnel support (n=2), workshops with personnel support (n=1) and supervision (n=1). Note, some studies are duplicated throughout the table when multiple D&I strategies are compared. 22 53 56 86

The key D&I concepts that were reported within the included studies were adherence or adherence-related (eg, self-efficacy, translation and perception). Specific outcomes within these concepts were further examined from the individual study results. We did not have enough evidence to present a meta-analysis of the effect of D&I strategies on D&I outcomes. Therefore, online supplemental table S5 presents a qualitative summary of the relationships between reported D&I exposure and D&I outcomes. 40 studies reported adherence-related outcomes, of which 32 (80%) were indicated to have potential D&I effect. Studies using WRP (n=14) reported completing between 1.4 and 2.6 sessions/week and cumulative utilisation of 39–85.6%; 9 of these 14 studies have potential D&I effect. Studies using WR (n=24) presented utilisation frequency ranging from 0.8 to 3.2 sessions/week and cumulative utilisation of 55–98% of sessions; 16 of these 24 have potential D&I effect. In studies evaluating workshops only (n=9; 22%), frequency utilisation was reported between 1 and 2 sessions/week across eight of the nine studies and one study had 52% cumulative utilisation; two have potential D&I effect.

Effects of D&I strategies on injury outcomes

Three RCTs specifically examined the effects of the D&I strategies used to deliver NMT programmes on injury outcomes ( table 2 ). Two studies that compared both WR and WRP to supplementary resource only found no significant differences between strategies, 53 56 they reported reduced injury rates in the highest adherence groups by 56% and 72%, respectively. Another study comparing WR and WRP to a standard of practice warm-up found a 36% reduction of ankle and knee injuries when using WR and a 38% reduction in ankle and knee injuries without supervision. 22 There were no significant differences in injury rates between groups.

Injury Outcomes by D&I strategies and adherence

This study evaluated current literature to inform evidence-based best practices for the D&I of NMT programmes in youth team sport. To our knowledge, this is the first systematic review evaluating the D&I of NMT programmes in youth sport. To improve the practical implementation of NMT warm-ups, factors associated with implementation success and current best practices for delivering context-specific NMT programmes are required to be evaluated. 118 In this review, we found few D&I-related studies use D&I or behaviour change frameworks, theories or models to guide their research questions. We discovered the number of RCTs examining the effectiveness of D&I strategies for NMT programme delivery is limited. Common barriers to NMT implementation include programme flexibility and time restrictions; and the use of coach workshops and supplementary resources are currently the primary strategy in NMT programme D&I facilitation.

One-third of the included studies used a D&I framework or behaviour change model in their research work. The HAPA and RE-AIM models were the most frequently used. These models are a conceptual and organised combination of theories required to direct the design, evaluation and translation of evidence-based interventions (NMT programmes) and the context in which they are being implemented. 36 71 119 It is imperative for D&I studies to use these frameworks/models to fully understand specific implementation processes and contexts. Future D&I studies should consider using appropriate frameworks or models, including adaptations and combination of models to guide their specific aims.

Relevance to D&I

Across the relevant literature, a variety of designs and levels of evidence were included.

Of 68 studies, 7 (10.3%) were found to be ‘highly relevant’ toward informing D&I best practice (2 (2.9%) RCTs, 5 (7.4%) quasi-experimental). Other ‘relevant’ studies evaluated implementation as secondary objectives (Type 1 hybrid designs) and/or were of lower level of evidence. 33 observational studies were ‘highly relevant’ to D&I, assessing D&I outcomes and barriers and facilitators from a qualitative lens. While these studies are important for understanding D&I context, more high-quality and highly relevant studies such as RCTs and quasi-experimental designs using the Type 3 hybrid approach, or non-hybrid approach focused on solely evaluating the effectiveness of D&I strategies, are needed to advance the widespread adoption and continued use of NMT programmes in youth team sport.

Effectiveness versus efficacy

Effectiveness is indicative of an evidence-informed intervention’s readiness for practical implementation. 36 Findings from our RITES scores evaluation indicate that the majority of the RCTs had a primary focus on efficacy and not effectiveness. Although many RCT studies had a fair balance between efficacy and effectiveness for participant characteristics, trial settings and clinical relevance domains (≥50% of RCTs), there is a lack of flexibility in the development and evaluation of the evidence supporting current NMT warm-up programmes. These disparities regarding practical implementation have implications for D&I research and practice in this field. Current NMT programmes may need to be modified or adapted to the local context and evaluated further to improve implementation in youth sport settings.

Contextual considerations

In our Adapted Socioecological Model ( figure 2 ), we demonstrate that the utilisation of NMT programmes by individual players within youth team sport can depend on their coach adopting and implementing the warm-up, which may also be dependent on larger organisational systems. Barriers related to end-users’ success in wide-spread adoption and long-term maintenance can be moderated; however, researchers and implementers have to be intentional about tackling these recognised barriers and associated challenges 25 87 104 115 ; integrating the facilitators of successful implementation intending to reduce and address these obstacles is essential. The barriers and facilitators identified in this systematic review provide insight into the combination of D&I strategies that should be formulated and tested by D&I researchers in the sports injury prevention field.

Within the current review, lack of time, whether it be learning, instructing and/or practicing the programme, is a common barrier that plays a significant role in implementation. A recent narrative review focused solely on the barriers and facilitators associated with exercise-based warm-up programmes showed similar conclusions regarding time restrictions. 115 Collective themes within this literature for players, coaches and organisations found that reduced buy-in and support at different levels impacted the adoption of NMT warm-up programmes. The lack of awareness and knowledge of the injury prevention benefits of NMT warm-up programmes also presented major barriers to buy-in, leading to reduced implementation success. Future interventions should ensure that education about evidence-informed injury prevention outcomes associated with programme adherence is integrated within their D&I strategies.

D&I science is a growing field of study. A variety of D&I outcomes were identified such as self-efficacy, intention, reach, outcome expectancy and most commonly, adherence or adherence-related outcomes. These outcomes were evaluated using different D&I strategies for NMT warm-up programmes. The most commonly reported strategies were Workshops with supplementary Resources with/without in-season Personnel support. Evaluation of D&I outcomes showed that adherence or adherence-related outcomes were most frequently reported across studies. Various measures of adherence as defined by Owoeye et al (2020) were identified, including cumulative utilisation, utilisation frequency, utilisation fidelity, duration fidelity and exercise fidelity. 36

Adherence remains the most common D&I outcome in the sport injury prevention literature. 36 120 In this review, we defined adherence and adherence-related thresholds for a moderate-to-high dose-response to be ≥70% cumulative utilisation and/or ≥1.5 session/week to achieve the desired protective effects. This was done with consideration of pragmatism and a practical balance between programme efficacy and effectiveness given the existing literature. 24 91 32 of 40 studies (80%) from those with adherence or adherence-related outcomes had a potential D&I effect based on a moderate-to-high adherence or adherence-related outcome level. The use of WR and WRP was the most common D&I strategies for delivering NMT warm-up programmes. While there are several areas for improvement for the practical D&I of NMT warm-up programmes in youth sport settings, the use of comprehensive workshops and supplementary resources at various levels, particularly with coaches, appears to be the optimal best practice. However, only two ‘highly relevant’ D&I studies (RCTs) from the current systematic review presented conclusions based on the effectiveness of D&I strategies and outcomes specifically.

Many studies (n=26/68; 38.2%) included both male and female participants; however, no sex-differences were described. When examining D&I outcomes, only 7/26 (26.9%) had moderate-to-high adherence when looking at both male and female youth players. In total, 84.6% of the female-only (11/13) and 72.7% of the male-only studies (8/11) reported moderate-to-high adherence levels. These findings suggest greater attention and efforts for adherence and implementation of NMT programmes in the male youth team sport setting may need further consideration compared with the female youth sport context.

Of the preliminary evidence for Type 2 and 3 hybrid designs, the literature highlighted in the synthesis of this data that WR are effective strategies in injury prevention and showed more moderate-to-high adherence levels. Given that most studies are doing some form of WR, adding in-season personnel support does not increase the protective effect and may be less sustainable given that resources, time and support are significant barriers to the D&I of these programmes.

Additionally, greater implementation and programme buy-in were found in studies where uptake of these NMT programmes was supported across multiple stakeholders, particularly at the organisation level. 19 67 90 103 112 Catering to programme deliverers (coaches, organisations, parents) and evaluating their awareness, perception and self-efficacy may help further inform our understanding of D&I and how we can best work to promote programme uptake further.

D&I strategies and injury outcomes

The findings from this systematic review suggest that while various D&I intervention strategies are effective at reducing injuries in youth team sports, the ranges of injury rate ratios are similar across studies employing different strategies (32–88% lower injury rates across WR strategy studies and 41–77% lower injury rates across WRP strategy studies). 22 53 56 Although this was not the proposed evaluation of these studies, our findings demonstrate that the use of workshops may influence D&I success and the availability of supplementary resources alone may not be efficacious. Future evaluation of the influence of delivery strategies should be considered.

Future directions

Using facilitators to reduce barrier burden.

Regarding NMT strategy evaluation, our findings show that most of the current programmes focus on efficacy over effectiveness, particularly in the aspect of intervention flexibility; this suggests a need for the adaptation of NMT programmes to fit local contexts. NMT programme developers should consider more enjoyable and user-friendly exercises that include sport-specific activities (eg, ball work, partner drills, tags). Increasing variations also improves player buy-in and increases intrinsic motivation. At a coaching level, workshops on NMT programmes should include evidence-informed education on the injury prevention benefits and should incorporate content addressing coach self-efficacy to enhance implementation quality. 16 100 121 An ongoing pragmatic evaluation of NMT programme effectiveness is warranted as they undergo adaptation to local contexts.

Organisations have expressed limited knowledge and education for implementation as a significant barrier to successful NMT programme use. 90 99 101 105 112 115 122 Implementers should look to provide accessible resources and encourage further support from multiple stakeholders, including the governing bodies. This could lead to policy changes within the club and result in greater uptake of these programmes long-term. Collaborations among stakeholders (researchers, youth sport administrators, coaches and players) in programme development, evaluation, D&I are necessary to improve efforts for impactful practical translation of programmes.

Research recommendations

The support for NMT programmes within youth sport is extensive. 28 Although these programmes have been shown to be effective for injury prevention in many sports, 10 11 sport representation across D&I studies in our review was limited. Scaling up of NMT programmes and supporting continued research into other sports is vital for increased context-specific D&I of these programmes to reduce the overall burden of youth sport injuries.

Compliance and adherence were often used interchangeably, despite having distinct definitions. Although their mathematical calculations are similar, these two constructs are contextually different. Compliance refers to individuals conforming to prescribed recommendations in controlled intervention settings, 123 while adherence refers to the agreement of an individual’s behaviour to recommended evidence-based interventions in uncontrolled settings. 36 Standardised definitions should be considered more frequently by researchers to build on current knowledge and inform future D&I research.

Using D&I frameworks/models can improve NMT programme implementation success in a practical setting. 71 124 Application of D&I frameworks/models, including behaviour change models, 124 is limited in injury prevention and this is reflected in the current systematic review. Future studies should use D&I frameworks/models to help guide the implementation of these NMT programmes. In doing so, researchers can gain a better understanding of the contextual and behaviour change aspects related to youth sport injury prevention. 115

Limitations

Given the broad nature of our research question, specific results were required for inclusion. Despite being specific to our objectives, our limitations set for participant age range, team sport settings and English language studies only, may have resulted in missing other studies that evaluated D&I interventions and outcomes related to NMT programmes.

Due to the heterogeneous nature of studies, meta-analysis was not possible for any of our objectives. Inclusion of various study designs, although comprehensive, impeded this process and resulted in inconsistent injury and adherence definitions across our population of interest. Furthermore, the subjective nature of many qualitative studies included may have resulted in variability within the data extracted. With the varied definitions used for each specific outcome and design, we looked to consolidate the terminology used into more succinct and unified language and we encourage this to be employed by researchers.

Methodological flaws existed in the included studies that may warrant caution about the interpretation of our conclusions. For example, many of the included studies did not include power calculations or reported low power, increasing the chance of Type 2 error. Further, many studies did not consider confounding or effect modification in their analyses or failed to report the validity of measurement tools used for injury data collection. We also acknowledge that publication bias may have favoured the inclusion of studies demonstrating significant findings (eg, effectiveness, efficacy). By considering quality assessment as an objective, we aimed to account for these limitations.

There was limited evidence supporting the effect of D&I strategies on D&I-specific outcomes. There were only two high-level evidence (RCTs) studies in this review that directly discussed the matter of D&I strategies on D&I outcomes. 55 56 D&I-related outcomes were evaluated as secondary objectives in other high-level evidence studies, therefore, we could only examine the relationship between D&I strategy and outcome to assess if the strategy used resulted in moderate-to-high adherence levels, given our pre-established thresholds.

Conclusions

This systematic review demonstrates that: (1) Few D&I-related studies are based on D&I or behaviour change theories, frameworks or models; (2) few RCTs have examined the effectiveness of D&I strategies for delivering NMT programmes; (3) programme flexibility and time restrictions are the most common barriers to implementation and; (4) a combination of coach workshops and supplementary resources are currently the primary strategy facilitating NMT programme D&I; however their effectiveness is only evaluated in a few studies. This systematic review provides foundational evidence to facilitate evidence-informed knowledge translation practices in youth sport injury prevention. Transitioning to more high-quality D&I research RCTs and quasi-experimental designs that leverage current knowledge of barriers and facilitators, incorporates Type 2 or Type 3 hybrid approaches and uses behaviour change frameworks are important next steps to optimise the translation of NMT programmes into routine practice in youth team sport settings.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

• Cumming SP ,
• Australian Sports Commission
• Meeuwisse DW ,
• Eliason PH , et al
• Child and Adolescent Health Measurement Initiative
• Emmonds S ,
• Weaving D ,
• Lara-Bercial S , et al
• Merkel DL ,
• Raysmith BP ,
• Charlton PC
• Brunner R ,
• Friesenbichler B ,
• Casartelli NC , et al
• Whittaker JL , et al
• Meeuwisse WH
• Gianotti S ,
• Jacquet C ,
• Verhagen E , et al
• Marshall DA ,
• Lopatina E ,
• Lacny S , et al
• Owoeye OBA ,
• Akinbo SRA ,
• Tella BA , et al
• Palacios-Derflingher LM ,
• Pasanen K ,
• Parkkari J ,
• Pasanen M , et al
• Soligard T ,
• Myklebust G ,
• Steffen K , et al
• Thorborg K ,
• Krommes KK ,
• Esteve E , et al
• Webster KE ,
• Räisänen AM , et al
• Räisänen AM ,
• van den Berg C ,
• Owoeye OBA , et al
• Pfeifer K , et al
• Richmond SA ,
• Doyle-Baker PK , et al
• Barengo N ,
• Meneses-Echávez J ,
• Ramírez-Vélez R , et al
• Bizzini M ,
• Sadigursky D ,
• De Lira DNL , et al
• Åkerlund I ,
• Sonesson S , et al
• Hislop MD ,
• Stokes KA ,
• Williams S , et al
• Hopper AJ ,
• Joyce C , et al
• Bergeron MF ,
• Mountjoy M ,
• Armstrong N , et al
• De Ste Croix M ,
• Sanchez SH , et al
• Creech MJ ,
• Peterson DC , et al
• Rauvola RS ,
• Brownson RC
• Owoeye OB ,
• Olawale OA , et al
• Niederer D ,
• Vogt L , et al
• Ardern CL ,
• Büttner F ,
• Andrade R , et al
• Guyatt GH ,
• Vist GE , et al
• Kunz R , et al
• Siemieniuk R ,
• Maddox CD ,
• Subialka JA ,
• Young JL , et al
• Wieland LS ,
• Berman BM ,
• Altman DG , et al
• Landes SJ ,
• McBain SA ,
• Schwartz S ,
• Duncan DT , et al
• Ranganathan P
• Proctor E ,
• Silmere H ,
• Raghavan R , et al
• Schwarzer R
• Hägglund M ,
• Atroshi I ,
• Wagner P , et al
• Steffen K ,
• Romiti M , et al
• Sugimoto D ,
• Barber Foss KD , et al
• Schneider B , et al
• Meeuwisse WH ,
• Al Attar WSA ,
• Alzahrani H , et al
• Alizadeh MH ,
• Shahrbanian S , et al
• Waldén M , et al
• Akasaka K ,
• Otsudo T , et al
• Leppänen M ,
• Vasankari T , et al
• LaBella CR ,
• Huxford MR ,
• Grissom J , et al
• Loppini M ,
• Berton A , et al
• Parsons JL ,
• Carswell J ,
• Nwoba IM , et al
• Rössler R ,
• Bizzini M , et al
• Slauterbeck JR ,
• Choquette R ,
• Tourville TW , et al
• Olsen OE , et al
• Mattacola CG ,
• Bush HM , et al
• Hancock MV ,
• Stokes KA , et al
• RÄisÄnen AM , et al
• Barboza SD ,
• Emery C , et al
• Hellquist E ,
• Ahlqvist K , et al
• Tomsovsky L ,
• Whatman C , et al
• Verhagen E ,
• Gouttebarge V , et al
• Celebrini RG ,
• Miller WC , et al
• Franchina M ,
• Tercier S , et al
• Register-Mihalik J ,
• Choquette R , et al
• Lindblom H ,
• Carlfjord S ,
• Ljunggren G ,
• Perera NKP ,
• Merrett CK ,
• Linnéll J , et al
• Myklebust G , et al
• O’Brien J ,
• Santner E ,
• Befus K , et al
• Räisänen A ,
• Black AM , et al
• Sonesson S ,
• Lindblom H , et al
• Nilstad A ,
• Thein-Nissenbaum J ,
• Ageberg E ,
• Lucander K , et al
• Brodin EM ,
• Watkins R ,
• Cornelissen MH ,
• Baan A , et al
• Donaldson A ,
• Callaghan A ,
• Taylor JR ,
• Novak MA , et al
• MacFarlane AJ ,
• Weiss-Laxer NS , et al
• McGuine TA ,
• Pennuto A , et al
• Munoz-Plaza C ,
• Davis A , et al
• Norcross MF ,
• Johnson ST ,
• Bovbjerg VE , et al
• Jimenez P ,
• Oliver JL , et al
• Denegar CR , et al
• Lambert M , et al
• Withall AL , et al
• Minnig MC ,
• Hawkinson L ,
• Root HJ , et al
• Macpherson A , et al
• Fenelon D , et al
• Bruinsma A , et al
• Verhagen EALM , et al
• Brown JC , et al
• Bogardus RL ,
• Martin RJ ,
• Richman AR , et al

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

• Data supplement 1
• Data supplement 2

X @carlavdb_, @amraisanen, @KatiPasanen, @Kat_Schneider7, @CarolynAEmery, @owoeye_oba

Contributors DL, CE and OBAO contributed to development of study proposal and design. DL, CvdB, AMR, IJS, KV, JK, AH, CE and OBAO conducted search, study selection and screening, data extraction and synthesis and quality assessment. DL led the writing of the manuscript and was the guarantor for the project. All authors contributed to drafting and revising the final manuscript. All authors approved the submitted version of the manuscript.

Funding This study was funded by Canadian Institutes for Health Research Foundation Grant Program (PI CAE).

Competing interests OBAO is a Deputy Editor for the British Journal of Sports Medicine. CE, KJS and KP are Associate Editors for the British Journal of Sports Medicine.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF: