is a systematic review original research

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Systematic Review | Definition, Example, & Guide

Systematic Review | Definition, Example & Guide

Published on June 15, 2022 by Shaun Turney . Revised on November 20, 2023.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question “What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?”

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

Table of contents

What is a systematic review, systematic review vs. meta-analysis, systematic review vs. literature review, systematic review vs. scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, other interesting articles, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce bias . The methods are repeatable, and the approach is formal and systematic:

• Formulate a research question
• Develop a protocol
• Search for all relevant studies
• Apply the selection criteria
• Extract the data
• Synthesize the data
• Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesizing all available evidence and evaluating the quality of the evidence. Synthesizing means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Systematic reviews often quantitatively synthesize the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesize results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarize and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimize bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

Prevent plagiarism. Run a free check.

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

• A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
• If you’re doing a systematic review on your own (e.g., for a research paper or thesis ), you should take appropriate measures to ensure the validity and reliability of your research.
• Access to databases and journal archives. Often, your educational institution provides you with access.
• Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
• Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

• They minimize research bias by considering all available evidence and evaluating each study for bias.
• Their methods are transparent , so they can be scrutinized by others.
• They’re thorough : they summarize all available evidence.
• They can be replicated and updated by others.

Systematic reviews also have a few cons .

• They’re time-consuming .
• They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

• Allow you to more effectively communicate your research to other researchers and practitioners
• Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

• Population(s) or problem(s)
• Intervention(s)
• Comparison(s)

You can rearrange these four components to write your research question:

• What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fifth component, the type of study design . In this case, the acronym is PICOT .

• Type of study design(s)
• The population of patients with eczema
• The intervention of probiotics
• In comparison to no treatment, placebo , or non-probiotic treatment
• The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
• Randomized control trials, a type of study design

Their research question was:

• What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

• Background information : Provide the context of the research question, including why it’s important.
• Research objective (s) : Rephrase your research question as an objective.
• Selection criteria: State how you’ll decide which studies to include or exclude from your review.
• Search strategy: Discuss your plan for finding studies.
• Analysis: Explain what information you’ll collect from the studies and how you’ll synthesize the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

• Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
• Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
• Gray literature: Gray literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of gray literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of gray literature.
• Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

• Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
• Handsearch: Conference proceedings and reference lists of articles
• Gray literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
• Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

• Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
• Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarize what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

• Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
• Your judgment of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomized into the control and treatment groups.

Step 6: Synthesize the data

Synthesizing the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesizing the data:

• Narrative ( qualitative ): Summarize the information in words. You’ll need to discuss the studies and assess their overall quality.
• Quantitative : Use statistical methods to summarize and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analyzed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

• Abstract : A summary of the review
• Introduction : Including the rationale and objectives
• Methods : Including the selection criteria, search method, data extraction method, and synthesis method
• Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
• Discussion : Including interpretation of the results and limitations of the review
• Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

In their report, Boyle and colleagues concluded that probiotics cannot be recommended for reducing eczema symptoms or improving quality of life in patients with eczema. Note Generative AI tools like ChatGPT can be useful at various stages of the writing and research process and can help you to write your systematic review. However, we strongly advise against trying to pass AI-generated text off as your own work.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Prospective cohort study

Research bias

• Implicit bias
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic
• Social desirability bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Turney, S. (2023, November 20). Systematic Review | Definition, Example & Guide. Scribbr. Retrieved April 3, 2024, from https://www.scribbr.com/methodology/systematic-review/

Is this article helpful?

Shaun Turney

Other students also liked, how to write a literature review | guide, examples, & templates, how to write a research proposal | examples & templates, what is critical thinking | definition & examples, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Scientific Value of Systematic Reviews: Survey of Editors of Core Clinical Journals

Contributed equally to this work with: Joerg J. Meerpohl, Florian Herrle

* E-mail: [email protected]

Affiliations German Cochrane Center, Institute of Medical Biometry and Medical Informatics, University Medical Center Freiburg, Freiburg, Germany, Pediatric Hematology and Oncology, Center for Pediatrics and Adolescent Medicine, University Medical Center Freiburg, Freiburg, Germany

Affiliations German Cochrane Center, Institute of Medical Biometry and Medical Informatics, University Medical Center Freiburg, Freiburg, Germany, Department of Surgery, University Medical Center Mannheim, University of Heidelberg, Mannheim, Germany

Affiliation German Cochrane Center, Institute of Medical Biometry and Medical Informatics, University Medical Center Freiburg, Freiburg, Germany

Affiliations German Cochrane Center, Institute of Medical Biometry and Medical Informatics, University Medical Center Freiburg, Freiburg, Germany, Cochrane Switzerland, IUMSP, University Hospital Lausanne, Lausanne, Switzerland

• Joerg J. Meerpohl, 
• Florian Herrle, 
• Gerd Antes, 
• Erik von Elm

• Published: May 1, 2012
• https://doi.org/10.1371/journal.pone.0035732
• Reader Comments

4 Oct 2012: Meerpohl JJ, Herrle F, Reinders S, Antes G, von Elm E (2012) Correction: Scientific Value of Systematic Reviews: Survey of Editors of Core Clinical Journals. PLOS ONE 7(10): 10.1371/annotation/b9a9cb87-3d96-47e4-a073-a7e97a19f47c. https://doi.org/10.1371/annotation/b9a9cb87-3d96-47e4-a073-a7e97a19f47c View correction

Synthesizing research evidence using systematic and rigorous methods has become a key feature of evidence-based medicine and knowledge translation. Systematic reviews (SRs) may or may not include a meta-analysis depending on the suitability of available data. They are often being criticised as ‘secondary research’ and denied the status of original research. Scientific journals play an important role in the publication process. How they appraise a given type of research influences the status of that research in the scientific community. We investigated the attitudes of editors of core clinical journals towards SRs and their value for publication.

We identified the 118 journals labelled as “core clinical journals” by the National Library of Medicine, USA in April 2009. The journals’ editors were surveyed by email in 2009 and asked whether they considered SRs as original research projects; whether they published SRs; and for which section of the journal they would consider a SR manuscript.

The editors of 65 journals (55%) responded. Most respondents considered SRs to be original research (71%) and almost all journals (93%) published SRs. Several editors regarded the use of Cochrane methodology or a meta-analysis as quality criteria; for some respondents these criteria were premises for the consideration of SRs as original research. Journals placed SRs in various sections such as “Review” or “Feature article”. Characterization of non-responding journals showed that about two thirds do publish systematic reviews.

Currently, the editors of most core clinical journals consider SRs original research. Our findings are limited by a non-responder rate of 45%. Individual comments suggest that this is a grey area and attitudes differ widely. A debate about the definition of ‘original research’ in the context of SRs is warranted.

Citation: Meerpohl JJ, Herrle F, Antes G, von Elm E (2012) Scientific Value of Systematic Reviews: Survey of Editors of Core Clinical Journals. PLoS ONE 7(5): e35732. https://doi.org/10.1371/journal.pone.0035732

Editor: Nitika Pant Pai, McGill University Health Centre, McGill University, Canada

Received: October 13, 2011; Accepted: March 21, 2012; Published: May 1, 2012

Copyright: © 2012 Meerpohl et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Funding: The authors have no support or funding to report.

Competing interests: The authors have read the journal’s policy and have the following conflicts: They declare that they have no financial competing interests. Erik von Elm is a PLoS ONE Editorial Board Member. All of the authors are or have been involved with The Cochrane Collaboration, which produces systematic reviews. This does not alter the authors’ adherence to all the PLoS ONE policies on sharing data and materials.

Introduction

Since the first comparative study to answer a therapeutic question by James Lind in the 18 th century [1] , the number of medical research studies is ever increasing. Accordingly, the need to synthesize research evidence has been recognized for over two centuries. A more formal approach to systematically synthesizing the research evidence that accumulates in medical science in a systematic review was not developed until the late 20 th century and gained momentum with the advent of evidence-based medicine. [2] Besides providing a comprehensive overview, systematic reviews help to identify areas where further research is needed or, inversely, might be unnecessary or even unethical [3] .

Evidence-based medicine has been called a “new paradigm” because it asks questions about health care in an answerable format and considers the best evidence available from clinical research. In order to keep abreast of the large quantity of new data being generated continuously, systematic reviews have become the central and indispensable tool of evidence-based medicine [4] .

In contrast to classical narrative reviews, systematic reviews use an explicit and rigorous methodology. They start with a clearly stated set of clinically relevant questions and pre-defined criteria for study inclusion. The scientific literature is then systematically searched with the aim of identifying all potentially relevant studies. After application of eligibility criteria, the included studies are assessed for their internal validity, in particular the risk of bias. If possible, data are combined using meta-analytic methods. [5] By statistically combining information from all or part of the included studies, meta-analyses can provide pooled estimates that are more precise than those derived from individual studies. [6] Presence or absence of a meta-analysis does not represent a quality criterion since it is directly dependent on the studies identified and data available for inclusion in the systematic review. Finally, results of any systematic review need to be interpreted in the light of random error and considering external validity or applicability of results.

The general concept of systematic reviews and their methodology were originally developed in the social sciences. In medicine they have been applied predominantly for the evaluation of treatments. [5] With some adaptations the general methodology of systematic reviews is also applicable to questions of diagnostic test accuracy and prognosis. [7] – [9] Systematic reviews usually provide more reliable findings than individual studies or non-systematic narrative reviews. [10] – [13] Consequently, more robust conclusions can be drawn which, in turn, may inform decision making on different levels from individual patient care to the organization of health care systems.

Over the last 10 to 15 years, the number of published systematic reviews has increased markedly. [14] When carried out before the start of new clinical studies, a systematic review can help to optimize the allocation of limited research resources. Consequently, leading funding agencies such as the UK Medical Research Council require systematic reviews as part of grant applications. [15] Leading medical journals now advocate a systematic overview of the evidence as part of published reports of new randomized trials. [16] In recent years the role of research syntheses has been further strengthened by the decision of the U.S. government in 2009 to allocate $1.1 billion to comparative-effectiveness research (CER) under the framework of the American Recovery and Reinvestment Act. [17] The Institute of Medicine (IOM) recently published two reports which underline the relevance of systematic review methodology both for CER and evidence-based clinical practice guidelines [18] , [19] .

Despite these important functions and the recent prominent government support, the status and value of systematic reviews is still being disputed in academia. The debate is partly fueled by persistent misconceptions. [20] In the past, systematic reviews have been dubbed “secondary research” in contrast to “primary or original research”, implying that they were less scientifically novel and required less methodological rigor than studies deemed primary research. Early opponents even spoke of “mega-silliness” and “statistical alchemy for the 21 st century” [21] .

At present there is considerable heterogeneity across countries with regard to both the funding available for systematic reviews and their academic recognition. While in some countries, such as the U.K. and The Netherlands, medical faculties have established professorships and academic units for systematic reviews the related methods still play only a minor role in medical student education in other countries. Similarly, the way methods of research synthesis are adopted and used varies widely across medical specialty fields.

Scientific journals play an important role in the dissemination of scientific knowledge by setting quality standards and determining the way in which research is being published. Journal editors can thus be considered gatekeepers. How they appraise a given type of research activity influences the recognition it receives in the scientific community. Given the influential role of journal editors, we set out to elucidate their attitudes towards systematic reviews. In particular, we were interested in the scientific status attributed to systematic reviews and their acceptability for publication in clinical journals.

• consider a systematic review manuscript an original research project
• publish a systematic review in their journal, and
• in which section of their journal they would publish a systematic review.

Two investigators then independently evaluated and classified the responses. If discrepancies occurred, consensus was reached in discussion with a third investigator. We extracted the ISI impact factor of the included journals from the Journal Citation Report 2009. [22] All data were collated in a spreadsheet in Microsoft Excel and used for descriptive statistics.

To characterize the group of non-responding journals, we developed a three step process that entailed 1) a PubMed search for systematic reviews classified as meta-analyses published in these journals in 2009, 2) hand-searching the content published in 2009 of journals for which we did not identify a meta-analysis in our PubMed search, 3) evaluation of author instructions of journals from point 2 (above) to determine whether they would have published systematic reviews.

Seventeen (14%) of the 118 journals were general medical journals and 101 (86%) were specialty journals. The majority of the journals were published in the USA ( Table 1 ). Editors of 65 journals (55%) responded to our survey. For three journals, not all questions were answered. The response rate was higher for editors of general medical journals (13/17; 76%) than for those of specialty journals (52/101; 52%). We received responses from 50% (51/101) of the U.S. journals and from 80% (12/15) of the British journals. The median ISI impact factor for responder journals was 2.99 (range 0.29 – 52.6) and for non-responder journals 3.61 (range 0.40 – 69.0).

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0035732.t001

Status of Systematic Reviews

Seventy-one percent (46/65) of editors regarded systematic reviews as original research projects. Nine of them (29%) did so only under certain premises ( Table 2 ). For some editors the use of Cochrane methodology [5] or meta-analytic methods were a criterion to decide whether a systematic review is considered original research. For illustration, Table 3 includes some excerpts from the responses.

https://doi.org/10.1371/journal.pone.0035732.t002

https://doi.org/10.1371/journal.pone.0035732.t003

Acceptability for Publication

Of 64 respondents, 60 (94%) published systematic reviews ( Table 2 ). Six (9%) did so only rarely and four (6%) not at all. About a third of the journals published systematic reviews in a section dedicated to original research articles and a third in a specific section for (systematic) reviews. Some journals either featured them as special articles or placed them in other sections ( Table 2 ).

Non-responding Journals

Of the 53 journals that did not respond to our survey, 30 (56.6%) published at least one systematic review in 2009, two (3.8%) would accept systematic reviews according to their author instructions, while 21 (39.6%) neither published any nor explicitly mention systematic reviews in their author instructions. Of those that published systematic reviews, in 18 of the 30 the systematic reviews were published in journal sections that contained original research articles.

We surveyed editors of core clinical journals and found that most of them regarded systematic reviews as original research. Nearly all of the journals represented by these editors published systematic reviews. The respective comments of the respondents indicate that this is an ill-defined area. This was mirrored by the variety of criteria used to decide whether a submitted systematic review manuscript represented original research or not. For some respondents, the inclusion of a meta-analysis was the key argument while others looked at the methods being used.

In our set of core clinical journals the general attitude towards systematic reviews was rather positive. It is conceivable that a broader sample of biomedical journals, e.g., including basic sciences journals, would have yielded a more conservative picture. The main limitation of our survey is that about 45% of the contacted journals did not respond which could potentially significantly change the results and affect the interpretation. While the proportion of non-responding journals that published systematic reviews was lower, the majority still published at least one in 2009. Interestingly, more than half of the non-responder journals that published systematic reviews seemed to consider them original research. If one assumes that the non-responding editors are more skeptical about systematic reviews than those who responded then the results may be less positive overall.

From the large spectrum of responses we conclude that a debate about the status and the academic recognition of systematic reviews is warranted. A next step should be an in-depth analysis of the views of different stakeholders including researchers, funders, users of systematic reviews (e.g., policy makers) and again journal editors.

Ideally, the clinical research community would accept systematic reviews as a research category of its own, which is defined by methodological criteria, as is the case for other types of research. With certain quality criteria being fulfilled, systematic reviews should not be denied the appropriate academic recognition they deserve. Under these premises their scientific value should be on par with conventional original research studies. This argument becomes even more compelling when one considers that systematic reviews are essentially observational studies of aggregate or individual data from previous studies.

Due to the continuous work of The Cochrane Collaboration and other international institutions and networks, the use and recognition of systematic reviews has increased considerably over the last 15 years. [4] However, the limited funding opportunities available for systematic review projects represent a main barrier to an even wider implementation. A clarification of the scientific status of systematic reviews might motivate researchers to undertake such projects to an even larger extent. If high-quality systematic reviews are accepted as valid research projects by the research community, then funding agencies might also be more open to financially support them e.g. by creating specific grant schemes.

In conclusion, the attitudes of editors of clinical journals vary with regard to the value given to systematic reviews. Most responding editors regarded systematic reviews as original research projects based on varying criteria. This interpretation is limited by a non-responder rate of 45%. A debate about the scientific value of systematic reviews and their academic recognition is warranted and would help establish sustainable programs of evidence synthesis across countries and different fields of clinical research.

Supporting Information

Journals invited to participate in survey.

https://doi.org/10.1371/journal.pone.0035732.s001

Acknowledgments

We would like to thank Stefan Reinders and Rebecca Weida for their assistance. The open access publication of this work was supported by the Deutsche Forschungsgemeinschaft.

Author Contributions

Conceived and designed the experiments: JJM FH GA EvE. Performed the experiments: JJM FH. Analyzed the data: JJM FH EvE. Wrote the paper: JJM FH GA EvE.

• 1. Lind J (1753) A treatise of the scurvy. In: nature InthreepartsContaininganinquiryintothe, causes , cure , editors. of that disease. Together with a critical and chronological view of what has been published on the subject. Edinburgh: Sands, Murray and Cochran for A Kincaid and A Donaldson.
• View Article
• Google Scholar
• 5. Higgins JP, Green S (2011) (2011) Cochrane Handbook for Systematic Reviews of Interventions; Version 5.1.0, available from http://www.cochrane-handbook.org . Chichester, UK John Wiley & Sons, Ltd.
• 6. Deeks J, Higgins JP, Altman D (2011) Analysing data and undertaking meta-analyses. In: Higgins JP, Green S, editors. Cochrane Handbook for systematic review of interventions. Chichester, UK: John Wiley & Sons.
• 18. Institute of Medicine (U.S.). Committee on Standards for Systematic Reviews of Comparative Effectiveness Research (2011) Finding What Works In Health Care: Standards For Systematic Reviews.
• 19. Institute of Medicine (U.S.). Committee on Standards for Systematic Reviews of Comparative Effectiveness Research (2011) Clinical Practice Guidelines We Can Trust.
• 22. ISI Web of Knowledge (2009) Journal Citation Report 2008; available at http://wokinfo.com/products_tools/analytical/jcr/ ; accessed in March 2010.

Conducting a Systematic Review

What is a systematic review, a systematic review is not a literature review, recent systematic reviews at nymc, where do i find published systematic reviews.

• Take the Class on Conducting a Systematic Review
• Starting a Systematic Review
• EndNote Desktop This link opens in a new window
• What is a Scoping Review
• Collaborating with a Librarian This link opens in a new window

In a nutshell, a systematic review is a secondary study from a collection of primary studies (original research) that pertain to a specific research question. Those primary studies have been analyzed, examined, appraised, and evaluated for the highest level of evidence and quality of methodology, in order to provide the best answer to a research question.

The methods and/or protocol of conducting a systematic review are clear, understandable, rigorous and transparent. In a systematic review, you should see:

• A clearly stated objective.
• Search Methods: The where, and the how. 
• Types of participants, problem, study design
• Interventions, exposures, outcomes.
• See the example below:

Additional Definitions: 

Systematic reviews are often confused with review articles. This type of publication are often called narrative reviews or literature reviews. 

Literature reviews (or narrative reviews or review articles) are valuable and help to build one's knowledge base on a topic. However, they are quite different from systematic reviews.

• There is no  research question.
• It is an overview or BROAD examination of the published literature on a topic.
• There is no critical appraisal of the research. No stated methods as it is not study, but rather a narrative review of the studies. 

1.      Aiman W, Ali MA, Jumean S, et al. BRAF Inhibitors in BRAF-Mutated Colorectal Cancer: A Systematic Review. J Clin Med. Dec 25 2023;13(1)doi:10.3390/jcm13010113

2.      Covell MM, Roy JM, Rumalla K, et al. The Limited Utility of the Hospital Frailty Risk Score as a Frailty Assessment Tool in Neurosurgery: A Systematic Review. Neurosurgery. Feb 1 2024;94(2):251-262.  doi:10.1227/neu.0000000000002668

3.      Drivas E, Gachabayov M, Kajmolli A, Stadlan Z, Felsenreich DM, Castaldi M. Quilting Suture Technique After Mastectomy: A Meta-Analysis. Am Surg. Dec 2023;89(12):6045-6052. doi:10.1177/00031348231173995

4.      Faden J, Citrome L. A systematic review of clozapine for aggression and violence in patients with schizophrenia or schizoaffective disorder. Schizophr Res. Jan 29 2024;doi:10.1016/j.schres.2023.11.008

5.      Gachabayov M, Kajmolli A, Quintero L, et al. Inadvertent laparoscopic lavage of perforated colon cancer: a systematic review. Langenbecks Arch Surg. Jan 10 2024;409(1):35. doi:10.1007/s00423-023-03224-5

6.      Ingrassia JP, Maqsood MH, Gelfand JM, et al. Cardiovascular and Venous Thromboembolic Risk With JAK Inhibitors in Immune-Mediated Inflammatory Skin Diseases: A Systematic Review and Meta-Analysis. JAMA Dermatol. Jan 1 2024;160(1):28-36. doi:10.1001/jamadermatol.2023.4090

7.      Ligi D, Della Franca C, Notarte KI, et al. Platelet distribution width (PDW) as a significant correlate of COVID-19 infection severity and mortality. Clin Chem Lab Med. Feb 26 2024;62(3):385-395. doi:10.1515/cclm-2023-0625

8.      Nduma BN, Al-Ajlouni YA, Njei B. The Application of Artificial Intelligence (AI)-Based Ultrasound for the Diagnosis of Fatty Liver Disease: A Systematic Review. Cureus. Dec 2023;15(12):e50601. doi:10.7759/cureus.50601

9.      Sreenivasan J, Reddy RK, Jamil Y, et al. Intravascular Imaging-Guided Versus Angiography-Guided Percutaneous Coronary Intervention: A Systematic Review and Meta-Analysis of Randomized Trials. J Am Heart Assoc. Jan 16 2024;13(2):e031111. doi:10.1161/jaha.123.031111

10.    Stifani BM, Lavelanet AF. Reversal of medication abortion with progesterone: a systematic review. BMJ Sex Reprod Health. Jan 9 2024;50(1):43-52. doi:10.1136/bmjsrh-2023-201875

• The  Campbell Collaboration  is an international research network that produces systematic reviews of the effects of social interventions focusing on education, crime and justice, and social welfare.
• The  Centre for Reviews and Dissemination  (CRD) databases are updated daily and provide decision-makers with access to quality assessed systematic reviews, economic evaluations, summaries of health technology assessments, summaries of all Cochrane reviews and protocols, and summaries of Campbell reviews.
• The Cochrane Database of Systematic Reviews (CDSR) includes the full text of regularly updated systematic reviews of the effects of healthcare prepared by The Cochrane Collaboration.
• The Database of Abstracts of Reviews of Effects (DARE) covers a broad range of health related interventions and complements the CDSR by quality-assessing and summarizing reviews that have not yet been carried out by the Cochrane Collaboration. Each abstract includes a summary of the review together with a critical commentary about the overall quality.
• Both  PubMed and PubMed Clinical Queries  can be used to locate systematic reviews. The Clinical Queries tool retrieves citations identified as systematic reviews, meta-analyses, reviews of clinical trials, evidence-based medicine, consensus development conferences, guidelines, and citations to articles from journals specializing in review studies of value to clinicians.

NOTE:  Linking to the Cochrane Databases and PubMed through the  Health Sciences Library  ensures that you have access to the full text of systematic reviews and articles when available.

• Next: Take the Class on Conducting a Systematic Review >>
• Last Updated: Feb 1, 2024 9:26 AM
• URL: https://guides.library.nymc.edu/systematic_review

Introduction to Systematic Reviews

• Reference work entry
• First Online: 20 July 2022
• pp 2159–2177
• Cite this reference work entry

• Tianjing Li 3 ,
• Ian J. Saldanha 4 &
• Karen A. Robinson 5  

196 Accesses

A systematic review identifies and synthesizes all relevant studies that fit prespecified criteria to answer a research question. Systematic review methods can be used to answer many types of research questions. The type of question most relevant to trialists is the effects of treatments and is thus the focus of this chapter. We discuss the motivation for and importance of performing systematic reviews and their relevance to trialists. We introduce the key steps in completing a systematic review, including framing the question, searching for and selecting studies, collecting data, assessing risk of bias in included studies, conducting a qualitative synthesis and a quantitative synthesis (i.e., meta-analysis), grading the certainty of evidence, and writing the systematic review report. We also describe how to identify systematic reviews and how to assess their methodological rigor. We discuss the challenges and criticisms of systematic reviews, and how technology and innovations, combined with a closer partnership between trialists and systematic reviewers, can help identify effective and safe evidence-based practices more quickly.

• Systematic review
• Meta-analysis
• Research synthesis
• Evidence-based
• Risk of bias

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

AHRQ (2015) Methods guide for effectiveness and comparative effectiveness reviews. Available from https://effectivehealthcare.ahrq.gov/products/cer-methods-guide/overview . Accessed on 27 Oct 2019

Andersen MZ, Gülen S, Fonnes S, Andresen K, Rosenberg J (2020) Half of Cochrane reviews were published more than two years after the protocol. J Clin Epidemiol 124:85–93. https://doi.org/10.1016/j.jclinepi.2020.05.011

Article   Google Scholar  

Berkman ND, Lohr KN, Ansari MT, Balk EM, Kane R, McDonagh M, Morton SC, Viswanathan M, Bass EB, Butler M, Gartlehner G, Hartling L, McPheeters M, Morgan LC, Reston J, Sista P, Whitlock E, Chang S (2015) Grading the strength of a body of evidence when assessing health care interventions: an EPC update. J Clin Epidemiol 68(11):1312–1324

Borah R, Brown AW, Capers PL, Kaiser KA (2017) Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open 7(2):e012545. https://doi.org/10.1136/bmjopen-2016-012545

Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, Howells DW, Ioannidis JP, Oliver S (2014) How to increase value and reduce waste when research priorities are set. Lancet 383(9912):156–165. https://doi.org/10.1016/S0140-6736(13)62229-1

Clarke M, Chalmers I (1998) Discussion sections in reports of controlled trials published in general medical journals: islands in search of continents? JAMA 280(3):280–282

Cooper NJ, Jones DR, Sutton AJ (2005) The use of systematic reviews when designing studies. Clin Trials 2(3):260–264

Djulbegovic B, Kumar A, Magazin A, Schroen AT, Soares H, Hozo I, Clarke M, Sargent D, Schell MJ (2011) Optimism bias leads to inconclusive results-an empirical study. J Clin Epidemiol 64(6):583–593. https://doi.org/10.1016/j.jclinepi.2010.09.007

Elliott JH, Synnot A, Turner T, Simmonds M, Akl EA, McDonald S, Salanti G, Meerpohl J, MacLehose H, Hilton J, Tovey D, Shemilt I, Thomas J (2017) Living systematic review network. Living systematic review: 1. Introduction-the why, what, when, and how. J Clin Epidemiol 91:23–30

Equator Network. Reporting guidelines for systematic reviews. Available from https://www.equator-network.org/?post_type=eq_guidelines&eq_guidelines_study_design=systematic-reviews-and-meta-analyses&eq_guidelines_clinical_specialty=0&eq_guidelines_report_section=0&s=+ . Accessed 9 Mar 2020

Garner P, Hopewell S, Chandler J, MacLehose H, Schünemann HJ, Akl EA, Beyene J, Chang S, Churchill R, Dearness K, Guyatt G, Lefebvre C, Liles B, Marshall R, Martínez García L, Mavergames C, Nasser M, Qaseem A, Sampson M, Soares-Weiser K, Takwoingi Y, Thabane L, Trivella M, Tugwell P, Welsh E, Wilson EC, Schünemann HJ (2016) Panel for updating guidance for systematic reviews (PUGs). When and how to update systematic reviews: consensus and checklist. BMJ 354:i3507. https://doi.org/10.1136/bmj.i3507 . Erratum in: BMJ 2016 Sep 06 354:i4853

Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, Norris S, Falck-Ytter Y, Glasziou P, DeBeer H, Jaeschke R, Rind D, Meerpohl J, Dahm P, Schünemann HJ (2011) GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol 64(4):383–394

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (eds) (2019a) Cochrane handbook for systematic reviews of interventions, 2nd edn. Wiley, Chichester

Google Scholar  

Higgins JPT, Lasserson T, Chandler J, Tovey D, Thomas J, Flemyng E, Churchill R (2019b) Standards for the conduct of new Cochrane intervention reviews. In: JPT H, Lasserson T, Chandler J, Tovey D, Thomas J, Flemyng E, Churchill R (eds) Methodological expectations of Cochrane intervention reviews. Cochrane, London

IOM (2011) Committee on standards for systematic reviews of comparative effectiveness research, board on health care services. In: Eden J, Levit L, Berg A, Morton S (eds) Finding what works in health care: standards for systematic reviews. National Academies Press, Washington, DC

Jonnalagadda SR, Goyal P, Huffman MD (2015) Automating data extraction in systematic reviews: a systematic review. Syst Rev 4:78

Krnic Martinic M, Pieper D, Glatt A, Puljak L (2019) Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks. BMC Med Res Methodol 19(1):203. Published 4 Nov 2019. https://doi.org/10.1186/s12874-019-0855-0

Lasserson TJ, Thomas J, Higgins JPT (2019) Chapter 1: Starting a review. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (eds) Cochrane handbook for systematic reviews of interventions version 6.0 (updated July 2019). Cochrane. Available from www.training.cochrane.org/handbook

Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers TC (1992) Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med 327(4):248–254

Lau J (2019) Editorial: systematic review automation thematic series. Syst Rev 8(1):70. Published 11 Mar 2019. https://doi.org/10.1186/s13643-019-0974-z

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 6(7):e1000100. https://doi.org/10.1371/journal.pmed.1000100

Lund H, Brunnhuber K, Juhl C, Robinson K, Leenaars M, Dorch BF, Jamtvedt G, Nortvedt MW, Christensen R, Chalmers I (2016) Towards evidence based research. BMJ 355:i5440. https://doi.org/10.1136/bmj.i5440

Marshall IJ, Noel-Storr A, Kuiper J, Thomas J, Wallace BC (2018) Machine learning for identifying randomized controlled trials: an evaluation and practitioner’s guide. Res Synth Methods 9(4):602–614. https://doi.org/10.1002/jrsm.1287

Michelson M, Reuter K (2019) The significant cost of systematic reviews and meta-analyses: a call for greater involvement of machine learning to assess the promise of clinical trials. Contemp Clin Trials Commun 16:100443. https://doi.org/10.1016/j.conctc.2019.100443 . Erratum in: Contemp Clin Trials Commun 2019 16:100450

Moher D, Liberati A, Tetzlaff J (2009) Altman DG; PRISMA group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med 151(4):264–269. W64

Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA, PRISMA-P Group (2015) Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 4(1):1. https://doi.org/10.1186/2046-4053-4-1

NIHR HTA Stage 1 guidance notes. Available from https://www.nihr.ac.uk/documents/hta-stage-1-guidance-notes/11743 ; Accessed 10 Mar 2020

Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, Catalá-López F, Li L, Reid EK, Sarkis-Onofre R, Moher D (2016) Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med 13(5):e1002028. https://doi.org/10.1371/journal.pmed.1002028

Page MJ, Higgins JPT, Sterne JAC (2019) Chapter 13: assessing risk of bias due to missing results in a synthesis. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ et al (eds) Cochrane handbook for systematic reviews of interventions, 2nd edn. Wiley, Chichester, pp 349–374

Chapter   Google Scholar  

Robinson KA (2009) Use of prior research in the justification and interpretation of clinical trials. Johns Hopkins University

Robinson KA, Goodman SN (2011) A systematic examination of the citation of prior research in reports of randomized, controlled trials. Ann Intern Med 154(1):50–55. https://doi.org/10.7326/0003-4819-154-1-201101040-00007

Rouse B, Cipriani A, Shi Q, Coleman AL, Dickersin K, Li T (2016) Network meta-analysis for clinical practice guidelines – a case study on first-line medical therapies for primary open-angle glaucoma. Ann Intern Med 164(10):674–682. https://doi.org/10.7326/M15-2367

Saldanha IJ, Lindsley K, Do DV et al (2017) Comparison of clinical trial and systematic review outcomes for the 4 most prevalent eye diseases. JAMA Ophthalmol 135(9):933–940. https://doi.org/10.1001/jamaophthalmol.2017.2583

Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM (2007) Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 7:10

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, Moher D, Tugwell P, Welch V, Kristjansson E, Henry DA (2017) AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 358:j4008. https://doi.org/10.1136/bmj.j4008

Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D (2007) How quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med 147(4):224–233

Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, Henry D, Altman DG, Ansari MT, Boutron I, Carpenter JR, Chan AW, Churchill R, Deeks JJ, Hróbjartsson A, Kirkham J, Jüni P, Loke YK, Pigott TD, Ramsay CR, Regidor D, Rothstein HR, Sandhu L, Santaguida PL, Schünemann HJ, Shea B, Shrier I, Tugwell P, Turner L, Valentine JC, Waddington H, Waters E, Wells GA, Whiting PF, Higgins JP (2016) ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 355:i4919. https://doi.org/10.1136/bmj.i4919

Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, Cates CJ, Cheng HY, Corbett MS, Eldridge SM, Emberson JR, Hernán MA, Hopewell S, Hróbjartsson A, Junqueira DR, Jüni P, Kirkham JJ, Lasserson T, Li T, McAleenan A, Reeves BC, Shepperd S, Shrier I, Stewart LA, Tilling K, White IR, Whiting PF, Higgins JPT (2019) RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 366:l4898. https://doi.org/10.1136/bmj.l4898

Thomas J, Kneale D, McKenzie JE, Brennan SE, Bhaumik S (2019) Chapter 2: determining the scope of the review and the questions it will address. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (eds) Cochrane handbook for systematic reviews of interventions version 6.0 (updated July 2019). Cochrane. Available from www.training.cochrane.org/handbook

USPSTF U.S. Preventive Services Task Force Procedure Manual (2017). Available from: https://www.uspreventiveservicestaskforce.org/uspstf/sites/default/files/inline-files/procedure-manual2017_update.pdf . Accessed 21 May 2020

Whitaker (2015) UCSF guides: systematic review: when will i be finished? https://guides.ucsf.edu/c.php?g=375744&p=3041343 , Accessed 13 May 2020

Whiting P, Savović J, Higgins JP, Caldwell DM, Reeves BC, Shea B, Davies P, Kleijnen J (2016) Churchill R; ROBIS group. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol 69:225–234. https://doi.org/10.1016/j.jclinepi.2015.06.005

Download references

Author information

Authors and affiliations.

Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, CO, USA

Tianjing Li

Department of Health Services, Policy, and Practice and Department of Epidemiology, Brown University School of Public Health, Providence, RI, USA

Ian J. Saldanha

Department of Medicine, Johns Hopkins University, Baltimore, MD, USA

Karen A. Robinson

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Tianjing Li .

Editor information

Editors and affiliations.

Department of Surgery, Division of Surgical Oncology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA

Steven Piantadosi

Department of Epidemiology, School of Public Health, Johns Hopkins University, Baltimore, MD, USA

Curtis L. Meinert

Section Editor information

Department of Epidemiology, University of Colorado Denver Anschutz Medical Campus, Aurora, CO, USA

The Johns Hopkins Center for Clinical Trials and Evidence Synthesis, Johns Hopkins University, Baltimore, MD, USA

Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this entry

Cite this entry.

Li, T., Saldanha, I.J., Robinson, K.A. (2022). Introduction to Systematic Reviews. In: Piantadosi, S., Meinert, C.L. (eds) Principles and Practice of Clinical Trials. Springer, Cham. https://doi.org/10.1007/978-3-319-52636-2_194

Download citation

DOI : https://doi.org/10.1007/978-3-319-52636-2_194

Published : 20 July 2022

Publisher Name : Springer, Cham

Print ISBN : 978-3-319-52635-5

Online ISBN : 978-3-319-52636-2

eBook Packages : Mathematics and Statistics Reference Module Computer Science and Engineering

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Jump to navigation

• Bahasa Malaysia

What are systematic reviews?

Watch this video from   Cochrane Consumers and Communication to learn what systematic reviews are, how researchers prepare them, and why they’re an important part of making informed decisions about health - for everyone. 

Cochrane evidence, including our systematic reviews, provides a powerful tool to enhance your healthcare knowledge and decision making. This video from Cochrane Sweden explains a bit about how we create health evidence and what Cochrane does. 

• Search our Plain Language Summaries of health evidence
• Learn more about Cochrane and our work

Research articles include any original primary research relating to the design, conduct or reporting of systematic reviews, as well as traditional systematic review results papers.

Systematic Reviews strongly encourages that all datasets on which the conclusions of the paper rely should be available to readers. We encourage authors to ensure that their datasets are either deposited in publicly available repositories (where available and appropriate) or presented in the main manuscript or additional supporting files whenever possible. Please see Springer Nature’s information on recommended repositories .

From the Editor re: Living Systematic Reviews Living systematic reviews are gaining traction as a method for engaging with topics where the evidence base is updating on a regular basis, particularly where such updates are likely to result in changes in effect size or direction, or where policy requirements include frequent updates due to changing knowledge needs. Systematic Reviews is doing due diligence on what is required to adequately, transparently and comprehensively support the publication of living systematic reviews; this is work in progress. Once we have the details resolved, there will be another public notice and our author guidelines will be updated. If you have an interest in publishing living systematic reviews in the journal and you wish to be updated on progress, please contact the Publisher, Alexandra Hay ( [email protected] ).

Kind regards, Craig Lockwood, Editor in Chief

Reporting standards

Systematic Reviews supports the complete and transparent reporting of research. The Editors require the submission of a populated checklist and figure from the relevant reporting guidelines, including PRISMA checklist and flow diagram or the most the appropriate PRISMA extension   for variations to the standard systematic reviews methodology. The flow diagram should be included in the main body of the text and the checklist should be provided as an additional file , both the flow diagram and the checklist should be referenced in the text. Submissions received without these elements will be returned to the authors as incomplete. A Word file of the checklist and flow diagram can be downloaded here .

It is understood that for some systematic reviews certain aspects of the report may not comply fully with the PRISMA checklist. The checklist will not be used as a tool for judging the suitability of manuscripts for publication in Systematic Reviews , but is intended as an aid to authors to clearly, completely, and transparently let reviewers and readers know what authors did and found. Using the PRISMA guideline to write the completed systematic review report, completing the PRISMA checklist, and constructing a flow diagram are likely to optimize the quality of reporting and make the peer review process more efficient.

Preparing your manuscript

The information below details the section headings that you should include in your manuscript and what information should be within each section.

Please note that your manuscript must include a 'Declarations' section including all of the subheadings (please see below for more information).

The title page should:

• "A versus B in the treatment of C: a randomized controlled trial", "X is a risk factor for Y: a case control study", "What is the impact of factor X on subject Y: A systematic review"
• or for non-clinical or non-research studies a description of what the article reports
• if a collaboration group should be listed as an author, please list the Group name as an author. If you would like the names of the individual members of the Group to be searchable through their individual PubMed records, please include this information in the “Acknowledgements” section in accordance with the instructions below
• indicate the corresponding author

The Abstract should not exceed 350 words. Please minimize the use of abbreviations and do not cite references in the abstract. Reports of systematic reviews should follow the PRISMA extension for abstracts. The abstract must include the following separate sections:

• Background: the context and purpose of the study
• Methods: how the study was performed and statistical tests used
• Results: the main findings
• Conclusions: brief summary and potential implications
• Systematic review registration: if your systematic review is registered in a publicly accessible registry, include the name of the registry and registration number.

Three to ten keywords representing the main content of the article.

The Background section should explain the background to the study, its aims, a summary of the existing literature and why this study was necessary or its contribution to the field.

The methods section should include:

• the aim, design and setting of the study
• the characteristics of participants or description of materials
• a clear description of all processes, interventions and comparisons. Generic drug names should generally be used. When proprietary brands are used in research, include the brand names in parentheses
• the type of statistical analysis used, including a power calculation if appropriate

This should include the findings of the study including, if appropriate, results of statistical analysis which must be included either in the text or as tables and figures.

This section should discuss the implications of the findings in context of existing research and highlight limitations of the study.

Conclusions

This should state clearly the main conclusions and provide an explanation of the importance and relevance of the study reported.

List of abbreviations

If abbreviations are used in the text they should be defined in the text at first use, and a list of abbreviations should be provided.

Declarations

All manuscripts must contain the following sections under the heading 'Declarations':

Ethics approval and consent to participate

Consent for publication, availability of data and materials, competing interests, authors' contributions, acknowledgements.

• Authors' information (optional)

Please see below for details on the information to be included in these sections.

If any of the sections are not relevant to your manuscript, please include the heading and write 'Not applicable' for that section. 

Manuscripts reporting studies involving human participants, human data or human tissue must:

• include a statement on ethics approval and consent (even where the need for approval was waived)
• include the name of the ethics committee that approved the study and the committee’s reference number if appropriate

Studies involving animals must include a statement on ethics approval and for experimental studies involving client-owned animals, authors must also include a statement on informed consent from the client or owner.

See our editorial policies for more information.

If your manuscript does not report on or involve the use of any animal or human data or tissue, please state “Not applicable” in this section.

If your manuscript contains any individual person’s data in any form (including any individual details, images or videos), consent for publication must be obtained from that person, or in the case of children, their parent or legal guardian. All presentations of case reports must have consent for publication.

You can use your institutional consent form or our consent form if you prefer. You should not send the form to us on submission, but we may request to see a copy at any stage (including after publication).

See our editorial policies for more information on consent for publication.

If your manuscript does not contain data from any individual person, please state “Not applicable” in this section.

All manuscripts must include an ‘Availability of data and materials’ statement. Data availability statements should include information on where data supporting the results reported in the article can be found including, where applicable, hyperlinks to publicly archived datasets analysed or generated during the study. By data we mean the minimal dataset that would be necessary to interpret, replicate and build upon the findings reported in the article. We recognise it is not always possible to share research data publicly, for instance when individual privacy could be compromised, and in such instances data availability should still be stated in the manuscript along with any conditions for access.

Authors are also encouraged to preserve search strings on searchRxiv https://searchrxiv.org/ , an archive to support researchers to report, store and share their searches consistently and to enable them to review and re-use existing searches. searchRxiv enables researchers to obtain a digital object identifier (DOI) for their search, allowing it to be cited. 

Data availability statements can take one of the following forms (or a combination of more than one if required for multiple datasets):

• The datasets generated and/or analysed during the current study are available in the [NAME] repository, [PERSISTENT WEB LINK TO DATASETS]
• The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
• All data generated or analysed during this study are included in this published article [and its supplementary information files].
• The datasets generated and/or analysed during the current study are not publicly available due [REASON WHY DATA ARE NOT PUBLIC] but are available from the corresponding author on reasonable request.
• Data sharing is not applicable to this article as no datasets were generated or analysed during the current study.
• The data that support the findings of this study are available from [third party name] but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of [third party name].
• Not applicable. If your manuscript does not contain any data, please state 'Not applicable' in this section.

More examples of template data availability statements, which include examples of openly available and restricted access datasets, are available here .

BioMed Central strongly encourages the citation of any publicly available data on which the conclusions of the paper rely in the manuscript. Data citations should include a persistent identifier (such as a DOI) and should ideally be included in the reference list. Citations of datasets, when they appear in the reference list, should include the minimum information recommended by DataCite and follow journal style. Dataset identifiers including DOIs should be expressed as full URLs. For example:

Hao Z, AghaKouchak A, Nakhjiri N, Farahmand A. Global integrated drought monitoring and prediction system (GIDMaPS) data sets. figshare. 2014. http://dx.doi.org/10.6084/m9.figshare.853801

With the corresponding text in the Availability of data and materials statement:

The datasets generated during and/or analysed during the current study are available in the [NAME] repository, [PERSISTENT WEB LINK TO DATASETS]. [Reference number]  

If you wish to co-submit a data note describing your data to be published in BMC Research Notes , you can do so by visiting our submission portal . Data notes support open data and help authors to comply with funder policies on data sharing. Co-published data notes will be linked to the research article the data support ( example ).

All financial and non-financial competing interests must be declared in this section.

See our editorial policies for a full explanation of competing interests. If you are unsure whether you or any of your co-authors have a competing interest please contact the editorial office.

Please use the authors initials to refer to each authors' competing interests in this section.

If you do not have any competing interests, please state "The authors declare that they have no competing interests" in this section.

All sources of funding for the research reported should be declared. If the funder has a specific role in the conceptualization, design, data collection, analysis, decision to publish, or preparation of the manuscript, this should be declared.

The individual contributions of authors to the manuscript should be specified in this section. Guidance and criteria for authorship can be found in our editorial policies .

Please use initials to refer to each author's contribution in this section, for example: "FC analyzed and interpreted the patient data regarding the hematological disease and the transplant. RH performed the histological examination of the kidney, and was a major contributor in writing the manuscript. All authors read and approved the final manuscript."

Please acknowledge anyone who contributed towards the article who does not meet the criteria for authorship including anyone who provided professional writing services or materials.

Authors should obtain permission to acknowledge from all those mentioned in the Acknowledgements section.

See our editorial policies for a full explanation of acknowledgements and authorship criteria.

If you do not have anyone to acknowledge, please write "Not applicable" in this section.

Group authorship (for manuscripts involving a collaboration group): if you would like the names of the individual members of a collaboration Group to be searchable through their individual PubMed records, please ensure that the title of the collaboration Group is included on the title page and in the submission system and also include collaborating author names as the last paragraph of the “Acknowledgements” section. Please add authors in the format First Name, Middle initial(s) (optional), Last Name. You can add institution or country information for each author if you wish, but this should be consistent across all authors.

Please note that individual names may not be present in the PubMed record at the time a published article is initially included in PubMed as it takes PubMed additional time to code this information.

Authors' information

This section is optional.

You may choose to use this section to include any relevant information about the author(s) that may aid the reader's interpretation of the article, and understand the standpoint of the author(s). This may include details about the authors' qualifications, current positions they hold at institutions or societies, or any other relevant background information. Please refer to authors using their initials. Note this section should not be used to describe any competing interests.

Footnotes can be used to give additional information, which may include the citation of a reference included in the reference list. They should not consist solely of a reference citation, and they should never include the bibliographic details of a reference. They should also not contain any figures or tables.

Footnotes to the text are numbered consecutively; those to tables should be indicated by superscript lower-case letters (or asterisks for significance values and other statistical data). Footnotes to the title or the authors of the article are not given reference symbols.

Always use footnotes instead of endnotes.

Examples of the Vancouver reference style are shown below.

See our editorial policies for author guidance on good citation practice

Web links and URLs: All web links and URLs, including links to the authors' own websites, should be given a reference number and included in the reference list rather than within the text of the manuscript. They should be provided in full, including both the title of the site and the URL, as well as the date the site was accessed, in the following format: The Mouse Tumor Biology Database. http://tumor.informatics.jax.org/mtbwi/index.do . Accessed 20 May 2013. If an author or group of authors can clearly be associated with a web link, such as for weblogs, then they should be included in the reference.

Example reference style:

Article within a journal

Smith JJ. The world of science. Am J Sci. 1999;36:234-5.

Article within a journal (no page numbers)

Rohrmann S, Overvad K, Bueno-de-Mesquita HB, Jakobsen MU, Egeberg R, Tjønneland A, et al. Meat consumption and mortality - results from the European Prospective Investigation into Cancer and Nutrition. BMC Medicine. 2013;11:63.

Article within a journal by DOI

Slifka MK, Whitton JL. Clinical implications of dysregulated cytokine production. Dig J Mol Med. 2000; doi:10.1007/s801090000086.

Article within a journal supplement

Frumin AM, Nussbaum J, Esposito M. Functional asplenia: demonstration of splenic activity by bone marrow scan. Blood 1979;59 Suppl 1:26-32.

Book chapter, or an article within a book

Wyllie AH, Kerr JFR, Currie AR. Cell death: the significance of apoptosis. In: Bourne GH, Danielli JF, Jeon KW, editors. International review of cytology. London: Academic; 1980. p. 251-306.

OnlineFirst chapter in a series (without a volume designation but with a DOI)

Saito Y, Hyuga H. Rate equation approaches to amplification of enantiomeric excess and chiral symmetry breaking. Top Curr Chem. 2007. doi:10.1007/128_2006_108.

Complete book, authored

Blenkinsopp A, Paxton P. Symptoms in the pharmacy: a guide to the management of common illness. 3rd ed. Oxford: Blackwell Science; 1998.

Online document

Doe J. Title of subordinate document. In: The dictionary of substances and their effects. Royal Society of Chemistry. 1999. http://www.rsc.org/dose/title of subordinate document. Accessed 15 Jan 1999.

Online database

Healthwise Knowledgebase. US Pharmacopeia, Rockville. 1998. http://www.healthwise.org. Accessed 21 Sept 1998.

Supplementary material/private homepage

Doe J. Title of supplementary material. 2000. http://www.privatehomepage.com. Accessed 22 Feb 2000.

University site

Doe, J: Title of preprint. http://www.uni-heidelberg.de/mydata.html (1999). Accessed 25 Dec 1999.

Doe, J: Trivial HTTP, RFC2169. ftp://ftp.isi.edu/in-notes/rfc2169.txt (1999). Accessed 12 Nov 1999.

Organization site

ISSN International Centre: The ISSN register. http://www.issn.org (2006). Accessed 20 Feb 2007.

Dataset with persistent identifier

Zheng L-Y, Guo X-S, He B, Sun L-J, Peng Y, Dong S-S, et al. Genome data from sweet and grain sorghum (Sorghum bicolor). GigaScience Database. 2011. http://dx.doi.org/10.5524/100012 .

Figures, tables and additional files

See  General formatting guidelines  for information on how to format figures, tables and additional files.

Submit manuscript

• Editorial Board
• Manuscript editing services
• Instructions for Editors
• Sign up for article alerts and news from this journal
• Follow us on Twitter

Annual Journal Metrics

2022 Citation Impact 3.7 - 2-year Impact Factor 3.8 - 5-year Impact Factor 1.561 - SNIP (Source Normalized Impact per Paper) 1.269 - SJR (SCImago Journal Rank)

2023 Speed 88 days submission to first editorial decision for all manuscripts (Median) 296 days submission to accept (Median)

2023 Usage  3,531,065 downloads 3,533 Altmetric mentions

• More about our metrics

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Cochrane Colloquium Abstracts

Review or original article the manuscript category of systematic review and meta-analysis in high-impact biomedical journals.

Article type Poster Year 2011 Madrid Authors Tam K 1 , Tsai L 1 , K U 1 , Chen C 1 , Lo H 1 1 Center of Evidence-based Medicine, College of Medicine, Taipei Medical University, Taiwan Abstract Background: Unlike a narrative review, a systematic review involves the application of scientific strategies, in ways that limit bias, to the assembly and critical appraisal of all relevant studies that address a specific clinical question. A meta-analysis is a type of systematic review that uses a statistical strategy for assembling the results of several studies into a single estimate. However, when an author submits a systematic review and meta-analysis to journals, the manuscript category between a review and original article is indistinct. Objectives: To investigate the manuscript category of systematic reviews and meta-analysis in biomedical journals. Methods: Biomedical journals (impact factor >6) that consider systematic reviews and meta-analyses in the field of clinical sciences for publication were included. The Instructions to Authors of biomedical journals and the article category printed on the front page of the literature were reviewed for evidence of an editorial policy on the manuscript category. Results: 63 of 311 biomedical journals publish systematic reviews and meta-analyses of clinical issues. In the Instructions to Authors, 4.76% classified a systematic review and meta-analysis as an original article, 15.9% as a review, 20.6% as an independent type of manuscript, and 58.7% did not mention any policy on the article type for systematic review and meta-analysis. For the article category posted at the front page of the literature, 31.7% printed systematic reviews and meta-analyses as an original article, 9.52% as a review, 4.76% as a meta-analysis, and 39.7% did not reveal the article type on the front page. Conclusions: Most of the high-impact clinical biomedical journals did not mention their policy on classification of systematic reviews and meta-analyses in the Instructions to Authors. However, a relatively large proportion of journals recognize a systematic review and meta-analysis as an original article.

Systematic Reviews & Expert Reviews

• What are Expert Reviews?
• What Type of Review Should I Choose?
• Setting the Groundwork
• Finding the Evidence
• Selecting the Evidence
• Study Appraisal
• Data Extraction & Abstraction
• Reporting Your Findings
• Systematic Review Toolbox
• Librarian Involvement

Types of Evidence

In the Health Sciences, we generally categorize evidence into two main types: Filtered and Unfiltered resources. When you look at the traditional evidence pyramid, we are looking at the research cycle: starting with unfiltered information (the original research), once a body of work is established, we can start analyzing and synthesizing the existing research, creating  filtered information  (i.e., systematic reviews and meta-analyses). This top-tier of evidence includes Systematic Reviews and other "expert reviews."

Image from https://openmd.com/guide/levels-of-evidence

Expert Review Types

Systematic reviews.

A systematic review uses systematic and explicit methods to identify, select, and critically appraise relevant research and to collect and analyze data from included studies.  It traditionally brings together evidence from the quantitative literature to answer questions on the effectiveness of a specific intervention for a particular condition.

• Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors).  Cochrane Handbook for Systematic Reviews of Interventions  version 6.2 (updated February 2021). Cochrane, 2021. Available from  http://www.training.cochrane.org/handbook .

INTEGRATIVE REVIEWS

An integrative review critiques and synthesizes the literature on a topic in an integrated way to generate new frameworks or perspectives on the topic.  It allows for the inclusion of several study designs (e.g. experimental/nonexperimental, theoretical studies/empirical literature).  It is also known as a “comprehensive review” or a “critical overview.”

• Torraco, R. J. (2005). Writing integrative literature reviews: guidelines and examples.  Human Resource Development Review , 4(3), 356–367.  http://doi.org/10.1177/1534484305278283

SCOPING REVIEWS

A scoping review maps the body of literature on a topic (often a broad topic) and identifies key concepts and research gaps.  It may include data from any type of evidence and research methodology.  It can be used as a standalone project or as a preliminary step to a systematic review.

• Munn, Z., Peters, M., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC medical research methodology, 18(1), 143.  https://doi.org/10.1186/s12874-018-0611-x
• Arksey, H., & O’Malley, L. (2005). Scoping studies: towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19–32.  https://doi.org/10.1080/1364557032000119616

RAPID REVIEW

A realist review looks to identify and explain social interventions or programs and the interactions between context, mechanisms, and outcomes for policy makers.  It seeks to answer the question, “What works, for whom, in what circumstances?”  It embraces multiple methods (both qualitative and quantitative).

• Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005). Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy, 10 Suppl1, 21–34.  http://doi.org/10.1258/1355819054308530

UMBRELLA REVIEW

An overview of reviews, or umbrella review, summarizes the evidence from multiple research syntheses into one accessible and usable document.  It is based on high-quality, reliable systematic reviews on a specific health problem or topic, and it explores the consistency of findings across reviews

• Aromataris, E., et al. (2015). Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. International Journal of Evidence-Based Healthcare, 13(3), 132–140.  http://doi.org/10.1097/XEB.0000000000000055
• Next: What Type of Review Should I Choose? >>
• Last Updated: Apr 4, 2024 2:19 PM
• URL: https://researchguides.gonzaga.edu/systematic-reviews

Systematic Reviews

• What is a Systematic Review?

A systematic review is an evidence synthesis that uses explicit, reproducible methods to perform a comprehensive literature search and critical appraisal of individual studies and that uses appropriate statistical techniques to combine these valid studies.

Key Characteristics of a Systematic Review:

Generally, systematic reviews must have:

• a clearly stated set of objectives with pre-defined eligibility criteria for studies
• an explicit, reproducible methodology
• a systematic search that attempts to identify all studies that would meet the eligibility criteria
• an assessment of the validity of the findings of the included studies, for example through the assessment of the risk of bias
• a systematic presentation, and synthesis, of the characteristics and findings of the included studies.

A meta-analysis is a systematic review that uses quantitative methods to synthesize and summarize the pooled data from included studies.

Additional Information

• How-to Books
• Beyond Health Sciences

• Cochrane Handbook For Systematic Reviews of Interventions Provides guidance to authors for the preparation of Cochrane Intervention reviews. Chapter 6 covers searching for reviews.
• Systematic Reviews: CRD’s Guidance for Undertaking Reviews in Health Care From The University of York Centre for Reviews and Dissemination: Provides practical guidance for undertaking evidence synthesis based on a thorough understanding of systematic review methodology. It presents the core principles of systematic reviewing, and in complementary chapters, highlights issues that are specific to reviews of clinical tests, public health interventions, adverse effects, and economic evaluations.
• Cornell, Sytematic Reviews and Evidence Synthesis Beyond the Health Sciences Video series geared for librarians but very informative about searching outside medicine.
• << Previous: Getting Started
• Next: Levels of Evidence >>
• Getting Started
• Levels of Evidence
• Locating Systematic Reviews
• Searching Systematically
• Developing Answerable Questions
• Identifying Synonyms & Related Terms
• Using Truncation and Wildcards
• Identifying Search Limits/Exclusion Criteria
• Keyword vs. Subject Searching
• Where to Search
• Search Filters
• Sensitivity vs. Precision
• Core Databases
• Other Databases
• Clinical Trial Registries
• Conference Presentations
• Databases Indexing Grey Literature
• Web Searching
• Handsearching
• Citation Indexes
• Documenting the Search Process
• Managing your Review

Research Support

• Last Updated: Feb 29, 2024 3:16 PM
• URL: https://guides.library.ucdavis.edu/systematic-reviews

Easy guide to conducting a systematic review

Affiliations.

• 1 Discipline of Child and Adolescent Health, University of Sydney, Sydney, New South Wales, Australia.
• 2 Department of Nephrology, The Children's Hospital at Westmead, Sydney, New South Wales, Australia.
• 3 Education Department, The Children's Hospital at Westmead, Sydney, New South Wales, Australia.
• PMID: 32364273
• DOI: 10.1111/jpc.14853

A systematic review is a type of study that synthesises research that has been conducted on a particular topic. Systematic reviews are considered to provide the highest level of evidence on the hierarchy of evidence pyramid. Systematic reviews are conducted following rigorous research methodology. To minimise bias, systematic reviews utilise a predefined search strategy to identify and appraise all available published literature on a specific topic. The meticulous nature of the systematic review research methodology differentiates a systematic review from a narrative review (literature review or authoritative review). This paper provides a brief step by step summary of how to conduct a systematic review, which may be of interest for clinicians and researchers.

Keywords: research; research design; systematic review.

© 2020 Paediatrics and Child Health Division (The Royal Australasian College of Physicians).

Publication types

• Systematic Review
• Research Design*

• Open access
• Published: 05 April 2024

The impact of health insurance on maternal and reproductive health service utilization and financial protection in low- and lower middle-income countries: a systematic review of the evidence

• Joseph Kazibwe 1 ,
• Phuong Bich Tran 2 ,
• Andrea Hannah Kaiser 1 ,
• Simon Peter Kasagga 3 ,
• Felix Masiye 4 ,
• Björn Ekman 1 &
• Jesper Sundewall 1 , 5  

BMC Health Services Research volume  24 , Article number:  432 ( 2024 ) Cite this article

Metrics details

Low- and middle-income countries have committed to achieving universal health coverage (UHC) as a means to enhance access to services and improve financial protection. One of the key health financing reforms to achieve UHC is the introduction or expansion of health insurance to enhance access to basic health services, including maternal and reproductive health care. However, there is a paucity of evidence of the extent to which these reforms have had impact on the main policy objectives of enhancing service utilization and financial protection. The aim of this systematic review is to assess the existing evidence on the causal impact of health insurance on maternal and reproductive health service utilization and financial protection in low- and lower middle-income countries.

The review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The search included six databases: Medline, Embase, Web of Science, Cochrane, CINAHL, and Scopus as of 23rd May 2023. The keywords included health insurance, impact, utilisation, financial protection, and maternal and reproductive health. The search was followed by independent title and abstract screening and full text review by two reviewers using the Covidence software. Studies published in English since 2010, which reported on the impact of health insurance on maternal and reproductive health utilisation and or financial protection were included in the review. The ROBINS-I tool was used to assess the quality of the included studies.

A total of 17 studies fulfilled the inclusion criteria. The majority of the studies (82.4%, n  = 14) were nationally representative. Most studies found that health insurance had a significant positive impact on having at least four antenatal care (ANC) visits, delivery at a health facility and having a delivery assisted by a skilled attendant with average treatment effects ranging from 0.02 to 0.11, 0.03 to 0.34 and 0.03 to 0.23 respectively. There was no evidence that health insurance had increased postnatal care, access to contraception and financial protection for maternal and reproductive health services. Various maternal and reproductive health indicators were reported in studies. ANC had the greatest number of reported indicators ( n  = 10), followed by financial protection ( n  = 6), postnatal care ( n  = 5), and delivery care ( n  = 4). The overall quality of the evidence was moderate based on the risk of bias assessment.

The introduction or expansion of various types of health insurance can be a useful intervention to improve ANC (receiving at least four ANC visits) and delivery care (delivery at health facility and delivery assisted by skilled birth attendant) service utilization in low- and lower-middle-income countries. Implementation of health insurance could enable countries’ progress towards UHC and reduce maternal mortality. However, more research using rigorous impact evaluation methods is needed to investigate the causal impact of health insurance coverage on postnatal care utilization, contraceptive use and financial protection both in the general population and by socioeconomic status.

Trial registration

This study was registered with Prospero (CRD42021285776).

Peer Review reports

Introduction

Low- and middle-income countries (LMICs) have committed to making progress towards universal health coverage (UHC) as part of the Sustainable Development Goals (SDGs). UHC has been defined by the World Health Organization (WHO) as a state where all people and communities receive the quality health services they need, when they need them, without experiencing financial hardship due to health care costs [ 1 ]. Generally, high income countries have attained high levels of service coverage (UHC service coverage index of at least 80 out of 100), however a majority of low- and lower-middle income countries (LLMICs) are still lagging behind (UHC service coverage index of less than 60 out of 100) as of 2022 [ 2 ]. The health service coverage index is the average coverage of essential services based on 14 tracer indicators of health service coverage (encompassing reproductive, maternal, newborn and child health, infectious diseases, non-communicable diseases and service capacity and access) among the general and the most disadvantaged population [ 3 , 4 ]. Similarly, while efforts have been made to decrease catastrophic health expenditure globally, LLMICs continue to face the greatest burden of people being thrust into extreme poverty (spending less than international dollars 1.9 per day) due to out of pocket payments (OOP) on healthcare [ 5 ].

In order to advance towards UHC, several countries especially LLMICs, are planning or implementing health financing reforms with a view to introduce or expand some form of health insurance (i.e. prepayment and pooling of funds). Countries that have opted for health insurance schemes – specifically social health insurance (SHI) – have seen an increment in their health expenditure compared to those that have a tax-based model of financing [ 6 , 7 ]. However, the choice of health financing mechanism does not necessarily have a clear effect on health outcomes (such as increased immunization coverage, reduced under-five mortality) or financial protection [ 7 ]. For example, Wagstaff who looked at Organization for Economic Co-operation and Development (OECD) countries found that neither a tax-funded health system nor a SHI system had a significant effect on health outcomes [ 6 ] while Gabani et al. who looked at over 124 countries found that transitions from predominantly OOP financing to tax-funded health systems yielded significantly better health outcomes than transitions from predominantly OOP financing to health insurance [ 7 ].

An increasing number of LLMICs have started implementing, or are planning to implement health insurance reforms to advance UHC [ 8 , 9 , 10 ]. Health insurance can go by different names including SHI, publicly funded health insurance (PFHI), community-based health insurance (CBHI) and private for-profit health insurance based on the pre-payment arrangement within an insurance scheme [ 11 ]. The intention, however, is the same for all health insurance systems (especially not for profit health insurance), which is to pool the risk of high-cost health care across a large number of people in order to protect individuals from high unexpected medical costs. Through a system of prepayment for guaranteed access to a predetermined package of health benefits, individuals can benefit from more predictable health care expenses and be protected from catastrophic health expenditure. A number of countries are opting for SHI. SHI refers to a health insurance system where contributions in form of premiums are collected from employees, employers and or government and pooled into an insurance fund [ 12 ]. Over time, SHI has been defined to mean insurance schemes where employees and employers both contribute premiums to the insurance fund. In instances where contributions/premiums are paid by government, such insurance has been referred to as PFHI for example in India [ 13 , 14 ]. PFHI has been implemented in some LLMIC settings, where there is a large informal sector, and inability to pay or collect premiums. In some cases, a health insurance scheme can be a combination of tiered contributions by members and subsidies from the government for example contributory and non-contributory.

A core component of UHC is maternal and reproductive health services (MRH), which has received a lot of attention in the past few decades. It was central to the Millennium Development Goals, specifically Goal 5 aimed at improving maternal health [ 15 ]; and it is currently well stipulated within the SDGs. MRH is one of the four categories measured for the UHC service coverage index. The other health services areas under the index are infectious diseases, non-communicable diseases and service capacity and access [ 16 ]. Several interventions have been implemented to improve MRH, including sexual and reproductive health and rights interventions. These endeavors have led to the improvement of MRH globally [ 17 ]. However, several LLMICs continue to face high maternal mortality ratios (accounting for 94% of all maternal deaths globally) [ 18 ], which is far from achieving the target of reducing maternal mortality to 70 deaths per 100,000 live births. Furthermore, women have continued to experience financial barriers when seeking healthcare, and they are found to be more vulnerable to facing financial hardships when accessing care, compared to men [ 19 ].

Despite the increasing interest surrounding health insurance, our understanding of the actual causal impact of the implemented reforms remains limited. Several reviews have examined the existing evidence on the impact of health insurance on service utilization and financial protection, but the results are inconclusive [ 20 , 21 , 22 , 23 ]. A review by Comfort et al. [ 24 ] analyzed the effects of health insurance on maternal health services in LMICs. Insurance (a mix of different types of insurance) was found to be consistently associated with increased utilization of facility-based child delivery and delivery assisted by a skilled health worker.

However, Comfort et al.’s study did not address the impact of health insurance on financial protection. In addition, the study examined various types of insurance schemes, including a mix of both for-profit and not-for-profit models. The study also included conditional cash transfers (or CCTs, a kind of demand-side financing). Therefore, based on Comfort et al.'s findings, it can be challenging to discern the specific impact of individual types of insurance. Our study differs from that of Comfort et al. as we specifically focus on well-specified not-for-profit health insurance as the intervention in LLMICs. Furthermore, we have also examined and reported on the impact of insurance on the financial protection of women of reproductive age. Our review constitutes a contribution to the current evidence base on this topic as no previous review has specifically examined the impact of not-for-profit health insurance on maternal and reproductive service utilization and financial protection in LLMICs, despite the recognition that MRH is among the four core categories of essential health services under UHC [ 4 ].

Our study aims to review the existing evidence of the causal impact of health insurance on maternal and reproductive service utilization and financial protection in LLMICs to inform ongoing health financing reform discussions and identify evidence gaps for future research.

The systematic review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [ 25 ]. In addition, we used the PICO framework [ 26 ] to guide the study scope. The study is registered with PROSPERO, registration number CRD42021285776. We searched electronic databases to identify published articles, and bibliographies of included articles were screened to identify missed articles that fulfilled the inclusion criteria.

To narrow the scope of our study, we employed the PICO framework with the following parameters – Population: Women of reproductive age residing in low- and lower middle-income countries as classified by the World Bank [ 27 ] as of 1st July 2021 – Intervention: health insurance – Comparator: uninsured women – Outcomes: two types of outcomes were considered, i) utilisation of maternal and reproductive health services, and ii) financial protection. Outcomes on utilisation of MRH included contraceptive use, number of antenatal care (ANC) visits, delivery at health facility. Outcomes on financial protection included catastrophic health expenditure and impoverishment impact of out-of-pocket expenditure (OOPE).”

Studies reporting impact

In this study, we reviewed studies that reported impact, i.e. studies that employed a research design enabling the identification of a causal effect of the intervention on an outcome of relevance. We defined studies that report impact as studies that estimate the causal influence the intervention (health insurance) has on a given outcome (MRH and financial protection indicators).

Intervention

The intervention in this study is any not-for-profit health insurance. Health insurance is the protection of registered members (beneficiaries) from high costs of/expenditure on health services by pooling resources through payment of agreed periodic premiums. A person or entity (employer or government) pays a periodic premium to enable them access to health services without requiring them to pay for the services. In some cases where there is a co-payment, the beneficiary pays a small amount or portion of the cost of services they receive. This arrangement aims at sharing the financial risks associated with falling ill and needing medical care.

Inclusion criteria

Our review applied the following inclusion criteria. We included articles that:

Reported on any specific type of not-for-profit health insurance. The reason was to focus on health insurance implemented with the intention of progressing towards UHC.

Reported on the impact of health insurance on MRH service utilization and/or financial protection of people accessing MRH services, and where any MRH service is included in the health benefit package of the insurance scheme. MRH in this study covers contraception, pre- and postnatal services, and delivery care.

Used experimental and/or quasi experimental study designs.

Were published from 2010 onwards and in English. We included studies from 2010 to capture the most recent evidence, as insurance schemes undergo reforms over time with likely implications on their respective performance. Relevant unpublished studies in the form of reports were also considered.

Exclusion criteria

Articles were excluded if they:

Reported on the impact of private for-profit insurance only or aggregated all types of insurance (both for-profit and not-for-profit) as one. Excluding such articles was necessary because grouping different insurance types as a single intervention makes it challenging to differentiate the specific impact of each insurance type.

Adopted a non-experimental study design prohibiting the identification of a causal effect.

Reported on the impact of health insurance qualitatively.

Were published in languages other than English.

Had unavailable full texts.

Databases searched

We searched for published literature in selected electronic databases and bibliographies. Databases included Medline (PubMed), Embase, Web of Science, CINAHL, Cochrane and Scopus. Additionally, we reviewed bibliographies of included articles to find other relevant articles that might have been missed in the search. It should be noted that although no time restrictions were included in the search strategy, studies published before 2010 were excluded at the screening stage.

Search strategy

The six electronic databases were searched on the 31st of October 2021, with an update of the search carried out on 23rd May 2023. The search strategy included all keywords and respective keyword variations for the five keyword domains: health insurance, impact, utilisation, financial protection, and maternal and reproductive health. Search strategies were customised for the respective databases (Supplementary Material 1 : Appendix 1).

Study selection

The PRISMA guidelines [ 28 ] were followed in the articles screening and selection process. The articles retrieved from the search were uploaded to Covidence systematic review software [ 29 ], where duplicates were removed. Initially, at least two independent researchers (PT, SPK, and JK) carried out screening for each title and abstract. Subsequently, full text screening was conducted by PT and JK, following a standard protocol. In the event of any conflict, a fourth researcher (JS) was available to review the conflict and make the final decision.

Data extraction

We developed a data extraction template in Microsoft Excel, which was piloted on ten randomly selected articles and necessary adjustments were made. We extracted data on author, year of publication, target group, study design, country, geographic location, setting (rural/urban/mixed), level of health facility, study participants, type of insurance, year of implementation of insurance, source of data, year of data collection, analysis methods used, description of the insurance, type of membership (voluntary/compulsory), enrolment requirements, services covered by insurance, services received, insurance coverage, premium, reimbursement rates, co-payments, OOPE, indicators used in measuring utilisation, financial protection and their definitions, proportion of households experiencing catastrophic heath expenditure (CHE), measures used for impact, theoretical framework used, reported impact (adjusted and unadjusted), and correction of self-selection among others.

Quality assessment and risk of bias in individual studies

The quality of the evidence was assessed through a two-step process, including: 1) using a tool for assessing the risk of bias in each study and 2) using the GRADE criteria to determine the level of certainty of the evidence.

The study design of an article being assessed determined the quality assessment tool to be used. Since we did not find any randomised studies, we resorted to a tool suitable for non-randomised studies. We used the Risk of Bias in Non-randomised Studies of Interventions (ROBINS-I) tool developed by the Development and Evaluation (GRADE) working group [ 30 ]. The tool rates the risk of bias in seven domains, 1) Bias due to confounding, 2) Bias in selection of participants into the study, 3) Bias in classification of interventions, 4) Bias due to departures from intended interventions, 5) Bias due to missing data, 6) Bias in measurement of outcomes, and 7) Bias in selection of reported results [ 31 ]. The study can be rated as low risk of bias, moderate risk of bias, or serious risk of bias based on the respective guiding questions in the tool for each domain. The overall extent of bias of a study is determined by the respective domain ratings, following the algorithm in the guide. The above tool was selected following the findings of a systematic review by Ma et al [ 32 ], describing the ROBINS-I tool as one of the most reliable tools available for quality assessment.

The certainty level of evidence of each study was then determined following the GRADE criteria based on the ROBINS-I tool [ 33 ]. It involved three steps: 1) establishing the initial level of certainty as advised by GRADE, 2) considering lowering or raising the level of certainty and 3) determining the final certainty rating. The certainty level of the evidence could be high, moderate, low, or very low depending on the rating. A non-randomised study’s evidence is initially rated as high level of certainty, which is then downgraded by a level or two depending on how it performs in the following domains: limitations in the detailed study design and execution; inconsistency (or heterogeneity); indirectness (PICO and applicability); imprecision; and publication bias. The downgrade of the certainty level of the evidence is mitigated (increased) by the magnitude of the effect estimates.

Reporting, summary measures and synthesis of results

The reporting was both descriptive and analytical. For extracted quantitative data, we reported summary measures. Indicators reported by the different studies were categorised into MRH service utilisation indicators and financial protection indicators with several subcategories each. Additionally, we elicited the covariates used in the adjustment of results from the studies and categorised these into characteristics including mother’s demographic, households, partners, communities, and mother’s perceptions. The reported impact of the insurance on MRH service utilisation and financial protection was summarised in five groups: positive and significant impact, positive and not significant impact, no impact, negative and not significant impact, and negative and significant impact. The impact findings were summarised in a table showing the proportion of studies that reported a positive significant impact for each respective indicator. Studies that had a high risk of bias were excluded in the calculation of the proportions as shown in Table 3 .

The systematic literature search yielded a total of 11,988 studies after deduplication. Following title and abstract screening and full text review, we included 17 studies that fulfilled our eligibility criteria. Figure 1 shows the flow of selection process including reasons for exclusion of articles at the full text reading stage.

PRISMA flow chart

Characteristics of included studies

Table 1 provides an overview of key characteristics of the included studies.

All the included studies were quasi experimental studies ( n  = 17). Quasi experimental studies are non-randomised studies that evaluate an intervention with the aim of demonstrating causality between the intervention and outcome [ 51 ]. Ghana and Indonesia had the most studies with three each. These were followed by the Philippines ( n  = 2), and Mauritania ( n  = 2). The review included one study from each of the following countries: Tanzania, Egypt, Rwanda, Ethiopia, India, and Senegal (Table 1 ). Additionally, there was a multi-country study that encompassed three countries: Ghana, Rwanda, and Indonesia [ 46 ]. Most studies ( n  = 14) were nationally representative of the population, while the remaining three were carried out in specific region(s) within the specified country [ 42 , 48 , 49 ]. Three studies were specifically conducted in rural settings [ 42 , 48 , 49 ], and no studies focused on urban settings exclusively. The rest of the studies ( n  = 14) covered both rural and urban areas. All the studies included in the review involved female participants of childbearing age from 15 years old. The majority of the studies ( n  = 13) specifically focused on women aged between 15 and 49. The studies focused on three types of health insurance, including social health insurance (e.g. Ghana), community-based health insurance (e.g. Rwanda), and publicly-funded health insurance (e.g. India). The data sources used were mostly secondary data ( n  = 14), specifically demographic health survey (DHS) data [ 52 ], Multiple Indicator Cluster Survey (MICS) and Family Life Survey (FLS).

Quality assessment of included studies

The overall quality of assessed studies was rated as moderate. A total of 12 studies were rated as having a high level of certainty of evidence [ 36 , 37 , 38 , 39 , 41 , 43 , 44 , 45 , 46 , 48 , 49 , 50 ], two studies had moderate while those rated low and very low were two [ 40 , 42 ] and one [ 34 ], respectively. This assessment was based on the categorization of the risk of bias using the ROBINS-I tool. Overall, the majority of the studies ( n  = 15; 88.2%) were categorised as having moderate risk of bias, and two studies were rated as having serious risk [ 34 , 42 ]. No study was found with an overall low risk of bias. All studies were rated as low risk of bias in three domains: bias in classification of interventions, bias due to departures from intended interventions, and bias due to missing data as shown in Fig. 2 . For the domain of bias in selection of reported results, the majority of the studies ( n  = 16, 94.1%) were rated as low risk of bias, while the rest ( n  = 1, 5.9%) was rated as moderate risk [ 40 ]. One study showed serious risk of bias due to confounding [ 34 ], and one [ 42 ] study was assessed to have serious risk of bias in the measurement of outcomes. The table with the assessment results is included in Supplementary Material 1 : Appendix 2, and Fig. 2 shows the ratings by domain as well as the overall rating of bias.

Assessment of the risk of bias of the studies according to the seven domains, using the ROBINS-I tool

Indicators used to measure the impact

Table 2 shows the indicators used to measure the impact of health insurance on MRH service utilisation and financial protection. There was a large variation in the number of indicators per category of MRH services, and the frequency to which they were reported in the studies. Regarding the number of indicators per category, ANC had the greatest number of reported indicators ( n  = 10), followed by financial protection ( n  = 6), postnatal care ( n  = 5), and delivery care ( n  = 4). Contraception had only one indicator, with three studies reporting on this indicator [ 34 , 40 , 42 ]. Regarding the frequency of use of the indicators, under the ANC category, the most common indicator was having at least four ANC visits during pregnancy ( n  = 9, 52.9%), which was also the second most reported MRH indicator in this review. For delivery care, delivery at a health facility was the most reported indicator in the delivery care category (and the most reported indicator in this review) ( n  = 14, 82.4%); followed by delivery by skilled attendant ( n  = 7, 41.2%). For the postnatal care category, having postnatal care (without specifying the point or time of access) was the most common indicator used in this category ( n  = 4, 23.5%) [ 38 , 40 , 41 , 45 ]. For financial protection, six indicators were reported. OOPE on delivery services was reported in four studies (23.5%) [ 34 , 42 , 43 , 50 ]. OOPE due to ANC, OOPE due to C-section, financial distress after C-section, and CHE were each reported in one study. Financial distress was defined by Garg et al. as a situation where a patient, or their household member, borrowed money or sold their assets to cover the OOPE due to seeking maternal health care [ 50 ].

Impact of health insurance on MRH service utilisation and financial protection

Studies reported a positive effect of the health insurances on ANC and delivery care indicators, with a clear significant positive impact reported for the most used indicators (having at least four ANC visits, delivery at a health facility, and delivery assisted by a skilled attendant) as shown in Table 3 . Specifically, 85.7% of the studies found a significant positive impact between health insurance and delivery with assistance from a skilled attendant, 83.3% reported a significant positive impact on delivery at a health facility, and 75.0% indicated a significant positive impact on having at least four ANC visits during pregnancy. In contrast, the evidence on the impact of health insurance on contraceptive use [ 40 ], postnatal care [ 36 , 38 , 40 , 41 , 45 ] and financial protection [ 43 , 48 , 50 ] indicators was scanty, variable and inconclusive.

Certain indicators (ANC at health facility, postnatal care visit at health facility in less than 2 months after delivery, OOPE due to ANC, and OOPE due to PNC) were not included in the analysis, because these indicators were only reported in articles that were excluded due to their serious risk of bias.

Table 4 shows the magnitude of the impact reported by each study for indicators that were reported by more than one study.

ANC: Health insurance increased the chance of a pregnant woman having at least four ANC visits. The magnitude of the positive significant impact of health insurance on receiving at least four ANC visits during a pregnancy ranged between approximately 2% [ 46 ] and 11% [ 36 ]. Insurance increased the total number of ANC visits during pregnancy. The magnitude of positive significant impact of health insurance on the number of ANC during pregnancy ranged from 43% [ 47 ] to 56% [ 48 ]. On the other hand, insurance did not have a significant positive impact on having an ANC visit in the first trimester except for Indonesia [ 46 ].

Delivery care: Health insurance increased chances of having a delivery at a health facility and delivery by a skilled attendant. Studies that reported a significant positive impact of health insurance on delivery at a health facility found a magnitude ranging from approximately 3% [ 38 ] to 34% [ 48 ]. The magnitude of the impact ranged from 3% [ 41 ] to 23% [ 45 ] for having a delivery assisted by a skilled attendant.

Postnatal care: Health insurance showed an increase in the chance of receiving postnatal care but only 50% of the studies reporting on the impact of health insurance on postnatal reported a significant positive increase. The magnitude of the health insurance on postnatal care among studies that reported positive significant impact was 4% [ 41 ] and 9% [ 45 ].

Reduction of OOPE: Evidence suggests that health insurance has generally reduced OOP payments for MRH services. However, of the two studies that reported on OOPE only one found a significant reduction in OOPE of 1,136,966 Indonesian Rupiah (IDR) and 676,402 IDR for non-contributory and contributory health insurance in Indonesia respectively [ 43 ].

Methods used to estimate the impact of health insurance on MRH service utilisation

No study used randomisation in allocating participants to the intervention or control groups.

A wide range of statistical methods were applied in the studies (Table 5 ). Propensity Score Analysis/Matching (PSM) was the most used statistical methods (58.8%), followed by difference-in-difference (DID) analysis (11.8%). Some studies utilised more than one method; for example, Samarakoon et al [ 34 ] used both PSM and DID. The effect measures used were mostly Average Treatment Effect (ATE) (47.1%), and Average Treatment Effect on the Treated (ATT) (35.3%).

Several methods were used to adjust for self-selection, such as PSM, DID, conditional mixed process framework (CMP) (e.g. Agbanyo et al [ 35 ]), entropy balance weighting of observed characteristics (e.g. Aizawa [ 43 ]) and coarsened exact matching (CEM) methods (e.g. Chang et al [ 37 ]). Anindya et al [ 41 ] used more than one method, specifically PSM followed by CEM for sensitivity analysis and robustness check.

Covariates adjusted for in the studies

Table 5 presents the covariates that were adjusted for in the studies. Overall, the most used covariates were age, the education level of the woman, and wealth status of the household; with each being used in 76.5% of all studies. This was followed by place of residence (rural/urban) and marital status, with each at 58.8%. The other covariates were used in less than 50% of the studies that adjusted for covariates.

Our review shows that there is considerable evidence on the impact of health insurance on ANC and delivery care service utilisation. However, there is a scarcity of evidence on the impact of health insurance on the financial protection of women seeking MRH services, utilisation of postnatal care, and contraception. We found that health insurance has a significant positive impact on ANC and delivery care service utilisation specifically having at least four ANC visits, delivery at a health facility and having a delivery assisted by a skilled attendant. However, findings regarding its impact on financial protection, contraception, and postnatal care were inconclusive.

ANC and delivery care utilisation

Among the articles reviewed, recent evidence shows that health insurance generally exhibits a positive impact on ANC and delivery care service utilisation. This is in line with the findings of Spaan et al. and Erlangga et al. reporting that social health insurance and CBHI improved general health service utilisation [ 20 , 21 ]. With comparison to Acharya et al [ 22 ] – who reported inconclusive results on the impact of health insurance on general health service utilisation among the informal sector – the evidence that was reported on MRH service utilisation in that study concurs with our findings.

On the other hand, our findings differ from Comfort et al., who stated that there was no evidence that insurance increased maternal health service utilisation [ 24 ]. The statement was premised on the fact that Comfort et al. did not identify any studies that used randomised methods. Comfort et al. argued that causality could not be established without randomisation of the intervention. However, quasi experimental studies can estimate causation which are the only studies we included in our review. In addition, as shown in Table 1 , all the studies included in our review were published after the publication of Comfort et al.’s review (2013). This indicates that studies which estimated the causal relationship between health insurance and MRH are recent.

For countries that are still experiencing high MMR [ 53 , 54 ], the evidence available on the positive impact of health insurance on at least four ANC visits, delivery at a health facility and having a delivery assisted by a skilled attendant can inform the country’s health financing reforms, encourage implementation, and expansion of such insurance schemes as an intervention to increase access to care and reduce MMR. MRH services such as attending ANC and having a health facility-based delivery have been highlighted as some of the ways to counter occurrence of maternal mortality[ 54 ], and investment in these services was found to be cost-effective [ 17 , 55 ].

Limited evidence on financial protection when accessing MRH

The available evidence suggests that health insurance plays a role in reducing OOPE. However, it is important to note that the evidence in this area is weak, with only a limited number of studies reporting on OOPE indicators. The findings are variable and inconclusive, particularly regarding the likelihood of CHE and the reduction of OOPE specifically related to delivery care. This finding contrasts with the results of a previous systematic review examining financial protection in a broader context [ 20 ]. Health insurance is known to reduce CHE generally. However, we did not find any evidence of a positive impact of health insurance in reducing CHE in the MRH context. It should be noted that this review found very few studies (less than five) that investigated the impact of health insurance on the financial protection of women seeking MRH services in LLMICs. Globally, LMICs bear the highest proportion of OOPE on health. OOPE on health was 43.21% of the total in low-income countries, and 48.17% for lower middle-income countries; meanwhile, the global average is at 18.01% based on the World Bank estimates of 2019 [ 56 ]. Countries that channel larger shares of total health expenditure through prepayment schemes such as health insurance tend to have lower levels of OOPE. As an example, in 2019, the level of OOPE as a proportion of current health expenditure in Indonesia was 34.76%, while in Ghana it was 36.22% which is lower than to the LMIC average. The OOP costs to the patient are found to increase with the increasing level of care. For example in Vietnam, community health facilities had a lower cost for deliveries compared to district and higher-level hospitals [ 57 ]. Health insurance could be key in protecting populations from financial hardship, although, more evidence is necessary to see whether there is substantive impact of health insurance on the financial protection of mothers or women seeking MRH services, especially among the different wealth quintiles, underserved and vulnerable groups of the population.

Inconclusive results on contraception and postnatal care utilisation

The evidence on the impact of health insurance on contraception and postnatal care service utilisation was scarce and inconclusive. Specifically, there was very little evidence on the impact of health insurance on the use of contraception. These findings differ from that of Comfort et al., who found a positive association between health insurance and postnatal care utilisation [ 24 ]. The difference in findings between our study and that of Comfort et al. could potentially be attributed to their inclusion of cross-sectional studies with less rigorous methods.

For contraceptive use, the inconclusive results could be partly explained by the limited insurance coverage for contraceptives in some countries, where the reimbursable contraceptive options are few. Moreover, the reimbursable contraceptive options may not be the most preferred by the society. For example, Ghana has just officially included long-term contraceptive options (such as permanent methods, intrauterine devices (IUDs), implants, and injectables) in the National Health Insurance Scheme benefit package [ 58 ]. On the other hand, cultural, social, and normative practices surrounding postnatal care, as well as the lack of awareness of the clinical postnatal care guidelines may partly explain the inconclusive evidence on the use of postnatal care [ 59 , 60 ]. In addition, despite the importance of postnatal care and contraceptive use in reducing maternal mortality [ 18 ], few studies have evaluated indicators in these areas and the quality of studies examining contraceptive use was moderate to low. The finding regarding the scarcity of evidence on postnatal care in LMICs is not unique to this study, as it has been reported in recent research as well [ 61 ]. Further research is needed to better understand the impact of health insurance on postnatal care and contraceptive utilisation.

Indicators used to measure MRH and the mismatch with international recommendations

Most of the indicators used to measure MRH service utilisation were related to ANC. This may be in part due to the well-established evidence regarding the positive effect of ANC on maternal health related outcomes. Moreover, this aligns with the long standing WHO ANC model (sometimes called basic or focused ANC) introduced in the 1990s, which recommended that a pregnant woman should have at least four ANC visits/contacts during pregnancy [ 62 , 63 ]. However, WHO recently updated their recommendations, increasing the number of ANC visits/contacts to eight [ 64 ]. Unfortunately, our review did not identify any articles that specifically used at least eight visits as an indicator for ANC.

For postnatal care, WHO recommended a minimum of four postnatal care contacts for mothers. These recommended contacts include the first contact within 24 h after delivery, the second contact between 48 and 72 h, the third contact between seven and 14 days, and the fourth contact in the sixth week after delivery [ 65 , 66 ]. However, there was a mismatch between the WHO recommended indicators and the indicators reported in these studies. This indicates that more publicity/sensitization on this important component of the MRH service delivery spectrum is vital. Authors should be encouraged to use recommended indicators to measure the impact of an intervention (health insurance) towards the achievement of global targets and allow for comparison across countries.

Methods used by studies

Propensity score matching was the most popular method used in studies. This conforms to the assertion of Abadie and Cattaneo (2018) that noted an increasing use of matching techniques by researcher partly because of the flexibility of the methods and the failure of ordinary linear regression to estimate conventional treatment effect parameters like ATE and ATET [ 67 ]. In addition, matching makes it possible to estimate treatment effects in the absence of experimental data in evaluation research [ 68 ].Despite the importance of propensity score matching in determining causal inference, it relies on the assumption of conditional independence which may not hold in some instances especially when there are unobservable variables that influence both the treatment and outcome [ 68 ].

Different covariates were used to construct statistical models. Some authors selected covariates based on variable significance level, while others based their selection on the confounding relationship between the exposure and outcome. To have evidence of high certainty, it is necessary to adjust the results based on confounders which can be identified using the directed acyclic graphs [ 67 , 69 ].

Quality of evidence

The quality of the studies included in this review, with regards to the risk of bias, was generally assessed as moderate. It is important to note that increasing the quality of studies in this context can be challenging, as randomised controlled trials are often not feasible or ethically permissible for evaluating policy-related public health interventions, such as health insurance schemes. The absence of randomisation in the allocation of the intervention to participants can introduce various forms of bias, including confounding, which may impact the validity of the study results. Recognizing this, it is essential to thoroughly assess potential drawbacks and biases using appropriate tools [ 30 , 31 ]. The authors tried to overcome this likely consequence of non-randomisation by adjusting for confounders; however, it is difficult to control for all the likely bias. The overall quality of a study can be improved through the randomisation of the intervention (where possible) and the use of causal inference statistical methods that address the potential selection problems that may arise [ 67 , 69 ].

Future research

Although we find that health insurance has a positive impact on the utilisation of ANC; we should be conscious of the intersectionality of evidence. Health insurance interventions may have varying effects across different subgroups within the population. Factors such as age, economic status, and the rural/urban setting can influence how individuals experience and benefit from health insurance coverage [ 47 , 70 ]. A study by Barasa et al. reported that most insurance schemes in sub–Saharan Africa are pro-rich and have minimal benefits for the poor given the low insurance coverage [ 71 ]. The impact of health insurance schemes on utilization and financial protection may vary based on the characteristics/features of the schemes for example organization/design, implementation, enrolment levels, premiums, target population, benefit package [ 21 , 72 ]. If countries are to advance UHC, there is need to understand the intersectionality of the impact, thus conduct more research to investigate the impact of health insurance across geographical domains (rural/urban), across type and level of health providers (private vs public; community-level providers vs secondary- and tertiary-level providers) and vulnerable population subgroups (e.g., people in lower socio-economic quintiles).

Limitations

Our review included studies that were published in English after 2009, which could have led to the omission of studies published in other languages, such as those conducted in French-speaking countries in West Africa or studies before 2010 that may have reported relevant results. We acknowledge that in some contexts, individuals may have private health insurance in addition to the type of health insurance examined in this study, which may have affected the results reported in the included studies. We included studies of various designs, which may have led to variations in the interpretation of results. The use of different covariates in the models employed by the studies could have influenced the magnitude of the reported impact of health insurance.

This study focused on the direction of impact (positive, no change or negative) and significance level of the impact but did not cover the magnitude of the impact. Furthermore, due to the heterogeneity in study design and other characteristics of the included studies, it was not feasible for us to conduct a meta-analysis.

The majority of the included studies used pre-existing datasets to estimate the impact of health insurance. The datasets utilised in this regard were not developed or collected to specifically evaluate health insurance schemes. Such datasets may not be comprehensive in collecting all the relevant data points needed for a robust evaluation of the impact of health insurance.

The quality assessment of the studies was conducted using the ROBINS-I tool – a validated tool recommended by Cochrane for the quality assessment of non-randomised studies [ 20 ]. However the tool does not address problems relating to imprecision of results, where statistical analyses fail to account for clustering or matching of participants [ 31 ]. Such shortfalls may have been overlooked. Therefore, studies that were found to have serious risk of bias were not included in the causal impact analysis, to avoid increasing biases in the summary results.

This review finds evidence supporting the positive impact of health insurance on the utilisation of ANC and delivery care services in low- and lower middle-income settings specially regarding receiving at least four ANC visits, delivery at a health facility and having a delivery assisted by a skilled attendant. Health insurance may contribute to making progress towards UHC, through improving access and utilisation of health services for all. The evidence on financial protection, contraception, and postnatal care is limited and inconclusive. Future evaluations of the impact of health insurance are crucial for countries to identify areas that require improvement, particularly in terms of its impact on vulnerable groups. Further research is needed to assess the impact of health insurance on contraception, postnatal care, and the financial protection of women seeking maternal and reproductive health services. Such work would contribute to a deeper understanding of the potential benefits and limitations of health insurance in these critical areas.

Availability of data and materials

The extracted data analysed during the current study is available from corresponding author on reasonable request.

World Health Organisation. Universal health coverage (UHC). Published April 1, 2021. https://www.who.int/news-room/fact-sheets/detail/universal-health-coverage-(uhc) .  Accessed 12 Jun 2022.

World Health Organization. Coverage of essential health services (SDG 3.8.1). Published 2022. https://www.who.int/data/gho/data/themes/topics/service-coverage . Accessed 18 December 2022.

World Health Organisation. UHC service coverage index (3.8.1). Published 2023. https://www.who.int/data/gho/indicator-metadata-registry/imr-details/4834 . Accessed 20 May 2023.

United Nations. SDG indicator metadata. https://unstats.un.org/sdgs/metadata/files/Metadata-03-08-01.pdf . Accessed 12 Jun 2022.

World Bank, World Health Organization. Global Monitoring Report on Financial Protection in Health 2021.  https://www.who.int/publications/i/item/9789240040953 . Accessed 11 Jun 2023.

Wagstaff A. Social Health Insurance Vs. Tax-Financed Health Systems - Evidence From The OECD. Published online 22 Jan 2013. https://doi.org/10.1596/1813-9450-4821 .

Gabani J, Mazumdar S, Suhrcke M. The effect of health financing systems on health system outcomes: a cross-country panel analysis. Health Econ. Published online 8 Dec 2022. https://doi.org/10.1002/HEC.4635 .

Fusheini A. Healthcare Financing Reforms: Ghana’s National Health Insurance. Heal Reforms Across World. Published online March 2020:25–54.  https://doi.org/10.1142/9789811208928_0002 .

Masiye F, Chansa C. Health Financing in Zambia. 2019.

Book   Google Scholar  

Cashin C, Dossou J-P. Can National Health Insurance Pave the Way to Universal Health Coverage in Sub-Saharan Africa? 2021;7(1). https://doi.org/10.1080/23288604.2021.2006122 .

Gottret P, Schieber G. Health financing revisited: a practitioner’s guide. 2006. https://documents.worldbank.org/en/publication/documents-reports/documentdetail/874011468313782370/health-financing-revisited-a-practitioners-guide . Accessed 18 May 2023.

World Health Assembly 58. Social Health Insurance: Sustainable Health Financing, Universal Coverage and Social Health Insurance: Report by the Secretariat. 2005. https://apps.who.int/iris/handle/10665/20302 . Accessed 5 June 2023.

Government of India. Rashtriya Swasthya Bima Yojana. https://www.india.gov.in/spotlight/rashtriya-swasthya-bima-yojana . Accessed 20 May 2023.

Ranjan A, Dixit P, Mukhopadhyay I, Thiagarajan S. Effectiveness of government strategies for financial protection against costs of hospitalization Care in India. BMC Public Health. 2018;18(1):501. https://doi.org/10.1186/s12889-018-5431-8 .

Article   PubMed   PubMed Central   Google Scholar  

United Nations (UN). United Nations Millennium Development Goals. https://www.un.org/millenniumgoals/ . Accessed 5 Jun 2023.

Watkins DA, Jamison DT, Mills A, et al. Universal Health Coverage and Essential Packages of Care. In: Disease Control Priorities, Third Edition (Volume 9): Improving Health and Reducing Poverty. The World Bank. 2017:43–65. https://doi.org/10.1596/978-1-4648-0527-1_ch3 .

Kaiser AH, Ekman B, Dimarco M, Sundewall J. The cost-effectiveness of sexual and reproductive health and rights interventions in low- and middle-income countries: a scoping review. Sex Reprod Heal Matters. 2021;29(1):90–103. https://doi.org/10.1080/26410397.2021.1983107/SUPPL_FILE/ZRHM_A_1983107_SM2657.DOCX .

Article   Google Scholar  

World Health Organisation. Maternal mortality fact sheet. Published September 2019.  https://www.who.int/news-room/fact-sheets/detail/maternal-mortality . Accessed 9 June 2022.

UN Women. Universal Health Coverage, Gender Equality and Social Protection. A Health Systems Approach. 2020. https://www.unwomen.org/sites/default/files/Headquarters/Attachments/Sections/Library/Publications/2020/Discussion-paper-Universal-health-coverage-gender-equality-and-social-protection-en.pdf . Accessed 22 May 2023.

Spaan E, Mathijssen J, Tromp N, McBain F, ten Have A, Baltussen R. The impact of health insurance in Africa and Asia: a systematic review. Bull World Health Organ. 2012;90(9):685–92. https://doi.org/10.2471/BLT.12.102301 .

Erlangga D, Suhrcke M, Ali S, Bloor K. The impact of public health insurance on health care utilisation, financial protection and health status in low- And middle-income countries: A systematic review. PLoS ONE. 2019;14(8):e0219731. https://doi.org/10.1371/journal.pone.0219731 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Acharya A, Vellakkal S, Taylor F, et al. The impact of health insurance schemes for the informal sector in low-and middle-income countries: a systematic review. World Bank Res Obs. 2013;28(2):236–66. https://doi.org/10.1093/wbro/lks009 .

Zhang C, Fu C, Song Y, Feng R, Wu X, Li Y. Utilization of public health care by people with private health insurance: a systematic review and meta-analysis. BMC Public Health. 2020;20(1):1–12. https://doi.org/10.1186/s12889-020-08861-9 .

Comfort AB, Peterson LA, Hatt LE. Effect of Health Insurance on the Use and Provision of Maternal Health Services and Maternal and Neonatal Health Outcomes: a Systematic Review. J Health Popul Nutr. 2013;31(4 Suppl 2):S81.

PubMed Central   Google Scholar  

Page MJ, McKenzie JE, Bossuyt PM, The PRISMA, et al. statement: an updated guideline for reporting systematic reviews. BMJ. 2020;2021:372. https://doi.org/10.1136/BMJ.N71 .

Schardt C, Adams MB, Owens T, Keitz S, Fontelo P. Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med Inform Decis Mak. 2007;7. https://doi.org/10.1186/1472-6947-7-16 .

World Bank Group. World Bank Country and Lending Groups – World Bank Data. Published 2021. https://datahelpdesk.worldbank.org/knowledgebase/articles/906519 . Accessed 11 Aug  2021.

Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement (Chinese edition). J Chinese Integr Med. 2009;7(9):889–96. https://doi.org/10.3736/jcim20090918 .

Veritas Health Innovation. Covidence systematic review software. https://www.covidence.org/ . Accessed 31 Dec 2021.

Sterne JA, Hernán MA, McAleenan A, Reeves BC. Chapter 25: assessing risk of bias in a non-randomized study | Cochrane Training. Cochrane. https://training.cochrane.org/handbook/current/chapter-25 . Accessed 20 Dec 2022.

Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355. https://doi.org/10.1136/BMJ.I4919 .

Ma LL, Wang YY, Yang ZH, Huang D, Weng H, Zeng XT. Methodological quality (risk of bias) assessment tools for primary and secondary medical studies: what are they and which is better? Mil Med Res. 2020;7(1):1–11. https://doi.org/10.1186/S40779-020-00238-8/TABLES/1 .

Schünemann HJ, Cuello C, Akl EA, et al. GRADE guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence. J Clin Epidemiol. 2019;111:105–14. https://doi.org/10.1016/J.JCLINEPI.2018.01.012 .

Article   PubMed   Google Scholar  

Samarakoon S, Parinduri RA. The effects of social health insurance on women’s healthcare use: evidence from Indonesia. Singapore Econ Rev . Published online Dec 18, 2020. https://doi.org/10.1142/S0217590820500733 .

Agbanyo R, Peprah JA. National health insurance and the choice of delivery facility among expectant mothers in Ghana. Int J Heal Econ Manag. 2021;21(1):27–49.

Ravit M, Ravalihasy A, Audibert M, et al. The impact of the obstetrical risk insurance scheme in Mauritania on maternal healthcare utilization: a propensity score matching analysis. Health Policy Plan. 2020;35(4):388–98. https://doi.org/10.1093/HEAPOL/CZZ150 .

Chang AY, Li Y, Ogbuoji O. Modest improvements in skilled birth attendants at delivery with increased Mutuelles coverage in Rwanda. J Public Health (Bangkok). 2018;40(3):623–9. https://doi.org/10.1093/PUBMED/FDX110 .

Rashad AS, Sharaf MF, Mansour EI. Does Public Health Insurance Increase Maternal Health Care Utilization in Egypt? J Int Dev. 2019;31(6):516–20. https://doi.org/10.1002/JID.3414 .

Gouda HN, Hodge A, Bermejo R, Zeck W, Jimenez-Soto E. The Impact of Healthcare Insurance on the Utilisation of Facility-Based Delivery for Childbirth in the Philippines. PLoS ONE. 2016;11(12):167268. https://doi.org/10.1371/JOURNAL.PONE.0167268 .

Philibert A, Ravit M, Ridde V, et al. Maternal and neonatal health impact of obstetrical risk insurance scheme in Mauritania: a quasi experimental before-and-after study. Health Policy Plan. 2017;32(3):405–17. https://doi.org/10.1093/HEAPOL/CZW142 .

Anindya K, Lee JT, McPake B, Wilopo SA, Millett C, Carvalho N. Impact of Indonesia’s national health insurance scheme on inequality in access to maternal health services: a propensity score matched analysis. J Glob Health. 2020;10(1).  https://doi.org/10.7189/JOGH.10.010429 .

Kuwawenaruwa A, Ramsey K, Binyaruka P, Baraka J, Manzi F, Borghi J. Implementation and effectiveness of free health insurance for the poor pregnant women in Tanzania: a mixed methods evaluation. Soc Sci Med. 2019;225:17–25. https://doi.org/10.1016/J.SOCSCIMED.2019.02.005 .

Aizawa T. The impact of health insurance on out-of-pocket expenditure on delivery in Indonesia. 2019;40(12):1374–95. https://doi.org/10.1080/07399332.2019.1578778 .

Bonfrer I, Breebaart L, De Poel E, Van. The Effects of Ghana’s National Health Insurance Scheme on Maternal and Infant Health Care Utilization. PLoS ONE. 2016;11(11):e0165623. https://doi.org/10.1371/JOURNAL.PONE.0165623 .

El Omari S, Karasneh M. Social health insurance in the Philippines: do the poor really benefit? J Econ Financ. 2021;45(1):171–87. https://doi.org/10.1007/S12197-020-09525-5/TABLES/11 .

Wang W, Temsah G, Mallick L. The impact of health insurance on maternal health care utilization: evidence from Ghana. Indonesia and Rwanda Health Policy Plan. 2017;32(3):366–75. https://doi.org/10.1093/HEAPOL/CZW135 .

Kofinti RE, Asmah EE, Ameyaw EK. Comparative study of the effect of National Health Insurance Scheme on use of delivery and antenatal care services between rural and urban women in Ghana. Health Econ Rev. 2022;12(1):1–19. https://doi.org/10.1186/S13561-022-00357-Z/TABLES/6 .

Bousmah MAQ, Diakhaté P, Toulao GÀD, Le Hesran JY, Lalou R. Effects of a free health insurance programme for the poor on health service utilisation and financial protection in Senegal. BMJ Glob Heal. 2022;7(Suppl 9):e009977. https://doi.org/10.1136/BMJGH-2022-009977 .

Mussa EC, Palermo T, Angeles G, et al. Impact of community-based health insurance on health services utilisation among vulnerable households in Amhara region. Ethiopia BMC Health Serv Res. 2023;23(1):1–15. https://doi.org/10.1186/S12913-023-09024-3/TABLES/3 .

Garg S, Tripathi N, Bebarta KK. BMC Res Notes. 2023;16(1):85. https://doi.org/10.1186/S13104-023-06335-W/TABLES/3 .

Harris AD, McGregor JC, Perencevich EN, et al. The Use and Interpretation of Quasi-Experimental Studies in Medical Informatics. J Am Med Inform Assoc. 2006;13(1):16. https://doi.org/10.1197/JAMIA.M1749 .

The Demographic and Health Surveys (DHS) Program. The Demographic and Health Surveys (DHS) Program. https://dhsprogram.com/ . Accessed 3, Jul 2022.

Bauserman M, Thorsten VR, Nolen TL, et al. Maternal mortality in six low and lower-middle income countries from 2010 to 2018: risk factors and trends. Reprod Health. 2020;17(3):1–10. https://doi.org/10.1186/S12978-020-00990-Z/FIGURES/2 .

Goldenberg RL, McClure EM, Saleem S. Improving pregnancy outcomes in low- and middle-income countries. Reprod Health. 2018;15(Suppl 1). https://doi.org/10.1186/S12978-018-0524-5 .

Horton S, Levin C. Cost-Effectiveness of Interventions for Reproductive, Maternal, Neonatal, and Child Health. In: Disease Control Priorities, Third Edition (Volume 2): Reproductive, Maternal, Newborn, and Child Health. The World Bank. 2016:319–334.  https://doi.org/10.1596/978-1-4648-0348-2_ch17 .

The World BanK. Out-of-pocket expenditure (% of current health expenditure). https://data.worldbank.org/indicator/SH.XPD.OOPC.CH.ZS . Accessed 19 Jun 2022.

Vu PH, Sepehri A, Tran LTT. Trends in out-of-pocket expenditure on facility-based delivery and financial protection of health insurance: findings from Vietnam’s Household Living Standard Survey 2006–2018. Int J Heal Econ Manag. 2022;23(2):237–54. https://doi.org/10.1007/S10754-022-09330-Z/TABLES/3 .

International Planned Parenthood Federation. Ghana offers free long-term contraception in a ‘game changer’ for women’s reproductive health rights. Published 2022. https://www.ippf.org/blogs/ghana-offers-free-long-term-contraception-game-changer-womens-reproductive-health-rights . Accessed 6 June 2023.

Priyadarshani P, Borovac-Pinheiro A, Burke TF. Glob J Reprod Med Access and Quality of Postnatal Care in Low-and Middle-Income Countries: a Pragmatic Review. J Reprod Med. 2021;8(3). https://doi.org/10.19080/GJORM.2021.08.5556740 .

Sacks E, Finlayson K, Brizuela V, et al. Factors that influence uptake of routine postnatal care: findings on women’s perspectives from a qualitative evidence synthesis. PLoS ONE. 2022;17(8):e0270264. https://doi.org/10.1371/JOURNAL.PONE.0270264 .

Clarke-Deelder E, Opondo K, Oguttu M, Burke T, Cohen JL, McConnell M. Immediate postpartum care in low- and middle-income countries: a gap in healthcare quality research and practice. Am J Obstet Gynecol MFM. 2023;5(2):100764. https://doi.org/10.1016/J.AJOGMF.2022.100764 .

Beeckman K, Louckx F, Downe S, Putman K. The relationship between antenatal care and preterm birth: the importance of content of care. Eur J Public Health. 2013;23(3):366–71. https://doi.org/10.1093/EURPUB/CKS123 .

Tunçalp, Pena-Rosas JP, Lawrie T, et al. WHO recommendations on antenatal care for a positive pregnancy experience-going beyond survival. BJOG. 2017;124(6):860–2. https://doi.org/10.1111/1471-0528.14599 .

World Health Health Organisation. WHO Recommendations on antenatal care for a positive pregnancy experience. 2016. https://www.who.int/publications/i/item/9789241549912 . Accessed 28 May 2023.

World Health Organisation. WHO recommendations on maternal and newborn care for a positive postnatal experience: executive summary. Published 2022.  https://www.who.int/publications/i/item/9789240044074 . Accessed 19 Jun 2022.

Wojcieszek AM, Bonet M, Portela A, et al. WHO recommendations on maternal and newborn care for a positive postnatal experience: strengthening the maternal and newborn care continuum. BMJ Glob Heal. 2023;8:10992. https://doi.org/10.1136/bmjgh-2022-010992 .

Abadie A, Cattaneo MD. Econometric Methods for Program Evaluation. Published online. 2018. https://doi.org/10.1146/annurev-economics .

Abadie A, Imbens GW. Bias-Corrected Matching Estimators for Average Treatment Effects. J Bus Econ Stat. 2012;29(1):1–11. https://doi.org/10.1198/JBES.2009.07333 .

Cunningham S. Causal Inference The Mixtape - 3 Directed Acyclic Graphs. In: Causal Inference. 2021 https://mixtape.scunning.com/03-directed_acyclical_graphs .

Google Scholar  

Ronsmans C, Graham WJ. Maternal mortality: who, when, where, and why. Lancet. 2006;368(9542):1189–200. https://doi.org/10.1016/S0140-6736(06)69380-X .

Barasa E, Kazungu J, Nguhiu P, Ravishankar N. Examining the level and inequality in health insurance coverage in 36 sub-Saharan African countries. BMJ Glob Heal. 2021;6(4):e004712. https://doi.org/10.1136/BMJGH-2020-004712 .

Eze P, Ilechukwu S, Lawani LO. Impact of community-based health insurance in low- and middle-income countries: a systematic review and meta-analysis. Vojnov L, ed. PLoS One. 2023;18(6):e0287600. https://doi.org/10.1371/journal.pone.0287600 .

Download references

Acknowledgements

Open access funding provided by Lund University. This study was funded by the Swedish Research Council.

The funder of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report.

Author information

Authors and affiliations.

Department of Clinical Sciences, Lund University, Jan Waldenströms Gata, 35205 02, Malmö, Sweden

Joseph Kazibwe, Andrea Hannah Kaiser, Björn Ekman & Jesper Sundewall

Department of Family Medicine and Population Health, University of Antwerp, Antwerp, Belgium

Phuong Bich Tran

Department of Social Science, Makerere University, Kampala, Uganda

Simon Peter Kasagga

Department of Economics, University of Zambia, Lusaka, Zambia

Felix Masiye

HEARD, University of KwaZulu-Natal, Durban, South Africa

Jesper Sundewall

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualisation: – JS, BE, FM, JK. Developing and carrying out search strategy: – JK, PBT. Screening of articles: – SPK, JK, PBT. Data extraction, and quality assessment: – JK, PBT. Validation: – JK, PBT, JS. Data curation and analysis: – JK, PBT, AHK. Funding acquisition: – JS, BE, FM. Methodology: – JK, PBT, AHK. Project administration: – JK. Supervision: – JS, BE, FM. writing – original draft: – JK. writing review & editing: – JK, PBT, AHK, FM, BE, JS.

Corresponding author

Correspondence to Joseph Kazibwe .

Ethics declarations

Ethics approval and consent to participate.

The study utilised secondary data/literature that is publicly available and did not use any personal or private data.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Kazibwe, J., Tran, P.B., Kaiser, A.H. et al. The impact of health insurance on maternal and reproductive health service utilization and financial protection in low- and lower middle-income countries: a systematic review of the evidence. BMC Health Serv Res 24 , 432 (2024). https://doi.org/10.1186/s12913-024-10815-5

Download citation

Received : 13 October 2022

Accepted : 01 March 2024

Published : 05 April 2024

DOI : https://doi.org/10.1186/s12913-024-10815-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Health insurance
• Low and lower middle-income countries
• Maternal and reproductive health
• Financial protection

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

• Open access
• Published: 23 August 2022

Prognostic risk factors for moderate-to-severe exacerbations in patients with chronic obstructive pulmonary disease: a systematic literature review

• John R. Hurst 1 ,
• MeiLan K. Han 2 ,
• Barinder Singh 3 ,
• Sakshi Sharma 4 ,
• Gagandeep Kaur 3 ,
• Enrico de Nigris 5 ,
• Ulf Holmgren 6 &
• Mohd Kashif Siddiqui 3  

Respiratory Research volume  23 , Article number:  213 ( 2022 ) Cite this article

6309 Accesses

20 Citations

33 Altmetric

Metrics details

Chronic obstructive pulmonary disease (COPD) is a leading cause of morbidity and mortality worldwide. COPD exacerbations are associated with a worsening of lung function, increased disease burden, and mortality, and, therefore, preventing their occurrence is an important goal of COPD management. This review was conducted to identify the evidence base regarding risk factors and predictors of moderate-to-severe exacerbations in patients with COPD.

A literature review was performed in Embase, MEDLINE, MEDLINE In-Process, and the Cochrane Central Register of Controlled Trials (CENTRAL). Searches were conducted from January 2015 to July 2019. Eligible publications were peer-reviewed journal articles, published in English, that reported risk factors or predictors for the occurrence of moderate-to-severe exacerbations in adults age ≥ 40 years with a diagnosis of COPD.

The literature review identified 5112 references, of which 113 publications (reporting results for 76 studies) met the eligibility criteria and were included in the review. Among the 76 studies included, 61 were observational and 15 were randomized controlled clinical trials. Exacerbation history was the strongest predictor of future exacerbations, with 34 studies reporting a significant association between history of exacerbations and risk of future moderate or severe exacerbations. Other significant risk factors identified in multiple studies included disease severity or bronchodilator reversibility (39 studies), comorbidities (34 studies), higher symptom burden (17 studies), and higher blood eosinophil count (16 studies).

Conclusions

This systematic literature review identified several demographic and clinical characteristics that predict the future risk of COPD exacerbations. Prior exacerbation history was confirmed as the most important predictor of future exacerbations. These prognostic factors may help clinicians identify patients at high risk of exacerbations, which are a major driver of the global burden of COPD, including morbidity and mortality.

Chronic obstructive pulmonary disease (COPD) is the third leading cause of death worldwide [ 1 ]. Based upon disability-adjusted life-years, COPD ranked sixth out of 369 causes of global disease burden in 2019 [ 2 ]. COPD exacerbations are associated with a worsening of lung function, and increased disease burden and mortality (of those patients hospitalized for the first time with an exacerbation, > 20% die within 1 year of being discharged) [ 3 ]. Furthermore, patients with COPD consider exacerbations or hospitalization due to exacerbations to be the most important disease outcome, having a large impact on their lives [ 4 ]. Therefore, reducing the future risk of COPD exacerbations is a key goal of COPD management [ 5 ].

Being able to predict the level of risk for each patient allows clinicians to adapt treatment and patients to adjust their lifestyle (e.g., through a smoking cessation program) to prevent exacerbations [ 3 ]. As such, identifying high-risk patients using measurable risk factors and predictors that correlate with exacerbations is critical to reduce the burden of disease and prevent a cycle of decline encompassing irreversible lung damage, worsening quality of life (QoL), increasing disease burden, high healthcare costs, and early death.

Prior history of exacerbations is generally thought to be the best predictor of future exacerbations; however, there is a growing body of evidence suggesting other demographic and clinical characteristics, including symptom burden, airflow obstruction, comorbidities, and inflammatory biomarkers, also influence risk [ 6 , 7 , 8 , 9 ]. For example, in the prospective ECLIPSE observational study, the likelihood of patients experiencing an exacerbation within 1 year of follow-up increased significantly depending upon several factors, including prior exacerbation history, forced expiratory volume in 1 s (FEV 1 ), St. George’s Respiratory Questionnaire (SGRQ) score, gastroesophageal reflux, and white blood cell count [ 9 ].

Many studies have assessed predictors of COPD exacerbations across a variety of countries and patient populations. This systematic literature review (SLR) was conducted to identify and compile the evidence base regarding risk factors and predictors of moderate-to-severe exacerbations in patients with COPD.

• Systematic literature review

A comprehensive search strategy was designed to identify English-language studies published in peer-reviewed journals providing data on risk factors or predictors of moderate or severe exacerbations in adults aged ≥ 40 years with a diagnosis of COPD (sample size ≥ 100). The protocol is summarized in Table 1 and the search strategy is listed in Additional file 1 : Table S1. Key biomedical electronic literature databases were searched from January 2015 until July 2019. Other sources were identified via bibliographic searching of relevant systematic reviews.

Study selection process

Implementation and reporting followed the recommendations and standards of the Preferred Reporting Items for Systematic reviews and Meta-analyses (PRISMA) statement [ 10 ]. An independent reviewer conducted the first screening based on titles and abstracts, and a second reviewer performed a quality check of the excluded evidence. A single independent reviewer also conducted the second screening based on full-text articles, with a quality check of excluded evidence performed by a second reviewer. Likewise, data tables of the included studies were generated by one reviewer, and another reviewer performed a quality check of extracted data. Where more than one publication was identified describing a single study or trial, data were compiled into a single entry in the data-extraction table to avoid double counting of patients and studies. One publication was designated as the ‘primary publication’ for the purposes of the SLR, based on the following criteria: most recently published evidence and/or the article that presented the majority of data (e.g., journal articles were preferred over conference abstracts; articles that reported results for the full population were preferred over later articles providing results of subpopulations). Other publications reporting results from the same study were designated as ‘linked publications’; any additional data in the linked publications that were not included in the primary publication were captured in the SLR. Conference abstracts were excluded from the SLR unless they were a ‘linked publication.’

Included studies

A total of 5112 references (Fig.  1 ) were identified from the database searches. In total, 76 studies from 113 publications were included in the review. Primary publications and ‘linked publications’ for each study are detailed in Additional file 1 : Table S2, and study characteristics are shown in Additional file 1 : Table S3. The studies included clinical trials, registry studies, cross-sectional studies, cohort studies, database studies, and case–control studies. All 76 included studies were published in peer-reviewed journals. Regarding study design, 61 of the studies were observational (34 retrospective observational studies, 19 prospective observational studies, four cross-sectional studies, two studies with both retrospective and prospective cohort data, one case–control study, and one with cross-sectional and longitudinal data) and 15 were randomized controlled clinical trials.

PRISMA flow diagram of studies through the systematic review process. CA conference abstract, CENTRAL Cochrane Central Register of Controlled Trials, PRISMA  Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Of the 76 studies, 16 were conducted in North America (13 studies in the USA, two in Canada, and one in Mexico); 26 were conducted in Europe (seven studies in Spain, four in the UK, three in Denmark, two studies each in Bulgaria, the Netherlands, and Switzerland, and one study each in Sweden, Serbia, Portugal, Greece, Germany, and France) and 17 were conducted in Asia (six studies in South Korea, four in China, three in Taiwan, two in Japan, and one study each in Singapore and Israel). One study each was conducted in Turkey and Australia. Fifteen studies were conducted across multiple countries.

The majority of the studies (n = 54) were conducted in a multicenter setting, while 22 studies were conducted in a single-center setting. The sample size among the included studies varied from 118 to 339,389 patients.

Patient characteristics

A total of 75 studies reported patient characteristics (Additional file 1 : Table S4). The mean age was reported in 65 studies and ranged from 58.0 to 75.2 years. The proportion of male patients ranged from 39.7 to 97.6%. The majority of included studies (85.3%) had a higher proportion of males than females.

Exacerbation history (as defined per each study) was reported in 18 of 76 included studies. The proportion of patients with no prior exacerbation was reported in ten studies (range, 0.1–79.5% of patients), one or fewer prior exacerbation in ten studies (range, 46–100%), one or more prior exacerbation in eight studies (range, 18.4–100%), and two or more prior exacerbations in 12 studies (range, 6.1–55.0%).

Prognostic factors of exacerbations

A summary of the risk factors and predictors reported across the included studies is provided in Tables 2 and 3 . The overall findings of the SLR are summarized in Figs. 2 and 3 .

Risk factors for moderate-to-severe exacerbations in patients with COPD. Factors with > 30 supporting studies shown as large circles; factors with ≤ 30 supporting studies shown as small circles and should be interpreted cautiously. BDR bronchodilator reversibility, BMI body mass index, COPD chronic obstructive pulmonary disease, EOS eosinophil, QoL quality of life

Summary of risk factors for exacerbation events. a Treatment impact studies removed. BDR bronchodilator reversibility, BMI body mass index, COPD chronic obstructive pulmonary disease, EOS eosinophil, QoL quality of life

Exacerbation history within the past 12 months was the strongest predictor of future exacerbations. Across the studies assessing this predictor, 34 out of 35 studies (97.1%) reported a significant association between history of exacerbations and risk of future moderate-to-severe exacerbations (Table 3 ). Specifically, two or more exacerbations in the previous year or at least one hospitalization for COPD in the previous year were identified as reliable predictors of future moderate or severe exacerbations. Even one moderate exacerbation increased the risk of a future exacerbation, with the risk increasing further with each subsequent exacerbation (Fig.  4 ). A severe exacerbation was also found to increase the risk of subsequent exacerbation and hospitalization (Fig.  5 ). Patients experiencing one or more severe exacerbations were more likely to experience further severe exacerbations than moderate exacerbations [ 11 , 12 ]. In contrast, patients with a history of one or more moderate exacerbations were more likely to experience further moderate exacerbations than severe exacerbations [ 11 , 12 ].

Exacerbation history as a risk factor for moderate-to-severe exacerbations. Yun 2018 included two studies; the study from which data were extracted (COPDGene or ECLIPSE) is listed in parentheses. CI confidence interval, ES effect size

Exacerbation history as a risk factor for severe exacerbations. Where data have been extracted from a linked publication rather than the primary publication, the linked publication is listed in parentheses. CI confidence interval, ES , effect size

Overall, 35 studies assessed the association of comorbidities with the risk of exacerbation. All studies except one (97.1%) reported a positive association between comorbidities and the occurrence of moderate-to-severe exacerbations (Table 3 ). In addition to the presence of any comorbidity, specific comorbidities that were found to significantly increase the risk of moderate-to-severe exacerbations included anxiety and depression, cardiovascular comorbidities, gastroesophageal reflux disease/dyspepsia, and respiratory comorbidities (Fig.  6 ). Comorbidities that were significant risk factors for severe exacerbations included cardiovascular, musculoskeletal, and respiratory comorbidities, diabetes, and malignancy (Fig.  7 ). Overall, the strongest association between comorbidities and COPD readmissions in the emergency department was with cardiovascular disease. The degree of risk for both moderate-to-severe and severe exacerbations also increased with the number of comorbidities. A Dutch cohort study found that 88% of patients with COPD had at least one comorbidity, with hypertension (35%) and coronary heart disease (19%) being the most prevalent. In this cohort, the comorbidities with the greatest risk of frequent exacerbations were pulmonary cancer (odds ratio [OR] 1.85) and heart failure (OR 1.72) [ 7 ].

Comorbidities as risk factors for moderate-to-severe exacerbations. Yun 2018 included two studies; the study from which data were extracted (COPDGene or ECLIPSE) is listed in parentheses. Where data have been extracted from a linked publication rather than the primary publication, the linked publication is listed in parentheses. CI confidence interval, ES effect size, GERD gastroesophageal disease

Comorbidities as risk factors for severe exacerbations. Where data have been extracted from a linked publication rather than the primary publication, the linked publication is listed in parentheses. CI confidence interval, CKD , chronic kidney disease, ES effect size

The majority of studies assessing disease severity or bronchodilator reversibility (39/41; 95.1%) indicated a significant positive relation between risk of future exacerbations and greater disease severity, as assessed by greater lung function impairment (in terms of lower FEV 1 , FEV 1 /forced vital capacity ratio, or forced expiratory flow [25–75]/forced vital capacity ratio) or more severe Global Initiative for Chronic Obstructive Lung Disease (GOLD) class A − D, and a positive relationship between risk of future exacerbations and lack of bronchodilator reversibility (Table 3 , Figs. 8 and 9 ).

Disease severity as a risk factor for moderate-to-severe exacerbations. Yun 2018 included two studies; the study from which data were extracted (COPDGene or ECLIPSE) is listed in parentheses. Where data have been extracted from a linked publication rather than the primary publication, the linked publication is listed in parentheses. CI confidence interval, ES effect size, FEV 1 f orced expiratory volume in 1 s, FVC , forced vital capacity, GOLD Global Initiative for Obstructive Lung Disease, HR hazard ratio, OR odds ratio

Disease severity and BDR as risk factors for severe exacerbations. ACCP American College of Chest Physicians, ACOS Asthma-COPD overlap syndrome, ATS  American Thoracic Society, BDR bronchodilator reversibility, CI confidence interval, ERS  European Respiratory Society, ES effect size, FEV 1 forced expiratory volume in 1 s, FVC  forced vital capacity, GINA Global Initiative for Asthma, GOLD Global Initiative for Obstructive Lung Disease

Of 21 studies assessing the relationship between blood eosinophil count and exacerbations (Table 3 ), 16 reported estimates for the risk of moderate or severe exacerbations by eosinophil count. A positive association was observed between higher eosinophil count and a higher risk of moderate or severe exacerbations, particularly in patients not treated with an inhaled corticosteroid (ICS); however, five studies reported a significant positive association irrespective of intervention effects. The risk of moderate-to-severe exacerbations was observed to be positively associated with various definitions of higher eosinophil levels (absolute counts: ≥ 200, ≥ 300, ≥ 340, ≥ 400, and ≥ 500 cells/mm 3 ; % of blood eosinophil count: ≥ 2%, ≥ 3%, ≥ 4%, and ≥ 5%). Of note, one study found reduced efficacy of ICS in lowering moderate-to-severe exacerbation rates for current smokers versus former smokers at all eosinophil levels [ 13 ].

Of 12 studies assessing QoL scales, 11 (91.7%) studies reported a significant association between the worsening of QoL scores and the risk of future exacerbations (Table 3 ). Baseline SGRQ [ 14 , 15 ], Center for Epidemiologic Studies Depression Scale (for which increased scores may indicate impaired QoL) [ 16 ], and Clinical COPD Questionnaire [ 17 , 18 ] scores were found to be associated with future risk of moderate and/or severe COPD exacerbations. For symptom scores, six out of eight studies assessing the association between moderate-to-severe or severe exacerbations with COPD Assessment Test (CAT) scores reported a significant and positive relationship. Furthermore, the risk of moderate-to-severe exacerbations was found to be significantly higher in patients with higher CAT scores (≥ 10) [ 15 , 19 , 20 , 21 ], with one study demonstrating that a CAT score of 15 increased predictive ability for exacerbations compared with a score of 10 or more [ 18 ]. Among 15 studies that assessed the association of modified Medical Research Council (mMRC) scores with the risk of moderate-to-severe or severe exacerbation, 11 found that the risk of moderate-to-severe or severe exacerbations was significantly associated with higher mMRC scores (≥ 2) versus lower scores. Furthermore, morning and night symptoms (measured by Clinical COPD Questionnaire) were associated with poor health status and predicted future exacerbations [ 17 ].

Of 36 studies reporting the relationship between smoking status and moderate-to-severe or severe exacerbations, 22 studies (61.1%) reported a significant positive association (Table 3 ). Passive smoking was also significantly associated with an increased risk of severe exacerbations (OR 1.49) [ 20 ]. Of note, three studies reported a significantly lower rate of moderate-to-severe exacerbations in current smokers compared with former smokers [ 22 , 23 , 24 ].

A total of 14 studies assessed the association of body mass index (BMI) with the occurrence of frequent moderate-to-severe exacerbations in patients with COPD. Six out of 14 studies (42.9%) reported a significant negative association between exacerbations and BMI (Table 3 ). The risk of moderate and/or severe COPD exacerbations was highest among underweight patients compared with normal and overweight patients [ 23 , 25 , 26 , 27 , 28 ].

In the 29 studies reporting an association between age and moderate or severe exacerbations, more than half found an association of older age with an increased risk of moderate-to-severe exacerbations (58.6%; Table 3 ). Four of these studies noted a significant increase in the risk of moderate-to-severe or severe exacerbations for every 10-year increase in age [ 25 , 26 , 29 , 30 ]. However, 12 studies reported no significant association between age and moderate-to-severe or severe exacerbation risk.

Sixteen out of 33 studies investigating the impact of sex on exacerbation risk found a significant association (48.5%; Table 3 ). Among these, ten studies reported that female sex was associated with an increased risk of moderate-to-severe exacerbations, while six studies showed a higher exacerbation risk in males compared with females. There was some variation in findings by geographic location and exacerbation severity (Additional file 2 : Figs. S1 and S2). Notably, when assessing the risk of severe exacerbations, more studies found an association with male sex compared with female sex (6/13 studies vs 1/13 studies, respectively).

Both studies evaluating associations between exacerbations and environmental factors reported that colder temperature and exposure to major air pollution (NO 2 , O 3 , CO, and/or particulate matter ≤ 10 μm in diameter) increased hospital admissions due to severe exacerbations and moderate-to-severe exacerbation rates [ 31 , 32 ].

Four studies assessed the association of 6-min walk distance with the occurrence of frequent moderate-to-severe exacerbations (Table 3 ). One study (25.0%) found that shorter 6-min walk distance (representing low physical activity) was significantly associated with a shortened time to severe exacerbation, but the effect size was small (hazard ratio 0.99) [ 33 ].

Five out of six studies assessing the relationship between race or ethnicity and exacerbation risk reported significant associations (Table 3 ). Additionally, one study reported an association between geographic location in the US and exacerbations, with living in the Northeast region being the strongest predictor of severe COPD exacerbations versus living in the Midwest and South regions [ 34 ].

Overall, seven studies assessed the association of biomarkers with risk of future exacerbations (Table 3 ), with the majority identifying significant associations between inflammatory biomarkers and increased exacerbation risk, including higher C-reactive protein levels [ 8 , 35 ], fibrinogen levels [ 8 , 30 ], and white blood cell count [ 8 , 15 , 16 ].

This SLR has identified several demographic and clinical characteristics that predict the future risk of COPD exacerbations. Key factors associated with an increased risk of future moderate-to-severe exacerbations included a history of prior exacerbations, worse disease severity and bronchodilator reversibility, the presence of comorbidities, a higher eosinophil count, and older age (Fig.  2 ). These prognostic factors may help clinicians identify patients at high risk of exacerbations, which are a major driver of the burden of COPD, including morbidity and mortality [ 36 ].

Findings from this review summarize the existing evidence, validating the previously published literature [ 6 , 9 , 23 ] and suggesting that the best predictor of future exacerbations is a history of exacerbations in the prior year [ 8 , 11 , 12 , 13 , 14 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 26 , 29 , 34 , 35 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 ]. In addition, the effect size generally increased with the number of prior exacerbations, with a stronger effect observed with prior severe versus moderate exacerbations. This effect was observed across regions, including in Europe and North America, and in several global studies. This relationship represents a vicious circle, whereby one exacerbation predisposes a patient to experience future exacerbations and leading to an ever-increasing disease burden, and emphasizes the importance of preventing the first exacerbation event through early, proactive exacerbation prevention. The finding that prior exacerbations tended to be associated with future exacerbations of the same severity suggests that the severity of the underlying disease may influence exacerbation severity. However, the validity of the traditional classification of exacerbation severity has recently been challenged [ 61 ], and further work is required to understand relationships with objective assessments of exacerbation severity.

In addition to exacerbation history, disease severity and bronchodilator reversibility were also strong predictors for future exacerbations [ 8 , 14 , 16 , 18 , 19 , 20 , 22 , 23 , 24 , 26 , 28 , 29 , 33 , 37 , 40 , 43 , 44 , 45 , 46 , 48 , 50 , 51 , 52 , 56 , 59 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 ]. The association with disease severity was noted in studies that used GOLD disease stages 1–4 and those that used FEV 1 percent predicted and other lung function assessments as continuous variables. Again, this risk factor is self-perpetuating, as evidence shows that even a single moderate or severe exacerbation may almost double the rate of lung function decline [ 79 ]. Accordingly, disease severity and exacerbation history may be correlated. Margüello et al. concluded that the severity of COPD could be associated with a higher risk of exacerbations, but this effect was partly determined by the exacerbations suffered in the previous year [ 23 ]. It should be noted that FEV 1 is not recommended by GOLD for use as a predictor of exacerbation risk or mortality alone due to insufficient precision when used at the individual patient level [ 5 ].

Another factor that should be considered when assessing individual exacerbation risk is the presence of comorbidities [ 7 , 14 , 16 , 18 , 19 , 20 , 21 , 22 , 24 , 25 , 26 , 27 , 28 , 30 , 33 , 34 , 35 , 40 , 41 , 44 , 45 , 46 , 47 , 48 , 51 , 52 , 53 , 54 , 56 , 58 , 59 , 63 , 64 , 73 , 74 , 76 , 77 , 80 , 81 , 82 , 83 , 84 , 85 ]. Comorbidities are common in COPD, in part due to common risk factors (e.g., age, smoking, lifestyle factors) that also increase the risk of other chronic diseases [ 7 ]. Significant associations were observed between exacerbation risk and comorbidities, such as anxiety and depression, cardiovascular disease, diabetes, and respiratory comorbidities. As with prior exacerbations, the strength of the association increased with the number of comorbidities. Some comorbidities that were found to be associated with COPD exacerbations share a common biological mechanism of systemic inflammation, such as cardiovascular disease, diabetes, and depression [ 86 ]. Furthermore, other respiratory comorbidities, including asthma and bronchiectasis, involve inflammation of the airways [ 87 ]. In these patients, optimal management of comorbidities may reduce the risk of future COPD exacerbations (and improve QoL), although further research is needed to confirm the efficacy of this approach to exacerbation prevention. As cardiovascular conditions, including hypertension and coronary heart disease, are the most common comorbidities in people with COPD [ 7 ], reducing cardiovascular risk may be a key goal in reducing the occurrence of exacerbations. For other comorbidities, the mechanism for the association with exacerbation risk may be related to non-biological factors. For example, in depression, it has been suggested that the mechanism may relate to greater sensitivity to symptom changes or more frequent physician visits [ 88 ].

There is now a growing body of evidence reporting the relationship between blood eosinophil count and exacerbation risk [ 8 , 13 , 14 , 20 , 37 , 48 , 52 , 56 , 59 , 60 , 62 , 89 , 90 , 91 , 92 , 93 , 94 , 95 , 96 , 97 , 98 , 99 ]. Data from many large clinical trials (SUNSET [ 89 ], FLAME [ 96 ], WISDOM [ 98 ], IMPACT [ 13 ], TRISTAN [ 99 ], INSPIRE [ 99 ], KRONOS [ 91 ], TRIBUTE [ 48 ], TRILOGY [ 52 ], TRINITY [ 56 ]) have also shown relationships between treatment, eosinophil count, and exacerbation rates. Evidence shows that eosinophil count, along with other effect modifiers (e.g., exacerbation history), can be used to predict reductions in exacerbations with ICS treatment. Identifying patients most likely to respond to ICS should contribute to personalized medicine approaches to treat COPD. One challenge in drawing a strong conclusion from eosinophil counts is the choice of a cut-off value, with a variety of absolute and percentage values observed to be positively associated with the risk of moderate-to-severe exacerbations. The use of absolute counts may be more practical, as these are not affected by variations in other immune cell numbers; however, there is a lack of consensus on this point [ 100 ].

Across the studies examined, associations between sex and the risk of moderate and/or severe exacerbations were variable [ 14 , 16 , 18 , 20 , 21 , 22 , 23 , 24 , 26 , 27 , 28 , 29 , 37 , 40 , 42 , 44 , 45 , 46 , 47 , 48 , 51 , 52 , 56 , 58 , 59 , 63 , 73 , 74 , 77 , 80 , 83 , 84 , 85 ]. A greater number of studies showed an increased risk of exacerbations in females compared with males. In contrast, some studies failed to detect a relationship, suggesting that country-specific or cultural factors may play a role. A majority of the included studies evaluated more male patients than female patients; to further elucidate the relationship between sex and exacerbations, more studies in female patients are warranted. Over half of the studies that assessed the relationship between age and exacerbation risk found an association between increasing age and increasing risk of moderate-to-severe COPD exacerbations [ 14 , 16 , 18 , 20 , 21 , 22 , 23 , 24 , 26 , 27 , 28 , 29 , 33 , 40 , 42 , 44 , 45 , 47 , 51 , 52 , 54 , 56 , 63 , 73 , 74 , 77 , 80 , 83 , 85 ].

Our findings also suggested that patients with low BMI have greater risk of moderate and/or severe exacerbations. The mechanism underlying this increased risk in underweight patients is poorly understood; however, loss of lean body mass in patients with COPD may be related to ongoing systemic inflammation that impacts skeletal muscle mass [ 101 , 102 , 103 ].

A limitation of this SLR, that may have resulted in some studies with valid results being missed, was the exclusion of non-English-language studies and the limitation by date; however, the search strategy was otherwise broad, resulting in the review of a large number of studies. The majority of studies captured in this SLR were from Europe, North America, and Asia. The findings may therefore be less generalizable to patients in other regions, such as Africa or South America. Given that one study reported an association between geographic location within different regions of the US and exacerbations [ 34 ], it is plausible that risk of exacerbations may be impacted by global location. As no formal meta-analysis was planned, the assessments are based on a qualitative synthesis of studies. A majority of the included studies looked at exposures of certain factors (e.g., history of exacerbations) at baseline; however, some of these factors change over time, calling into question whether a more sophisticated statistical analysis should have been conducted in some cases to consider time-varying covariates. Our results can only inform on associations, not causation, and there are likely bidirectional relationships between many factors and exacerbation risk (e.g., health status). Finally, while our review of the literature captured a large number of prognostic factors, other variables such as genetic factors, lung microbiome composition, and changes in therapy over time have not been widely studied to date, but might also influence exacerbation frequency [ 104 ]. Further research is needed to assess the contribution of these factors to exacerbation risk.

This SLR captured publications up to July 2019. However, further studies have since been published that further support the prognostic factors identified here. For example, recent studies have reported an increased risk of exacerbations in patients with a history of exacerbations [ 105 ], comorbidities [ 106 ], poorer lung function (GOLD stage) [ 105 ], higher symptomatic burden [ 107 ], female sex [ 105 ], and lower BMI [ 106 , 108 ].

In summary, the literature assessing risk factors for moderate-to-severe COPD exacerbations shows that there are associations between several demographic and disease characteristics with COPD exacerbations, potentially allowing clinicians to identify patients most at risk of future exacerbations. Exacerbation history, comorbidities, and disease severity or bronchodilator reversibility were the factors most strongly associated with exacerbation risk, and should be considered in future research efforts to develop prognostic tools to estimate the likelihood of exacerbation occurrence. Importantly, many prognostic factors for exacerbations, such as symptom burden, QoL, and comorbidities, are modifiable with optimal pharmacologic and non-pharmacologic treatments or lifestyle modifications. Overall, the evidence suggests that, taken together, predicting and reducing exacerbation risk is an achievable goal in COPD.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Body mass index

COPD Assessment Test

Chronic obstructive pulmonary disease

Forced expiratory volume in 1 s

Global Initiative for Chronic Obstructive Lung Disease

Inhaled corticosteroid

Modified Medical Research Council

Quality of life

St. George’s Respiratory Questionnaire

World Health Organization. The top 10 causes of death. 2018. https://www.who.int/news-room/fact-sheets/detail/the-top-10-causes-of-death . Accessed 22 Jul 2020.

GBD 2019 Diseases and Injuries Collaborators. Global burden of 369 diseases and injuries in 204 countries and territories, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019. The Lancet. 2020;396:1204–22.

Article   Google Scholar  

Hurst JR, Skolnik N, Hansen GJ, Anzueto A, Donaldson GC, Dransfield MT, Varghese P. Understanding the impact of chronic obstructive pulmonary disease exacerbations on patient health and quality of life. Eur J Intern Med. 2020;73:1–6.

Article   PubMed   Google Scholar  

Zhang Y, Morgan RL, Alonso-Coello P, Wiercioch W, Bała MM, Jaeschke RR, Styczeń K, Pardo-Hernandez H, Selva A, Ara Begum H, et al. A systematic review of how patients value COPD outcomes. Eur Respir J. 2018;52:1800222.

Global Initiative for Chronic Obstructive Lung Disease. 2022 GOLD Report. Global strategy for the diagnosis, management and prevention of COPD. 2022. https://goldcopd.org/2022-gold-reports-2/ . Accessed 02 Feb 2022.

Müllerová H, Shukla A, Hawkins A, Quint J. Risk factors for acute exacerbations of COPD in a primary care population: a retrospective observational cohort study. BMJ Open. 2014;4: e006171.

Article   PubMed   PubMed Central   Google Scholar  

Westerik JAM, Metting EI, van Boven JFM, Tiersma W, Kocks JWH, Schermer TR. Associations between chronic comorbidity and exacerbation risk in primary care patients with COPD. Respir Res. 2017;18:31.

Vedel-Krogh S, Nielsen SF, Lange P, Vestbo J, Nordestgaard BG. Blood eosinophils and exacerbations in chronic obstructive pulmonary disease. The Copenhagen General Population Study. Am J Respir Crit Care Med. 2016;193:965–74.

Hurst JR, Vestbo J, Anzueto A, Locantore N, Müllerová H, Tal-Singer R, Miller B, Lomas DA, Agusti A, Macnee W, et al. Susceptibility to exacerbation in chronic obstructive pulmonary disease. N Engl J Med. 2010;363:1128–38.

Article   CAS   PubMed   Google Scholar  

Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097.

Çolak Y, Afzal S, Marott JL, Nordestgaard BG, Vestbo J, Ingebrigtsen TS, Lange P. Prognosis of COPD depends on severity of exacerbation history: a population-based analysis. Respir Med. 2019;155:141–7.

Rothnie KJ, Müllerová H, Smeeth L, Quint JK. Natural history of chronic obstructive pulmonary disease exacerbations in a general practice-based population with chronic obstructive pulmonary disease. Am J Respir Crit Care Med. 2018;198:464–71.

Pascoe S, Barnes N, Brusselle G, Compton C, Criner GJ, Dransfield MT, Halpin DMG, Han MK, Hartley B, Lange P, et al. Blood eosinophils and treatment response with triple and dual combination therapy in chronic obstructive pulmonary disease: analysis of the IMPACT trial. Lancet Respir Med. 2019;7:745–56.

Yun JH, Lamb A, Chase R, Singh D, Parker MM, Saferali A, Vestbo J, Tal-Singer R, Castaldi PJ, Silverman EK, et al. Blood eosinophil count thresholds and exacerbations in patients with chronic obstructive pulmonary disease. J Allergy Clin Immunol. 2018;141:2037-2047.e10.

Yoon HY, Park SY, Lee CH, Byun MK, Na JO, Lee JS, Lee WY, Yoo KH, Jung KS, Lee JH. Prediction of first acute exacerbation using COPD subtypes identified by cluster analysis. Int J Chron Obstruct Pulmon Dis. 2019;14:1389–97.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Yohannes AM, Mulerova H, Lavoie K, Vestbo J, Rennard SI, Wouters E, Hanania NA. The association of depressive symptoms with rates of acute exacerbations in patients with COPD: results from a 3-year longitudinal follow-up of the ECLIPSE cohort. J Am Med Dir Assoc. 2017;18:955-959.e6.

Tsiligianni I, Metting E, van der Molen T, Chavannes N, Kocks J. Morning and night symptoms in primary care COPD patients: a cross-sectional and longitudinal study. An UNLOCK study from the IPCRG. NPJ Prim Care Respir Med. 2016;26:16040.

Jo YS, Yoon HI, Kim DK, Yoo CG, Lee CH. Comparison of COPD Assessment Test and Clinical COPD Questionnaire to predict the risk of exacerbation. Int J Chron Obstruct Pulmon Dis. 2018;13:101–7.

Marçôa R, Rodrigues DM, Dias M, Ladeira I, Vaz AP, Lima R, Guimarães M. Classification of Chronic Obstructive Pulmonary Disease (COPD) according to the new Global Initiative for Chronic Obstructive Lung Disease (GOLD) 2017: comparison with GOLD 2011. COPD. 2018;15:21–6.

Han MK, Quibrera PM, Carretta EE, Barr RG, Bleecker ER, Bowler RP, Cooper CB, Comellas A, Couper DJ, Curtis JL, et al. Frequency of exacerbations in patients with chronic obstructive pulmonary disease: an analysis of the SPIROMICS cohort. Lancet Respir Med. 2017;5:619–26.

Yii ACA, Loh CH, Tiew PY, Xu H, Taha AAM, Koh J, Tan J, Lapperre TS, Anzueto A, Tee AKH. A clinical prediction model for hospitalized COPD exacerbations based on “treatable traits.” Int J Chron Obstruct Pulmon Dis. 2019;14:719–28.

McGarvey L, Lee AJ, Roberts J, Gruffydd-Jones K, McKnight E, Haughney J. Characterisation of the frequent exacerbator phenotype in COPD patients in a large UK primary care population. Respir Med. 2015;109:228–37.

Margüello MS, Garrastazu R, Ruiz-Nuñez M, Helguera JM, Arenal S, Bonnardeux C, León C, Miravitlles M, García-Rivero JL. Independent effect of prior exacerbation frequency and disease severity on the risk of future exacerbations of COPD: a retrospective cohort study. NPJ Prim Care Respir Med. 2016;26:16046.

Engel B, Schindler C, Leuppi JD, Rutishauser J. Predictors of re-exacerbation after an index exacerbation of chronic obstructive pulmonary disease in the REDUCE randomised clinical trial. Swiss Med Wkly. 2017;147: w14439.

PubMed   Google Scholar  

Benson VS, Müllerová H, Vestbo J, Wedzicha JA, Patel A, Hurst JR. Evaluation of COPD longitudinally to identify predictive surrogate endpoints (ECLIPSE) investigators. Associations between gastro-oesophageal reflux, its management and exacerbations of chronic obstructive pulmonary disease. Respir Med. 2015;109:1147–54.

Santibáñez M, Garrastazu R, Ruiz-Nuñez M, Helguera JM, Arenal S, Bonnardeux C, León C, García-Rivero JL. Predictors of hospitalized exacerbations and mortality in chronic obstructive pulmonary disease. PLoS ONE. 2016;11: e0158727.

Article   PubMed   PubMed Central   CAS   Google Scholar  

Jo YS, Kim YH, Lee JY, Kim K, Jung KS, Yoo KH, Rhee CK. Impact of BMI on exacerbation and medical care expenses in subjects with mild to moderate airflow obstruction. Int J Chron Obstruct Pulmon Dis. 2018;13:2261–9.

Alexopoulos EC, Malli F, Mitsiki E, Bania EG, Varounis C, Gourgoulianis KI. Frequency and risk factors of COPD exacerbations and hospitalizations: a nationwide study in Greece (Greek Obstructive Lung Disease Epidemiology and health ecoNomics: GOLDEN study). Int J Chron Obstruct Pulmon Dis. 2015;10:2665–74.

PubMed   PubMed Central   Google Scholar  

Liu D, Peng SH, Zhang J, Bai SH, Liu HX, Qu JM. Prediction of short term re-exacerbation in patients with acute exacerbation of chronic obstructive pulmonary disease. Int J Chron Obstruct Pulmon Dis. 2015;10:1265–73.

Müllerová H, Maselli DJ, Locantore N, Vestbo J, Hurst JR, Wedzicha JA, Bakke P, Agusti A, Anzueto A. Hospitalized exacerbations of COPD: risk factors and outcomes in the ECLIPSE cohort. Chest. 2015;147:999–1007.

de Miguel-Díez J, Hernández-Vázquez J, López-de-Andrés A, Álvaro-Meca A, Hernández-Barrera V, Jiménez-García R. Analysis of environmental risk factors for chronic obstructive pulmonary disease exacerbation: a case-crossover study (2004–2013). PLoS ONE. 2019;14: e0217143.

Krachunov II, Kyuchukov NH, Ivanova ZI, Yanev NA, Hristova PA, Borisova ED, Popova TP, Pavlov PS, Nikolova PT, Ivanov YY. Impact of air pollution and outdoor temperature on the rate of chronic obstructive pulmonary disease exacerbations. Folia Med (Plovdiv). 2017;59:423–9.

Article   CAS   Google Scholar  

Baumeler L, Papakonstantinou E, Milenkovic B, Lacoma A, Louis R, Aerts JG, Welte T, Kostikas K, Blasi F, Boersma W, et al. Therapy with proton-pump inhibitors for gastroesophageal reflux disease does not reduce the risk for severe exacerbations in COPD. Respirology. 2016;21:883–90.

Annavarapu S, Goldfarb S, Gelb M, Moretz C, Renda A, Kaila S. Development and validation of a predictive model to identify patients at risk of severe COPD exacerbations using administrative claims data. Int J Chron Obstruct Pulmon Dis. 2018;13:2121–30.

Crisafulli E, Torres A, Huerta A, Méndez R, Guerrero M, Martinez R, Liapikou A, Soler N, Sethi S, Menéndez R. C-reactive protein at discharge, diabetes mellitus and ≥1 hospitalization during previous year predict early readmission in patients with acute exacerbation of chronic obstructive pulmonary disease. COPD. 2015;12:311–20.

Bollmeier SG, Hartmann AP. Management of chronic obstructive pulmonary disease: a review focusing on exacerbations. Am J Health Syst Pharm. 2020;77:259–68.

Bafadhel M, Peterson S, De Blas MA, Calverley PM, Rennard SI, Richter K, Fagerås M. Predictors of exacerbation risk and response to budesonide in patients with chronic obstructive pulmonary disease: a post-hoc analysis of three randomised trials. Lancet Respir Med. 2018;6:117–26.

Calverley PM, Anzueto AR, Dusser D, Mueller A, Metzdorf N, Wise RA. Treatment of exacerbations as a predictor of subsequent outcomes in patients with COPD. Int J Chron Obstruct Pulmon Dis. 2018;13:1297–308.

Calverley PM, Tetzlaff K, Dusser D, Wise RA, Mueller A, Metzdorf N, Anzueto A. Determinants of exacerbation risk in patients with COPD in the TIOSPIR study. Int J Chron Obstruct Pulmon Dis. 2017;12:3391–405.

Eklöf J, Sørensen R, Ingebrigtsen TS, Sivapalan P, Achir I, Boel JB, Bangsborg J, Ostergaard C, Dessau RB, Jensen US, et al. Pseudomonas aeruginosa and risk of death and exacerbations in patients with chronic obstructive pulmonary disease: an observational cohort study of 22 053 patients. Clin Microbiol Infect. 2020;26:227–34.

Estirado C, Ceccato A, Guerrero M, Huerta A, Cilloniz C, Vilaró O, Gabarrús A, Gea J, Crisafulli E, Soler N, Torres A. Microorganisms resistant to conventional antimicrobials in acute exacerbations of chronic obstructive pulmonary disease. Respir Res. 2018;19:119.

Fuhrman C, Moutengou E, Roche N, Delmas MC. Prognostic factors after hospitalization for COPD exacerbation. Rev Mal Respir. 2017;34:1–18.

Krachunov I, Kyuchukov N, Ivanova Z, Yanev NA, Hristova PA, Pavlov P, Glogovska P, Popova T, Ivanov YY. Stability of frequent exacerbator phenotype in patients with chronic obstructive pulmonary disease. Folia Med (Plovdiv). 2018;60:536–45.

Make BJ, Eriksson G, Calverley PM, Jenkins CR, Postma DS, Peterson S, Östlund O, Anzueto A. A score to predict short-term risk of COPD exacerbations (SCOPEX). Int J Chron Obstruct Pulmon Dis. 2015;10:201–9.

Montserrat-Capdevila J, Godoy P, Marsal JR, Barbé F. Predictive model of hospital admission for COPD exacerbation. Respir Care. 2015;60:1288–94.

Montserrat-Capdevila J, Godoy P, Marsal JR, Barbé F, Galván L. Risk factors for exacerbation in chronic obstructive pulmonary disease: a prospective study. Int J Tuberc Lung Dis. 2016;20:389–95.

Orea-Tejeda A, Navarrete-Peñaloza AG, Verdeja-Vendrell L, Jiménez-Cepeda A, González-Islas DG, Hernández-Zenteno R, Keirns-Davis C, Sánchez-Santillán R, Velazquez-Montero A, Puentes RG. Right heart failure as a risk factor for severe exacerbation in patients with chronic obstructive pulmonary disease: prospective cohort study. Clin Respir J. 2018;12:2635–41.

Papi A, Vestbo J, Fabbri L, Corradi M, Prunier H, Cohuet G, Guasconi A, Montagna I, Vezzoli S, Petruzzelli S, et al. Extrafine inhaled triple therapy versus dual bronchodilator therapy in chronic obstructive pulmonary disease (TRIBUTE): a double-blind, parallel group, randomised controlled trial. Lancet. 2018;391:1076–84.

Lipson DA, Barnhart F, Brealey N, Brooks J, Criner GJ, Day NC, Dransfield MT, Halpin DMG, Han MK, Jones CE, et al. Once-daily single-inhaler triple versus dual therapy in patients with COPD. N Engl J Med. 2018;378:1671–80.

Pasquale MK, Xu Y, Baker CL, Zou KH, Teeter JG, Renda AM, Davis CC, Lee TC, Bobula J. COPD exacerbations associated with the modified Medical Research Council scale and COPD assessment test among Humana Medicare members. Int J Chron Obstruct Pulmon Dis. 2016;11:111–21.

Schuler M, Wittmann M, Faller H, Schultz K. Including changes in dyspnea after inpatient rehabilitation improves prediction models of exacerbations in COPD. Respir Med. 2018;141:87–93.

Singh D, Papi A, Corradi M, Pavlišová I, Montagna I, Francisco C, Cohuet G, Vezzoli S, Scuri M, Vestbo J. Single inhaler triple therapy versus inhaled corticosteroid plus long-acting β 2 -agonist therapy for chronic obstructive pulmonary disease (TRILOGY): a double-blind, parallel group, randomised controlled trial. Lancet. 2016;388:963–73.

Søgaard M, Madsen M, Løkke A, Hilberg O, Sørensen HT, Thomsen RW. Incidence and outcomes of patients hospitalized with COPD exacerbation with and without pneumonia. Int J Chron Obstruct Pulmon Dis. 2016;11:455–65.

Stanford RH, Nag A, Mapel DW, Lee TA, Rosiello R, Schatz M, Vekeman F, Gauthier-Loiselle M, Merrigan JFP, Duh MS. Claims-based risk model for first severe COPD exacerbation. Am J Manag Care. 2018;24:e45–53.

Stanford RH, Lau MS, Li Y, Stemkowski S. External validation of a COPD risk measure in a commercial and medicare population: the COPD treatment ratio. J Manag Care Spec Pharm. 2019;25:58–69.

Vestbo J, Papi A, Corradi M, Blazhko V, Montagna I, Francisco C, Cohuet G, Vezzoli S, Scuri M, Singh D. Single inhaler extrafine triple therapy versus long-acting muscarinic antagonist therapy for chronic obstructive pulmonary disease (TRINITY): a double-blind, parallel group, randomised controlled trial. Lancet. 2017;389:1919–29.

Wei X, Ma Z, Yu N, Ren J, Jin C, Mi J, Shi M, Tian L, Gao Y, Guo Y. Risk factors predict frequent hospitalization in patients with acute exacerbation of COPD. Int J Chron Obstruct Pulmon Dis. 2018;13:121–9.

Whalley D, Svedsater H, Doward L, Crawford R, Leather D, Lay-Flurrie J, Bosanquet N. Follow-up interviews from The Salford Lung Study (COPD) and analyses per treatment and exacerbations. NPJ Prim Care Respir Med. 2019;29:20.

Zeiger RS, Tran TN, Butler RK, Schatz M, Li Q, Khatry DB, Martin U, Kawatkar AA, Chen W. Relationship of blood eosinophil count to exacerbations in chronic obstructive pulmonary disease. J Allergy Clin Immunol Pract. 2018;6:944-954.e945.

Vogelmeier CF, Kostikas K, Fang J, Tian H, Jones B, Morgan CL, Fogel R, Gutzwiller FS, Cao H. Evaluation of exacerbations and blood eosinophils in UK and US COPD populations. Respir Res. 2019;20:178.

Celli BR, Fabbri LM, Aaron SD, Agusti A, Brook R, Criner GJ, Franssen FME, Humbert M, Hurst JR, O’Donnell D, et al. An updated definition and severity classification of COPD exacerbations: the Rome proposal. Am J Respir Crit Care Med. 2021;204:1251–8.

Adir Y, Hakrush O, Shteinberg M, Schneer S, Agusti A. Circulating eosinophil levels do not predict severe exacerbations in COPD: a retrospective study. ERJ Open Research. 2018;4:00022–2018.

Bartels W, Adamson S, Leung L, Sin DD, van Eeden SF. Emergency department management of acute exacerbations of chronic obstructive pulmonary disease: factors predicting readmission. Int J Chron Obstruct Pulmon Dis. 2018;13:1647–54.

Kim V, Zhao H, Regan E, Han MK, Make BJ, Crapo JD, Jones PW, Curtis JL, Silverman EK, Criner GJ, COPDGene Investigators. The St. George’s Respiratory Questionnaire definition of chronic bronchitis may be a better predictor of COPD exacerbations compared with the classic definition. Chest. 2019;156:685–95.

Abston E, Comellas A, Reed RM, Kim V, Wise RA, Brower R, Fortis S, Beichel R, Bhatt S, Zabner J, et al. Higher BMI is associated with higher expiratory airflow normalised for lung volume (FEF25-75/FVC) in COPD. BMJ Open Respir Res. 2017;4: e000231.

Emura I, Usuda H, Satou K. Appearance of large scavenger receptor A-positive cells in peripheral blood: a potential risk factor for severe exacerbation of chronic obstructive pulmonary disease. Pathol Int. 2019;69:187–92.

Erol S, Sen E, Gizem Kilic Y, Yousif A, Akkoca Yildiz O, Acican T, Saryal S. Does the 2017 revision improve the ability of GOLD to predict risk of future moderate and severe exacerbation? Clin Respir J. 2018;12:2354–60.

Han MZ, Hsiue TR, Tsai SH, Huang TH, Liao XM, Chen CZ. Validation of the GOLD 2017 and new 16 subgroups (1A–4D) classifications in predicting exacerbation and mortality in COPD patients. Int J Chron Obstruct Pulmon Dis. 2018;13:3425–33.

Huang TH, Hsiue TR, Lin SH, Liao XM, Su PL, Chen CZ. Comparison of different staging methods for COPD in predicting outcomes. Eur Resp J. 2018;51:1700577.

Jung YH, Lee DY, Kim DW, Park SS, Heo EY, Chung HS, Kim DK. Clinical significance of laryngopharyngeal reflux in patients with chronic obstructive pulmonary disease. Int J Chron Obstruct Pulmon Dis. 2015;10:1343–51.

CAS   PubMed   PubMed Central   Google Scholar  

Kim J, Kim WJ, Lee CH, Lee SH, Lee MG, Shin KC, Yoo KH, Lee JH, Lim SY, Na JO, et al. Which bronchodilator reversibility criteria can predict severe acute exacerbation in chronic obstructive pulmonary disease patients? Respir Res. 2017;18:107.

Kobayashi S, Hanagama M, Ishida M, Sato H, Ono M, Yamanda S, Yamada M, Aizawa H, Yanai M. Clinical characteristics and outcomes in Japanese patients with COPD according to the 2017 GOLD classification: the Ishinomaki COPD Network Registry. Int J Chron Obstruct Pulmon Dis. 2018;13:3947–55.

Lee SH, Lee JH, Yoon HI, Park HY, Kim TH, Yoo KH, Oh YM, Jung KS, Lee SD, Lee SW. Change in inhaled corticosteroid treatment and COPD exacerbations: an analysis of real-world data from the KOLD/KOCOSS cohorts. Respir Res. 2019;20:62.

Pavlovic R, Stefanovic S, Lazic Z, Jankovic S. Factors associated with the rate of COPD exacerbations that require hospitalization. Turk J Med Sci. 2017;47:134–41.

Song JH, Lee CH, Um SJ, Park YB, Yoo KH, Jung KS, Lee SD, Oh YM, Lee JH, Kim EK, Kim DK. Clinical impacts of the classification by 2017 GOLD guideline comparing previous ones on outcomes of COPD in real-world cohorts. Int J Chron Obstruct Pulmon Dis. 2018;13:3473–84.

Sundh J, Johansson G, Larsson K, Lindén A, Löfdahl CG, Sandström T, Janson C. The phenotype of concurrent chronic bronchitis and frequent exacerbations in patients with severe COPD attending Swedish secondary care units. Int J Chron Obstruct Pulmon Dis. 2015;10:2327–34.

Urwyler P, Hussein NA, Bridevaux PO, Chhajed PN, Geiser T, Grendelmeier P, Zellweger LJ, Kohler M, Maier S, Miedinger D, et al. Predictive factors for exacerbation and reexacerbation in chronic obstructive pulmonary disease: an extension of the Cox model to analyze data from the Swiss COPD cohort. Multidiscip Respir Med. 2019;14:7.

Wallace AE, Kaila S, Bayer V, Shaikh A, Shinde MU, Willey VJ, Napier MB, Singer JR. Health care resource utilization and exacerbation rates in patients with COPD stratified by disease severity in a commercially insured population. J Manag Care Spec Pharm. 2019;25:205–17.

Halpin DMG, Decramer M, Celli BR, Mueller A, Metzdorf N, Tashkin DP. Effect of a single exacerbation on decline in lung function in COPD. Respir Med. 2017;128:85–91.

Bade BC, DeRycke EC, Ramsey C, Skanderson M, Crothers K, Haskell S, Bean-Mayberry B, Brandt C, Bastian LA, Akgün KM. Sex differences in veterans admitted to the hospital for chronic obstructive pulmonary disease exacerbation. Ann Am Thorac Soc. 2019;16:707–14.

Iyer AS, Bhatt SP, Dransfield M, Kinney G, Holm K, Wamboldt FS, Hanania N, Martinez C, Regan E, Foreman MG, et al. Psychological distress prospectively predicts severe exacerbations in smokers with and without airflow limitation—a longitudinal follow-up study of the COPDGene cohort [abstract]. Am J Respir Crit Care Med. 2017. https://doi.org/10.1164/ajrccm-conference.2017.195.1_MeetingAbstracts.A4709 .

Diamond M, Zhao H, Armstrong HF, Morrison M, Bailey KL, Carretta EE, Criner GJ, Han MK, Bleeker E, Cooper CB, et al. Anxiety and depression, either alone or in combination, are associated with respiratory exacerbations in smokers with and without COPD [abstract]. Am J Respir Crit Care Med. 2017;195:1615–31.

Google Scholar  

Lau CS, Siracuse BL, Chamberlain RS. Readmission After COPD Exacerbation Scale: determining 30-day readmission risk for COPD patients. Int J Chron Obstruct Pulmon Dis. 2017;12:1891–902.

Pikoula M, Quint JK, Nissen F, Hemingway H, Smeeth L, Denaxas S. Identifying clinically important COPD sub-types using data-driven approaches in primary care population based electronic health records. BMC Med Inform Decis Mak. 2019;19:86.

Wei YF, Tsai YH, Wang CC, Kuo PH. Impact of overweight and obesity on acute exacerbations of COPD—subgroup analysis of the Taiwan Obstructive Lung Disease cohort. Int J Chron Obstruct Pulmon Dis. 2017;12:2723–9.

Barnes PJ, Celli BR. Systemic manifestations and comorbidities of COPD. Eur Resp J. 2009;33:1165–85.

Polverino E, Dimakou K, Hurst J, Martinez-Garcia MA, Miravitlles M, Paggiaro P, Shteinberg M, Aliberti S, Chalmers JD. The overlap between bronchiectasis and chronic airway diseases: state of the art and future directions. Eur Respir J. 2018;52:1800328.

Xu W, Collet JP, Shapiro S, Lin Y, Yang T, Platt RW, Wang C, Bourbeau J. Independent effect of depression and anxiety on chronic obstructive pulmonary disease exacerbations and hospitalizations. Am J Respir Crit Care Med. 2008;178:913–20.

Chapman KR, Hurst JR, Frent SM, Larbig M, Fogel R, Guerin T, Banerji D, Patalano F, Goyal P, Pfister P, et al. Long-term triple therapy de-escalation to indacaterol/glycopyrronium in patients with chronic obstructive pulmonary disease (SUNSET): a randomized, double-blind, triple-dummy clinical trial. Am J Respir Crit Care Med. 2018;198:329–39.

Couillard S, Larivée P, Courteau J, Vanasse A. Eosinophils in COPD exacerbations are associated with increased readmissions. Chest. 2017;151:366–73.

Ferguson GT, Rabe KF, Martinez FJ, Fabbri LM, Wang C, Ichinose M, Bourne E, Ballal S, Darken P, DeAngelis K, et al. Triple therapy with budesonide/glycopyrrolate/formoterol fumarate with co-suspension delivery technology versus dual therapies in chronic obstructive pulmonary disease (KRONOS): a double-blind, parallel-group, multicentre, phase 3 randomised controlled trial. Lancet Respir Med. 2018;6:747–58.

Ko FWS, Chan KP, Ngai J, Ng SS, Yip WH, Ip A, Chan TO, Hui DSC. Blood eosinophil count as a predictor of hospital length of stay in COPD exacerbations. Respirology. 2019;25:259–66.

MacDonald MI, Osadnik CR, Bulfin L, Hamza K, Leong P, Wong A, King PT, Bardin PG. Low and high blood eosinophil counts as biomarkers in hospitalized acute exacerbations of COPD. Chest. 2019;156:92–100.

Müllerová H, Hahn B, Simard EP, Mu G, Hatipoğlu U. Exacerbations and health care resource use among patients with COPD in relation to blood eosinophil counts. Int J Chron Obstruct Pulmon Dis. 2019;14:683–92.

Bafadhel M, Greening NJ, Harvey-Dunstan TC, Williams JEA, Morgan MD, Brightling CE, Hussain SF, Pavord ID, Singh SJ, Steiner MC. Blood eosinophils and outcomes in severe hospitalised exacerbations of COPD. Chest. 2016;150:320–8.

Roche N, Chapman KR, Vogelmeier CF, Herth FJF, Thach C, Fogel R, Olsson P, Patalano F, Banerji D, Wedzicha JA. Blood eosinophils and response to maintenance chronic obstructive pulmonary disease treatment. Data from the FLAME trial. Am J Respir Crit Care Med. 2017;195:1189–97.

Vestbo J, Vogelmeier CF, Small M, Siddall J, Fogel R, Kostikas K. Inhaled corticosteroid use by exacerbations and eosinophils: a real-world COPD population. Int J Chron Obstruct Pulmon Dis. 2019;14:853–61.

Watz H, Tetzlaff K, Wouters EFM, Kirsten A, Magnussen H, Rodriguez-Roisin R, Vogelmeier C, Fabbri LM, Chanez P, Dahl R, et al. Blood eosinophil count and exacerbations in severe chronic obstructive pulmonary disease after withdrawal of inhaled corticosteroids: a post-hoc analysis of the WISDOM trial. Lancet Respir Med. 2016;4:390–8.

Pavord ID, Lettis S, Locantore N, Pascoe S, Jones PW, Wedzicha JA, Barnes NC. Blood eosinophils and inhaled corticosteroid/long-acting beta-2 agonist efficacy in COPD. Thorax. 2016;71:118–25.

Singh D. Predicting corticosteroid response in chronic obstructive pulmonary disease. Blood eosinophils gain momentum. Am J Respir Crit Care Med. 2017;196:1098–100.

Vestbo J, Prescott E, Almdal T, Dahl M, Nordestgaard BG, Andersen T, Sørensen TIA, Lange P. Body mass, fat-free body mass, and prognosis in patients with chronic obstructive pulmonary disease from a random population sample: findings from the Copenhagen City Heart Study. Am J Respir Crit Care Med. 2006;173:79–83.

Agustí AGN, Noguera A, Sauleda J, Sala E, Pons J, Busquets X. Systemic effects of chronic obstructive pulmonary disease. Eur Respir J. 2003;21:347–60.

Agustí AGN, Sauleda J, Miralles C, Gomez C, Togores B, Sala E, Batle S, Busquets X. Skeletal muscle apoptosis and weight loss in chronic obstructive pulmonary disease. Am J Respir Crit Care Med. 2002;166:485–9.

Labaki WW, Martinez FJ. Time to understand the infrequency of the frequent exacerbator phenotype in COPD. Chest. 2018;153:1087–8.

Hartley BF, Barnes NC, Lettis S, Compton CH, Papi A, Jones P. Risk factors for exacerbations and pneumonia in patients with chronic obstructive pulmonary disease: a pooled analysis. Respir Res. 2020;21:5.

Kim Y, Kim YJ, Kang YM, Cho WK. Exploring the impact of number and type of comorbidities on the risk of severe COPD exacerbations in Korean Population: a Nationwide Cohort Study. BMC Pulm Med. 2021;21:151.

Mackay AJ, Kostikas K, Roche N, Frent SM, Olsson P, Pfister P, Gupta P, Patalano F, Banerji D, Wedzicha JA. Impact of baseline symptoms and health status on COPD exacerbations in the FLAME study. Respir Res. 2020;21:93.

Smulders L, van der Aalst A, Neuhaus EDET, Polman S, Franssen FME, van Vliet M, de Kruif MD. Decreased risk of COPD exacerbations in obese patients. COPD. 2020;17:485–91.

Battisti WP, Wager E, Baltzer L, Bridges D, Cairns A, Carswell CI, Citrome L, Gurr JA, Mooney LA, Moore BJ, et al. Good publication practice for communicating company-sponsored medical research: GPP3. Ann Intern Med. 2015;163:461–4.

Putcha N, Barr RG, Han M, Woodruff PG, Bleecker ER, Kanner RE, Martinez FJ, Tashkin DP, Rennard SI, Breysse P, et al. Understanding the impact of passive smoke exposure on outcomes in COPD [abstract]. Am J Respir Crit Care Med. 2015;191:411–20.

Wu Z, Yang D, Ge Z, Yan M, Wu N, Liu Y. Body mass index of patients with chronic obstructive pulmonary disease is associated with pulmonary function and exacerbations: a retrospective real world research. J Thorac Dis. 2018;10:5086–99.

Download references

Acknowledgements

Medical writing support, under the direction of the authors, was provided by Julia King, PhD, and Sarah Piggott, MChem, CMC Connect, McCann Health Medical Communications, funded by AstraZeneca in accordance with Good Publication Practice (GPP3) guidelines [ 109 ].

This study was supported by AstraZeneca.

Author information

Authors and affiliations.

UCL Respiratory, University College London, London, WC1E 6BT, UK

John R. Hurst

Division of Pulmonary and Critical Care, University of Michigan, Ann Arbor, MI, USA

MeiLan K. Han

Formerly of Parexel International, Mohali, India

Barinder Singh, Gagandeep Kaur & Mohd Kashif Siddiqui

Parexel International, Mohali, India

Sakshi Sharma

Formerly of AstraZeneca, Cambridge, UK

Enrico de Nigris

AstraZeneca, Gothenburg, Sweden

Ulf Holmgren

You can also search for this author in PubMed   Google Scholar

Contributions

The authors have made the following declaration about their contributions. JRH and MKH made substantial contributions to the interpretation of data; BS, SS, GK, and MKS made substantial contributions to the acquisition, analysis, and interpretation of data; EdN and UH made substantial contributions to the conception and design of the work and the interpretation of data. All authors contributed to drafting or critically revising the article, have approved the submitted version, and agree to be personally accountable for their own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. All authors read and approved the final manuscript.

Corresponding author

Correspondence to John R. Hurst .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

JRH reports consulting fees from AstraZeneca; speaker fees from AstraZeneca, Chiesi, Pfizer, and Takeda; and travel support from GlaxoSmithKline and AstraZeneca. MKH reports assistance with conduction of this research and publication from AstraZeneca; personal fees from Aerogen, Altesa Biopharma, AstraZeneca, Boehringer Ingelheim, Chiesi, Cipla, DevPro, GlaxoSmithKline, Integrity, Medscape, Merck, Mylan, NACE, Novartis, Polarean, Pulmonx, Regeneron, Sanofi, Teva, Verona, United Therapeutics, and UpToDate; either in kind research support or funds paid to the institution from the American Lung Association, AstraZeneca, Biodesix, Boehringer Ingelheim, the COPD Foundation, Gala Therapeutics, the NIH, Novartis, Nuvaira, Sanofi, and Sunovion; participation in Data Safety Monitoring Boards for Novartis and Medtronic with funds paid to the institution; and stock options from Altesa Biopharma and Meissa Vaccines. BS, GK, and MKS are former employees of Parexel International. SS is an employee of Parexel International, which was funded by AstraZeneca to conduct this analysis. EdN is a former employee of AstraZeneca and previously held stock and/or stock options in the company. UH is an employee of AstraZeneca and holds stock and/or stock options in the company.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file1: table s1..

Search strategies. Table S2. List of included studies with linked publications. Table S3. Study characteristics across the 76 included studies. Table S4. Clinical characteristics of the patients assessed across the included studies.

Additional file 2: Fig. S1.

Sex (male vs female) as a risk factor for moderate-to-severe exacerbations. Fig. S2. Sex (male vs female) as a risk factor for severe exacerbations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Hurst, J.R., Han, M.K., Singh, B. et al. Prognostic risk factors for moderate-to-severe exacerbations in patients with chronic obstructive pulmonary disease: a systematic literature review. Respir Res 23 , 213 (2022). https://doi.org/10.1186/s12931-022-02123-5

Download citation

Received : 02 March 2022

Accepted : 20 July 2022

Published : 23 August 2022

DOI : https://doi.org/10.1186/s12931-022-02123-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Exacerbations
• Comorbidities
• Hospitalization

Respiratory Research

ISSN: 1465-993X

• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Family Med Prim Care
• v.2(1); Jan-Mar 2013

Systematic Reviews and Meta-analysis: Understanding the Best Evidence in Primary Healthcare

S. gopalakrishnan.

Department of Community Medicine, SRM Medical College, Hospital and Research Centre, Kattankulathur, Tamil Nadu, India

P. Ganeshkumar

Healthcare decisions for individual patients and for public health policies should be informed by the best available research evidence. The practice of evidence-based medicine is the integration of individual clinical expertise with the best available external clinical evidence from systematic research and patient's values and expectations. Primary care physicians need evidence for both clinical practice and for public health decision making. The evidence comes from good reviews which is a state-of-the-art synthesis of current evidence on a given research question. Given the explosion of medical literature, and the fact that time is always scarce, review articles play a vital role in decision making in evidence-based medical practice. Given that most clinicians and public health professionals do not have the time to track down all the original articles, critically read them, and obtain the evidence they need for their questions, systematic reviews and clinical practice guidelines may be their best source of evidence. Systematic reviews aim to identify, evaluate, and summarize the findings of all relevant individual studies over a health-related issue, thereby making the available evidence more accessible to decision makers. The objective of this article is to introduce the primary care physicians about the concept of systematic reviews and meta-analysis, outlining why they are important, describing their methods and terminologies used, and thereby helping them with the skills to recognize and understand a reliable review which will be helpful for their day-to-day clinical practice and research activities.

Introduction

Evidence-based healthcare is the integration of best research evidence with clinical expertise and patient values. Green denotes, “Using evidence from reliable research, to inform healthcare decisions, has the potential to ensure best practice and reduce variations in healthcare delivery.” However, incorporating research into practice is time consuming, and so we need methods of facilitating easy access to evidence for busy clinicians.[ 1 ] Ganeshkumar et al . mentioned that nearly half of the private practitioners in India were consulting more than 4 h per day in a locality,[ 2 ] which explains the difficulty of them in spending time in searching evidence during consultation. Ideally, clinical decision making ought to be based on the latest evidence available. However, to keep abreast with the continuously increasing number of publications in health research, a primary healthcare professional would need to read an insurmountable number of articles every day, covered in more than 13 million references and over 4800 biomedical and health journals in Medline alone. With the view to address this challenge, the systematic review method was developed. Systematic reviews aim to inform and facilitate this process through research synthesis of multiple studies, enabling increased and efficient access to evidence.[ 1 , 3 , 4 ]

Systematic reviews and meta-analyses have become increasingly important in healthcare settings. Clinicians read them to keep up-to-date with their field and they are often used as a starting point for developing clinical practice guidelines. Granting agencies may require a systematic review to ensure there is justification for further research and some healthcare journals are moving in this direction.[ 5 ]

This article is intended to provide an easy guide to understand the concept of systematic reviews and meta-analysis, which has been prepared with the aim of capacity building for general practitioners and other primary healthcare professionals in research methodology and day-to-day clinical practice.

The purpose of this article is to introduce readers to:

• The two approaches of evaluating all the available evidence on an issue i.e., systematic reviews and meta-analysis,
• Discuss the steps in doing a systematic review,
• Introduce the terms used in systematic reviews and meta-analysis,
• Interpret results of a meta-analysis, and
• The advantages and disadvantages of systematic review and meta-analysis.

Application

What is the effect of antiviral treatment in dengue fever? Most often a primary care physician needs to know convincing answers to questions like this in a primary care setting.

To find out the solutions or answers to a clinical question like this, one has to refer textbooks, ask a colleague, or search electronic database for reports of clinical trials. Doctors need reliable information on such problems and on the effectiveness of large number of therapeutic interventions, but the information sources are too many, i.e., nearly 20,000 journals publishing 2 million articles per year with unclear or confusing results. Because no study, regardless of its type, should be interpreted in isolation, a systematic review is generally the best form of evidence.[ 6 ] So, the preferred method is a good summary of research reports, i.e., systematic reviews and meta-analysis, which will give evidence-based answers to clinical situations.

There are two fundamental categories of research: Primary research and secondary research. Primary research is collecting data directly from patients or population, while secondary research is the analysis of data already collected through primary research. A review is an article that summarizes a number of primary studies and may draw conclusions on the topic of interest which can be traditional (unsystematic) or systematic.

Terminologies

Systematic review.

A systematic review is a summary of the medical literature that uses explicit and reproducible methods to systematically search, critically appraise, and synthesize on a specific issue. It synthesizes the results of multiple primary studies related to each other by using strategies that reduce biases and random errors.[ 7 ] To this end, systematic reviews may or may not include a statistical synthesis called meta-analysis, depending on whether the studies are similar enough so that combining their results is meaningful.[ 8 ] Systematic reviews are often called overviews.

The evidence-based practitioner, David Sackett, defines the following terminologies.[ 3 ]

• Review: The general term for all attempts to synthesize the results and conclusions of two or more publications on a given topic.
• Overview: When a review strives to comprehensively identify and track down all the literature on a given topic (also called “systematic literature review”).
• Meta-analysis: A specific statistical strategy for assembling the results of several studies into a single estimate.

Systematic reviews adhere to a strict scientific design based on explicit, pre-specified, and reproducible methods. Because of this, when carried out well, they provide reliable estimates about the effects of interventions so that conclusions are defensible. Systematic reviews can also demonstrate where knowledge is lacking. This can then be used to guide future research. Systematic reviews are usually carried out in the areas of clinical tests (diagnostic, screening, and prognostic), public health interventions, adverse (harm) effects, economic (cost) evaluations, and how and why interventions work.[ 9 ]

Cochrane reviews

Cochrane reviews are systematic reviews undertaken by members of the Cochrane Collaboration which is an international not-for-profit organization that aims to help people to make well-informed decisions about healthcare by preparing, maintaining, and promoting the accessibility of systematic reviews of the effects of healthcare interventions.

Cochrane Primary Health Care Field is a systematic review of primary healthcare research on prevention, treatment, rehabilitation, and diagnostic test accuracy. The overall aim and mission of the Primary Health Care Field is to promote the quality, quantity, dissemination, accessibility, applicability, and impact of Cochrane systematic reviews relevant to people who work in primary care and to ensure proper representation in the interests of primary care clinicians and consumers in Cochrane reviews and review groups, and in other entities. This field would serve to coordinate and promote the mission of the Cochrane Collaboration within the primary healthcare disciplines, as well as ensuring that primary care perspectives are adequately represented within the Collaboration.[ 10 ]

Meta-analysis

A meta-analysis is the combination of data from several independent primary studies that address the same question to produce a single estimate like the effect of treatment or risk factor. It is the statistical analysis of a large collection of analysis and results from individual studies for the purpose of integrating the findings.[ 11 ] The term meta-analysis has been used to denote the full range of quantitative methods for research reviews.[ 12 ] Meta-analyses are studies of studies.[ 13 ] Meta-analysis provides a logical framework to a research review where similar measures from comparable studies are listed systematically and the available effect measures are combined wherever possible.[ 14 ]

The fundamental rationale of meta-analysis is that it reduces the quantity of data by summarizing data from multiple resources and helps to plan research as well as to frame guidelines. It also helps to make efficient use of existing data, ensuring generalizability, helping to check consistency of relationships, explaining data inconsistency, and quantifies the data. It helps to improve the precision in estimating the risk by using explicit methods.

Therefore, “systematic review” will refer to the entire process of collecting, reviewing, and presenting all available evidence, while the term “meta-analysis” will refer to the statistical technique involved in extracting and combining data to produce a summary result.[ 15 ]

Steps in doing systematic reviews/meta-analysis

Following are the six fundamental essential steps while doing systematic review and meta-analysis.[ 16 ]

Define the question

This is the most important part of systematic reviews/meta-analysis. The research question for the systematic reviews may be related to a major public health problem or a controversial clinical situation which requires acceptable intervention as a possible solution to the present healthcare need of the community. This step is most important since the remaining steps will be based on this.

Reviewing the literature

This can be done by going through scientific resources such as electronic database, controlled clinical trials registers, other biomedical databases, non-English literatures, “gray literatures” (thesis, internal reports, non–peer-reviewed journals, pharmaceutical industry files), references listed in primary sources, raw data from published trials and other unpublished sources known to experts in the field. Among the available electronic scientific database, the popular ones are PUBMED, MEDLINE, and EMBASE.

Sift the studies to select relevant ones

To select the relevant studies from the searches, we need to sift through the studies thus identified. The first sift is pre-screening, i.e., to decide which studies to retrieve in full, and the second sift is selection which is to look again at these studies and decide which are to be included in the review. The next step is selecting the eligible studies based on similar study designs, year of publication, language, choice among multiple articles, sample size or follow-up issues, similarity of exposure, and or treatment and completeness of information.

It is necessary to ensure that the sifting includes all relevant studies like the unpublished studies (desk drawer problem), studies which came with negative conclusions or were published in non-English journals, and studies with small sample size.

Assess the quality of studies

The steps undertaken in evaluating the study quality are early definition of study quality and criteria, setting up a good scoring system, developing a standard form for assessment, calculating quality for each study, and finally using this for sensitivity analysis.

For example, the quality of a randomized controlled trial can be assessed by finding out the answers to the following questions:

• Was the assignment to the treatment groups really random?
• Was the treatment allocation concealed?
• Were the groups similar at baseline in terms of prognostic factors?
• Were the eligibility criteria specified?
• Were the assessors, the care provider, and the patient blinded?
• Were the point estimates and measure of variability presented for the primary outcome measure?
• Did the analyses include intention-to-treat analysis?

Calculate the outcome measures of each study and combine them

We need a standard measure of outcome which can be applied to each study on the basis of its effect size. Based on their type of outcome, following are the measures of outcome: Studies with binary outcomes (cured/not cured) have odds ratio, risk ratio; studies with continuous outcomes (blood pressure) have means, difference in means, standardized difference in means (effect sizes); and survival or time-to-event data have hazard ratios.

Combining studies

Homogeneity of different studies can be estimated at a glance from a forest plot (explained below). For example, if the lower confidence interval of every trial is below the upper of all the others, i.e., the lines all overlap to some extent, then the trials are homogeneous. If some lines do not overlap at all, these trials may be said to be heterogeneous.

The definitive test for assessing the heterogeneity of studies is a variant of Chi-square test (Mantel–Haenszel test). The final step is calculating the common estimate and its confidence interval with the original data or with the summary statistics from all the studies. The best estimate of treatment effect can be derived from the weighted summary statistics of all studies which will be based on weighting to sample size, standard errors, and other summary statistics. Log scale is used to combine the data to estimate the weighting.

Interpret results: Graph

The results of a meta-analysis are usually presented as a graph called forest plot because the typical forest plots appear as forest of lines. It provides a simple visual presentation of individual studies that went into the meta-analysis at a glance. It shows the variation between the studies and an estimate of the overall result of all the studies together.

Forest plot

Meta-analysis graphs can principally be divided into six columns [ Figure 1 ]. Individual study results are displayed in rows. The first column (“study”) lists the individual study IDs included in the meta-analysis; usually the first author and year are displayed. The second column relates to the intervention groups and the third column to the control groups. The fourth column visually displays the study results. The line in the middle is called “the line of no effect.” The weight (in %) in the fifth column indicates the weighting or influence of the study on the overall results of the meta-analysis of all included studies. The higher the percentage weight, the bigger the box, the more influence the study has on the overall results. The sixth column gives the numerical results for each study (e.g., odds ratio or relative risk and 95% confidence interval), which are identical to the graphical display in the fourth column. The diamond in the last row of the graph illustrates the overall result of the meta-analysis.[ 4 ]

Interpretation of meta-analysis[ 4 ]

Thus, the horizontal lines represent individual studies. Length of line is the confidence interval (usually 95%), squares on the line represent effect size (risk ratio) for the study, with area of the square being the study size (proportional to weight given) and position as point estimate (relative risk) of the study.[ 7 ]

For example, the forest plot of the effectiveness of dexamethasone compared with placebo in preventing the recurrence of acute severe migraine headache in adults is shown in Figure 2 .[ 17 ]

Forest plot of the effectiveness of dexamethasone compared with placebo in preventing the recurrence of acute severe migraine headache in adults[ 17 ]

The overall effect is shown as diamond where the position toward the center represents pooled point estimate, the width represents estimated 95% confidence interval for all studies, and the black plain line vertically in the middle of plot is the “line of no effect” (e.g., relative risk = 1).

Therefore, when examining the results of a systematic reviews/meta-analysis, the following questions should be kept in mind:

• Heterogeneity among studies may make any pooled estimate meaningless.
• The quality of a meta-analysis cannot be any better than the quality of the studies it is summarizing.
• An incomplete search of the literature can bias the findings of a meta-analysis.
• Make sure that the meta-analysis quantifies the size of the effect in units that you can understand.

Subgroup analysis and sensitivity analysis

Subgroup analysis looks at the results of different subgroups of trials, e.g., by considering trials on adults and children separately. This should be planned at the protocol stage itself which is based on good scientific reasoning and is to be kept to a minimum.

Sensitivity analysis is used to determine how results of a systematic review/meta-analysis change by fiddling with data, for example, what is the implication if the exclusion criteria or excluded unpublished studies or weightings are assigned differently. Thus, after the analysis, if changing makes little or no difference to the overall results, the reviewer's conclusions are robust. If the key findings disappear, then the conclusions need to be expressed more cautiously.

Advantages of Systematic Reviews

Systematic reviews have specific advantages because of using explicit methods which limit bias, draw reliable and accurate conclusions, easily deliver required information to healthcare providers, researchers, and policymakers, help to reduce the time delay in the research discoveries to implementation, improve the generalizability and consistency of results, generation of new hypotheses about subgroups of the study population, and overall they increase precision of the results.[ 18 ]

Limitations in Systematic Reviews/Meta-analysis

As with all research, the value of a systematic review depends on what was done, what was found, and the clarity of reporting. As with other publications, the reporting quality of systematic reviews varies, limiting readers’ ability to assess the strengths and weaknesses of those reviews.[ 5 ]

Even though systematic review and meta-analysis are considered the best evidence for getting a definitive answer to a research question, there are certain inherent flaws associated with it, such as the location and selection of studies, heterogeneity, loss of information on important outcomes, inappropriate subgroup analyses, conflict with new experimental data, and duplication of publication.

Publication Bias

Publication bias results in it being easier to find studies with a “positive” result.[ 19 ] This occurs particularly due to inappropriate sifting of the studies where there is always a tendency towards the studies with positive (significant) outcomes. This effect occurs more commonly in systematic reviews/meta-analysis which need to be eliminated.

The quality of reporting of systematic reviews is still not optimal. In a recent review of 300 systematic reviews, few authors reported assessing possible publication bias even though there is overwhelming evidence both for its existence and its impact on the results of systematic reviews. Even when the possibility of publication bias is assessed, there is no guarantee that systematic reviewers have assessed or interpreted it appropriately.[ 20 ]

To overcome certain limitations mentioned above, the Cochrane reviews are currently reported in a format where at the end of every review, findings are summarized in the author's point of view and also give an overall picture of the outcome by means of plain language summary. This is found to be much helpful to understand the existing evidence about the topic more easily by the reader.

A systematic review is an overview of primary studies which contains an explicit statement of objectives, materials, and methods, and has been conducted according to explicit and reproducible methodology. A meta-analysis is a mathematical synthesis of the results of two or more primary studies that addressed the same hypothesis in the same way. Although meta-analysis can increase the precision of a result, it is important to ensure that the methods used for the reviews were valid and reliable.

High-quality systematic reviews and meta-analyses take great care to find all relevant studies, critically assess each study, synthesize the findings from individual studies in an unbiased manner, and present balanced important summary of findings with due consideration of any flaws in the evidence. Systematic review and meta-analysis is a way of summarizing research evidence, which is generally the best form of evidence, and hence positioned at the top of the hierarchy of evidence.

Systematic reviews can be very useful decision-making tools for primary care/family physicians. They objectively summarize large amounts of information, identifying gaps in medical research, and identifying beneficial or harmful interventions which will be useful for clinicians, researchers, and even for public and policymakers.

Source of Support: Nil

Conflict of Interest: None declared.

This paper is in the following e-collection/theme issue:

Published on 4.4.2024 in Vol 11 (2024)

Studies of Social Anxiety Using Ambulatory Assessment: Systematic Review

Authors of this article:

• Javier Fernández-Álvarez 1, 2 , PhD   ; 
• Desirée Colombo 1 , PhD   ; 
• Juan Martín Gómez Penedo 3 , PhD   ; 
• Maitena Pierantonelli 4 , MSc   ; 
• Rosa María Baños 4, 5, 6 , Prof Dr   ; 
• Cristina Botella 1, 6 , Prof Dr  

1 Department of Basic and Clinical Psychology and Psychobiology, Jaume I University, Castellon de la Plana, Spain

2 Fundación Aiglé, Buenos Aires, Argentina

3 Facultad de Psicología, Universidad de Buenos Aires (CONICET), Buenos Aires, Argentina

4 Polibienestar Research Institute, University of Valencia, Valencia, Spain

5 Department of Personality, Evaluation, and Psychological Treatments, University of Valencia, Valencia, Spain

6 Ciber Fisiopatologia Obesidad y Nutricion (CB06/03 Instituto Salud Carlos III), Madrid, Spain

Corresponding Author:

Javier Fernández-Álvarez, PhD

Department of Basic and Clinical Psychology and Psychobiology

Jaume I University

Avda. Vicent Sos Baynat s/n

Castellon de la Plana, 12071

Phone: 34 964 72 80 0

Email: [email protected]

Background: There has been an increased interest in understanding social anxiety (SA) and SA disorder (SAD) antecedents and consequences as they occur in real time, resulting in a proliferation of studies using ambulatory assessment (AA). Despite the exponential growth of research in this area, these studies have not been synthesized yet.

Objective: This review aimed to identify and describe the latest advances in the understanding of SA and SAD through the use of AA.

Methods: Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, a systematic literature search was conducted in Scopus, PubMed, and Web of Science.

Results: A total of 70 articles met the inclusion criteria. The qualitative synthesis of these studies showed that AA permitted the exploration of the emotional, cognitive, and behavioral dynamics associated with the experience of SA and SAD. In line with the available models of SA and SAD, emotion regulation, perseverative cognition, cognitive factors, substance use, and interactional patterns were the principal topics of the included studies. In addition, the incorporation of AA to study psychological interventions, multimodal assessment using sensors and biosensors, and transcultural differences were some of the identified emerging topics.

Conclusions: AA constitutes a very powerful methodology to grasp SA from a complementary perspective to laboratory experiments and usual self-report measures, shedding light on the cognitive, emotional, and behavioral antecedents and consequences of SA and the development and maintenance of SAD as a mental disorder.

Introduction

Social anxiety (SA) is a normal and adaptive manifestation that all human beings experience in anticipation of a potential interactional threat. Similar to any adaptive response, it is enormously beneficial, particularly in protecting people from potential dangers that may arise from social interactions [ 1 ]. However, this adaptive response is sometimes exacerbated, and instead of preparing the individual for optimal performance, it becomes paralyzing, triggering intense fear, catastrophic thoughts, and avoidance behaviors, among other characteristic manifestations. When this pathological response becomes habitual in anticipation and confrontation of social interactions, it is referred to as SA disorder (SAD).

Accordingly, SAD is understood as a prevalent clinical condition characterized by intense fear and avoidance of social situations. SAD is a heterogeneous clinical condition that usually entails high levels of dysfunction in the lives of people who experience it. This heterogeneous nature of SAD is marked by the dynamic deployment of cognitions, emotions, and behaviors in the interaction with others and in different contexts [ 2 ].

Ambulatory Assessment

Ambulatory assessment (AA) serves as an umbrella term that includes specific techniques such as experience sampling methods, ecological momentary assessment, or daily retrospective methods. These techniques constitute a research methodology of paramount significance in the field of clinical psychology and psychotherapy, enabling researchers and clinicians to gather in-depth, ecologically valid data from individuals. Unlike traditional assessment methods that rely on retrospective self-reporting, AA involves the repeated real-time measurement and gauging of various aspects of an individual’s experiences, behaviors, and physiological responses. The fact that it entails multiple assessments over a certain period makes AA an optimal tool to explore within-person fluctuations and trajectories of these experiences and behaviors.

Moreover, AA circumvents the biases that usual retrospective reports may have because it is usually implemented in momentary assessments or recent retrospective reports (eg, daily diaries [ 3 ]). Owing to the variability of psychological processes such as affective and emotional dynamics [ 4 ], AA can more precisely determine the fluctuation of symptoms. In addition, owing to the possibility of incorporating sensors and biosensors, AA can provide multimodal assessment, overcoming the biases of self-reports [ 5 ].

Over the last few years, there has been an increasing interest in exploring the use of AA in clinical psychology and psychotherapy. Owing to the incorporation of digital technologies, namely, mobile phones, AA has become a very powerful add-on for clinical researchers [ 6 ], first and foremost due to the possibility that AA provides of capturing data in real time, thus grasping contextual aspects from naturalistic settings that laboratory-based assessment does not allow for [ 7 , 8 ]. SA symptoms are particularly contextually bound, and thus, it is of utmost relevance to consider the sensitivity of the context [ 2 ].

All these characteristics have a central and common pursuit to personalize models of psychopathology [ 9 ]. There is no doubt that every person has a certain and unique composition of traits that may lead to functional and dysfunctional states in continuous interaction with the context. Therefore, the revolution of AA is helping foster the creation of tailored models with intensive longitudinal data, which can shed light on the factors that may lead people to experience and behave in adaptive or maladaptive ways.

In this sense, AA proves to be a suitable tool for capturing the co-occurrence of symptoms and psychological processes, offering crucial insights into the antecedents and consequences of SA with enhanced accuracy. This, in turn, facilitates a comprehensive understanding of the factors contributing to the appearance and maintenance of SA and SAD. Given the dimensional nature of SA, AA can shed light on the differences among clinical, subclinical, and healthy participants.

The understanding of SAD in ecological settings may also lead to improving psychological treatments. The insights gained through AA can contribute to refining therapeutic approaches by investigating mechanisms of change [ 10 ]. However, to the best of our knowledge, no systematic review has synthesized the available evidence analyzing the strengths and limitations of the current state of the art.

For all the aforementioned reasons, AA provides a range of advantages that justify its exponential growth as a research methodology in clinical psychology. Although a plethora of research has implemented AA in individuals with SA, these data have not been synthesized yet. Hence, the main aim of this review was to identify studies that used AA to explore SA.

This review followed the recommendations of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement [ 11 ]. The full protocol was registered before the data analysis [ 12 ].

Literature Search

Scopus, PubMed, and Web of Science literature searches were conducted. The search strategy for PubMed is available in Multimedia Appendix 1 and was adapted to the syntax requirements of each database. No filters were included in the searches. The reference lists of eligible articles were manually reviewed to identify additional relevant publications.

Inclusion Criteria

To be included, the studies had to fulfill the following criteria: (1) studies that focused on participants with SAD or SA symptoms; (2) studies that used AA to explore the dynamics of SA, correlates of SA, or the feasibility of AA for SA assessment; (3) articles that included any sort of discussion regarding SA in AA (this criterion was included given that some articles fulfilled all the previous criteria but featured no specific discussion on SA); (4) articles published in peer-reviewed journals in English; and (5) studies that presented at least one active AA or passive data collected via sensor signals (eg, geo-mapping, accelerometer, or GPS) from a phone or smartwatch multiple times per day. Articles were excluded if they (1) were not empirical studies; (2) used a child or adolescent sample; and (3) did not use a daily diary, experience sampling, or AA approach that included assessment and discussion of SA.

Study Selection and Data Extraction

A database search was conducted until August 31, 2023. The literature search produced a total of 14,504 articles, 8926 (61.54%) of which were retained after removing duplicates. A total of 1.23% (110/8926) of these articles were retrieved after title and abstract screening. The subsequent application of the selection criteria resulted in the inclusion of 50% (55/110) of the articles, adding another 15 studies that were identified through citation searching. In total, we included 70 studies ( Figure 1 ). Studies were independently selected by 2 researchers, and disagreements were resolved through consensus.

Using premade tables, the following information was extracted: author and year of publication, study design, follow-up period and sample size and characteristics, aims of the study, measures collected through AA, app name in the case of smartphone-based AA, participation rate (ie, response to recruitment), attrition rate, compliance with AA questions, incentives, outcomes explored, AA measures, frequency of AA, and main findings.

Finally, to summarize the main information, 2 of the authors proposed categories. To form a category, it was required that at least 2 of the included studies addressed the same or a similar topic. To reach an agreement, the resulting categories underwent thorough discussion between the first and fourth authors. This criterion was applied to the “Principal themes explored” section. For the “Emerging topics” section, the criterion for category inclusion was based on relevance regarding addressing a key aspect of the literature that had not been explored before.

Sample Characteristics

Of the 70 studies selected for this systematic review, 29 (41%) used the same sample ( Textbox 1 ). Given that a large proportion of the studies were conducted in university settings, most of the included participants were undergraduate students and, therefore, young adults. Moreover, all the studies (70/70, 100%) were conducted in high-income countries, principally in the United States. A total of 43% (30/70) of the studies included both healthy and clinical populations. In total, 21% (15/70) of the studies were conducted only with clinical participants, whereas 36% (25/70) were conducted with healthy individuals exploring the dynamics of SA. In addition, 33% (23/70) of the studies included comorbid samples, mainly those with other anxiety or depressive disorders. Regarding gender, most studies (57/70, 81%) included more women than men. This reflects the prevalence rates of SAD, which have been shown to be higher in women than in men [ 13 ]. Table S1 in Multimedia Appendix 2 [ 14 - 81 ] and Table S2 in Multimedia Appendix 3 [ 14 - 81 ] summarize the characteristics and main findings of the included studies.

Studies with the same samples

• Blalock et al [ 14 , 15 ], Farmer and Kashdan [ 16 ], Goodman et al [ 17 , 18 ], Kashdan and Farmer [ 19 ], and Kashdan et al [ 20 , 21 ]
• Daniel et al [ 22 , 82 , 83 ] and Beltzer et al [ 23 ]
• Daniel et al [ 24 ], Daros et al [ 25 ], and Ladis et al [ 26 ]
• Doorley et al [ 27 , 28 ]
• Goodman et al [ 29 - 32 ]
• Oren-Yagoda et al [ 33 - 36 ]
• O’Toole et al [ 37 , 38 ]
• Villanueva et al [ 39 ] and Rinner et al [ 40 ]

Methodological Characteristics of the Studies

Sampling frequency.

There were 2 types of sampling identified among the included studies. On the one hand, there were some studies that implemented a daily diary, which typically entails an end-of-day report. On the other hand, other AA studies comprised different numbers of daily assessments and diverse types of prompt contingencies. The duration of the AA ranged from 4 to 35 days. The most commonly implemented study duration was 14 days.

Although a large proportion of the studies (63/70, 90%) used a fixed or random scheduled prompt structure, a range of studies incorporated an event-contingent design [ 40 - 45 ]. Indeed, SA determinants, correlates, and consequences are particularly triggered in interpersonal situations, and event-contingent designs may be suitable for detecting relevant moments. Finally, studies such as those by Daniel et al [ 24 ], Helbig-Lang et [ 46 ], and Kashdan et al [ 20 ] included a mix of different types of data collection, combining random prompt, event-contingent, and end-of-day records.

Compliance rates typically indicate the percentage of prompts or days that the participants completed on average. Compliance rates ranged from 40% to 95%, with average compliance across studies of 73.72% of prompts (SD 15.93%; median 80%).

Statistical Analysis

The vast majority of the studies (61/70, 87%) used hierarchical linear models to account for the nested nature of the collected data. A total of 4% (3/70) of the studies used a combination of hierarchical linear models and structural equation modeling [ 43 , 45 , 79 ], 1% (1/70) of the studies used machine learning techniques [ 47 ], 3% (2/70) of the studies calculated ANOVA models [ 23 , 42 ], and 1% (1/70) of the studies ran ordinary least squares regression [ 48 ].

Methodological Design Characteristics

Table 1 summarizes the principal methodological characteristics of each study. Power analysis to calculate the needed sample size (which was rarely conducted; 15/70, 21% of the studies) and the psychometric properties of the AA questions (37/70, 53% of the studies) were the 2 criteria that were reported the least, whereas the percentage of AA compliance (56/70, 80% of the studies) and attrition rates of AA (46/70, 66% of the studies) were the criteria that were reported the most among the studies in this review.

a AA: ambulatory assessment.

b Presence.

d N/A: not applicable.

Principal Themes Explored

Affective and emotional dynamics.

Naragon-Gainey [ 43 ] explored structural models of affect and internalizing symptoms. While the between-person variance in negative affect (NA) and concurrent levels of NA predicted SA, positive affect (PA) did not. This lack of significance contradicted many other AA studies that yielded significant between-person associations between lower levels of PA and higher levels of NA with SA [ 16 , 20 , 30 , 54 , 57 , 58 , 61 , 69 , 70 ].

Individuals with SAD showed not only higher overall levels of NA but also more emotional instability, which was not the case for PA [ 16 ]. These authors even suggested that the interaction between NA and instability could explain the appearance of SAD.

Discrete Emotions

As suggested by Rozen and Aderka [ 84 ], there is a wide range of discrete emotions that have not been integrated into classic models of SAD. In that sense, AA allows for a nuanced study of single emotions and how they are interconnected not only with each other but also with other cognitive-affective processes and behaviors.

First, Kashdan and Collins [ 70 ] revealed that SA was related to less time spent feeling happy and relaxed and more time spent feeling angry. The results also showed that happy moments were aroused in companion to others. The fact that SA was associated with fewer and less intense positive emotions and more anger episodes was independent of being with others or alone.

Oren-Yagoda et al [ 35 ] investigated the role of envy, showing that visual modes of communication are related to elevated envy compared to voice or text communication. In addition, envy predicted subsequent anxiety above and beyond previous anxiety as well as other negative emotions.

Loneliness is a defining emotion of people with SAD, and Oren-Yagoda et al [ 36 ] found that this relationship was also confirmed in an ecological setting using AA. A significant association was predicted in certain social situations (ie, negativity, positivity, and meaningfulness). Moreover, the relationship between loneliness and anxiety was shown to be reciprocal in individuals with SAD (loneliness predicted anxiety and anxiety predicted loneliness). This deleterious reciprocity was not found in healthy controls.

Finally, Oren-Yagoda et al [ 34 ] investigated the fluctuations of pride in individuals diagnosed with SAD. The results indicated that levels of pride were lower in patients with SAD than in nondiagnosed controls, although when pride was experienced, it predicted a reduction in anxiety levels.

Emotion Regulation

Emotion regulation (ER) was by far the most explored construct among the studies that implemented AA in individuals with SA. This may be explained by the fact that ER is a dynamic process, and AA is very suitable for detecting affect changes and fluctuations in daily life. In addition, not only are ER and SA symptoms highly context sensitive, but mounting research has also shown their interdependence [ 85 ], which justifies the importance of exploring how individuals with SAD use ER in daily life.

Discrete Strategies

Several studies explored only 1 strategy. For example, Kashdan et al [ 20 ] found that individuals with SAD experienced greater experiential avoidance than healthy individuals. Using the same sample, Kashdan et al [ 21 ] found that momentary experiential avoidance presented a stronger association with anxiety during social interactions for individuals with SAD than for individuals without SAD.

Similarly, another study explored the role of emotional suppression, demonstrating that, on days in which SA symptoms increased, the use of suppression tended also to increase [ 61 , 69 ]. The same was found by Beltzer et al [ 23 ], who identified that days with high levels of SA and expressive suppression led to fewer positive emotions.

Farmer and Kashdan [ 61 ] implemented a 2-week diary, showing that high SA was related to positive emotion suppression, fewer positive social events, and fewer positive emotions on subsequent days. In contrast, low SA was associated with fewer negative social events on the days after cognitive reappraisal was used to reduce distress. However, the use of cognitive reappraisal did not lead to any changes in people with high SA.

Meanwhile, Farmer and Kashdan [ 16 ] found that individuals with SAD were 3 times more likely to present acute shifts in NA, which may be a particular experience of this anxiety disorder and not others. This acute shift in NA may lead to experiencing emotions as uncontrollable and threatening, which in turn could explain the higher use of suppression.

Goodman et al [ 63 ] showed that alcohol consumption moderated the negative association between SA and a range of healthy social interactions such as laughter or feelings of acceptance; that is, SA was not related to the perceived quality of interpersonal interactions when participants consumed alcohol, suggesting that alcohol consumption may be a reinforcer of SA. These findings suggest that ER plays a central role in the experience of individuals with SAD who try to explicitly control their emotions and are aware of the effort they make to do so. In a different but related study, Goodman and Kashdan [ 29 ] found that both anxiety and pain were interfering factors in goal attainment, as well as finding an inverse association between daily meaning in life and perceived emotion-related goal interference.

Substance Use

Alcohol consumption and SAD are highly comorbid given that, among individuals with SAD, it is frequent to resort to alcohol as a coping strategy or, in other words, as an ER strategy. By means of AA, it is possible to detect how these 2 phenomena are interrelated. Therefore, several studies explored this topic. Battista et al [ 52 ] examined the relationship between alcohol consumption and SA, revealing that, after each alcoholic drink consumed, SA tended to decrease 2 hours later. Contrary to what the authors expected, this association was not explained by the level of trait SA.

Goodman et al [ 63 ] showed that alcohol consumption moderated the negative association between SA and a range of healthy social interactions such as laughter or feelings of acceptance; that is, alcohol consumption SA was no longer related to the perceived quality of interpersonal interactions, suggesting that SA may be a reinforcer of SA. This reinforcement could be either negative (attenuation of anxiety) or positive (better perception of social situations), which was the main finding of Goodman et al [ 64 ]. The results of this study yielded evidence supporting the negative reinforcement hypothesis such that people with SAD presented higher coping motives (negative reinforcers) but equal levels of affiliation motives (positive reinforcers).

Walukevich-Dienst et al [ 80 ] revealed that consuming substances as a coping strategy in SA is related to heavier consumption, especially on drinking days, which may be a risk factor for the development of an alcohol use disorder. Moreover, when there were SA coping motives, the consequences were more negative compared to days without SA coping motives. Interestingly, levels of SA at baseline were not moderators of these associations, indicating that the coping motive is more important than the antecedent levels of SA.

Kim and Kwon [ 73 ] showed that individuals with SAD had a higher increase rate of alcohol craving when they were tense and lonely and experiencing SA in comparison to individuals without a diagnosis of SAD. These results were moderated by the rate of rumination in the SAD group and avoidance in the non-SAD group. O’Grady et al [ 75 ] explored the role of social-contextual events in the relationship between trait SA and drinking. The results revealed a positive association between trait SA and drinking on the evenings of days in which the individuals experienced an embarrassing situation. This was significantly higher than in healthy participants, suggesting a behavioral maladaptive ER strategy to cope with the intensified levels of SA.

In addition, Buckner et al [ 56 ] studied the association between SA, cannabis use, cannabis craving, and situational variables. This study showed that SA interacted with cannabis craving predicting cannabis use, which sets out a complex relationship between SA and cannabis use and not a simple association (ie, cannabis use as a response to increased state SA).

In the study by Buckner et al [ 57 ], they showed that baseline SA was associated with increases in NA throughout the days in which the participants were monitored but was also significantly associated with postquit withdrawal. Participants with higher levels of SA presented more severe postquit withdrawal symptoms as well as an increase in NA during a cessation attempt. For this reason, the authors suggested that those participants may particularly benefit from intervention and treatment strategies.

Finally, Papp et al [ 76 ] found that students with higher levels of SA presented a stronger association between NA and prescription misuse. This study included externalizing and internalizing symptoms (depression and SA symptoms), and moderation was shown to be significant only for internalizing symptoms. This may indicate differential patterns of substance use as an ER strategy according to different personality traits.

The Complexity of ER: Polyregulation and Flexibility

Some studies incorporated a range of ER strategies in the AA process, which is in line with the current idea of polyregulation , that is, that individuals usually implement a range of strategies simultaneously [ 86 ]. For example, Daros et al [ 25 ] measured various strategies, and after clustering through a factorial analysis, 2 categories of ER strategies were considered for the analyses: avoidant and engagement strategies. However, the authors did not find any significant results for these 2 macrocategories.

On the basis of the model by Gross, Blalock et al [ 14 ] examined both suppression and cognitive reappraisal in SA in comparison to healthy individuals. Both groups presented worse emotional experiences when they suppressed positive versus negative emotions as well as when reappraising negative versus positive emotions. This suggests that suppression may be an adaptive strategy not to feel negative emotions, whereas reappraisal may be effective in increasing positive states. Interestingly, individuals with SAD showed more positive emotions after reappraising negative emotional states to feel fewer negative emotions than the healthy controls.

However, none of these previous examples explicitly framed their studies under the concept of emotion polyregulation, as did Ladis et al [ 26 ]. By including 8 strategies (problem-solving, introspection, distraction, acceptance, thought suppression, seeking advice, cognitive reappraisal, and expressive suppression), the authors systematically explored how often polyregulation occurs in daily life. Overall, the results yielded nonsignificant SA correlates of polyregulation, suggesting that it may be more dependent on within-person differences.

In addition to polyregulation, flexibility has been shown to be central in ER literature to distinguish adaptive from maladaptive regulatory processes [ 87 ]. Given that flexibility is mostly dependent on context, AA emerges as a very useful tool. O’Toole et al [ 38 ] investigated this specific topic in individuals with high and low SA and considered type and intensity as 2 contextual factors. This study revealed that SA moderated the relationship between emotion intensity and experiential avoidance. In individuals with high SA, there was a stronger association between experiential avoidance and specific emotions, such as guilt, nervousness, and sadness.

The study conducted by Beltzer et al [ 23 ] presents an illustrative example of how ER can be better explained by contextual triggers than by the adaptative or maladaptive continuum. The authors created an algorithm based on the 10 most used strategies to generate an ecological momentary intervention (EMI) based on contextual triggers. This algorithm was tested with a group of strategies (disengagement, engagement, and aversive cognitive perseveration) and with 10 individual single strategies compared with a random policy and a behavior policy, with ER effectiveness being the observed outcome. The contextual algorithm improved other policies in cases in which the top 10 strategies were considered separately. However, when the strategies were grouped into categories, the algorithm did not outperform the random recommender or the observed ER strategies.

Goodman et al [ 32 ] also explored ER flexibility considering 2 components of flexibility: the evaluation of contextual demands and matching regulatory strategies to contextual demands. The results indicated that people with SAD considered momentary assessments to be more anxiety provoking while presenting similar patterns to those of control participants when gauging contextual demands, particularly those related to perceived controllability. That is, some disengagement strategies (rumination, thought suppression, and expressive suppression) were found to be unrelated to perceived controllability. This means that these strategies were used independently of perceived controllability in a certain context. However, contrary to previous results, participants with SAD yielded similar patterns to those of control participants in response to anxiety intensity.

Higher anxiety ratings predicted greater use of all strategies regardless of the type of strategy used. In addition, this study showed that people with SAD may be more prone to use thought suppression but not engagement strategies, which is inconsistent with previous studies.

In other studies, a specific component of the heterogeneous and complex process of ER was explored. Although most clinical research on ER tends to simplify the discussion on putatively maladaptive (eg, expressive suppression) or adaptive (eg, cognitive reappraisal) strategies, there is a range of potential explanatory variables that set a more complex scenario than implementing certain strategies or not. For example, one study [ 24 ] investigated the perception of ER effectiveness, which is based on a robust research line that revolves around how goals shape and determine ER deployment and outcomes [ 88 ]. Daniel et al [ 24 ] found that, depending on the way in which effectiveness is measured (either judgment of effectiveness or change in affect), the results differ. While the judgment of effectiveness indicates that avoidance‐oriented ER attempts are less effective than engagement‐oriented ER attempts, changes in self-reported effects following ER attempts present the opposite results.

Similarly, Goodman et al [ 18 ] explored the extent to which beliefs of ER determine the use of specific patterns. In laboratory settings, De Castella et al [ 89 ] demonstrated that individuals with SAD presented low emotional self-efficacy, or the belief that emotions cannot be changed, but the results obtained by Goodman et al [ 18 ] provide ecological validity to a result that confirms the interdependency between cognitions and emotions [ 90 ].

Another approach is to calculate ER diversity, defined as the variety, frequency, and evenness of the implemented ER. Finally, Daniel et al [ 22 ] studied whether ER diversity predicted SA severity. The results showed that diversity within avoidance-oriented strategies was associated with both trait and state SA levels. At a more specific level of analysis, participants who responded more evenly and deployed a vast array of avoidance-oriented strategies more frequently were more prone to belonging to the high-SA group.

Emotion Clarity and Differentiation

Another important aspect that has been of increasing interest in the ER literature is linked to convergent processes such as emotion clarity and differentiation. Emotion clarity is defined as the ability to identify, distinguish, and describe specific emotions [ 91 ]. Park and Naragon-Gainey [ 45 ] found that lower momentary clarity was related to increases in subsequent momentary internalizing symptoms (ie, anxiety and depressive symptoms). This association was explained by an unsuccessful use of ER.

A total of 4% (3/70) of the studies examined emotion differentiation in individuals with SAD [ 19 , 37 , 48 ]. Emotion differentiation is conceptualized as the ability to recognize, identify, and label broad emotional experiences into discrete emotion categories [ 92 ]. Kashdan and Farmer [ 19 ] demonstrated that an increase in SA symptoms is linked to an impairment in negative emotion differentiation (ie, the ability to label and describe differences among negative emotions). In particular, it was found that negative emotion differentiation plays a relevant role in implementing more effective ER strategies. For example, Seah et al [ 48 ] conducted 2 studies that showed that negative emotion differentiation moderated the positive relationship between rumination and social avoidance. Similarly, O’Toole et al [ 37 ] showed that individuals with both high SA and poor negative emotion differentiation presented the least use of cognitive reappraisal. In addition, individuals with high SA used more suppression strategies despite the ability to differentiate positive emotions.

Perseverative Cognition and Mind Wandering

Although the most common variable that has been studied in individuals with SAD is postevent processing (PEP), there is a great overlap between PEP and rumination. In essence, both revolve around the perseverative thinking of past events, but PEP is strictly related to social interactions, including an important component of the actual interventions that both the individual and the others may have done or said. In total, 6% (4/70) of the studies used AA to examine processes related to PEP and rumination.

Helbig-Lang et al [ 46 ] studied individuals diagnosed with SAD to explore predictors of higher PEP levels. The results showed that higher PEP was predicted by self-attention, NA, social performance situations, and the use of safety behaviors. In addition, Badra et al [ 50 ] investigated PEP in a nonclinical sample of undergraduate students with high and low SA scores. Although no differences were detected between the groups, PEP was reduced to a single item, and no additional information on the social context was collected, which could have affected the results.

Another study that explored PEP showed that it decreased after a speech task [ 68 ]. The between-level average differed from the person-specific trajectories. This was the case not only for the decrease in PEP after the speech task but also in the temporal cascading relationship between PEP and the next measurement of PEP. The level of anxiety in the speech task predicted engagement in PEP, and this activated a more intense experience of the negative balanced memory, indicating the interconnectedness of cognitive-affective processes.

Bailey et al [ 51 ] studied perseverative cognition (worry and rumination) using physiological measures and found that there was a higher use of this type of repetitive thinking related to lower heart rate variability after negative social interactions. Finally, potential changes in momentary PEP throughout treatment were explored by Katz et al [ 41 ] showing that cognitive behavioral group therapy reduced levels of PEP.

In total, 3% (2/70) of the studies explored mind wandering, which is both an interesting and controversial topic. Traditionally, it was considered that mind wandering was just a negative process given that it was linked to the opposite of having a mindful disposition. However, the latest research has shown that it could be related to less boredom, more creativity, and a better mental health state [ 93 ].

In the study by Arch et al [ 49 ], the authors investigated differences in the frequency, range of content, and correlates of internal off-task thinking (ie, mind wandering). Relative to on-task thinking, internal off-task thinking was associated with worse mood, more self-focus, and less thought controllability for those with SAD compared to healthy controls. In addition, participants with SAD engaged in internal unrelated task thinking more frequently than those in the control group and presented more unintentional mind wandering on a trait questionnaire.

Specific Interactional Triggers, Patterns, and Activities

Geyer et al [ 62 ] investigated the association between NA in social interactions and perceived enjoyment of those interactions to explore specific real-time contributors to negative perceptions often experienced by individuals with SA. The results revealed that this association was more negative when SA was more severe, although the sample consisted of undiagnosed individuals.

Meanwhile, Blalock et al [ 15 ] studied the experience of flow in social and nonsocial situations in individuals with SAD and healthy participants. The results were contrary to the hypotheses, revealing that individuals with SAD presented flow more frequently in social situations than healthy participants. The authors explained this unexpected result using the concept of flow, which includes the component of experiencing a challenging situation as a defining feature. It is reasonable to think that people with SAD experience normal social interactions as more challenging than healthy participants.

In one study, the association between sexual activity and SA was explored in nonclinical individuals [ 71 ]. As could be expected, sexual activity was influenced by SA such that individuals with SA reported their sexual episodes as less pleasurable and reported being less connected with their partners as well as presenting a lower frequency of sexual activity. Overall, these results suggest that sexual activity is not fulfilling when experiencing SA.

Goodman et al [ 30 ] conducted 2 AA studies in which they concluded that, although individuals with SAD present fewer and less satisfying social relationships, they enjoy social interactions when they occur, which might indicate that they are happier with others than alone. In other words, experiencing SA and the relative concern about socializing does not hinder the pleasure of socializing.

These results are aligned with those of other 2 studies conducted on undergraduate students [ 54 , 83 ], which showed that SA was not related to a lower desire to be with others. However, in the study by Brown et al [ 54 ], a preference for solitude was found in interactions with unfamiliar people.

Villanueva et al [ 39 ] revealed that individuals with SAD presented a higher number of social interactions through their mobile phones than the control group. They were also the group with the least number of social interactions (vs the control group and individuals with depression). In-person interactions (ie, face-to-face) were revealed to be less related to increases in NA and decreases in PA.

In the study by Doorley et al [ 27 ], no significant association was found between the medium of communication (ie, digital vs face-to-face communication) and SA. That is, in both media, there was an association between SA and less positive and more negative emotions. Oren-Yagoda and Aderka [ 33 ] also explored media of communication in individuals diagnosed with SAD. In this case, the focus was on the media of communication and the associated perceptions and emotions. Individuals with SAD usually preferred to use voice and text media to a greater extent than visual media. However, the authors found that, despite this preference, when visual media were implemented, immediate increases in positive perceptions and emotions were experienced by people with SAD. These results support the idea that the selected medium functions as a safety behavior.

Meanwhile, Russell et al [ 44 ] revealed that individuals with SAD presented higher levels of submissive behavior and lower levels of dominant behavior compared to control participants. However, this was true in the presence of anxiety-eliciting cues, which means that there are certain situations that can be perceived as safe environments. In contexts of emotional security, all individuals with SAD presented an enhanced agreeable and deceased quarrelsome behavior, also meaning that there might be situations of security in which individuals with SAD can respond to positive appraisals with enhanced affiliative behavior.

Hur et al [ 66 ] found that individuals with SAD benefit more from having close friends, family members, and romantic partners in terms of resulting in lower levels of NA, anxiety, and depression compared to control participants. However, they tend to spend less time with those companions. These results emphasize the intact capacity of individuals with SAD to enhance their mood through social interactions.

In this line, Hannah Lee [ 65 ] obtained consistent results, suggesting a tight connection between the levels of SA and the degree of unfamiliarity and judgmentalness in the interactions. Namely, individuals with higher levels of SA presented a stronger association with the 2 processes as a consequence of being more sensitive to experiencing anxiety when facing interactions of the same level of unfamiliarity and judgmentalness compared to people with lower levels of trait SA.

Čolić et al [ 59 ] were the first to explore depersonalization and derealization in embarrassing situations. They showed that people with SAD presented more embarrassing social interactions than control participants, and as a result, they also presented more depersonalization and derealization, which can be seen as responses to strong emotions (including embarrassment) as well as attempts to cope with situational challenges.

Cognitive Factors

Social comparison is another important aspect to explore in individuals with SAD given that they usually tend to see themselves with a negative self-image [ 94 , 95 ]. In another study, Brown et al [ 54 ] showed that SA was associated with greater self-consciousness, which can also be aligned with the negative self-view that characterizes SAD.

Goodman et al [ 31 ] found that SA is related to less favorable and more unstable social comparisons, which can be explained by a negative self-image. Moreover, they demonstrated that, when people with SAD make less favorable social comparisons, they are especially fearful of others’ evaluations.

In a recent study, Brown et al [ 55 ] investigated interpersonal distress in heightened SA symptoms as predictors of suicidal ideation. Specifically, this study showed that hurdles to seeking social support and social comparisons mediated suicidal ideation.

Models of SAD have emphasized the central role of fear of evaluation in the appearance and maintenance of this clinical condition. This construct has been included in cognitive models related to attentional biases and negative interpretations of the self [ 96 - 98 ]. Although negative evaluation was considered a core dimension in early models of SAD, fear of positive evaluation has emerged as an important topic in recent years [ 78 , 99 ].

Another study explored anxiety sensitivity cognitive concerns and fear of negative evaluation as 2 potential predictors of SA amplification. Anxiety sensitivity cognitive concerns were shown to uniquely amplify arousal as a consequence of social stress, whereas fear of negative evaluation predicted anxiety fluctuations, indicating that these 2 cognitive constructs may be associated with SA in different ways [ 79 ].

By implementing 2 AA studies, Reichenberger et al [ 78 ] explored the interaction of both positive and negative evaluation, affect, and stress reactivity. Although the results were not fully in line with the hypotheses, fear of negative evaluation was negatively associated with PA. In addition, the results revealed that the closeness of the relationships was paramount to determine when the interaction was significant, with closer relationships being less anxiety provoking. Consistent with these results, positive and negative feedback seeking has been shown to be higher in individuals with SAD than in healthy participants [ 81 ], all of which is aligned with the mounting evidence developed in cross-sectional or laboratory settings. Similarly, Doorley et al [ 28 ] demonstrated that self-perceived intense positive events, which are normally reduced in SA, paradoxically provided more psychological benefits (reduced anxiety and motivation toward social situations as well as an increased sense of belonging).

Moreover, Nanamori et al [ 74 ] studied triggers of self-focused attention, which is another key component of classic cognitive models of SAD. The results showed that perception of gaze, evaluation, and authority predicted self-focused attention from the observer’s perspective, whereas perception of gaze also predicted self-focus on body sensation. Moreover, the perception of positive response and that of a stranger predicted self-focus on body sensation hinged on sex, suggesting that the positive response perception of female participants acted as a predictor of the self-focus on body sensation.

Emerging Topics

Use of aa to assess psychological interventions.

Daniel et al [ 83 ], Katz et al [ 41 ], and Kivity and Huppert [ 72 ] implemented AA to explore the course of treatment. In the case of Katz et al [ 41 ], PEP, a putative maintenance factor of SA symptoms, was assessed over the course of a cognitive behavioral therapy intervention with a subset of the 60 included participants answering an AA. The intervention yielded significant reductions in both general and momentary PEP, and both types of PEP were significant predictors of SA severity after treatment.

Another study was conducted by Kivity and Huppert [ 72 ]. It explored how ER in individuals with high and low SA responded to a practice of cognitive reappraisal using self-report, laboratory tasks, and daily diaries. Although the group with high SA presented lower symptom severity and greater self-efficacy of reappraisal, daily anxiety was not significantly different.

Daniel et al [ 83 ] conducted a randomized controlled trial testing whether cognitive bias modification for interpretation could decrease negative interpretation bias. Both the active and control conditions received an AA throughout the treatment period (5 weeks). While the active group also received cognitive bias modification training, the control group only answered the AA. A total of 2 publications were identified from this study, reporting self-report and passive sensing outcome measures. Despite the interesting approach of incorporating multimodal assessment, both analyses yielded nonsignificant results.

Use of Sensors and Biosensors in AA

Over the last few years, new advancements in wearable sensors and biosensors have enabled us to incorporate them into AA studies. In the case of SA, this has led to a considerable body of evidence. Specifically, Bailey et al [ 51 ], Boukhechba et al [ 53 ], Chow et al [ 58 ], Daniel et al [ 82 ], Di Matteo et al [ 60 ], and Jacobson et al [ 47 ] conducted studies using sensors (GPS location and accelerometers) and biosensors (heart rate and heart rate variability).

Bailey et al [ 51 ] investigated perseverative cognition in relation to the parasympathetic nervous system, which is considered of utmost relevance in the regulation of stress and emotions [ 100 ]. In this study, individuals with both depression and SAD were monitored, and individuals with SAD presented the highest frequency of daily perseverative cognition, which was statistically associated with lower heart rate variability, moderated by negative social interactions. Jacobson and Bhattacharya [ 67 ] showed that spending time indoors was associated with anxiety and avoidance symptoms, and this association was significantly higher in individuals with SAD than in those with generalized anxiety disorder.

Boukhechba et al [ 53 ] and Chow et al [ 58 ] used GPS and Jacobson et al [ 47 ] used an accelerometer to demonstrate the capability of passive sensing to predict the severity of SA symptomology according to the level of activity. Moreover, Di Matteo et al [ 60 ] designed an app to capture ambient audio, GPS location, screen state, and light sensor data, and this app was shown to be able to identify SAD patterns of behavior in a relatively accurate way.

Exploring Idiographic Comorbidity Patterns

Although a vast array of studies included heterogeneous samples in terms of their diagnosis, only Piccirillo and Rodebaugh [ 77 ] had the objective of exploring SAD and major depressive disorder comorbidity, aiming to model person-specific trajectories of cognitive-affective and behavioral dimensions related to these disorders. By including only cisgender women with comorbid SAD and major depressive disorder, this study showed the utmost relevance in disentangling between-person, within-person, and person-specific patterns. For example, loneliness was revealed to be a common predictor of depressive mood and social avoidance at the group level; however, this was not the case when examining the idiographic networks.

Transcultural Differences

Only 1% (1/70) of the studies examined potential variations between different cultural groups. Lee et al [ 42 ] explored differences between European Americans and Asian Americans, showing that both groups experienced the same number of anxious events during social situations but Asian Americans presented more negative emotions in those moments.

Principal Findings

Our review of 70 original studies using AA to explore SA showed that this methodology provides valuable real-time information on the momentary association of SA with several variables, such as context, affective dynamics, emotional states and regulation, social interactions, and other consequences and antecedents. This comprehensive understanding can contribute to better insights into the appearance and maintenance of symptoms and of this clinical disorder.

Aligned with the burgeoning literature on AA, affect and emotional dynamics emerged as the most studied topics. These investigations revealed a trend of an increase in NA levels leading to a heightened experience of SA, as well as a growing attention to positive emotions and PA deficits in individuals with SAD. This review also supports the notion that negative emotions and affect are not enough to distinguish normal from pathological SA. As demonstrated by Park and Naragon-Gainey [ 45 ], AA may help shed light on the structural models of affect both between and within individuals’ variances, evidence that traditional cross-sectional research or long-term longitudinal research may not capture.

Most interestingly, most of the studies exploring affect trends explored them coupled with ER strategies, consistently extending the vast literature in this regard. Exacerbated NA and PA and dysfunctional strategies to cope with them form a dysfunctional pattern that may be responsible for the appearance and maintenance of SA and SAD [ 85 ]. The studies revealed that the interpersonal encounters of individuals with SAD may differ from those of controls in terms of ER use and type of ER strategy. Specifically, both intra- and interpersonal regulatory mechanisms have been shown to be associated with increasing levels of SA. In clinical populations, what was shown to influence the levels of SA was not the use of certain strategies but rather the lack of effectiveness of their use. However, this was not the case in healthy populations. Accordingly, a potential difference between SA symptoms in healthy and clinical populations may lie on the effectiveness or underuse of ER strategies.

In line with the mounting evidence exploring suppression and both experiential and behavioral avoidance in individuals with SAD, this systematic review showed coherent and robust results across the included studies concerning the maladaptive use of these 2 strategies. People with SAD may present an overreliance on the use of suppression and avoidance [ 85 ], resulting in a range of negative outcomes such as an increase in NA and a decrease in PA.

In contrast, cognitive reappraisal, a putatively effective strategy, does not yield straightforward results. The problem with individuals with SAD is more the ineffective use of cognitive reappraisal rather than the scarce use of this strategy, although the studies did not seem to coherently yield conclusive results.

Taken together, the results on affective dynamics and ER indicate how appropriate AA can be to study these processes, especially in the case of SA and SAD. AA may be particularly helpful in disentangling between- and within-person effects, which the literature demonstrates can have different or even contrary results, especially in the context of psychological interventions [ 101 ].

As a key takeaway message, AA shows that individuals with high SA symptoms or a diagnosis of SAD may not use a narrower repertoire of ER strategies but rather implement that repertoire with less skillfulness or less ability to identify when it is appropriate to implement a certain strategy. In this sense, it is essential to continue exploring the role of polyregulation and flexibility to identify which specific facet of ER contributes as a mechanism of action of SAD.

In that vein, substance use can be seen as a maladaptive behavior that functions as a behavioral strategy to regulate emotions and cope with situations that elicit symptoms of SA. This is particularly recurrent in SA-provoking situations, constituting a reinforcement cycle that operates similarly to other safety behaviors. In particular, alcohol consumption may function as a negative reinforcer, attenuating the negative self-perceived quality of interpersonal encounters and anxiety levels. When this occurs, alcohol use becomes a rapidly established maladaptive behavior with negative consequences.

In addition, maladaptive cognitions, emotional mechanisms, and behaviors were found to be activated when levels of anxiety increased. More specifically, cognitive aspects such as beliefs in capacity, effectiveness in regulating emotions, and the ability to differentiate emotions appear to be relevant in explaining how SA is activated.

As a general takeaway message, there is ample evidence showing the mutual directionality between cognitive and affective or emotional facets and behaviors in the appearance and maintenance of SAD. The several sections in which the studies were categorized constitute just one way of organizing the information. However, many of these studies can be understood as forms of cognition or ER or specific interactional patterns. A clear example is alcohol consumption, which is a behavior that, in the context of SAD, can be understood as an ER strategy.

In that sense, ER is currently a trending topic in AA, but it is important to integrate this increasing amount of knowledge into classic cognitive models. This is pertinent for psychopathological developments in general, and SAD is not an exception. Over the years, the most influential developments in SAD have been cognitive and behavioral models [ 96 , 98 ]. Paradoxically, in this review, cognitive processes remained an underexplored area. An example of this is mental imagery, which has been shown to be a transdiagnostic process that explains the appearance and maintenance of a range of clinical conditions, including SAD [ 94 ]; however, there is a dearth of studies on this crucial construct.

An additional line of research that needs to be further explored is comorbidity to explore the mutual dependency of certain groups of signs and symptoms. However, given the lack of network analyses, this mutual dependency was not explored in depth. For example, there was only one study exploring suicidal ideation and attempts despite the ample existing literature on AA in suicide research [ 102 ] and the strong association between suicide and SA [ 103 ]. In the same vein, the relationship among personality, personality pathology, and SA is a relevant topic in contemporary psychopathology that could be further explored using AA strategies. Indeed, there is a growing body of evidence exploring personality and personality pathology dynamics and states, which should be considered in future studies of SA [ 9 ].

Another issue worth discussing revolves around the incorporation of AA into psychological interventions. This is an increasingly used practice and may be well integrated with routine outcome monitoring procedures that have been shown to yield significant effects when implemented in both controlled and naturalistic interventions [ 104 ]. Routine outcome monitoring is an increasingly implemented strategy that can connect research and practice in unprecedented ways. With that aim, it is necessary to create simple visualization interfaces to feed back the trajectories of certain patient variables. Some efforts have already been made in this direction [ 105 ]. If these endeavors are further developed, they can be used by clinicians as a clinical tool, and at the same time, researchers can collect naturalistic data.

Another topic with a lot of potential is the incorporation of behavioral and physiological processes by means of sensors and biosensors. Multimethod measurements that incorporate both passive and active assessments can be of tremendous relevance to harness the affordances of each approach. Together with the development of machine learning algorithms, the proliferation of EMIs is more plausible. This is very important to enhance the personalization of possible treatments.

Regarding data collection, there are now software solutions that are opening up unprecedented opportunities for future research. Older studies usually included PDAs or similar devices, which implies not only spending more resources to implement an AA study but also some degree of training in order for participants to be able to use these devices. Currently, there are studies that harness existing survey platforms such as Qualtrics to program either random or fixed prompts without the need for any specifically developed software.

Given that most of the studies were conducted in the United States and the rest were conducted in other Western high-income countries, the results should be generalized to other contexts. Cultural and contextual factors are determinant in all psychopathological conditions, and SAD is not an exception [ 106 ]. The proliferation of open-source platforms (eg, m-Path [ 107 ]) will permit the dissemination of this methodology to researchers without large budgets, such as researchers from low- and middle-income countries. This is crucial to guarantee that knowledge is not restricted to certain populations, fundamentally populations from Western, educated, industrialized, rich, and democratic countries. In addition, most of the studies (54/70, 77%) paid the participants to enhance the compliance rates, which turned out to be in line with the average compliance in AA literature (approximately 75% [ 108 ]). However, this should be considered in future research on AA that seeks to increase the external validity by means of ecological designs.

Methodological Design

Regarding the methodological design of the studies included in this systematic review, there are important aspects to discuss. Most of the studies were well designed; advanced statistical strategies were applied; and, accordingly, most of this research was published in journals with a high impact factor. However, there is a methodological pitfall in AA research that revolves around the lack of psychometrically sound instruments, usually because of trying to reduce participant burden as much as possible. According to Hopwood et al [ 109 ], four aspects are essential when discussing the theoretical and methodological implications for AA research: (1) How should time be scaled? (2) How many assessments are needed? (3) How frequently should assessments be conducted? and (4) When should the assessments occur? Researchers using AA methods to conduct research on SA and SAD should carefully consider these questions both theoretically and empirically. Moreover, there is a wider consensus on the need to conduct more theory-driven hypothesis testing [ 110 ]. All these methodological aspects should be considered with caution together with the importance of increasing the transparency of reporting the results of AA research [ 111 ].

Power analysis was revealed as a weak methodological aspect, with many of the studies not calculating the required sample sizes to anticipate the number of needed participants. Potential limitations concerning the quality of the studies seem to be related to the lack of clear guidelines and standards, which have only recently started to emerge [ 112 ].

Regarding the data analysis, most of the studies used multilevel or hierarchical linear models [ 113 ]. Indeed, in cases in which ANOVAs or ordinary least squares models were used instead of hierarchical models, the results should be interpreted with more caution. They do not account for the dependency of the data, and in longitudinal assessments such as AA in which data are essentially nested, using these strategies may yield inaccurate pictures of the data [ 114 ]. In addition, AAs generally entail mounting random missing data, and multilevel mixed models are appropriate to deal with that data structure.

Future studies need to incorporate new modalities of data analysis that might provide more complex information to understand the dynamics of SA. For example, multilevel network analyses [ 115 ] would allow for shedding light not only on the nested structure of the symptoms but also on the interconnectedness at every moment of the individuals’ experiences and behaviors. In addition, new-generation time-series analyses such as the time-varying change point autoregressive models would allow for the detection of gradual and abrupt changes in SA markers over the course of the AAs [ 116 ]. Furthermore, the recently developed dynamic structural equation modeling method [ 117 ] is particularly suitable for intensive longitudinal data from AA. This method allows for a more accurate estimation of individual differences in means and autoregressive effects from AA data. Finally, machine learning strategies will be paramount not only to build predictive models that can better explain SA but also to implement EMIs based on people’s needs [ 118 ]. In the field of SAD, there is a dearth of studies on EMIs, which is surprising given the ample evidence that has been found using AA.

Limitations

The results of this review should be considered in light of certain limitations. The first limitation concerns the inclusion criteria. Gray literature, including dissertations and preprint depositories, was not considered. Given the growing interest in this topic, we may have missed other relevant studies from these sources. However, this decision had the main aim of ensuring the rigor of including articles that had undergone a peer review process. In addition, we only included revised articles published in English, excluding articles published in different languages.

A second limitation is that this is the first synthesis that summarizes the literature on AA for SA, but further quantitative syntheses (ie, meta-analyses) should be conducted on the specific topics identified in this study. Thus, a qualitative review is a first step that contributes to taking stock of the principal topics studied in the field of SA and AA, but no conclusive statements should be drawn.

Conclusions

This systematic review shows that AA constitutes a very powerful modality to grasp SA from a complementary perspective to laboratory experiments and usual self-report measures. Over the last few years, mounting research has been conducted showing important trends that are shedding light on the understanding of SA and SAD using this ecological tool that is revolutionizing the field.

Acknowledgments

This work was supported by Margarita Salas postdoctoral contracts MGS/2021/37 (UP2021-021) and MGS/2022/18 (UP2022-024) financed by the European Union NextGenerationEU. This study was also funded by the Prometeo Programme grant for Research Groups of Excellence (CIPROM/2021/041—Project “IMPULSA”), Conselleria d’Innovació, Universitats, Ciència i Societat Digital, Generalitat Valenciana. Finally, this work was supported by the Generalitat Valenciana through the Santiago Grisolía predoctoral program (GRISOLIAP/2021/009).

Data Availability

All data generated or analyzed during this study are included in this published article and its supplementary information files.

Conflicts of Interest

None declared.

Syntax used in database searches.

Characteristics of the included studies.

Social anxiety disorder ambulatory assessment research design overview.

PRISMA Checklist.

• Gilboa-Schechtman E, Shachar I, Sahar Y. Positivity impairment as a broad-based feature of social anxiety. In: Weeks JW, editor. The Wiley Blackwell Handbook of Social Anxiety Disorder. Hoboken, NJ. John Wiley & Sons; 2014;409-432.
• Morrison AS, Heimberg RG. Social anxiety and social anxiety disorder. Annu Rev Clin Psychol. Mar 28, 2013;9(1):249-274. [ CrossRef ] [ Medline ]
• Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annu Rev Clin Psychol. Apr 2008;4(1):1-32. [ CrossRef ] [ Medline ]
• Kuppens P, Oravecz Z, Tuerlinckx F. Feelings change: accounting for individual differences in the temporal dynamics of affect. J Pers Soc Psychol. Dec 2010;99(6):1042-1060. [ CrossRef ] [ Medline ]
• Trull TJ, Ebner-Priemer UW. Using experience sampling methods/ecological momentary assessment (ESM/EMA) in clinical assessment and clinical research: introduction to the special section. Psychol Assess. Dec 2009;21(4):457-462. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Colombo D, Suso-Ribera C, Fernández-Álvarez J, Cipresso P. Technology based ecological momentary assessment in clinical psychology. In: Asmundson JG, editor. Comprehensive Clinical Psychology. New York, NY. Elsevier Academic Press; 2022;106-120.
• Fonseca-Pedrero E, Ródenas-Perea G, Pérez-Albéniz A, Al-Halabí S, Pérez M, Muñiz J. La hora de la evaluación ambulatoria. Pap Psicol. 2021;42(2):21-28. [ CrossRef ]
• Palmier-Claus JE, Myin-Germeys I, Barkus E, Bentley L, Udachina A, Delespaul PA, et al. Experience sampling research in individuals with mental illness: reflections and guidance. Acta Psychiatr Scand. Jan 2011;123(1):12-20. [ CrossRef ] [ Medline ]
• Wright AG, Woods WC. Personalized models of psychopathology. Annu Rev Clin Psychol. May 07, 2020;16:49-74. [ CrossRef ] [ Medline ]
• Myin-Germeys I, Kasanova Z, Vaessen T, Vachon H, Kirtley O, Viechtbauer W, et al. Experience sampling methodology in mental health research: new insights and technical developments. World Psychiatry. Jun 24, 2018;17(2):123-132. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. Jul 21, 2009;339(jul21 1):b2535. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Social anxiety is experienced in the wild: a protocol for a systematic review of studies using ambulatory assessment. Open Science Framework. URL: https://osf.io/47hc8/ [accessed 2024-02-17]
• Asher M, Asnaani A, Aderka IM. Gender differences in social anxiety disorder: a review. Clin Psychol Rev. Aug 2017;56:1-12. [ CrossRef ] [ Medline ]
• Blalock DV, Kashdan TB, Farmer AS. Trait and daily emotion regulation in social anxiety disorder. Cogn Ther Res. Dec 8, 2015;40(3):416-425. [ CrossRef ]
• Blalock DV, Kashdan TB, McKnight PE. High risk, high reward: daily perceptions of social challenge and performance in social anxiety disorder. J Anxiety Disord. Mar 2018;54:57-64. [ CrossRef ] [ Medline ]
• Farmer AS, Kashdan TB. Affective and self-esteem instability in the daily lives of people with generalized social anxiety disorder. Clin Psychol Sci. Mar 01, 2014;2(2):187-201. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Goodman FR, Kashdan TB, Stiksma MC, Blalock DV. Personal strivings to understand anxiety disorders: social anxiety as an exemplar. Clin Psychol Sci. Nov 14, 2018;7(2):283-301. [ CrossRef ]
• Goodman FR, Kashdan TB, İmamoğlu A. Valuing emotional control in social anxiety disorder: a multimethod study of emotion beliefs and emotion regulation. Emotion. Jun 2021;21(4):842-855. [ CrossRef ] [ Medline ]
• Kashdan TB, Farmer AS. Differentiating emotions across contexts: comparing adults with and without social anxiety disorder using random, social interaction, and daily experience sampling. Emotion. Jun 2014;14(3):629-638. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kashdan TB, Farmer AS, Adams LM, Ferssizidis P, McKnight PE, Nezlek JB. Distinguishing healthy adults from people with social anxiety disorder: evidence for the value of experiential avoidance and positive emotions in everyday social interactions. J Abnorm Psychol. Aug 2013;122(3):645-655. [ CrossRef ] [ Medline ]
• Kashdan TB, Goodman FR, Machell KA, Kleiman EM, Monfort SS, Ciarrochi J, et al. A contextual approach to experiential avoidance and social anxiety: evidence from an experimental interaction and daily interactions of people with social anxiety disorder. Emotion. Aug 2014;14(4):769-781. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Daniel KE, Larrazabal MA, Boukhechba M, Barnes LE, Teachman BA. State and trait emotion regulation diversity in social anxiety. Clin Psychol Sci. Sep 28, 2023;11(5):894-909. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Beltzer ML, Ameko MK, Daniel KE, Daros AR, Boukhechba M, Barnes LE, et al. Building an emotion regulation recommender algorithm for socially anxious individuals using contextual bandits. Br J Clin Psychol. Jan 14, 2022;61 Suppl 1(S1):51-72. [ CrossRef ] [ Medline ]
• Daniel KE, Baee S, Boukhechba M, Barnes LE, Teachman BA. Do I really feel better? Effectiveness of emotion regulation strategies depends on the measure and social anxiety. Depress Anxiety. Dec 25, 2019;36(12):1182-1190. [ CrossRef ] [ Medline ]
• Daros AR, Daniel KE, Meyer MJ, Chow PI, Barnes LE, Teachman BA. Impact of social anxiety and social context on college students’ emotion regulation strategy use: an experience sampling study. Motiv Emot. May 27, 2019;43(5):844-855. [ CrossRef ]
• Ladis I, Toner ER, Daros AR, Daniel KE, Boukhechba M, Chow PI, et al. Assessing emotion polyregulation in daily life: who uses it, when is it used, and how effective is it? Affect Sci. Jun 08, 2023;4(2):248-259. [ CrossRef ] [ Medline ]
• Doorley JD, Volgenau KM, Kelso KC, Kashdan TB, Shackman AJ. Do people with elevated social anxiety respond differently to digital and face-to-face communications? Two daily diary studies with null effects. J Affect Disord. Nov 01, 2020;276:859-865. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Doorley JD, Goodman FR, Disabato DJ, Kashdan TB, Weinstein JS, Shackman AJ. The momentary benefits of positive events for individuals with elevated social anxiety. Emotion. Apr 2021;21(3):595-606. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Goodman FR, Kashdan TB. The most important life goals of people with and without social anxiety disorder: focusing on emotional interference and uncovering meaning in life. J Posit Psychol. Nov 11, 2021;16(2):272-281. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Goodman FR, Rum R, Silva G, Kashdan TB. Are people with social anxiety disorder happier alone? J Anxiety Disord. Dec 2021;84:102474. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Goodman FR, Kelso KC, Wiernik BM, Kashdan TB. Social comparisons and social anxiety in daily life: an experience-sampling approach. J Abnorm Psychol. Jul 2021;130(5):468-489. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Goodman FR, Daniel KE, Eldesouky L, Brown BA, Kneeland ET. How do people with social anxiety disorder manage daily stressors? Deconstructing emotion regulation flexibility in daily life. J Affect Disord Rep. Dec 2021;6:100210. [ CrossRef ]
• Oren-Yagoda R, Aderka IM. The medium is the message: effects of mediums of communication on perceptions and emotions in social anxiety disorder. J Anxiety Disord. Oct 2021;83:102458. [ CrossRef ] [ Medline ]
• Oren-Yagoda R, Paz N, Aderka IM. Pride in social anxiety disorder. J Anxiety Disord. Oct 2023;99:102759. [ CrossRef ] [ Medline ]
• Oren-Yagoda R, Schwartz M, Aderka IM. The grass is always greener: envy in social anxiety disorder. J Anxiety Disord. Aug 2021;82:102445. [ CrossRef ] [ Medline ]
• Oren-Yagoda R, Melamud-Ganani I, Aderka IM. All by myself: loneliness in social anxiety disorder. J Psychopathol Clin Sci. Jan 2022;131(1):4-13. [ CrossRef ] [ Medline ]
• O’Toole MS, Jensen MB, Fentz HN, Zachariae R, Hougaard E. Emotion differentiation and emotion regulation in high and low socially anxious individuals: an experience-sampling study. Cogn Ther Res. Mar 13, 2014;38(4):428-438. [ CrossRef ]
• O'Toole MS, Zachariae R, Mennin DS. Social anxiety and emotion regulation flexibility: considering emotion intensity and type as contextual factors. Anxiety Stress Coping. Nov 30, 2017;30(6):716-724. [ CrossRef ] [ Medline ]
• Villanueva J, Meyer AH, Miché M, Wersebe H, Mikoteit T, Hoyer J, et al. Social interaction in major depressive disorder, social phobia, and controls: the importance of affect. J Technol Behav Sci. Dec 10, 2019;5(2):139-148. [ CrossRef ]
• Rinner MT, Meyer AH, Mikoteit T, Hoyer J, Imboden C, Hatzinger M, et al. General or specific? The memory-experience gap for individuals diagnosed with a major depressive disorder or a social phobia diagnosis, and individuals without such diagnoses. Memory. Oct 16, 2019;27(9):1194-1203. [ CrossRef ] [ Medline ]
• Katz DE, Cassin S, Weerasinghe R, Rector NA. Changes in post-event processing during cognitive behavioural therapy for social anxiety disorder: a longitudinal analysis using post-session measurement and experience sampling methodology. J Anxiety Disord. Aug 2019;66:102107. [ CrossRef ] [ Medline ]
• Lee MR, Okazaki S, Yoo HC. Frequency and intensity of social anxiety in Asian Americans and European Americans. Cultur Divers Ethnic Minor Psychol. Apr 2006;12(2):291-305. [ CrossRef ] [ Medline ]
• Naragon-Gainey K. Affective models of depression and anxiety: extension to within-person processes in daily life. J Affect Disord. Jan 15, 2019;243:241-248. [ CrossRef ] [ Medline ]
• Russell JJ, Moskowitz DS, Zuroff DC, Bleau P, Pinard G, Young SN. Anxiety, emotional security and the interpersonal behavior of individuals with social anxiety disorder. Psychol Med. May 12, 2010;41(3):545-554. [ CrossRef ]
• Park J, Naragon-Gainey K. Daily experiences of emotional clarity and their association with internalizing symptoms in naturalistic settings. Emotion. Aug 2019;19(5):764-775. [ CrossRef ] [ Medline ]
• Helbig-Lang S, von Auer M, Neubauer K, Murray E, Gerlach AL. Post-event processing in social anxiety disorder after real-life social situations - an ambulatory assessment study. Behav Res Ther. Sep 2016;84:27-34. [ CrossRef ] [ Medline ]
• Jacobson NC, Summers B, Wilhelm S. Digital biomarkers of social anxiety severity: digital phenotyping using passive smartphone sensors. J Med Internet Res. May 29, 2020;22(5):e16875. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Seah TH, Aurora P, Coifman KG. Emotion differentiation as a protective factor against the behavioral consequences of rumination: a conceptual replication and extension in the context of social anxiety. Behav Ther. Jan 2020;51(1):135-148. [ CrossRef ] [ Medline ]
• Arch JJ, Wilcox RR, Ives LT, Sroloff A, Andrews-Hanna JR. Off-task thinking among adults with and without social anxiety disorder: an ecological momentary assessment study. Cogn Emot. Mar 19, 2021;35(2):269-281. [ CrossRef ] [ Medline ]
• Badra M, Schulze L, Becker ES, Vrijsen JN, Renneberg B, Zetsche U. The association between ruminative thinking and negative interpretation bias in social anxiety. Cogn Emot. Sep 2017;31(6):1234-1242. [ CrossRef ] [ Medline ]
• Bailey T, Shahabi L, Tarvainen M, Shapiro D, Ottaviani C. Moderating effects of the valence of social interaction on the dysfunctional consequences of perseverative cognition: an ecological study in major depression and social anxiety disorder. Anxiety Stress Coping. Mar 22, 2019;32(2):179-195. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Battista SR, MacKinnon SP, Sherry SB, Barrett SP, MacNevin PD, Stewart SH. Does alcohol reduce social anxiety in daily life? A 22-day experience sampling study. J Soc Clin Psychol. Jun 2015;34(6):508-528. [ CrossRef ]
• Boukhechba M, Chow P, Fua K, Teachman BA, Barnes LE. Predicting social anxiety from global positioning system traces of college students: feasibility study. JMIR Ment Health. Jul 04, 2018;5(3):e10101. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Brown LH, Silvia PJ, Myin-Germeys I, Kwapil TR. When the need to belong goes wrong: the expression of social anhedonia and social anxiety in daily life. Psychol Sci. Sep 06, 2007;18(9):778-782. [ CrossRef ] [ Medline ]
• Brown BA, Rottenberg J, Goodman FR. Social anxiety and interpersonal risk for suicidal ideation: a longitudinal daily diary analysis. Suicide Life Threat Behav. Dec 28, 2023;53(6):968-980. [ CrossRef ] [ Medline ]
• Buckner JD, Crosby RD, Wonderlich SA, Schmidt NB. Social anxiety and cannabis use: an analysis from ecological momentary assessment. J Anxiety Disord. Mar 2012;26(2):297-304. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Buckner JD, Langdon KJ, Jeffries ER, Zvolensky MJ. Socially anxious smokers experience greater negative affect and withdrawal during self-quit attempts. Addict Behav. Apr 2016;55:46-49. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chow PI, Fua K, Huang Y, Bonelli W, Xiong H, Barnes LE, et al. Using mobile sensing to test clinical models of depression, social anxiety, state affect, and social isolation among college students. J Med Internet Res. Mar 03, 2017;19(3):e62. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Čolić J, Bassett TR, Latysheva A, Imboden C, Bader K, Hatzinger M, et al. Depersonalization and derealization in embarrassing social interactions: an experience sampling study in social phobia, major depression and controls. J Anxiety Disord. Mar 2020;70:102189. [ CrossRef ] [ Medline ]
• Di Matteo D, Fotinos K, Lokuge S, Mason G, Sternat T, Katzman MA, et al. Automated screening for social anxiety, generalized anxiety, and depression from objective smartphone-collected data: cross-sectional study. J Med Internet Res. Aug 13, 2021;23(8):e28918. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Farmer AS, Kashdan TB. Social anxiety and emotion regulation in daily life: spillover effects on positive and negative social events. Cogn Behav Ther. 2012;41(2):152-162. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Geyer EC, Fua KC, Daniel KE, Chow PI, Bonelli W, Huang Y, et al. I did ok, but did I like it? Using ecological momentary assessment to examine perceptions of social interactions associated with severity of social anxiety and depression. Behav Ther. Nov 2018;49(6):866-880. [ CrossRef ] [ Medline ]
• Goodman FR, Stiksma MC, Kashdan TB. Social anxiety and the quality of everyday social interactions: the moderating influence of alcohol consumption. Behav Ther. May 2018;49(3):373-387. [ CrossRef ] [ Medline ]
• Goodman FR, Brown BA, Silva GM, Bradford DE, Tennen H, Kashdan TB. Motives and consequences of alcohol use in people with social anxiety disorder: a daily diary study. Behav Ther. Jul 2022;53(4):600-613. [ CrossRef ] [ Medline ]
• Hannah Lee J. Perceptions towards an interaction partner predict social anxiety: an ecological momentary assessment study. Cogn Emot. Dec 30, 2021;35(8):1479-1498. [ CrossRef ] [ Medline ]
• Hur J, DeYoung KA, Islam S, Anderson AS, Barstead MG, Shackman AJ. Social context and the real-world consequences of social anxiety. Psychol Med. Aug 19, 2019;50(12):1989-2000. [ CrossRef ]
• Jacobson NC, Bhattacharya S. Digital biomarkers of anxiety disorder symptom changes: personalized deep learning models using smartphone sensors accurately predict anxiety symptoms from ecological momentary assessments. Behav Res Ther. Feb 2022;149:104013. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kane L, Ashbaugh AR. Ecological momentary assessment of post-event processing in between two speech tasks: relationships with cognitive and affective factors involved in the maintenance of social anxiety. Behav Res Ther. Dec 2022;159:104208. [ CrossRef ] [ Medline ]
• Kashdan TB, Steger MF. Expanding the topography of social anxiety. An experience-sampling assessment of positive emotions, positive events, and emotion suppression. Psychol Sci. Feb 06, 2006;17(2):120-128. [ CrossRef ] [ Medline ]
• Kashdan TB, Collins RL. Social anxiety and the experience of positive emotion and anger in everyday life: an ecological momentary assessment approach. Anxiety Stress Coping. May 2010;23(3):259-272. [ CrossRef ] [ Medline ]
• Kashdan TB, Adams L, Savostyanova A, Ferssizidis P, McKnight PE, Nezlek JB. Effects of social anxiety and depressive symptoms on the frequency and quality of sexual activity: a daily process approach. Behav Res Ther. May 2011;49(5):352-360. [ CrossRef ] [ Medline ]
• Kivity Y, Huppert JD. Does cognitive reappraisal reduce anxiety? A daily diary study of a micro-intervention with individuals with high social anxiety. J Consult Clin Psychol. Mar 2016;84(3):269-283. [ CrossRef ] [ Medline ]
• Kim S, Kwon JH. The impact of negative emotions on drinking among individuals with social anxiety disorder in daily life: the moderating effect of maladaptive emotion regulation strategies. Cogn Ther Res. Oct 09, 2019;44(2):345-359. [ CrossRef ]
• Nanamori M, Tomita N, Kametani C, Matsuda N, Kumano H. Triggers of self-focused attention: an ecological momentary assessment study. Biopsychosoc Med. Apr 23, 2023;17(1):16. [ FREE Full text ] [ CrossRef ] [ Medline ]
• O'Grady MA, Cullum J, Armeli S, Tennen H. Putting the relationship between social anxiety and alcohol use into context: a daily diary investigation of drinking in response to embarrassing events. J Soc Clin Psychol. Jun 2011;30(6):599-615. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Papp LM, Kouros CD, Witt HK, Curtin JJ, Blumenstock SM, Barringer A. Real-time momentary mood as a predictor of college students’ prescription drug misuse in daily life: direct links and the moderating role of background mental health. Exp Clin Psychopharmacol. Dec 2022;30(6):787-796. [ CrossRef ]
• Piccirillo ML, Rodebaugh TL. Personalized networks of social anxiety disorder and depression and implications for treatment. J Affect Disord. Feb 01, 2022;298(Pt A):262-276. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Reichenberger J, Smyth JM, Blechert J. Fear of evaluation unpacked: day-to-day correlates of fear of negative and positive evaluation. Anxiety Stress Coping. Mar 09, 2018;31(2):159-174. [ CrossRef ] [ Medline ]
• Saulnier KG, Saulnier SJ, Allan NP. Cognitive risk factors and the experience of acute anxiety following social stressors: an ecological momentary assessment study. J Anxiety Disord. May 2022;88:102571. [ CrossRef ] [ Medline ]
• Walukevich-Dienst K, Calhoun BH, Fairlie AM, Cadigan JM, Patrick ME, Lee CM. Using substances to cope with social anxiety: associations with use and consequences in daily life. Psychol Addict Behav. Jun 2023;37(4):581-591. [ CrossRef ] [ Medline ]
• Wilson GA, Koerner N, Antony MM. An examination of feedback seeking in individuals with social anxiety disorder, generalized anxiety disorder, or no history of mental disorder using a daily diary method. J Cogn Psychother. Apr 01, 2018;32(1):15-37. [ CrossRef ] [ Medline ]
• Daniel KE, Mendu S, Baglione A, Cai L, Teachman BA, Barnes LE, et al. Cognitive bias modification for threat interpretations: using passive Mobile Sensing to detect intervention effects in daily life. Anxiety Stress Coping. May 31, 2022;35(3):298-312. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Daniel KE, Daros AR, Beltzer ML, Boukhechba M, Barnes LE, Teachman BA. How anxious are you right now? Using ecological momentary assessment to evaluate the effects of cognitive bias modification for social threat interpretations. Cogn Ther Res. Mar 11, 2020;44(3):538-556. [ CrossRef ]
• Rozen N, Aderka IM. Emotions in social anxiety disorder: a review. J Anxiety Disord. Apr 2023;95:102696. [ CrossRef ] [ Medline ]
• Dryman MT, Heimberg RG. Emotion regulation in social anxiety and depression: a systematic review of expressive suppression and cognitive reappraisal. Clin Psychol Rev. Nov 2018;65:17-42. [ CrossRef ] [ Medline ]
• Ford BQ, Gross JJ, Gruber J. Broadening our field of view: the role of emotion polyregulation. Emot Rev. Jun 23, 2019;11(3):197-208. [ CrossRef ]
• Aldao A, Sheppes G, Gross JJ. Emotion regulation flexibility. Cogn Ther Res. Jan 1, 2015;39(3):263-278. [ CrossRef ]
• Tamir M, Mitchell C, Gross JJ. Hedonic and instrumental motives in anger regulation: research report. Psychol Sci. Apr 01, 2008;19(4):324-328. [ CrossRef ] [ Medline ]
• De castella K, Goldin P, Jazaieri H, Ziv M, Heimberg RG, Gross JJ. Emotion beliefs in social anxiety disorder: associations with stress, anxiety, and well‐being. Aust J Psychol. Nov 20, 2020;66(2):139-148. [ FREE Full text ] [ CrossRef ]
• McRae K, Zarolia P. Cognition and emotion in emotion dysregulation. In: Beauchaine TP, Crowell SE, editors. The Oxford Handbook of Emotion Dysregulation. Oxford, UK. Oxford University Press; 2018;39-52.
• Gohm CL, Clore GL. Individual differences in emotional experience: mapping available scales to processes. Pers Soc Psychol Bull. Jul 02, 2016;26(6):679-697. [ CrossRef ]
• Barrett LF, Gross J, Christensen TC, Benvenuto M. Knowing what you're feeling and knowing what to do about it: mapping the relation between emotion differentiation and emotion regulation. Cogn Emot. Nov 2001;15(6):713-724. [ CrossRef ]
• Ottaviani C, Couyoumdjian A. Pros and cons of a wandering mind: a prospective study. Front Psychol. 2013;4:524. [ FREE Full text ] [ CrossRef ] [ Medline ]
• McEvoy PM, Erceg-Hurn DM, Barber KC, Dupasquier JR, Moscovitch DA. Transportability of imagery-enhanced CBT for social anxiety disorder. Behav Res Ther. Jul 2018;106:86-94. [ CrossRef ] [ Medline ]
• Hirsch CR, Clark DM, Mathews A, Williams R. Self-images play a causal role in social phobia. Behav Res Ther. Aug 2003;41(8):909-921. [ CrossRef ] [ Medline ]
• Clark DM, Wells A. A cognitive model of social phobia. PsycNET. URL: https://psycnet.apa.org/record/1995-98887-004 [accessed 2024-09-15]
• Moscovitch DA. What is the core fear in social phobia? A new model to facilitate individualized case conceptualization and treatment. Cogn Behav Pract. May 2009;16(2):123-134. [ CrossRef ]
• Rapee RM, Heimberg RG. A cognitive-behavioral model of anxiety in social phobia. Behav Res Ther. Aug 1997;35(8):741-756. [ CrossRef ] [ Medline ]
• Weeks JW, Howell AN. Fear of positive evaluation: the neglected fear domain in social anxiety. In: Weeks JW, editor. The Wiley Blackwell Handbook of Social Anxiety Disorder. Hoboken, NJ. John Wiley & Sons; 2014.
• Mather M, Thayer JF. How heart rate variability affects emotion regulation brain networks. Curr Opin Behav Sci. Feb 2018;19:98-104. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zilcha-Mano S. Toward personalized psychotherapy: the importance of the trait-like/state-like distinction for understanding therapeutic change. Am Psychol. Apr 2021;76(3):516-528. [ CrossRef ] [ Medline ]
• Sedano-Capdevila A, Porras-Segovia A, Bello HJ, Baca-García E, Barrigon ML. Use of ecological momentary assessment to study suicidal thoughts and behavior: a systematic review. Curr Psychiatry Rep. May 18, 2021;23(7):41. [ CrossRef ] [ Medline ]
• Buckner JD, Lemke AW, Jeffries ER, Shah SM. Social anxiety and suicidal ideation: test of the utility of the interpersonal-psychological theory of suicide. J Anxiety Disord. Jan 2017;45:60-63. [ FREE Full text ] [ CrossRef ] [ Medline ]
• de Jong K, Conijn JM, Gallagher RA, Reshetnikova AS, Heij M, Lutz MC. Using progress feedback to improve outcomes and reduce drop-out, treatment duration, and deterioration: a multilevel meta-analysis. Clin Psychol Rev. Apr 2021;85:102002. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bringmann LF, van der Veen DC, Wichers M, Riese H, Stulp G. ESMvis: a tool for visualizing individual experience sampling method (ESM) data. Qual Life Res. Nov 22, 2021;30(11):3179-3188. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Soto JA, Dawson-Andoh NA, BeLue R. The relationship between perceived discrimination and generalized anxiety disorder among African Americans, Afro Caribbeans, and non-Hispanic Whites. J Anxiety Disord. Mar 2011;25(2):258-265. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mestdagh M, Verdonck S, Piot M, Niemeijer K, Tuerlinckx F, Kuppens P, et al. m-Path: an easy-to-use and highly tailorable platform for ecological momentary assessment and intervention in behavioral research and clinical practice. PsyArXiv. Preprints posted online January 25, 2022. [ FREE Full text ] [ CrossRef ]
• Wrzus C, Neubauer AB. Ecological momentary assessment: a meta-analysis on designs, samples, and compliance across research fields. Assessment. Apr 11, 2023;30(3):825-846. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hopwood CJ, Bleidorn W, Wright AG. Connecting theory to methods in longitudinal research. Perspect Psychol Sci. May 05, 2022;17(3):884-894. [ CrossRef ] [ Medline ]
• Kuppens P, Dejonckheere E, Kalokerinos EK, Koval P. Some recommendations on the use of daily life methods in affective science. Affect Sci. Jun 19, 2022;3(2):505-515. [ CrossRef ] [ Medline ]
• Hall M, Scherner PV, Kreidel Y, Rubel JA. A systematic review of momentary assessment designs for mood and anxiety symptoms. Front Psychol. May 17, 2021;12:642044. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Trull TJ, Ebner-Priemer UW. Ambulatory assessment in psychopathology research: a review of recommended reporting guidelines and current practices. J Abnorm Psychol. Jan 2020;129(1):56-63. [ CrossRef ] [ Medline ]
• Raudenbush SW, Bryk AS. Hierarchical Linear Models: Applications and Data Analysis Methods. 2nd edition. Thousand Oaks, CA. Sage Publications; 2002.
• Gómez Penedo JM, Muiños R, Hirsch P, Roussos A. La aplicación de modelos lineales jerárquicos para el estudio de la eficacia en psicoterapia. Rev Argent Cienc Comport. Apr 24, 2019;11(1):25. [ CrossRef ]
• Bringmann LF, Vissers N, Wichers M, Geschwind N, Kuppens P, Peeters F, et al. A network approach to psychopathology: new insights into clinical longitudinal data. PLoS One. Apr 4, 2013;8(4):e60188. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hehlmann MI, Schwartz B, Lutz T, Gómez Penedo JM, Rubel JA, Lutz W. The use of digitally assessed stress levels to model change processes in CBT - a feasibility study on seven case examples. Front Psychiatry. Mar 9, 2021;12:613085. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hamaker EL, Asparouhov T, Brose A, Schmiedek F, Muthén B. At the frontiers of modeling intensive longitudinal data: dynamic structural equation models for the affective measurements from the COGITO study. Multivariate Behav Res. Apr 06, 2018;53(6):820-841. [ CrossRef ] [ Medline ]
• Delgadillo J. Machine learning: a primer for psychotherapy researchers. Psychother Res. Jan 29, 2021;31(1):1-4. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by J Torous; submitted 17.02.23; peer-reviewed by K Daniel, T Ranjan; comments to author 03.09.23; revised version received 28.01.24; accepted 07.02.24; published 04.04.24.

©Javier Fernández-Álvarez, Desirée Colombo, Juan Martín Gómez Penedo, Maitena Pierantonelli, Rosa María Baños, Cristina Botella. Originally published in JMIR Mental Health (https://mental.jmir.org), 04.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.