writing a research paper about covid

• Research article
• Open access
• Published: 04 June 2021

Coronavirus disease (COVID-19) pandemic: an overview of systematic reviews

• Israel Júnior Borges do Nascimento 1 , 2 ,
• Dónal P. O’Mathúna 3 , 4 ,
• Thilo Caspar von Groote 5 ,
• Hebatullah Mohamed Abdulazeem 6 ,
• Ishanka Weerasekara 7 , 8 ,
• Ana Marusic 9 ,
• Livia Puljak   ORCID: orcid.org/0000-0002-8467-6061 10 ,
• Vinicius Tassoni Civile 11 ,
• Irena Zakarija-Grkovic 9 ,
• Tina Poklepovic Pericic 9 ,
• Alvaro Nagib Atallah 11 ,
• Santino Filoso 12 ,
• Nicola Luigi Bragazzi 13 &
• Milena Soriano Marcolino 1

On behalf of the International Network of Coronavirus Disease 2019 (InterNetCOVID-19)

BMC Infectious Diseases volume  21 , Article number:  525 ( 2021 ) Cite this article

16k Accesses

29 Citations

13 Altmetric

Metrics details

Navigating the rapidly growing body of scientific literature on the SARS-CoV-2 pandemic is challenging, and ongoing critical appraisal of this output is essential. We aimed to summarize and critically appraise systematic reviews of coronavirus disease (COVID-19) in humans that were available at the beginning of the pandemic.

Nine databases (Medline, EMBASE, Cochrane Library, CINAHL, Web of Sciences, PDQ-Evidence, WHO’s Global Research, LILACS, and Epistemonikos) were searched from December 1, 2019, to March 24, 2020. Systematic reviews analyzing primary studies of COVID-19 were included. Two authors independently undertook screening, selection, extraction (data on clinical symptoms, prevalence, pharmacological and non-pharmacological interventions, diagnostic test assessment, laboratory, and radiological findings), and quality assessment (AMSTAR 2). A meta-analysis was performed of the prevalence of clinical outcomes.

Eighteen systematic reviews were included; one was empty (did not identify any relevant study). Using AMSTAR 2, confidence in the results of all 18 reviews was rated as “critically low”. Identified symptoms of COVID-19 were (range values of point estimates): fever (82–95%), cough with or without sputum (58–72%), dyspnea (26–59%), myalgia or muscle fatigue (29–51%), sore throat (10–13%), headache (8–12%) and gastrointestinal complaints (5–9%). Severe symptoms were more common in men. Elevated C-reactive protein and lactate dehydrogenase, and slightly elevated aspartate and alanine aminotransferase, were commonly described. Thrombocytopenia and elevated levels of procalcitonin and cardiac troponin I were associated with severe disease. A frequent finding on chest imaging was uni- or bilateral multilobar ground-glass opacity. A single review investigated the impact of medication (chloroquine) but found no verifiable clinical data. All-cause mortality ranged from 0.3 to 13.9%.

Conclusions

In this overview of systematic reviews, we analyzed evidence from the first 18 systematic reviews that were published after the emergence of COVID-19. However, confidence in the results of all reviews was “critically low”. Thus, systematic reviews that were published early on in the pandemic were of questionable usefulness. Even during public health emergencies, studies and systematic reviews should adhere to established methodological standards.

Peer Review reports

The spread of the “Severe Acute Respiratory Coronavirus 2” (SARS-CoV-2), the causal agent of COVID-19, was characterized as a pandemic by the World Health Organization (WHO) in March 2020 and has triggered an international public health emergency [ 1 ]. The numbers of confirmed cases and deaths due to COVID-19 are rapidly escalating, counting in millions [ 2 ], causing massive economic strain, and escalating healthcare and public health expenses [ 3 , 4 ].

The research community has responded by publishing an impressive number of scientific reports related to COVID-19. The world was alerted to the new disease at the beginning of 2020 [ 1 ], and by mid-March 2020, more than 2000 articles had been published on COVID-19 in scholarly journals, with 25% of them containing original data [ 5 ]. The living map of COVID-19 evidence, curated by the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre), contained more than 40,000 records by February 2021 [ 6 ]. More than 100,000 records on PubMed were labeled as “SARS-CoV-2 literature, sequence, and clinical content” by February 2021 [ 7 ].

Due to publication speed, the research community has voiced concerns regarding the quality and reproducibility of evidence produced during the COVID-19 pandemic, warning of the potential damaging approach of “publish first, retract later” [ 8 ]. It appears that these concerns are not unfounded, as it has been reported that COVID-19 articles were overrepresented in the pool of retracted articles in 2020 [ 9 ]. These concerns about inadequate evidence are of major importance because they can lead to poor clinical practice and inappropriate policies [ 10 ].

Systematic reviews are a cornerstone of today’s evidence-informed decision-making. By synthesizing all relevant evidence regarding a particular topic, systematic reviews reflect the current scientific knowledge. Systematic reviews are considered to be at the highest level in the hierarchy of evidence and should be used to make informed decisions. However, with high numbers of systematic reviews of different scope and methodological quality being published, overviews of multiple systematic reviews that assess their methodological quality are essential [ 11 , 12 , 13 ]. An overview of systematic reviews helps identify and organize the literature and highlights areas of priority in decision-making.

In this overview of systematic reviews, we aimed to summarize and critically appraise systematic reviews of coronavirus disease (COVID-19) in humans that were available at the beginning of the pandemic.

Methodology

Research question.

This overview’s primary objective was to summarize and critically appraise systematic reviews that assessed any type of primary clinical data from patients infected with SARS-CoV-2. Our research question was purposefully broad because we wanted to analyze as many systematic reviews as possible that were available early following the COVID-19 outbreak.

Study design

We conducted an overview of systematic reviews. The idea for this overview originated in a protocol for a systematic review submitted to PROSPERO (CRD42020170623), which indicated a plan to conduct an overview.

Overviews of systematic reviews use explicit and systematic methods for searching and identifying multiple systematic reviews addressing related research questions in the same field to extract and analyze evidence across important outcomes. Overviews of systematic reviews are in principle similar to systematic reviews of interventions, but the unit of analysis is a systematic review [ 14 , 15 , 16 ].

We used the overview methodology instead of other evidence synthesis methods to allow us to collate and appraise multiple systematic reviews on this topic, and to extract and analyze their results across relevant topics [ 17 ]. The overview and meta-analysis of systematic reviews allowed us to investigate the methodological quality of included studies, summarize results, and identify specific areas of available or limited evidence, thereby strengthening the current understanding of this novel disease and guiding future research [ 13 ].

A reporting guideline for overviews of reviews is currently under development, i.e., Preferred Reporting Items for Overviews of Reviews (PRIOR) [ 18 ]. As the PRIOR checklist is still not published, this study was reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2009 statement [ 19 ]. The methodology used in this review was adapted from the Cochrane Handbook for Systematic Reviews of Interventions and also followed established methodological considerations for analyzing existing systematic reviews [ 14 ].

Approval of a research ethics committee was not necessary as the study analyzed only publicly available articles.

Eligibility criteria

Systematic reviews were included if they analyzed primary data from patients infected with SARS-CoV-2 as confirmed by RT-PCR or another pre-specified diagnostic technique. Eligible reviews covered all topics related to COVID-19 including, but not limited to, those that reported clinical symptoms, diagnostic methods, therapeutic interventions, laboratory findings, or radiological results. Both full manuscripts and abbreviated versions, such as letters, were eligible.

No restrictions were imposed on the design of the primary studies included within the systematic reviews, the last search date, whether the review included meta-analyses or language. Reviews related to SARS-CoV-2 and other coronaviruses were eligible, but from those reviews, we analyzed only data related to SARS-CoV-2.

No consensus definition exists for a systematic review [ 20 ], and debates continue about the defining characteristics of a systematic review [ 21 ]. Cochrane’s guidance for overviews of reviews recommends setting pre-established criteria for making decisions around inclusion [ 14 ]. That is supported by a recent scoping review about guidance for overviews of systematic reviews [ 22 ].

Thus, for this study, we defined a systematic review as a research report which searched for primary research studies on a specific topic using an explicit search strategy, had a detailed description of the methods with explicit inclusion criteria provided, and provided a summary of the included studies either in narrative or quantitative format (such as a meta-analysis). Cochrane and non-Cochrane systematic reviews were considered eligible for inclusion, with or without meta-analysis, and regardless of the study design, language restriction and methodology of the included primary studies. To be eligible for inclusion, reviews had to be clearly analyzing data related to SARS-CoV-2 (associated or not with other viruses). We excluded narrative reviews without those characteristics as these are less likely to be replicable and are more prone to bias.

Scoping reviews and rapid reviews were eligible for inclusion in this overview if they met our pre-defined inclusion criteria noted above. We included reviews that addressed SARS-CoV-2 and other coronaviruses if they reported separate data regarding SARS-CoV-2.

Information sources

Nine databases were searched for eligible records published between December 1, 2019, and March 24, 2020: Cochrane Database of Systematic Reviews via Cochrane Library, PubMed, EMBASE, CINAHL (Cumulative Index to Nursing and Allied Health Literature), Web of Sciences, LILACS (Latin American and Caribbean Health Sciences Literature), PDQ-Evidence, WHO’s Global Research on Coronavirus Disease (COVID-19), and Epistemonikos.

The comprehensive search strategy for each database is provided in Additional file 1 and was designed and conducted in collaboration with an information specialist. All retrieved records were primarily processed in EndNote, where duplicates were removed, and records were then imported into the Covidence platform [ 23 ]. In addition to database searches, we screened reference lists of reviews included after screening records retrieved via databases.

Study selection

All searches, screening of titles and abstracts, and record selection, were performed independently by two investigators using the Covidence platform [ 23 ]. Articles deemed potentially eligible were retrieved for full-text screening carried out independently by two investigators. Discrepancies at all stages were resolved by consensus. During the screening, records published in languages other than English were translated by a native/fluent speaker.

Data collection process

We custom designed a data extraction table for this study, which was piloted by two authors independently. Data extraction was performed independently by two authors. Conflicts were resolved by consensus or by consulting a third researcher.

We extracted the following data: article identification data (authors’ name and journal of publication), search period, number of databases searched, population or settings considered, main results and outcomes observed, and number of participants. From Web of Science (Clarivate Analytics, Philadelphia, PA, USA), we extracted journal rank (quartile) and Journal Impact Factor (JIF).

We categorized the following as primary outcomes: all-cause mortality, need for and length of mechanical ventilation, length of hospitalization (in days), admission to intensive care unit (yes/no), and length of stay in the intensive care unit.

The following outcomes were categorized as exploratory: diagnostic methods used for detection of the virus, male to female ratio, clinical symptoms, pharmacological and non-pharmacological interventions, laboratory findings (full blood count, liver enzymes, C-reactive protein, d-dimer, albumin, lipid profile, serum electrolytes, blood vitamin levels, glucose levels, and any other important biomarkers), and radiological findings (using radiography, computed tomography, magnetic resonance imaging or ultrasound).

We also collected data on reporting guidelines and requirements for the publication of systematic reviews and meta-analyses from journal websites where included reviews were published.

Quality assessment in individual reviews

Two researchers independently assessed the reviews’ quality using the “A MeaSurement Tool to Assess Systematic Reviews 2 (AMSTAR 2)”. We acknowledge that the AMSTAR 2 was created as “a critical appraisal tool for systematic reviews that include randomized or non-randomized studies of healthcare interventions, or both” [ 24 ]. However, since AMSTAR 2 was designed for systematic reviews of intervention trials, and we included additional types of systematic reviews, we adjusted some AMSTAR 2 ratings and reported these in Additional file 2 .

Adherence to each item was rated as follows: yes, partial yes, no, or not applicable (such as when a meta-analysis was not conducted). The overall confidence in the results of the review is rated as “critically low”, “low”, “moderate” or “high”, according to the AMSTAR 2 guidance based on seven critical domains, which are items 2, 4, 7, 9, 11, 13, 15 as defined by AMSTAR 2 authors [ 24 ]. We reported our adherence ratings for transparency of our decision with accompanying explanations, for each item, in each included review.

One of the included systematic reviews was conducted by some members of this author team [ 25 ]. This review was initially assessed independently by two authors who were not co-authors of that review to prevent the risk of bias in assessing this study.

Synthesis of results

For data synthesis, we prepared a table summarizing each systematic review. Graphs illustrating the mortality rate and clinical symptoms were created. We then prepared a narrative summary of the methods, findings, study strengths, and limitations.

For analysis of the prevalence of clinical outcomes, we extracted data on the number of events and the total number of patients to perform proportional meta-analysis using RStudio© software, with the “meta” package (version 4.9–6), using the “metaprop” function for reviews that did not perform a meta-analysis, excluding case studies because of the absence of variance. For reviews that did not perform a meta-analysis, we presented pooled results of proportions with their respective confidence intervals (95%) by the inverse variance method with a random-effects model, using the DerSimonian-Laird estimator for τ 2 . We adjusted data using Freeman-Tukey double arcosen transformation. Confidence intervals were calculated using the Clopper-Pearson method for individual studies. We created forest plots using the RStudio© software, with the “metafor” package (version 2.1–0) and “forest” function.

Managing overlapping systematic reviews

Some of the included systematic reviews that address the same or similar research questions may include the same primary studies in overviews. Including such overlapping reviews may introduce bias when outcome data from the same primary study are included in the analyses of an overview multiple times. Thus, in summaries of evidence, multiple-counting of the same outcome data will give data from some primary studies too much influence [ 14 ]. In this overview, we did not exclude overlapping systematic reviews because, according to Cochrane’s guidance, it may be appropriate to include all relevant reviews’ results if the purpose of the overview is to present and describe the current body of evidence on a topic [ 14 ]. To avoid any bias in summary estimates associated with overlapping reviews, we generated forest plots showing data from individual systematic reviews, but the results were not pooled because some primary studies were included in multiple reviews.

Our search retrieved 1063 publications, of which 175 were duplicates. Most publications were excluded after the title and abstract analysis ( n = 860). Among the 28 studies selected for full-text screening, 10 were excluded for the reasons described in Additional file 3 , and 18 were included in the final analysis (Fig. 1 ) [ 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 ]. Reference list screening did not retrieve any additional systematic reviews.

PRISMA flow diagram

Characteristics of included reviews

Summary features of 18 systematic reviews are presented in Table 1 . They were published in 14 different journals. Only four of these journals had specific requirements for systematic reviews (with or without meta-analysis): European Journal of Internal Medicine, Journal of Clinical Medicine, Ultrasound in Obstetrics and Gynecology, and Clinical Research in Cardiology . Two journals reported that they published only invited reviews ( Journal of Medical Virology and Clinica Chimica Acta ). Three systematic reviews in our study were published as letters; one was labeled as a scoping review and another as a rapid review (Table 2 ).

All reviews were published in English, in first quartile (Q1) journals, with JIF ranging from 1.692 to 6.062. One review was empty, meaning that its search did not identify any relevant studies; i.e., no primary studies were included [ 36 ]. The remaining 17 reviews included 269 unique studies; the majority ( N = 211; 78%) were included in only a single review included in our study (range: 1 to 12). Primary studies included in the reviews were published between December 2019 and March 18, 2020, and comprised case reports, case series, cohorts, and other observational studies. We found only one review that included randomized clinical trials [ 38 ]. In the included reviews, systematic literature searches were performed from 2019 (entire year) up to March 9, 2020. Ten systematic reviews included meta-analyses. The list of primary studies found in the included systematic reviews is shown in Additional file 4 , as well as the number of reviews in which each primary study was included.

Population and study designs

Most of the reviews analyzed data from patients with COVID-19 who developed pneumonia, acute respiratory distress syndrome (ARDS), or any other correlated complication. One review aimed to evaluate the effectiveness of using surgical masks on preventing transmission of the virus [ 36 ], one review was focused on pediatric patients [ 34 ], and one review investigated COVID-19 in pregnant women [ 37 ]. Most reviews assessed clinical symptoms, laboratory findings, or radiological results.

Systematic review findings

The summary of findings from individual reviews is shown in Table 2 . Overall, all-cause mortality ranged from 0.3 to 13.9% (Fig. 2 ).

A meta-analysis of the prevalence of mortality

Clinical symptoms

Seven reviews described the main clinical manifestations of COVID-19 [ 26 , 28 , 29 , 34 , 35 , 39 , 41 ]. Three of them provided only a narrative discussion of symptoms [ 26 , 34 , 35 ]. In the reviews that performed a statistical analysis of the incidence of different clinical symptoms, symptoms in patients with COVID-19 were (range values of point estimates): fever (82–95%), cough with or without sputum (58–72%), dyspnea (26–59%), myalgia or muscle fatigue (29–51%), sore throat (10–13%), headache (8–12%), gastrointestinal disorders, such as diarrhea, nausea or vomiting (5.0–9.0%), and others (including, in one study only: dizziness 12.1%) (Figs. 3 , 4 , 5 , 6 , 7 , 8 and 9 ). Three reviews assessed cough with and without sputum together; only one review assessed sputum production itself (28.5%).

A meta-analysis of the prevalence of fever

A meta-analysis of the prevalence of cough

A meta-analysis of the prevalence of dyspnea

A meta-analysis of the prevalence of fatigue or myalgia

A meta-analysis of the prevalence of headache

A meta-analysis of the prevalence of gastrointestinal disorders

A meta-analysis of the prevalence of sore throat

Diagnostic aspects

Three reviews described methodologies, protocols, and tools used for establishing the diagnosis of COVID-19 [ 26 , 34 , 38 ]. The use of respiratory swabs (nasal or pharyngeal) or blood specimens to assess the presence of SARS-CoV-2 nucleic acid using RT-PCR assays was the most commonly used diagnostic method mentioned in the included studies. These diagnostic tests have been widely used, but their precise sensitivity and specificity remain unknown. One review included a Chinese study with clinical diagnosis with no confirmation of SARS-CoV-2 infection (patients were diagnosed with COVID-19 if they presented with at least two symptoms suggestive of COVID-19, together with laboratory and chest radiography abnormalities) [ 34 ].

Therapeutic possibilities

Pharmacological and non-pharmacological interventions (supportive therapies) used in treating patients with COVID-19 were reported in five reviews [ 25 , 27 , 34 , 35 , 38 ]. Antivirals used empirically for COVID-19 treatment were reported in seven reviews [ 25 , 27 , 34 , 35 , 37 , 38 , 41 ]; most commonly used were protease inhibitors (lopinavir, ritonavir, darunavir), nucleoside reverse transcriptase inhibitor (tenofovir), nucleotide analogs (remdesivir, galidesivir, ganciclovir), and neuraminidase inhibitors (oseltamivir). Umifenovir, a membrane fusion inhibitor, was investigated in two studies [ 25 , 35 ]. Possible supportive interventions analyzed were different types of oxygen supplementation and breathing support (invasive or non-invasive ventilation) [ 25 ]. The use of antibiotics, both empirically and to treat secondary pneumonia, was reported in six studies [ 25 , 26 , 27 , 34 , 35 , 38 ]. One review specifically assessed evidence on the efficacy and safety of the anti-malaria drug chloroquine [ 27 ]. It identified 23 ongoing trials investigating the potential of chloroquine as a therapeutic option for COVID-19, but no verifiable clinical outcomes data. The use of mesenchymal stem cells, antifungals, and glucocorticoids were described in four reviews [ 25 , 34 , 35 , 38 ].

Laboratory and radiological findings

Of the 18 reviews included in this overview, eight analyzed laboratory parameters in patients with COVID-19 [ 25 , 29 , 30 , 32 , 33 , 34 , 35 , 39 ]; elevated C-reactive protein levels, associated with lymphocytopenia, elevated lactate dehydrogenase, as well as slightly elevated aspartate and alanine aminotransferase (AST, ALT) were commonly described in those eight reviews. Lippi et al. assessed cardiac troponin I (cTnI) [ 25 ], procalcitonin [ 32 ], and platelet count [ 33 ] in COVID-19 patients. Elevated levels of procalcitonin [ 32 ] and cTnI [ 30 ] were more likely to be associated with a severe disease course (requiring intensive care unit admission and intubation). Furthermore, thrombocytopenia was frequently observed in patients with complicated COVID-19 infections [ 33 ].

Chest imaging (chest radiography and/or computed tomography) features were assessed in six reviews, all of which described a frequent pattern of local or bilateral multilobar ground-glass opacity [ 25 , 34 , 35 , 39 , 40 , 41 ]. Those six reviews showed that septal thickening, bronchiectasis, pleural and cardiac effusions, halo signs, and pneumothorax were observed in patients suffering from COVID-19.

Quality of evidence in individual systematic reviews

Table 3 shows the detailed results of the quality assessment of 18 systematic reviews, including the assessment of individual items and summary assessment. A detailed explanation for each decision in each review is available in Additional file 5 .

Using AMSTAR 2 criteria, confidence in the results of all 18 reviews was rated as “critically low” (Table 3 ). Common methodological drawbacks were: omission of prospective protocol submission or publication; use of inappropriate search strategy: lack of independent and dual literature screening and data-extraction (or methodology unclear); absence of an explanation for heterogeneity among the studies included; lack of reasons for study exclusion (or rationale unclear).

Risk of bias assessment, based on a reported methodological tool, and quality of evidence appraisal, in line with the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) method, were reported only in one review [ 25 ]. Five reviews presented a table summarizing bias, using various risk of bias tools [ 25 , 29 , 39 , 40 , 41 ]. One review analyzed “study quality” [ 37 ]. One review mentioned the risk of bias assessment in the methodology but did not provide any related analysis [ 28 ].

This overview of systematic reviews analyzed the first 18 systematic reviews published after the onset of the COVID-19 pandemic, up to March 24, 2020, with primary studies involving more than 60,000 patients. Using AMSTAR-2, we judged that our confidence in all those reviews was “critically low”. Ten reviews included meta-analyses. The reviews presented data on clinical manifestations, laboratory and radiological findings, and interventions. We found no systematic reviews on the utility of diagnostic tests.

Symptoms were reported in seven reviews; most of the patients had a fever, cough, dyspnea, myalgia or muscle fatigue, and gastrointestinal disorders such as diarrhea, nausea, or vomiting. Olfactory dysfunction (anosmia or dysosmia) has been described in patients infected with COVID-19 [ 43 ]; however, this was not reported in any of the reviews included in this overview. During the SARS outbreak in 2002, there were reports of impairment of the sense of smell associated with the disease [ 44 , 45 ].

The reported mortality rates ranged from 0.3 to 14% in the included reviews. Mortality estimates are influenced by the transmissibility rate (basic reproduction number), availability of diagnostic tools, notification policies, asymptomatic presentations of the disease, resources for disease prevention and control, and treatment facilities; variability in the mortality rate fits the pattern of emerging infectious diseases [ 46 ]. Furthermore, the reported cases did not consider asymptomatic cases, mild cases where individuals have not sought medical treatment, and the fact that many countries had limited access to diagnostic tests or have implemented testing policies later than the others. Considering the lack of reviews assessing diagnostic testing (sensitivity, specificity, and predictive values of RT-PCT or immunoglobulin tests), and the preponderance of studies that assessed only symptomatic individuals, considerable imprecision around the calculated mortality rates existed in the early stage of the COVID-19 pandemic.

Few reviews included treatment data. Those reviews described studies considered to be at a very low level of evidence: usually small, retrospective studies with very heterogeneous populations. Seven reviews analyzed laboratory parameters; those reviews could have been useful for clinicians who attend patients suspected of COVID-19 in emergency services worldwide, such as assessing which patients need to be reassessed more frequently.

All systematic reviews scored poorly on the AMSTAR 2 critical appraisal tool for systematic reviews. Most of the original studies included in the reviews were case series and case reports, impacting the quality of evidence. Such evidence has major implications for clinical practice and the use of these reviews in evidence-based practice and policy. Clinicians, patients, and policymakers can only have the highest confidence in systematic review findings if high-quality systematic review methodologies are employed. The urgent need for information during a pandemic does not justify poor quality reporting.

We acknowledge that there are numerous challenges associated with analyzing COVID-19 data during a pandemic [ 47 ]. High-quality evidence syntheses are needed for decision-making, but each type of evidence syntheses is associated with its inherent challenges.

The creation of classic systematic reviews requires considerable time and effort; with massive research output, they quickly become outdated, and preparing updated versions also requires considerable time. A recent study showed that updates of non-Cochrane systematic reviews are published a median of 5 years after the publication of the previous version [ 48 ].

Authors may register a review and then abandon it [ 49 ], but the existence of a public record that is not updated may lead other authors to believe that the review is still ongoing. A quarter of Cochrane review protocols remains unpublished as completed systematic reviews 8 years after protocol publication [ 50 ].

Rapid reviews can be used to summarize the evidence, but they involve methodological sacrifices and simplifications to produce information promptly, with inconsistent methodological approaches [ 51 ]. However, rapid reviews are justified in times of public health emergencies, and even Cochrane has resorted to publishing rapid reviews in response to the COVID-19 crisis [ 52 ]. Rapid reviews were eligible for inclusion in this overview, but only one of the 18 reviews included in this study was labeled as a rapid review.

Ideally, COVID-19 evidence would be continually summarized in a series of high-quality living systematic reviews, types of evidence synthesis defined as “ a systematic review which is continually updated, incorporating relevant new evidence as it becomes available ” [ 53 ]. However, conducting living systematic reviews requires considerable resources, calling into question the sustainability of such evidence synthesis over long periods [ 54 ].

Research reports about COVID-19 will contribute to research waste if they are poorly designed, poorly reported, or simply not necessary. In principle, systematic reviews should help reduce research waste as they usually provide recommendations for further research that is needed or may advise that sufficient evidence exists on a particular topic [ 55 ]. However, systematic reviews can also contribute to growing research waste when they are not needed, or poorly conducted and reported. Our present study clearly shows that most of the systematic reviews that were published early on in the COVID-19 pandemic could be categorized as research waste, as our confidence in their results is critically low.

Our study has some limitations. One is that for AMSTAR 2 assessment we relied on information available in publications; we did not attempt to contact study authors for clarifications or additional data. In three reviews, the methodological quality appraisal was challenging because they were published as letters, or labeled as rapid communications. As a result, various details about their review process were not included, leading to AMSTAR 2 questions being answered as “not reported”, resulting in low confidence scores. Full manuscripts might have provided additional information that could have led to higher confidence in the results. In other words, low scores could reflect incomplete reporting, not necessarily low-quality review methods. To make their review available more rapidly and more concisely, the authors may have omitted methodological details. A general issue during a crisis is that speed and completeness must be balanced. However, maintaining high standards requires proper resourcing and commitment to ensure that the users of systematic reviews can have high confidence in the results.

Furthermore, we used adjusted AMSTAR 2 scoring, as the tool was designed for critical appraisal of reviews of interventions. Some reviews may have received lower scores than actually warranted in spite of these adjustments.

Another limitation of our study may be the inclusion of multiple overlapping reviews, as some included reviews included the same primary studies. According to the Cochrane Handbook, including overlapping reviews may be appropriate when the review’s aim is “ to present and describe the current body of systematic review evidence on a topic ” [ 12 ], which was our aim. To avoid bias with summarizing evidence from overlapping reviews, we presented the forest plots without summary estimates. The forest plots serve to inform readers about the effect sizes for outcomes that were reported in each review.

Several authors from this study have contributed to one of the reviews identified [ 25 ]. To reduce the risk of any bias, two authors who did not co-author the review in question initially assessed its quality and limitations.

Finally, we note that the systematic reviews included in our overview may have had issues that our analysis did not identify because we did not analyze their primary studies to verify the accuracy of the data and information they presented. We give two examples to substantiate this possibility. Lovato et al. wrote a commentary on the review of Sun et al. [ 41 ], in which they criticized the authors’ conclusion that sore throat is rare in COVID-19 patients [ 56 ]. Lovato et al. highlighted that multiple studies included in Sun et al. did not accurately describe participants’ clinical presentations, warning that only three studies clearly reported data on sore throat [ 56 ].

In another example, Leung [ 57 ] warned about the review of Li, L.Q. et al. [ 29 ]: “ it is possible that this statistic was computed using overlapped samples, therefore some patients were double counted ”. Li et al. responded to Leung that it is uncertain whether the data overlapped, as they used data from published articles and did not have access to the original data; they also reported that they requested original data and that they plan to re-do their analyses once they receive them; they also urged readers to treat the data with caution [ 58 ]. This points to the evolving nature of evidence during a crisis.

Our study’s strength is that this overview adds to the current knowledge by providing a comprehensive summary of all the evidence synthesis about COVID-19 available early after the onset of the pandemic. This overview followed strict methodological criteria, including a comprehensive and sensitive search strategy and a standard tool for methodological appraisal of systematic reviews.

In conclusion, in this overview of systematic reviews, we analyzed evidence from the first 18 systematic reviews that were published after the emergence of COVID-19. However, confidence in the results of all the reviews was “critically low”. Thus, systematic reviews that were published early on in the pandemic could be categorized as research waste. Even during public health emergencies, studies and systematic reviews should adhere to established methodological standards to provide patients, clinicians, and decision-makers trustworthy evidence.

Availability of data and materials

All data collected and analyzed within this study are available from the corresponding author on reasonable request.

World Health Organization. Timeline - COVID-19: Available at: https://www.who.int/news/item/29-06-2020-covidtimeline . Accessed 1 June 2021.

COVID-19 Dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JHU). Available at: https://coronavirus.jhu.edu/map.html . Accessed 1 June 2021.

Anzai A, Kobayashi T, Linton NM, Kinoshita R, Hayashi K, Suzuki A, et al. Assessing the Impact of Reduced Travel on Exportation Dynamics of Novel Coronavirus Infection (COVID-19). J Clin Med. 2020;9(2):601.

Chinazzi M, Davis JT, Ajelli M, Gioannini C, Litvinova M, Merler S, et al. The effect of travel restrictions on the spread of the 2019 novel coronavirus (COVID-19) outbreak. Science. 2020;368(6489):395–400. https://doi.org/10.1126/science.aba9757 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Fidahic M, Nujic D, Runjic R, Civljak M, Markotic F, Lovric Makaric Z, et al. Research methodology and characteristics of journal articles with original data, preprint articles and registered clinical trial protocols about COVID-19. BMC Med Res Methodol. 2020;20(1):161. https://doi.org/10.1186/s12874-020-01047-2 .

EPPI Centre . COVID-19: a living systematic map of the evidence. Available at: http://eppi.ioe.ac.uk/cms/Projects/DepartmentofHealthandSocialCare/Publishedreviews/COVID-19Livingsystematicmapoftheevidence/tabid/3765/Default.aspx . Accessed 1 June 2021.

NCBI SARS-CoV-2 Resources. Available at: https://www.ncbi.nlm.nih.gov/sars-cov-2/ . Accessed 1 June 2021.

Gustot T. Quality and reproducibility during the COVID-19 pandemic. JHEP Rep. 2020;2(4):100141. https://doi.org/10.1016/j.jhepr.2020.100141 .

Article   PubMed   PubMed Central   Google Scholar  

Kodvanj, I., et al., Publishing of COVID-19 Preprints in Peer-reviewed Journals, Preprinting Trends, Public Discussion and Quality Issues. Preprint article. bioRxiv 2020.11.23.394577; doi: https://doi.org/10.1101/2020.11.23.394577 .

Dobler CC. Poor quality research and clinical practice during COVID-19. Breathe (Sheff). 2020;16(2):200112. https://doi.org/10.1183/20734735.0112-2020 .

Article   Google Scholar  

Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326. https://doi.org/10.1371/journal.pmed.1000326 .

Lunny C, Brennan SE, McDonald S, McKenzie JE. Toward a comprehensive evidence map of overview of systematic review methods: paper 1-purpose, eligibility, search and data extraction. Syst Rev. 2017;6(1):231. https://doi.org/10.1186/s13643-017-0617-1 .

Pollock M, Fernandes RM, Becker LA, Pieper D, Hartling L. Chapter V: Overviews of Reviews. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). Cochrane. 2020. Available from www.training.cochrane.org/handbook .

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane handbook for systematic reviews of interventions version 6.1 (updated September 2020). Cochrane. 2020; Available from www.training.cochrane.org/handbook .

Pollock M, Fernandes RM, Newton AS, Scott SD, Hartling L. The impact of different inclusion decisions on the comprehensiveness and complexity of overviews of reviews of healthcare interventions. Syst Rev. 2019;8(1):18. https://doi.org/10.1186/s13643-018-0914-3 .

Pollock M, Fernandes RM, Newton AS, Scott SD, Hartling L. A decision tool to help researchers make decisions about including systematic reviews in overviews of reviews of healthcare interventions. Syst Rev. 2019;8(1):29. https://doi.org/10.1186/s13643-018-0768-8 .

Hunt H, Pollock A, Campbell P, Estcourt L, Brunton G. An introduction to overviews of reviews: planning a relevant research question and objective for an overview. Syst Rev. 2018;7(1):39. https://doi.org/10.1186/s13643-018-0695-8 .

Pollock M, Fernandes RM, Pieper D, Tricco AC, Gates M, Gates A, et al. Preferred reporting items for overviews of reviews (PRIOR): a protocol for development of a reporting guideline for overviews of reviews of healthcare interventions. Syst Rev. 2019;8(1):335. https://doi.org/10.1186/s13643-019-1252-9 .

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Open Med. 2009;3(3):e123–30.

Krnic Martinic M, Pieper D, Glatt A, Puljak L. Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks. BMC Med Res Methodol. 2019;19(1):203. https://doi.org/10.1186/s12874-019-0855-0 .

Puljak L. If there is only one author or only one database was searched, a study should not be called a systematic review. J Clin Epidemiol. 2017;91:4–5. https://doi.org/10.1016/j.jclinepi.2017.08.002 .

Article   PubMed   Google Scholar  

Gates M, Gates A, Guitard S, Pollock M, Hartling L. Guidance for overviews of reviews continues to accumulate, but important challenges remain: a scoping review. Syst Rev. 2020;9(1):254. https://doi.org/10.1186/s13643-020-01509-0 .

Covidence - systematic review software. Available at: https://www.covidence.org/ . Accessed 1 June 2021.

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008.

Borges do Nascimento IJ, et al. Novel Coronavirus Infection (COVID-19) in Humans: A Scoping Review and Meta-Analysis. J Clin Med. 2020;9(4):941.

Article   PubMed Central   Google Scholar  

Adhikari SP, Meng S, Wu YJ, Mao YP, Ye RX, Wang QZ, et al. Epidemiology, causes, clinical manifestation and diagnosis, prevention and control of coronavirus disease (COVID-19) during the early outbreak period: a scoping review. Infect Dis Poverty. 2020;9(1):29. https://doi.org/10.1186/s40249-020-00646-x .

Cortegiani A, Ingoglia G, Ippolito M, Giarratano A, Einav S. A systematic review on the efficacy and safety of chloroquine for the treatment of COVID-19. J Crit Care. 2020;57:279–83. https://doi.org/10.1016/j.jcrc.2020.03.005 .

Li B, Yang J, Zhao F, Zhi L, Wang X, Liu L, et al. Prevalence and impact of cardiovascular metabolic diseases on COVID-19 in China. Clin Res Cardiol. 2020;109(5):531–8. https://doi.org/10.1007/s00392-020-01626-9 .

Article   CAS   PubMed   Google Scholar  

Li LQ, Huang T, Wang YQ, Wang ZP, Liang Y, Huang TB, et al. COVID-19 patients’ clinical characteristics, discharge rate, and fatality rate of meta-analysis. J Med Virol. 2020;92(6):577–83. https://doi.org/10.1002/jmv.25757 .

Lippi G, Lavie CJ, Sanchis-Gomar F. Cardiac troponin I in patients with coronavirus disease 2019 (COVID-19): evidence from a meta-analysis. Prog Cardiovasc Dis. 2020;63(3):390–1. https://doi.org/10.1016/j.pcad.2020.03.001 .

Lippi G, Henry BM. Active smoking is not associated with severity of coronavirus disease 2019 (COVID-19). Eur J Intern Med. 2020;75:107–8. https://doi.org/10.1016/j.ejim.2020.03.014 .

Lippi G, Plebani M. Procalcitonin in patients with severe coronavirus disease 2019 (COVID-19): a meta-analysis. Clin Chim Acta. 2020;505:190–1. https://doi.org/10.1016/j.cca.2020.03.004 .

Lippi G, Plebani M, Henry BM. Thrombocytopenia is associated with severe coronavirus disease 2019 (COVID-19) infections: a meta-analysis. Clin Chim Acta. 2020;506:145–8. https://doi.org/10.1016/j.cca.2020.03.022 .

Ludvigsson JF. Systematic review of COVID-19 in children shows milder cases and a better prognosis than adults. Acta Paediatr. 2020;109(6):1088–95. https://doi.org/10.1111/apa.15270 .

Lupia T, Scabini S, Mornese Pinna S, di Perri G, de Rosa FG, Corcione S. 2019 novel coronavirus (2019-nCoV) outbreak: a new challenge. J Glob Antimicrob Resist. 2020;21:22–7. https://doi.org/10.1016/j.jgar.2020.02.021 .

Marasinghe, K.M., A systematic review investigating the effectiveness of face mask use in limiting the spread of COVID-19 among medically not diagnosed individuals: shedding light on current recommendations provided to individuals not medically diagnosed with COVID-19. Research Square. Preprint article. doi : https://doi.org/10.21203/rs.3.rs-16701/v1 . 2020 .

Mullins E, Evans D, Viner RM, O’Brien P, Morris E. Coronavirus in pregnancy and delivery: rapid review. Ultrasound Obstet Gynecol. 2020;55(5):586–92. https://doi.org/10.1002/uog.22014 .

Pang J, Wang MX, Ang IYH, Tan SHX, Lewis RF, Chen JIP, et al. Potential Rapid Diagnostics, Vaccine and Therapeutics for 2019 Novel coronavirus (2019-nCoV): a systematic review. J Clin Med. 2020;9(3):623.

Rodriguez-Morales AJ, Cardona-Ospina JA, Gutiérrez-Ocampo E, Villamizar-Peña R, Holguin-Rivera Y, Escalera-Antezana JP, et al. Clinical, laboratory and imaging features of COVID-19: a systematic review and meta-analysis. Travel Med Infect Dis. 2020;34:101623. https://doi.org/10.1016/j.tmaid.2020.101623 .

Salehi S, Abedi A, Balakrishnan S, Gholamrezanezhad A. Coronavirus disease 2019 (COVID-19): a systematic review of imaging findings in 919 patients. AJR Am J Roentgenol. 2020;215(1):87–93. https://doi.org/10.2214/AJR.20.23034 .

Sun P, Qie S, Liu Z, Ren J, Li K, Xi J. Clinical characteristics of hospitalized patients with SARS-CoV-2 infection: a single arm meta-analysis. J Med Virol. 2020;92(6):612–7. https://doi.org/10.1002/jmv.25735 .

Yang J, Zheng Y, Gou X, Pu K, Chen Z, Guo Q, et al. Prevalence of comorbidities and its effects in patients infected with SARS-CoV-2: a systematic review and meta-analysis. Int J Infect Dis. 2020;94:91–5. https://doi.org/10.1016/j.ijid.2020.03.017 .

Bassetti M, Vena A, Giacobbe DR. The novel Chinese coronavirus (2019-nCoV) infections: challenges for fighting the storm. Eur J Clin Investig. 2020;50(3):e13209. https://doi.org/10.1111/eci.13209 .

Article   CAS   Google Scholar  

Hwang CS. Olfactory neuropathy in severe acute respiratory syndrome: report of a case. Acta Neurol Taiwanica. 2006;15(1):26–8.

Google Scholar  

Suzuki M, Saito K, Min WP, Vladau C, Toida K, Itoh H, et al. Identification of viruses in patients with postviral olfactory dysfunction. Laryngoscope. 2007;117(2):272–7. https://doi.org/10.1097/01.mlg.0000249922.37381.1e .

Rajgor DD, Lee MH, Archuleta S, Bagdasarian N, Quek SC. The many estimates of the COVID-19 case fatality rate. Lancet Infect Dis. 2020;20(7):776–7. https://doi.org/10.1016/S1473-3099(20)30244-9 .

Wolkewitz M, Puljak L. Methodological challenges of analysing COVID-19 data during the pandemic. BMC Med Res Methodol. 2020;20(1):81. https://doi.org/10.1186/s12874-020-00972-6 .

Rombey T, Lochner V, Puljak L, Könsgen N, Mathes T, Pieper D. Epidemiology and reporting characteristics of non-Cochrane updates of systematic reviews: a cross-sectional study. Res Synth Methods. 2020;11(3):471–83. https://doi.org/10.1002/jrsm.1409 .

Runjic E, Rombey T, Pieper D, Puljak L. Half of systematic reviews about pain registered in PROSPERO were not published and the majority had inaccurate status. J Clin Epidemiol. 2019;116:114–21. https://doi.org/10.1016/j.jclinepi.2019.08.010 .

Runjic E, Behmen D, Pieper D, Mathes T, Tricco AC, Moher D, et al. Following Cochrane review protocols to completion 10 years later: a retrospective cohort study and author survey. J Clin Epidemiol. 2019;111:41–8. https://doi.org/10.1016/j.jclinepi.2019.03.006 .

Tricco AC, Antony J, Zarin W, Strifler L, Ghassemi M, Ivory J, et al. A scoping review of rapid review methods. BMC Med. 2015;13(1):224. https://doi.org/10.1186/s12916-015-0465-6 .

COVID-19 Rapid Reviews: Cochrane’s response so far. Available at: https://training.cochrane.org/resource/covid-19-rapid-reviews-cochrane-response-so-far . Accessed 1 June 2021.

Cochrane. Living systematic reviews. Available at: https://community.cochrane.org/review-production/production-resources/living-systematic-reviews . Accessed 1 June 2021.

Millard T, Synnot A, Elliott J, Green S, McDonald S, Turner T. Feasibility and acceptability of living systematic reviews: results from a mixed-methods evaluation. Syst Rev. 2019;8(1):325. https://doi.org/10.1186/s13643-019-1248-5 .

Babic A, Poklepovic Pericic T, Pieper D, Puljak L. How to decide whether a systematic review is stable and not in need of updating: analysis of Cochrane reviews. Res Synth Methods. 2020;11(6):884–90. https://doi.org/10.1002/jrsm.1451 .

Lovato A, Rossettini G, de Filippis C. Sore throat in COVID-19: comment on “clinical characteristics of hospitalized patients with SARS-CoV-2 infection: a single arm meta-analysis”. J Med Virol. 2020;92(7):714–5. https://doi.org/10.1002/jmv.25815 .

Leung C. Comment on Li et al: COVID-19 patients’ clinical characteristics, discharge rate, and fatality rate of meta-analysis. J Med Virol. 2020;92(9):1431–2. https://doi.org/10.1002/jmv.25912 .

Li LQ, Huang T, Wang YQ, Wang ZP, Liang Y, Huang TB, et al. Response to Char’s comment: comment on Li et al: COVID-19 patients’ clinical characteristics, discharge rate, and fatality rate of meta-analysis. J Med Virol. 2020;92(9):1433. https://doi.org/10.1002/jmv.25924 .

Download references

Acknowledgments

We thank Catherine Henderson DPhil from Swanscoe Communications for pro bono medical writing and editing support. We acknowledge support from the Covidence Team, specifically Anneliese Arno. We thank the whole International Network of Coronavirus Disease 2019 (InterNetCOVID-19) for their commitment and involvement. Members of the InterNetCOVID-19 are listed in Additional file 6 . We thank Pavel Cerny and Roger Crosthwaite for guiding the team supervisor (IJBN) on human resources management.

This research received no external funding.

Author information

Authors and affiliations.

University Hospital and School of Medicine, Universidade Federal de Minas Gerais, Belo Horizonte, Minas Gerais, Brazil

Israel Júnior Borges do Nascimento & Milena Soriano Marcolino

Medical College of Wisconsin, Milwaukee, WI, USA

Israel Júnior Borges do Nascimento

Helene Fuld Health Trust National Institute for Evidence-based Practice in Nursing and Healthcare, College of Nursing, The Ohio State University, Columbus, OH, USA

Dónal P. O’Mathúna

School of Nursing, Psychotherapy and Community Health, Dublin City University, Dublin, Ireland

Department of Anesthesiology, Intensive Care and Pain Medicine, University of Münster, Münster, Germany

Thilo Caspar von Groote

Department of Sport and Health Science, Technische Universität München, Munich, Germany

Hebatullah Mohamed Abdulazeem

School of Health Sciences, Faculty of Health and Medicine, The University of Newcastle, Callaghan, Australia

Ishanka Weerasekara

Department of Physiotherapy, Faculty of Allied Health Sciences, University of Peradeniya, Peradeniya, Sri Lanka

Cochrane Croatia, University of Split, School of Medicine, Split, Croatia

Ana Marusic, Irena Zakarija-Grkovic & Tina Poklepovic Pericic

Center for Evidence-Based Medicine and Health Care, Catholic University of Croatia, Ilica 242, 10000, Zagreb, Croatia

Livia Puljak

Cochrane Brazil, Evidence-Based Health Program, Universidade Federal de São Paulo, São Paulo, Brazil

Vinicius Tassoni Civile & Alvaro Nagib Atallah

Yorkville University, Fredericton, New Brunswick, Canada

Santino Filoso

Laboratory for Industrial and Applied Mathematics (LIAM), Department of Mathematics and Statistics, York University, Toronto, Ontario, Canada

Nicola Luigi Bragazzi

You can also search for this author in PubMed   Google Scholar

Contributions

IJBN conceived the research idea and worked as a project coordinator. DPOM, TCVG, HMA, IW, AM, LP, VTC, IZG, TPP, ANA, SF, NLB and MSM were involved in data curation, formal analysis, investigation, methodology, and initial draft writing. All authors revised the manuscript critically for the content. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Livia Puljak .

Ethics declarations

Ethics approval and consent to participate.

Not required as data was based on published studies.

Consent for publication

Not applicable.

Competing interests

The authors declare no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: appendix 1..

Search strategies used in the study.

Additional file 2: Appendix 2.

Adjusted scoring of AMSTAR 2 used in this study for systematic reviews of studies that did not analyze interventions.

Additional file 3: Appendix 3.

List of excluded studies, with reasons.

Additional file 4: Appendix 4.

Table of overlapping studies, containing the list of primary studies included, their visual overlap in individual systematic reviews, and the number in how many reviews each primary study was included.

Additional file 5: Appendix 5.

A detailed explanation of AMSTAR scoring for each item in each review.

Additional file 6: Appendix 6.

List of members and affiliates of International Network of Coronavirus Disease 2019 (InterNetCOVID-19).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Borges do Nascimento, I.J., O’Mathúna, D.P., von Groote, T.C. et al. Coronavirus disease (COVID-19) pandemic: an overview of systematic reviews. BMC Infect Dis 21 , 525 (2021). https://doi.org/10.1186/s12879-021-06214-4

Download citation

Received : 12 April 2020

Accepted : 19 May 2021

Published : 04 June 2021

DOI : https://doi.org/10.1186/s12879-021-06214-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Coronavirus
• Evidence-based medicine
• Infectious diseases

BMC Infectious Diseases

ISSN: 1471-2334

• Submission enquiries: [email protected]
• General enquiries: [email protected]

EDITORIAL article

Editorial: coronavirus disease (covid-19): the impact and role of mass media during the pandemic.

• 1 Department of Social and Organizational Psychology, Iscte-University Institute of Lisbon, CIS-IUL, Lisbon, Portugal
• 2 Department of Psychology and Social Work, Mid Sweden University, Östersund, Sweden
• 3 Department of Psychiatry and Psychotherapy, Medical School and University Hospital, Eberhard Karls University of Tübingen, Tübingen, Germany

Editorial on the Research Topic Coronavirus Disease (COVID-19): The Impact and Role of Mass Media During the Pandemic

The outbreak of the coronavirus disease 2019 (COVID-19) has created a global health crisis that had a deep impact on the way we perceive our world and our everyday lives. Not only has the rate of contagion and patterns of transmission threatened our sense of agency, but the safety measures to contain the spread of the virus also required social and physical distancing, preventing us from finding solace in the company of others. Within this context, we launched our Research Topic on March 27th, 2020, and invited researchers to address the Impact and Role of Mass Media During the Pandemic on our lives at individual and social levels.

Despite all the hardships, disruption, and uncertainty brought by the pandemic, we received diverse and insightful manuscript proposals. Frontiers in Psychology published 15 articles, involving 61 authors from 8 countries, which were included in distinct specialized sections, including Health Psychology, Personality and Social Psychology, Emotion Science, and Organizational Psychology. Despite the diversity of this collective endeavor, the contributions fall into four areas of research: (1) the use of media in public health communication; (2) the diffusion of false information; (3) the compliance with the health recommendations; and (4) how media use relates to mental health and well-being.

A first line of research includes contributions examining the use of media in public health communication. Drawing on media messages used in previous health crises, such as Ebola and Zika, Hauer and Sood describe how health organizations use media. They offer a set of recommendations for COVID-19 related media messages, including the importance of message framing, interactive public forums with up-to-date information, and an honest communication about what is known and unknown about the pandemic and the virus. Following a content analysis approach, Parvin et al. studied the representations of COVID-19 in the opinion section of five Asian e-newspapers. The authors identified eight main issues (health and drugs, preparedness and awareness, social welfare and humanity, governance and institutions, the environment and wildlife, politics, innovation and technology, and the economy) and examined how e-newspapers from these countries attributed different weights to these issues and how this relates to the countries' cultural specificity. Raccanello et al. show how the internet can be a platform to disseminate a public campaign devised to inform adults about coping strategies that could help children and teenagers deal with the challenges of the pandemic. The authors examined the dissemination of the program through the analysis of website traffic, showing that in the 40 days following publication, the website reached 6,090 visits.

A second related line of research that drew the concern of researchers was the diffusion of false information about COVID-19 through the media. Lobato et al. examined the role of distinct individual differences (political orientation, social dominance orientation, traditionalism, conspiracy ideation, attitudes about science) on the willingness to share misinformation about COVID-19 over social media. The misinformation topics varied between the severity and spread of COVID-19, treatment and prevention, conspiracy theories, and miscellaneous unverifiable claims. Their results from 296 adult participants (Mage = 36.23; 117 women) suggest two different profiles. One indicating that those reporting more liberal positions and lower social dominance were less willing to share conspiracy misinformation. The other profile indicated that participants scoring high on social dominance and low in traditionalism were more willing to share both conspiracy and other miscellaneous claims, but less willing to share misinformation about the severity and spread of COVID-19. Their findings can have relevant contributions for the identification of specific individual profiles related to the widespread of distinct types of misinformation. Dhanani and Franz examined a sample of 1,141 adults (Mage = 44.66; 46.9% female, 74.7% White ethnic identity) living in the United States in March 2020. The authors examined how media consumption and information source were related to knowledge about COVID-19, the endorsement of misinformation about COVID-19, and prejudice toward Asian Americans. Higher levels of trust in informational sources such as public health organizations (e.g., Center for Disease Control) was associated with greater knowledge, lower endorsement of misinformation, and less prejudice toward Asian Americans. Media source was associated with distinct levels of knowledge, willingness to endorsement misinformation and prejudice toward American Asians, with social media use (e.g., Twitter, Facebook) being related with a lower knowledge about COVID-19, higher endorsement of misinformation, and stronger prejudice toward Asian Americans.

A third line of research addressed the factors that could contribute to compliance with the health recommendations to avoid the spread of the disease. Vai et al. studied early pre-lockdown risk perceptions about COVID-19 and the trust in media sources among 2,223 Italians (Mage = 36.4, 69.2% female). They found that the perceived usefulness of the containment measures (e.g., social distancing) was related to threat perception and efficacy beliefs. Lower threat perception was associated with less perception of utility of the containment measures. Although most participants considered themselves and others capable of taking preventive measures, they saw the measures as generally ineffective. Participants acknowledged using the internet as their main source of information and considered health organizations' websites as the most trustworthy source. Albeit frequently used, social media was in general considered an unreliable source of information. Tomczyk et al. studied knowledge about preventive behaviors, risk perception, stigmatizing attitudes (support for discrimination and blame), and sociodemographic data (e.g., age, gender, country of origin, education level, region, persons per household) as predictors of compliance with the behavioral recommendations among 157 Germans, (age range: 18–77 years, 80% female). Low compliance was associated with male gender, younger age, and lower public stigma. Regarding stigmatizing attitudes, the authors only found a relation between support for discrimination (i.e., support for compulsory measures) and higher intention to comply with recommendations. Mahmood et al. studied the relation between social media use, risk perception, preventive behaviors, and self-efficacy in a sample of 310 Pakistani adults (54.2% female). The authors found social media use to be positively related to self-efficacy and perceived threat, which were both positively related to preventive behaviors (e.g., hand hygiene, social distancing). Information credibility was also related to compliance with health recommendations. Lep et al. examined the relationship between information source perceived credibility and trust, and participants' levels of self-protective behavior among 1,718 Slovenians (age range: 18–81 years, 81.7% female). The authors found that scientists, general practitioners (family doctors), and the National Institute of Public Health were perceived as the more credible source of information, while social media and government officials received the lowest ratings. Perceived information credibility was found to be associated with lower levels of negative emotional responses (e.g., nervousness, helplessness) and a higher level of observance of self-protective measures (e.g., hand washing). Siebenhaar et al. also studied the link between compliance, distress by information, and information avoidance. They examined the online survey responses of 1,059 adults living in Germany (Mage = 39.53, 79.4% female). Their results suggested that distress by information could lead to higher compliance with preventive measures. Distress by information was also associated with higher information avoidance, which in turn is related to less compliance. Gantiva et al. studied the effectiveness of different messages regarding the intentions toward self-care behaviors, perceived efficacy to motivate self-care behaviors in others, perceived risk, and perceived message strength, in a sample of 319 Colombians (age range: 18–60 years, 69.9% female). Their experiment included the manipulation of message framing (gain vs. loss) and message content (economy vs. health). Participants judged gain-frame health related messages to be stronger and more effective in changing self-behavior, whereas loss-framed health messages resulted in increased perceived risk. Rahn et al. offer a comparative view of compliance and risk perception, examining three hazard types: COVID-19 pandemic, violent acts, and severe weather. With a sample of 403 Germans (age range: 18–89 years, 72% female), they studied how age, gender, previous hazard experience and different components of risk appraisal (perceived severity, anticipated negative emotions, anticipatory worry, and risk perception) were related to the intention to comply with behavioral recommendations. They found that higher age predicted compliance with health recommendations to prevent COVID-19, anticipatory worry predicted compliance with warning messages regarding violent acts, and women complied more often with severe weather recommendations than men.

A fourth line of research examined media use, mental health and well-being during the COVID-19 pandemic. Gabbiadini et al. addressed the use of digital technology (e.g., voice/video calls, online games, watching movies in party mode) to stay connected with others during lockdown. Participants, 465 Italians (age range: 18–73 years, 348 female), reported more perceived social support associated with the use of these digital technologies, which in turn was associated with fewer feelings of loneliness, boredom, anger, and higher sense of belongingness. Muñiz-Velázquez et al. compared the media habits of 249 Spanish adults (Mage = 42.06, 53.8% female) before and during confinement. They compared the type of media consumed (e.g., watching TV series, listening to radio, watching news) and found the increased consumption of TV and social networking sites during confinement to be negatively associated with reported level of happiness. People who reported higher levels of well-being also reported watching less TV and less use of social networking sites. Majeed et al. , on the other hand, examined the relation between problematic social media use, fear of COVID-19, depression, and mindfulness. Their study, involving 267 Pakistani adults (90 female), suggested trait mindfulness had a buffer effect, reducing the impact of problematic media use and fear of COVID-19 on depression.

Taken together, these findings highlight how using different frames for mass media gives a more expansive view of its positive and negative roles, but also showcase the major concerns in the context of a pandemic crisis. As limitations we highlight the use of cross-sectional designs in most studies, not allowing to establish true inferences of causal relationships. The outcome of some studies may also be limited by the unbalanced number of female and male participants, by the non-probability sampling method used, and by the restricted time frame in which the research occurred. Nevertheless, we are confident that all the selected studies in our Research Topic bring important and enduring contributions to the understanding of how media, individual differences, and social factors intertwine to shape our lives, which can also be useful to guide public policies during these challenging times.

Author Contributions

PA: conceptualization, writing the original draft, funding acquisition, writing—review, and editing. FE: conceptualization, writing—review, and editing. MP: writing—review and editing. NP: conceptualization, writing the original draft, writing—review, and editing. All authors approved the submitted version.

PA and NP received partial support to work on this Research Topic through Fundação para a Ciência e Tecnologia (FCT) with reference to the project PTDC/CCI-INF/29234/2017. MP contribution was supported by the German Research Foundation (DFG, PA847/22-1 and PA847/25-1). The authors are independent of the funders.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We would like to express our gratitude to all the authors who proposed their work, all the researchers who reviewed the submissions to this Research Topic, and to Rob Richards for proofreading the Editorial manuscript.

Keywords: COVID-19, coronavirus disease, mass media, health communication, prevention, intervention, social behavioral changes

Citation: Arriaga P, Esteves F, Pavlova MA and Piçarra N (2021) Editorial: Coronavirus Disease (COVID-19): The Impact and Role of Mass Media During the Pandemic. Front. Psychol. 12:729238. doi: 10.3389/fpsyg.2021.729238

Received: 22 June 2021; Accepted: 30 July 2021; Published: 23 August 2021.

Edited and reviewed by: Eduard Brandstätter , Johannes Kepler University of Linz, Austria

Copyright © 2021 Arriaga, Esteves, Pavlova and Piçarra. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Patrícia Arriaga, patricia.arriaga@iscte-iul.pt

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Coronavirus (COVID-19)

How to publish in coronavirus (covid-19).

• The work is original. The manuscript (or substantial parts of it) must not have been published previously, or be under consideration or review by another journal. Manuscripts that were previously posted on a preprint server such as ArXiv or BioRxiv are welcome.
• At least one author must be formally affiliated with funding from Wellcome (for more details, see our Publication criteria ).
• The reported study meets all applicable research and publication standards . We strongly recommend that you consult our editorial policies for more detail on reporting guidelines and ethical requirements.
• All methodological details and relevant data are made available to allow others to replicate the study, and that the manuscript adheres to appropriate reporting guidelines and community standards. For more detail, please see our policies and Data preparation guidelines .
• All authors have understood Wellcome Open Research’s policies for article publication and its publishing model .
• Your manuscript includes full author and affiliation information, and a conflict of interest statement.

• ORCID allows identification beyond names. Globally, names can be very common, they can change, they can be transliterated into other alphabets and so reliably linking researchers with their research and organizations can be difficult - this is solved through a unique ORCID iD.
• An ORCID iD also allows you to keep a constantly updated digital curriculum vitae. Individuals decide to register, which research activities to connect to their ID, which organizations to allow access, what information to make publicly available, what to share with trusted parties, and what to keep private. Individuals can control their profiles and can change these settings and permissions at any time.
• we collect and store authenticated ORCID iDs for authors and reviewers
• we publicly display the iDs with the iD icon for those authors and reviewers, linked to their ORCID account
• we connect to the user's ORCID record and update it with new published works

Are you a Wellcome-funded researcher?

If you are a previous or current Wellcome grant holder, sign up for information about developments, publishing and publications from Wellcome Open Research.

We'll keep you updated on any major new updates to Wellcome Open Research

The email address should be the one you originally registered with F1000.

You registered with F1000 via Google, so we cannot reset your password.

To sign in, please click here .

If you still need help with your Google account password, please click here .

You registered with F1000 via Facebook, so we cannot reset your password.

If you still need help with your Facebook account password, please click here .

If your email address is registered with us, we will email you instructions to reset your password.

If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.

• All Publications
• Priorities Magazine Spring 2018
• The Next Plague and How Science Will Stop It
• Priorities Magazine Winter 2018
• Priorities Magazine Fall 2017
• Little Black Book of Junk Science
• Priorities Magazine Winter 2017
• Should You Worry About Artificial Flavors Or Colors?
• Should You Worry About Artificial Sweeteners?
• Summer Health and Safety Tips
• How Toxic Terrorists Scare You With Science Terms
• Adult Immunization: The Need for Enhanced Utilization
• Should You Worry About Salt?
• Priorities Magazine Spring 2016
• IARC Diesel Exhaust & Lung Cancer: An Analysis
• Teflon and Human Health: Do the Charges Stick?
• Helping Smokers Quit: The Science Behind Tobacco Harm Reduction
• Irradiated Foods
• Foods Are Not Cigarettes: Why Tobacco Lawsuits Are Not a Model for Obesity Lawsuits
• The Prevention and Treatment of Osteoporosis: A Review
• Are "Low Dose" Health Effects of Chemicals Real?
• The Effects of Nicotine on Human Health
• Traditional Holiday Dinner Replete with Natural Carcinogens - Even Organic Thanksgiving Dinners
• A Primer On Dental Care: Quality and Quackery
• Nuclear Energy and Health And the Benefits of Low-Dose Radiation Hormesis
• Priorities in Caring for Your Children: A Primer for Parents
• Endocrine Disrupters: A Scientific Perspective
• Good Stories, Bad Science: A Guide for Journalists to the Health Claims of "Consumer Activist" Groups
• A Comparison of the Health Effects of Alcohol Consumption and Tobacco Use in America
• Moderate Alcohol Consumption and Health
• Irradiated Foods Fifth Edition
• Media/Contact
• Write For Us

Our COVID-19 Research Summary - 2021         

The published literature on COVID now exceeds 211,000 papers, books, and documents, which include: 22,866 observational studies, 19,591 reviews, 1496 meta-analyses and 781 randomized control trials. These publications comprise the backdrop for our research and writing. The project began in the spring of 2020 based on a limited source of cumulative COVID-19 data and has broadened considerably. Here is what we have learned.

Our objectives remain to

• Describe trends and the geographic extent of the pandemic, including associated predictors
• Evaluate the effectiveness of vaccinations and exposure limitations.
• Provide public health perspectives.

“We know that people’s behavior, the mode of transmission, and the virus‘s characteristics all play a role but we don’t have a clear quantitative understanding of how all of these forces interact. With COVID, the biggest wild card has been human behavior.”

Dr. R. Rosenfeld , Head, Department of Machine Learning, Carnegie Mellon

Our findings along with other COVID studies are observational , focusing on numbers rather than people. Since the annual COVID mortality rate is about is only about 1 in 700, achieving reliable statistics by state would require a cohort approaching a million subjects, each of which would have to be tracked over time in order to estimate exposures. Observational studies are thus the only practical option and uncertainties about causality vs. association are inevitable.

COVID-19 trends continue to defy analysis in large part because of unpredictable variants, the latest of which is still unfolding. However, some aspects remain and will continue to dominate:

• Daily cases, deaths, and incidence of long-haul COVID can be reduced more than 10-fold by vaccination, notwithstanding deterioration over time that requires boosting. 
• Acceptance of vaccination remains a personal choice - a choice that may be associated with personal characteristics including income, education, and political perspective.
• COVID-19 comprises a major cause of death in the United States and may continue to do so.

Findings In 2021 .

January . We showed that COVID-19 cases increased 10-fold, by 30% per month, during the 2020-21 winter; deaths generally followed suit, while case fatality rates (CFRs) decreased up to 10-fold. The Northeast region had shifted from worst to best, so that urban predictor variables like population density were no longer important. Regional rates coalesced in January.

February . The regional trend analyses showed declining cases but not deaths, with steady CFRs. We reported that “No plausible hypotheses have been advanced for the order-of-magnitude increases in cases and deaths since September”, now referred to as the “winter surge”. We compared urban and rural rates and noted the shift towards higher cases, deaths, and CFRs in rural areas.

March . We found that cases and deaths declined while CFRs increased 5-fold. Regional trends, which had ranged 4-fold for cases and deaths now coalesced. We analyzed short-term deaths and found a strong day-of-week effect, probably due to reporting error, but no evidence of important holiday surges.

April . We tried explaining the cyclical behavior of cases in terms of “susceptibles;” predicting an underlying trend of 5000 new cases per million per month. By contrast, the average case rate is now about 8000 cases per million per month - 53 million cases in total. However, after vaccinations got underway and prior to the Delta variant, new cases dropped to levels similar to those at the beginning of the pandemic. CFR’s ranged from about 0.015 in northern regions to 0.05 in the Southwest by April. We showed that Caucasian and mixed-race subjects had far lower COVID death rates than persons of color, and that COVID death rates increased with age at the same rate as non-COVID deaths.

May . COVID rates remained low in May. Comparing states, we reported significant relationships between COVID rates and political preference along with situational factors like household crowding. An increase in Republican voters of 60 percentage points, used as a marker of political perspective, was associated with a doubling of cumulative cases.

June . We revisited our previous consideration of airborne virus transmission, which had been espoused by CDC and the epidemiological community. We estimated ventilation rates and concluded that exposures in a small apartment were likely worse than in subways or aircraft. We also revisited urban-rural differences in more detail and showed that regional COVID rates had continued to coalesce.

July . We did a detailed analysis of vaccination rates and benefits. Daily vaccination rates peaked in April, at about 50% higher in the Northeast than elsewhere. We showed a strong significant decrease in daily state-level cases associated with full vaccination. We estimated unvaccinated case rates to be hundred-folds higher than with full vaccinations. We compared vaccination effects with education, and air pollution concluding that such personal characteristics could also be important. We also showed a negative state-level relationship between voting Republican and cumulative vaccination rates. Interestingly, vaccination rates correlated with COVID rates in 2020  before  mass vaccinations began, and vaccinations at this time, also apparently, reduced mortality not associated with COVID.  “ Could the decision to vaccinate have been more critical than the vaccination itself? ”

August . We reported that COVID case rates showed a sharp upturn, followed by death, likely due to the arrival of the Delta variant. Death rates and cases had decreased steadily until July to about 30 per million or 10,000 per day – a level the CDC considered as a “tolerable” endemic. We have not had these low levels since then. CFRs peaked in July, growing six-fold with substantial geographic variability.

September . We examined cyclical variations in daily infection rates and found substantial heterogeneity. State-level vaccination rates predicted both cases and death; and complete vaccination decreased case and death rates about 100-fold, even in the presence of the Delta variant.

October . We compared October’s COVID rates with those of the 2018-19 influenza to obtain a public health perspective. Total COVID-19 and influenza cases were similar at about 30 million and both were controlled by vaccination.  Compare to influenza, COVID hospitalizations were 4-fold higher and deaths were 20-fold higher - COVID is clearly the more serious disease. We concluded that 178,000 lives may have been saved by COVID-19 vaccination.

November . We continued examining vaccine effects and found no difference in the real-world effectiveness of Pfizer or Moderna vaccines. We found that COVID vaccinations were associated with reduced non-COVID deaths by 3-fold. We built an empirical mathematical model of the temporal variation of cases that fit the existing data very well but grossly underestimated the current situation. We predicted that full vaccinations for the U.S. might reach 72% in the next year, but with a range of 50-90% among states.

December . Cases began a sharp upward trend at years end, with deaths lagging behind. Regional gradients shifted, with Northwest highest and Northeast lowest. Vaccination rates continued to increase slowly, led by the Northeast. Previous beneficial effects of vaccination had been overshadowed by the severity of the Delta variant. We reported that vaccine effectiveness appears to decrease substantially over time. Long-haul COVID, neglected by the epidemiology community, was inversely associated with  vaccination rates and the socioeconomic factors underlying vaccine reluctance or refusal. We estimated trends and the contributions of immunity acquired from previous infection, which we found to be statistically modest.

View the discussion thread.

By Fred Lipfert, PhD

Latest from Fred Lipfert, PhD :

Confused about COVID? Here’s how to read a research paper

Senior Lecturer in Evidence-Based Healthcare and University Ethics Advisor, University of Portsmouth

Disclosure statement

Simon Kolstoe does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.

University of Portsmouth provides funding as a member of The Conversation UK.

View all partners

Scientific evidence can be difficult to understand. Normally we can rely on experts to interpret it for us, or the media to accurately report any interesting new discoveries, but the pandemic has challenged this.

Almost daily we are faced with contradictory views claiming to be “based on the scientific evidence”. But if you’re not an academic, how can you go about checking the evidence for yourself?

Scientific research is communicated in the form of “research papers” published in professional journals. To ensure accuracy, each paper is carefully checked by both editors and outside academic experts in a process called “peer review”. Although peer review is not perfect , it does tend to ensure articles are more reliable compared with those produced in other types of publishing .

Therefore, to judge the scientific evidence for yourself, you need to read and understand peer-reviewed papers. This can be daunting, but if you approach research papers with the right strategy they can be easier to digest.

1. Find the research paper

Following the publication of new research, the results are often summarised by the media. Frustratingly, these summaries seldom provide a link to the original peer-reviewed paper itself.

To find the original paper, one good strategy is to track down the original press release from the university or company releasing the research. You can also use an academic search engine like Google scholar or PubMed to search for recent papers published by the authors, who are normally (although not always) named by journalists.

Historically readers have had to pay to read academic papers, but increasingly research papers are free to readers through “ open access ” arrangements. Unfortunately, if a paper is not open access, there is not much you can do to read it without paying a fee to the publisher.

2. Read the abstract and look at the pictures

Research papers are long and dense with a very different structure compared with articles in the normal media. Media articles start with the most important information in the first few lines and then add background or contextual information as the article progresses.

Research papers start off with an introduction describing the background, then sections describing the methods and results, a discussion (highlighting strengths and weaknesses of the research), and finally the conclusion – often only in the very last few sentences. However, to help speed up reading, a summary or “abstract” is always provided at the beginning.

The abstract is the best place to start (and is almost always available for free). If you are not an expert in the subject area, make sure you look up any words you do not understand, because everything mentioned in the abstract will be key to understanding the paper as a whole.

After reading the abstract you may find you have gathered all the information you need about the research, but if after reading it you still would like to find out more, have a quick look at the pictures, figures and diagrams (if available) to get a better idea of the experiments being reported.

3. Determine how good the journal is and who wrote the paper

After reading the abstract I normally look at who the authors are, what university or company they work for, and how good the journal publishing the paper is.

Academics with a track record of producing high-quality research are a good sign. The first and last authors listed in research papers are often the most important , so look them up to see what else they have produced.

Having the research published in a good journal is also important, because the better journals are able to access more experienced peer reviewers and editors. Here the “impact factor” of a journal is often quoted, which relates to how many other researchers refer to the papers published in it.

However, in recent years impact factors have been strongly criticised as a way of judging journals, even though it’s still true that the best research is published in a fairly small number of journals. One alternative to relying on the impact factor is to simply look up the journal title online to see what researchers say about it. As researchers spend a lot of time discussing which journals are best, this should allow you to find out fairly quickly whether the journal you’re looking at is a reputable one.

4. Read the discussion

If you have got this far you are probably convinced that the research paper is interesting and worth a bit more effort to read. So next, find the part of the paper that discusses the results (often called the discussion) and read through this carefully, flicking back to the methods or results sections if you need to understand in more detail how the experiments were done. Again, look up any terms you do not understand.

5. Read the introduction and check out some of the references

Once you have a good idea of what the paper is reporting, finish off by reading the introduction – this normally provides an overview of why the experiments were conducted in the first place. You should now have a very good idea of what the paper is reporting and some of the wider context.

If you are particularly interested in the topic, look too at some of the key references that the paper quotes. If the paper isn’t brand new, go back to an academic search engine to see whether others have since referenced (or cited) it, and what they are saying about the research.

6. When a paper is not a paper

A word of warning: not every article published in a journal reports new research. Journals also contain news articles, opinion pieces and reviews. These are seldom peer reviewed, and although still written for a professional audience, are not considered primary research.

Another thing to watch out for are versions of research papers that are made available online in advance of being checked by peer reviewers, in a form called “preprints”. Preprints can be very useful for finding out about new results quickly because the peer review and journal publication process can take up to a year. This has been necessary during the pandemic, for example. These preprints are normally clearly labelled, just as a warning that the information in them should not be relied upon in the same way as a full, peer-reviewed research paper.

• Coronavirus
• Academic research
• Coronavirus insights

Events and Communications Coordinator

Assistant Editor - 1 year cadetship

Executive Dean, Faculty of Health

Lecturer/Senior Lecturer, Earth System Science (School of Science)

Sydney Horizon Educators (Identified)

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Correspondence
• Published: 02 November 2020

The impact of COVID-19 on manuscript submissions to Pediatric Research

• Nicole B. Alkhouri 1 ,
• Maria C. Mutka 1 ,
• Matthew P. Stefanak 2 &
• Cynthia Bearer   ORCID: orcid.org/0000-0003-4809-2250 3  

Pediatric Research volume  90 ,  pages 6–7 ( 2021 ) Cite this article

3342 Accesses

10 Citations

3 Altmetric

Metrics details

The emergence of COVID-19 as a global pandemic in early 2020 has led to pervasive changes in the social and occupational behavior of scientists and physician-scientists. Perhaps expectedly, there has been a surge in biological, epidemiological, and clinical publications pertaining to COVID-19 in the worldwide scientific community 1 , 2 , 3 . However, many researchers have had limited or no access to their regular place of work and are under (at least partial) mandatory or voluntary stay-at-home orders. For some, this may have elicited a shift of focus from front-end study components, such as laboratory or clinical projects, to back-end study components, such as data analysis and manuscript preparation, which could elicit a greater amount of journal submissions. This has prompted an enquiry into the submission trends of Pediatric Research during the COVID-19 period. The total monthly submissions from 2017 to 2020 were compared by year to examine the overall trends in submission activity, while articles relating to COVID-19 were identified in order to determine their relative influence on the submissions in 2020.

Monthly manuscript submissions were obtained with permission from the editorial office of Pediatric Research for the period of February 2017 through July 2020. Revised submissions were not included in this analysis. Manuscripts regarding COVID-19 were isolated from the ScholarOne database by searching for articles submitted in 2020 containing at least one of the following terms in its title: “COVID”, “CORONA”, and “SARS”.

R (Version 3.6.2) was used for all statistical analyses and graphing 4 . The total monthly submission data including COVID-related articles were not normally distributed and were thus log-10 transformed to meet the assumption of normality. Monthly data excluding COVID-related articles were also not normally distributed and were natural-log transformed to meet the assumption of normality. Gaussian distribution was assessed using a Wilks-Shapiro test for normality. Homogeneity of variance between years was confirmed using a Levene’s Test 5 . Assessment of differences in mean submissions were averaged across the COVID-impacted period (February-July). One-way analysis of variance (ANOVA) was used to evaluate the differences in transformed averages between years (2017-2020). Post-hoc Tukey HSD tests were used to assess post-hoc comparisons between years.

The results of this analysis are shown in Figure 1. On average, there were 69 submissions to Pediatric Research in 2017, 78 in 2018, 90 in 2019, and 125 in 2020 during the 6-month study period. This led to greater average submissions during 2020 as compared to 2017-19 ( p  < 0.01 for all years), notably an 80.1% increase from 2017 to 2020, a 60.1% increase from 2018 to 2020, and a 39.2% increase from 2019 to 2020. Similar to 2020, the average submissions in 2019 were also greater than those in 2017 ( p  = 0.01), consistent with the general upward trend in submissions by year.

As of July 2020, there have been a total of 82 COVID-related articles submitted to Pediatric Research , with the highest being in April (22) and the lowest in February (1, Fig.  1 ). However, after excluding COVID-related articles, the average submissions in 2020 remained greater than those in 2017 and 2018 ( p  < 0.01), with the exception being only a trend towards a significant increase from 2019 to 2020 ( p  = 0.06).

Non-COVID related submissions are highlighted in dark red, while COVID related submissions are highlighted in light red.

Our results seem to suggest that, although COVID-19 submissions account for a notable proportion of Pediatric Research submissions since the onset of the pandemic, they do not fully explain the significantly higher number of journal submissions during 2020. The perpetuation of the 2020 increase after exclusion of COVID articles lends support to this interpretation. When considering the application of these results to occupational behavior during COVID-19, it appears many scientists may be spending more time writing and preparing manuscripts than before the pandemic began. However, some of the 2020 increase may be explained by the yearly rise in submissions to Pediatric Research , although the average increase from 2019 to 2020 (35) was much higher than between previous years. In summary, we suggest that:

Studies relating to COVID-19 are of interest to the authors, journal, and community of Pediatric Research ; however they have only accounted for a minority of the journal submissions.

COVID-19 may have altered the work focus of some scientists and physician-scientists towards manuscript production and publication.

Pediatric Research has seen a positive trend in yearly submission since 2017, which may partially explain the increase in average submissions during 2020.

As the effects of COVID-19 on infant and child health continue to be elucidated, Pediatric Research and other analogous journals are likely to see substantial submissions related to COVID-19. However, as researchers in many countries begin to transition back into their regular workplace setting, the overall submissions to journals may decline. The specific outcome of submission to Pediatric Research is more unclear, however, particularly when considering its incumbent upward trend in submissions. Moreover, the timeline of return to “business-as-usual” for Pediatric Research ’s submitting authors is uncertain as well, as many reside in the United States, where the COVID-19 pandemic has largely failed to be contained to date. Regardless of future direction, it is important that journals develop open and sustainable approaches to sharing and organizing novel information relating to COVID-19. Encouragingly, there is already much information published regarding COVID-19; however, some have coined this overwhelming amount of literature an ‘infodemic’ that can convolute aspects of the pandemic, especially treatment approaches 6 . Thus, coordination and translation of novel research by authors and journals domestically and internationally will not only aid in the successful suppression of COVID-19, but also lay a groundwork for potential future pandemics, which are only predicted to increase in frequency and intensity due to climate change and other anthropogenically amplified sources 7 , 8 , 9 .

Kambhampati, S. B., Vaishya, R. & Vaish, A. Unprecedented surge in publications related to COVID-19 in the first three months of pandemic: a bibliometric analytic report. J. Clin. Orthop. Trauma 11 (Suppl 3), S304–S306 (2020).

Article   Google Scholar  

Yu, Y. et al. A bibliometric analysis using VOSviewer of publications on COVID-19. Ann. Transl. Med. 8 , 816 (2020).

Article   CAS   Google Scholar  

Gotera, C. Treatment and research lines for the patient with COVID-19. Int. Braz. J. Urol. 46 (Suppl 1), 125–132 (2020).

R Core Team. R: A language and environment for statistical computing (R Foundation for Statistical Computing, Vienna, 2020).

Fox, J. & Weisberg, S. An R Companion to Applied Regression , 3rd edn (Sage, Thousand Oaks, 2019).

Gazendam, A. et al. The “infodemic” of journal publications associated with the novel coronavirus disease. J. Bone Jt. Surg. Am. 102 , e64(1-4) (2020).

Bouma, M. J. & van der Kaay, H. J. The El Niño Southern Oscillation and the historic malaria epidemics on the Indian subcontinent and Sri Lanka: an early warning system for future epidemics? Trop. Med. Int. Health 1 , 86–96 (1996).

Baker-Austin, C. et al. Emerging Vibrio risk at high latitudes in response to ocean warming. Nat . Climate Change 3 , 73–77 (2012).

Google Scholar  

Altizer, S. et al. Climate change and infectious diseases: from evidence to a predictive framework. Science 341 , 514–519 (2013).

Download references

Author information

Authors and affiliations.

Smith College, Northampton, MA, USA

Nicole B. Alkhouri & Maria C. Mutka

Hawai’i Institute of Marine Biology, University of Hawaii at Manoa’s School of Ocean and Earth Science and Technology, Kane’ohe, HI, USA

Matthew P. Stefanak

Division of Neonatology, Case Western Reserve University School of Medicine, Cleveland, OH, USA

Cynthia Bearer

You can also search for this author in PubMed   Google Scholar

Contributions

N.B.A. and M.C.M. collected and organized data for the editorial, in addition to writing and revising its drafts. M.P.S. completed the statistical analysis, offered interpretation of the data, and provided corresponding write-ups. C.B. revised drafts, as well as provided guidance and final approval of the editorial to be published.

Corresponding author

Correspondence to Matthew P. Stefanak .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Statement of Financial Support

Smith College affiliated authors were awarded Smith College’s Praxis Internship funding; William & Lois Briggs Endowment (CB).

There were no patients in this study, so no patient consent was required.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Alkhouri, N.B., Mutka, M.C., Stefanak, M.P. et al. The impact of COVID-19 on manuscript submissions to Pediatric Research . Pediatr Res 90 , 6–7 (2021). https://doi.org/10.1038/s41390-020-01220-9

Download citation

Received : 19 August 2020

Revised : 24 September 2020

Accepted : 03 October 2020

Published : 02 November 2020

Issue Date : July 2021

DOI : https://doi.org/10.1038/s41390-020-01220-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Pediatric research and covid-19: the changed landscape.

• E. J. Molloy
• C. B. Bearer

Pediatric Research (2022)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies