systematic literature review wikipedia

• Library Home
• Library Guides

Systematic Review Service

• What is a Systematic Review
• Systematic Review Process
• Scoping Review

What is a Systematic Review?

A systematic review is a comprehensive literature search, appraisal, and synthesis project that aims to answer a well-defined, specific, research question using existing primary research. Explicit methods are used to minimize bias and produce more reliable findings to inform decision making. Protocols of systematic reviews and meta-analyses allow researchers to plan and document their review methods and they help minimize arbitrary decision making regarding eligibility criteria and data extraction.

Types of Reviews

To determine if a systematic review is the right type of review, view this diagram .

Learn more about the different types of literature reviews:

• Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies . Health information and libraries journal , 26 (2), 91–108. https://doi-org.proxy.uchicago.edu/10.1111/j.1471-1842.2009.00848.x
• Sutton, A., Clowes, M., Preston, L., & Booth, A. (2019). Meeting the review family: exploring review types and associated information retrieval requirements. Health information and libraries journal , 36 (3), 202–222. https://doi-org.proxy.uchicago.edu/10.1111/hir.12276

Suggested Reading

Higgins JPT, Lasserson T, Chandler J, Tovey D, Thomas J, Flemyng E, Churchill R. Methodological Expectations of Cochrane Intervention Reviews . Cochrane: London, Version February 2022.

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.4 (updated August 2023). Cochrane, 2023.

Institute of Medicine. (2011). Finding What Works in Health Care: Standards for Systematic Reviews . Washington, DC: The National Academies Press.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. (2020). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews .

Umscheid, C. A. (2013). A Primer on Performing Systematic Reviews and Meta-analyses . Clinical Infectious Diseases, 57(5), 725-734.

Subject Specialist

Clinical Librarian

• << Previous: Home
• Next: Systematic Review Process >>
• Subjects: Biological Sciences , Medicine
• Tags: biology , medicine
• Updated: Mar 26, 2024 12:13 PM
• URL: https://guides.lib.uchicago.edu/systematicreviewservice
• Report a problem
• Login to LibApps

Open sourcetools

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• v.15(12); 2023 Dec
• PMC10828625

Ten Steps to Conduct a Systematic Review

Ernesto calderon martinez.

1 Digital Health, Universidad Nacional Autónoma de México, Ciudad de Mexico, MEX

Jose R Flores Valdés

2 General Medicine, Universidad Autonoma de Guadalajara, Guadalajara, MEX

Jaqueline L Castillo

Jennifer v castillo.

3 General Medicine, Universidad Autónoma de Guadalajara, Guadalajara, MEX

Ronald M Blanco Montecino

4 Research, University of Texas Southwestern Medical Center, Dallas, USA

Julio E Morin Jimenez

5 General Medicine, Universidad Autónoma del Estado de México, Ciudad de Mexico, MEX

David Arriaga Escamilla

6 Internal Medicine, Universidad Justo Sierra, Ciudad de Mexico, MEX

Edna Diarte

7 Medicine, Universidad Autonoma de Sinaloa, Culiacan, MEX

This article introduces a concise 10-step guide tailored for researchers engaged in systematic reviews within the field of medicine and health, aligning with the imperative for evidence-based healthcare. The guide underscores the importance of integrating research evidence, clinical proficiency, and patient preferences. It emphasizes the need for precision in formulating research questions, utilizing tools such as PICO(S)(Population Intervention Comparator Outcome), PEO (Population Exposure Outcome), SPICE (setting, perspective, intervention/exposure/interest, comparison, and evaluation), and SPIDER (expectation, client group, location, impact, professionals, service and evaluation), and advocates for the validation of research ideas through preliminary investigations. The guide prioritizes transparency by recommending the documentation and registration of protocols on various platforms. It highlights the significance of a well-organized literature search, encouraging the involvement of experts to ensure a high-quality search strategy. The critical stages of screening titles and abstracts are navigated using different tools, each characterized by its specific advantages. This diverse approach aims to enhance the effectiveness of the systematic review process. In conclusion, this 10-step guide provides a practical framework for the rigorous conduct of systematic reviews in the domain of medicine and health. It addresses the unique challenges inherent in this field, emphasizing the values of transparency, precision, and ongoing efforts to improve primary research practices. The guide aims to contribute to the establishment of a robust evidence base, facilitating informed decision-making in healthcare.

Introduction

The necessity of evidence-based healthcare, which prioritizes the integration of top-tier research evidence, clinical proficiency, and patient preferences, is increasingly recognized [ 1 , 2 ]. Due to the extensive amount and varied approaches of primary research, secondary research, particularly systematic reviews, is required to consolidate and interpret this information with minimal bias [ 3 , 4 ]. Systematic reviews, structured to reduce bias in the selection, examination, and consolidation of pertinent research studies, are highly regarded in the research evidence hierarchy. The aim is to enable objective, repeatable, and transparent healthcare decisions by reducing systematic errors.

To guarantee the quality and openness of systematic reviews, protocols are formulated, registered, and published prior to the commencement of the review process. Platforms such as PROSPERO (International Prospective Register of Systematic Reviews) aid in the registration of systematic review protocols, thereby enhancing transparency in the review process [ 5 ]. High-standard reviews comply with stringent peer review norms, ensuring that methodologies are revealed beforehand, thus reducing post hoc alterations for objective, repeatable, and transparent outcomes [ 6 ].

Nonetheless, the practical execution of systematic reviews, particularly in the field of medicine and health, poses difficulties for researchers. To address this, a succinct 10-step guide is offered to both seasoned and novice researchers, with the goal of improving the rigor and transparency of systematic reviews.

Technical report

Step 1: structure of your topic

When developing a research question for a systematic review or meta-analysis (SR/MA), it is essential to precisely outline the objectives of the study, taking into account potential effect modifiers. The research question should concentrate on and concisely explain the scientific elements and encapsulate the aim of the project.

Instruments such as PICO(S)(Population Intervention Comparator Outcome), PEO (Population Exposure Outcome), SPICE (setting, perspective, intervention/exposure/interest, comparison, and evaluation), and SPIDER (expectation, client group, location, impact, professionals, service and evaluation) assist in structuring research questions for evidence-based clinical practice, qualitative research, and mixed-methods research [ 7 - 9 ]. A joint strategy of employing SPIDER and PICO is suggested for exhaustive searches, subject to time and resource constraints. PICO and SPIDER are the frequently utilized tools. The selection between them is contingent on the research’s nature. The ability to frame and address research questions is crucial in evidence-based medicine. The "PICO format" extends to the "PICOTS" (Population Intervention Comparator Outcome Time Setting) (Table  1 ) design. Explicit delineation of these components is critical for systematic reviews, ensuring a balanced and pertinent research question with broad applicability.

This table gives a breakdown of the mnemonic for the elements required to formulate an adequate research question. Utilizing this mnemonic leads to a proper and non-biased search. Examples extracted from “The use and efficacy of oral phenylephrine versus placebo on adults treating nasal congestion over the years in a systematic review” [ 10 ].

RCT, randomized control trial; PICOTS, Population Intervention Comparator Outcome Time Setting

While there are various formats like SPICE and ECLIPSE, PICO continues to be favored due to its adaptability across research designs. The research question should be stated in the introduction of a systematic review, laying the groundwork for impartial interpretations. The PICOTS template is applicable to systematic reviews that tackle a variety of research questions.

Validation of the Idea

To bolster the solidity of our research, we advocate for the execution of preliminary investigations and the validation of ideas. An initial exploration, especially in esteemed databases like PubMed, is vital. This process serves several functions, including the discovery of pertinent articles, the verification of the suggested concept, the prevention of revisiting previously explored queries, and the assurance of a sufficient collection of articles for review.

Moreover, it is crucial to concentrate on topics that tackle significant healthcare challenges, align with worldwide necessities and principles, mirror the present scientific comprehension, and comply with established review methodologies. Gaining a profound comprehension of the research field through pertinent videos and discussions is crucial for enhancing result retrieval. Overlooking this step could lead to the unfortunate unearthing of a similar study published earlier, potentially leading to the termination of our research, a scenario where precious time would be squandered on an issue already thoroughly investigated.

For example, during our initial exploration using the terms “Silymarin AND Liver Enzyme Levels” on PubMed, we discovered a systematic review and meta-analysis discussing the impact of Silymarin on liver enzyme levels in humans [ 11 ]. This discovery acts as a safety net because we will not pursue this identical idea/approach and face rejection; instead, we can rephrase a more sophisticated research question or objective, shifting the focus on evaluating different aspects of the same idea by just altering a part of the PICOTS structure. We can evaluate a different population, a different comparator, and a different outcome and arrive at a completely novel idea. This strategic method guarantees the relevance and uniqueness of our research within the scientific community.

Step 2: databases

This procedure is consistently executed concurrently. A well-orchestrated and orderly team is essential for primary tasks such as literature review, screening, and risk of bias evaluation by independent reviewers. During the study inclusion phase, if disagreements arise, the involvement of a third independent reviewer often becomes vital for resolution. The team’s composition should strive to include individuals with a variety of skills.

The intricacy of the research question and the expected number of references dictate the team’s size. The final team structure is decided after the definitive search, with the participation of independent reviewers dependent on the number of hits obtained. It is crucial to maintain a balance of expertise among team members to avoid undue influence from a specific group of experts. Importantly, a team requires a competent leader who may not necessarily be the most senior member or a professor. The leader plays a central role in coordinating the project, ensuring compliance with the study protocol, keeping all team members updated, and promoting their active involvement.

Establishing solid selection criteria is the foundational step in a systematic review. These criteria act as the guiding principles during the screening process, ensuring a focused approach that conserves time, reduces errors, and maintains transparency and reproducibility, being a primary component of all systematic review protocols. Carefully designed to align with the research question, as in Table  1 , the selection criteria cover a range of study characteristics, including design, publication date, and geographical location. Importantly, they incorporate details related to the study population, exposure and outcome measures, and methodological approaches. Concurrently, researchers must develop a comprehensive search strategy to retrieve eligible studies. A well-organized strategy using various terms and Boolean operators is typically required (Figure  1 ). It involves crafting specific search queries for different online databases, such as Embase, MEDLINE, Web of Science, and Google Scholar. In these searches, we can include singulars and plurals of the terms, misspellings of the terms, and related terms, among others. However, it is crucial to strike a balance, avoiding overly extensive searches that yield unnecessary results and insufficient searches that may miss relevant evidence. In this process, collaborating with a librarian or search specialist improves the quality and reproducibility of the search. For this, it is important to understand the basic characteristics of the main databases (Table  2 ). It is important for the team to include in their methodology how they will collect the data and the tools they will use for their entire protocol so that there is a consensus about this among all of them.

Principal databases where the main articles of the whole body of the research can be gathered. This is an example of specialities and it can be used for the researchers to have a variety of databases to work.

NLM, National Library of Medicine; ICTRP, International Clinical Trials Registry Platform; LILACS, Literatura Latino-Americana e do Caribe em Ciências da Saúde

Boolean operators help break down and narrow down the search. "AND" will narrow your search so you get fewer results. It tells the database that your search results must include every one of your search terms. "OR" means MORE results. OR tells the database that you want results that mention one or both of your search terms. "NOT" means you are telling the database that you wish to have information related to the first term but not the second.

Image credits to authors of the articles (Created on  www.canva.com )

Documenting and registering the protocol early in the research process is crucial for transparency and avoiding duplication. The protocol serves as recorded guidance, encompassing elements like the research question, eligibility criteria, intervention details, quality assessment, and the analysis plan. Before uploading to registry sites, such as PROSPERO, it is advisable to have the protocol reviewed by the principal investigator. The comprehensive study protocol outlines research objectives, design, inclusion/exclusion criteria, electronic search strategy, and analysis plan, providing a framework for reviewers during the screening process. These are steps previously established in our process. Registration can be done on platforms like PROSPERO 5 for health and social care reviews or Cochrane 3 for interventions.

Step 3: search

In the process of conducting a systematic review, a well-organized literature search is a pivotal step. It is suggested to incorporate at least two to four online databases, such as Embase, MEDLINE, Web of Science, and Cochrane. As mentioned earlier, formulating search strategies for each database is crucial due to their distinct requirements. In line with AMSTAR (A Measurement Tool to Assess Systematic Reviews) guidelines, a minimum of two databases should be explored in systematic reviews/meta-analyses (SR/MA), but increasing this number improves the accuracy of the results [ 22 ]. We advise including databases from China as most studies exclude databases from this demographic [ 9 ]. The choice of databases, like Cochrane or ICTRP, is dependent on the review questions, especially in the case of clinical trials. These databases cater to various health-related aspects, and researchers should select based on the research subject. Additionally, it is important to consider unique search methods for each database, as some may not support the use of Boolean operators or quotations. Detailed search strategies for each database, including customization based on specific attributes, are provided for guidance. In general, systematic reviews involve searching through multiple databases and exploring additional sources, such as reference lists, clinical trial registries, and databases of non-indexed journals, to ensure a comprehensive review of both published and, in some instances, unpublished literature.

It is important to note that the extraction of information will also vary among databases. However, our goal is to obtain a RIS, BibText, CSV, bib, or txt file to import into any of the tools we will use in subsequent steps.

Step 4: tools

It is necessary to upload all our reference files into a predetermined tool like Rayyan, Covidence, EPPI, CADIMA, and DistillerSR for the collection and management of records (Table  3 ). The subsequent step entails the elimination of duplicates using a particular method. Duplicates are recognized if they have the same title and author published in the same year or if they have the same title and author published in the same journal. Tools such as Rayyan or Covidence assist in automatically identifying duplicates. The eradication of duplicate records is vital for lessening the workload during the screening of titles and abstracts.

The tools described above use artificial intelligence to help create keywords according to the inclusion and exclusion criteria defined previously by the researcher. This tool will help to reduce the amount of time to rule in or out efficiently.

Step 5: title and abstract screening

The process of a systematic review encompasses several steps, which include screening titles and abstracts and applying selection criteria. During the phase of title and abstract screening, a minimum of two reviewers independently evaluate the pertinence of each reference. Tools like Rayyan, Covidence, and DistillerSR are suggested for this phase due to their effectiveness. The decisions to further assess retrieved articles are made based on the selection criteria. It is recommended to involve at least three reviewers to minimize the likelihood of errors and resolve disagreements.

In the following stages of the systematic review process, the focus is on acquiring full-text articles. Numerous search engines provide links for free access to full-text articles, and in situations where this is not feasible, alternative routes such as ResearchGate are pursued for direct requests from authors. Additionally, a manual search is carried out to decrease bias, using methods like searching references from included studies, reaching out to authors and experts, and exploring related articles in PubMed and Google Scholar. This manual search is vital for identifying reports that might have been initially overlooked. The approach involves independent reviewing by assigning specific methods to each team member, with the results gathered for comparison, discussion, and minimizing bias.

Step 6: full-text screening

The second phase in the screening process is full-text screening. This involves a thorough examination of the study reports that were selected after the title and abstract screening stage. To prevent bias, it is essential that three individuals participate in the full-text screening. Two individuals will scrutinize the entire text to ensure that the initial research question is being addressed and that none of the previously determined exclusion criteria are present in the articles. They have the option to "include" or "exclude" an article. If an article is "excluded," the reviewer must provide a justification for its exclusion. The third reviewer is responsible for resolving any disagreements, which could arise if one reviewer "excludes" an article that another reviewer "includes." The articles that are "included" will be used in the systematic review.

The process of seeking additional references following the full-text screening in a systematic review involves identifying other potentially relevant studies that were not found in the initial literature search. This can be achieved by reviewing the reference lists of the studies that were included after the full-text screening. This step is crucial as it can help uncover additional studies that are relevant to your research question but might have been overlooked in the initial database search due to variations in keywords, indexing terms, or other factors [ 15 ]. 

A PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) chart, also referred to as a PRISMA flow diagram, is a visual tool that illustrates the steps involved in an SR/MA. These steps encompass the identification, screening, evaluation of eligibility, and inclusion of studies.

The PRISMA diagram provides a detailed overview of the information flow during the various stages of an SR/MA. It displays the count of records that were identified, included, and excluded, along with the reasons for any exclusions.

The typical stages represented on a PRISMA chart are as follows: 1) identification: this is where records are discovered through database searches. 2) screening: this stage involves going through the records after removing any duplicates. 3) eligibility: at this stage, full-text articles are evaluated for their suitability. 4) included: this refers to the studies that are incorporated into the qualitative and quantitative synthesis. The PRISMA chart serves as a valuable tool for researchers and readers alike, aiding in understanding the process of study selection in the review and the reasons for the exclusion of certain studies. It is usually the initial figure presented in the results section of your systematic review [ 4 ].

Step 7: data extraction

As the systematic review advances, the subsequent crucial steps involve data extraction from the studies included. This process involves a structured data extraction from the full texts included, guided by a pilot-tested Excel sheet, which aids two independent reviewers in meticulously extracting detailed information from each article [ 28 ]. This thorough process offers an initial comprehension of the common characteristics within the evidence body and sets the foundation for the following analytical and interpretive synthesis. The participation of two to three independent reviewers ensures a holistic approach, including the extraction of both adjusted and non-adjusted data to account for potential confounding factors in future analyses. Moreover, numerical data extracted, such as dichotomous or continuous data in intervention reviews or information on true and false results in diagnostic test reviews, undergoes a thorough process. The extracted data might be suitable for pooled analysis, depending on sufficiency and compatibility. Difficulties in harmonizing data formats might occur, and systematic review authors might resort to communication with study authors to resolve these issues and enhance the robustness of the synthesis. This multi-dimensional data extraction process ensures a comprehensive and nuanced understanding of the included studies, paving the way for the subsequent analysis and synthesis phases.

Step 8: risk of bias assessment

To conduct a risk of bias in medical research, it is crucial to adhere to a specific sequence: choose tools that are specifically designed for systematic reviews. These tools should have proven acceptable validity and reliability, specifically address items related to methodological quality (internal validity), and ideally be based on empirical evidence of bias [ 29 ]. These tools should be chosen once the full text is obtained. For easy organization, it can be helpful to compile a list of the retrieved articles and view the type of study because it is necessary to understand how to select and organize each one. The most common tools to evaluate the risk of bias can be found in Table  4 .

The table summarizes some of the different tools to appraise the different types of studies and their main characteristics.

ROB, risk of bias; RRB, risk of reporting bias; AMSTAR; A Measurement Tool to Assess Systematic Reviews; GRADE, Grading of Recommendations Assessment, Development and Evaluations; ROBINS, risk of bias in non-randomized studies; RCT, randomized controlled trials

After choosing the suitable tool for the type of study, you should know that a good risk of bias should be transparent and easily replicable. This necessitates the review protocol to include clear definitions of the biases that will be evaluated [ 30 ].

The subsequent step in determining the risk of bias is to understand the different categories of risk of bias. This will explicitly assess the risk of selection, performance, attrition, detection, and selective outcome reporting biases. It allows for separate risk of bias ratings by the outcome to account for the outcome-specific variations in detection bias and specific outcome reporting bias.

Keep in mind that assessing the risk of bias based on study design and conduct rather than reporting is very important. Poorly reported studies may be judged as unclear risk of bias. Avoid presenting the risk of bias assessment as a composite score. Finally, classifying the risk of bias as "low," "medium," or "high" is a more practical way to proceed. Methods for determining an overall categorization for the study limitations should be established a priori and documented clearly.

As a concluding statement or as a way to summarize the risk of bias, the assessment is to evaluate the internal validity of the studies included in the systematic review. This process helps to ensure that the conclusions drawn from the review are based on high-quality, reliable evidence.

Step 9: synthesis

This step can be broken down to simplify the concept of conducting a descriptive synthesis of a systematic review. 1) inclusion of studies: the final count of primary studies included in the review is established based on the screening process. 2) flowchart: the systematic review process flow is summarized in a flowchart. This includes the number of references discovered, the number of abstracts and full texts screened, and the final count of primary studies included. 3) study description: the characteristics of the included studies are detailed in a table in the main body of the manuscript. This includes the populations studied, types of exposures, intervention details, and outcomes. 4) results: if a meta-analysis is not possible, the results of the included studies are described. This includes the direction and magnitude of the effect, consistency of the effect across studies, and the strength of evidence for the effect. 5) reporting bias check: reporting bias is a systematic error that can influence the results of a systematic review. It happens when the nature and direction of the results affect the dissemination of research findings. Checking for this bias is an important part of the review process. 6) result verification: the results of the included studies should be verified for accuracy and consistency [ 36 , 37 ]. The descriptive synthesis primarily relies on words and text to summarize and explain the findings, necessitating careful planning and meticulous execution. 

Step 10: manuscript

When working on a systematic review and meta-analysis for submission, it is essential to keep the bibliographic database search current if more than six to 12 months have passed since the initial search to capture newly published articles. Guidelines like PRISMA and MOOSE provide flowcharts that visually depict the reporting process for systematic reviews and meta-analyses, promoting transparency, reproducibility, and comparability across studies [ 4 , 38 ]. The submission process requires a comprehensive PRISMA or MOOSE report with these flowcharts. Moreover, consulting with subject matter experts can improve the manuscript, and their contributions should be recognized in the final publication. A last review of the results' interpretation is suggested to further enhance the quality of the publication.

The composition process is organized into four main scientific sections: introduction, methods, results, and discussion, typically ending with a concluding section. After the manuscript, characteristics table, and PRISMA flow diagram are finalized, the team should forward the work to the principal investigator (PI) for comprehensive review and feedback. Finally, choosing an appropriate journal for the manuscript is vital, taking into account factors like impact factor and relevance to the discipline. Adherence to the author guidelines of journals is crucial before submitting the manuscript for publication.

The report emphasizes the increasing recognition of evidence-based healthcare, underscoring the integration of research evidence. The acknowledgment of the necessity for systematic reviews to consolidate and interpret extensive primary research aligns with the current emphasis on minimizing bias in evidence synthesis. The report highlights the role of systematic reviews in reducing systematic errors and enabling objective and transparent healthcare decisions. The detailed 10-step guide for conducting systematic reviews provides valuable insights for both experienced and novice researchers. The report emphasizes the importance of formulating precise research questions and suggests the use of tools for structuring questions in evidence-based clinical practice.

The validation of ideas through preliminary investigations is underscored, demonstrating a thorough approach to prevent redundancy in research efforts. The report provides a practical example of how an initial exploration of PubMed helped identify an existing systematic review, highlighting the importance of avoiding duplication. The systematic and well-coordinated team approach in the establishment of selection criteria, development of search strategies, and an organized methodology is evident. The detailed discussion on each step, such as data extraction, risk of bias assessment, and the importance of a descriptive synthesis, reflects a commitment to methodological rigor.

Conclusions

The systematic review process is a rigorous and methodical approach to synthesizing and evaluating existing research on a specific topic. The 10 steps we followed, from defining the research question to interpreting the results, ensured a comprehensive and unbiased review of the available literature. This process allowed us to identify key findings, recognize gaps in the current knowledge, and suggest areas for future research. Our work contributes to the evidence base in our field and can guide clinical decision-making and policy development. However, it is important to remember that systematic reviews are dependent on the quality of the original studies. Therefore, continual efforts to improve the design, reporting, and transparency of primary research are crucial.

The authors have declared that no competing interests exist.

Author Contributions

Concept and design:   Ernesto Calderon Martinez, Jennifer V. Castillo, Julio E. Morin Jimenez, Jaqueline L. Castillo, Edna Diarte

Acquisition, analysis, or interpretation of data:   Ernesto Calderon Martinez, Ronald M. Blanco Montecino , Jose R. Flores Valdés, David Arriaga Escamilla, Edna Diarte

Drafting of the manuscript:   Ernesto Calderon Martinez, Julio E. Morin Jimenez, Ronald M. Blanco Montecino , Jaqueline L. Castillo, David Arriaga Escamilla

Critical review of the manuscript for important intellectual content:   Ernesto Calderon Martinez, Jennifer V. Castillo, Jose R. Flores Valdés, Edna Diarte

Supervision:   Ernesto Calderon Martinez

Human Ethics

Consent was obtained or waived by all participants in this study

Animal Ethics

Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Systematic Review | Definition, Example, & Guide

Systematic Review | Definition, Example & Guide

Published on June 15, 2022 by Shaun Turney . Revised on November 20, 2023.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question “What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?”

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

Table of contents

What is a systematic review, systematic review vs. meta-analysis, systematic review vs. literature review, systematic review vs. scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, other interesting articles, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce bias . The methods are repeatable, and the approach is formal and systematic:

• Formulate a research question
• Develop a protocol
• Search for all relevant studies
• Apply the selection criteria
• Extract the data
• Synthesize the data
• Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesizing all available evidence and evaluating the quality of the evidence. Synthesizing means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Systematic reviews often quantitatively synthesize the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesize results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarize and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimize bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

• A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
• If you’re doing a systematic review on your own (e.g., for a research paper or thesis ), you should take appropriate measures to ensure the validity and reliability of your research.
• Access to databases and journal archives. Often, your educational institution provides you with access.
• Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
• Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

• They minimize research bias by considering all available evidence and evaluating each study for bias.
• Their methods are transparent , so they can be scrutinized by others.
• They’re thorough : they summarize all available evidence.
• They can be replicated and updated by others.

Systematic reviews also have a few cons .

• They’re time-consuming .
• They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

• Allow you to more effectively communicate your research to other researchers and practitioners
• Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

• Population(s) or problem(s)
• Intervention(s)
• Comparison(s)

You can rearrange these four components to write your research question:

• What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fifth component, the type of study design . In this case, the acronym is PICOT .

• Type of study design(s)
• The population of patients with eczema
• The intervention of probiotics
• In comparison to no treatment, placebo , or non-probiotic treatment
• The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
• Randomized control trials, a type of study design

Their research question was:

• What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

• Background information : Provide the context of the research question, including why it’s important.
• Research objective (s) : Rephrase your research question as an objective.
• Selection criteria: State how you’ll decide which studies to include or exclude from your review.
• Search strategy: Discuss your plan for finding studies.
• Analysis: Explain what information you’ll collect from the studies and how you’ll synthesize the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

• Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
• Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
• Gray literature: Gray literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of gray literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of gray literature.
• Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

• Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
• Handsearch: Conference proceedings and reference lists of articles
• Gray literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
• Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

• Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
• Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarize what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

• Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
• Your judgment of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomized into the control and treatment groups.

Step 6: Synthesize the data

Synthesizing the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesizing the data:

• Narrative ( qualitative ): Summarize the information in words. You’ll need to discuss the studies and assess their overall quality.
• Quantitative : Use statistical methods to summarize and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analyzed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

• Abstract : A summary of the review
• Introduction : Including the rationale and objectives
• Methods : Including the selection criteria, search method, data extraction method, and synthesis method
• Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
• Discussion : Including interpretation of the results and limitations of the review
• Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

In their report, Boyle and colleagues concluded that probiotics cannot be recommended for reducing eczema symptoms or improving quality of life in patients with eczema. Note Generative AI tools like ChatGPT can be useful at various stages of the writing and research process and can help you to write your systematic review. However, we strongly advise against trying to pass AI-generated text off as your own work.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Prospective cohort study

Research bias

• Implicit bias
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic
• Social desirability bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Turney, S. (2023, November 20). Systematic Review | Definition, Example & Guide. Scribbr. Retrieved July 4, 2024, from https://www.scribbr.com/methodology/systematic-review/

Is this article helpful?

Shaun Turney

Other students also liked, how to write a literature review | guide, examples, & templates, how to write a research proposal | examples & templates, what is critical thinking | definition & examples, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

• Advanced Search

Automatic Quality Assessment of Wikipedia Articles—A Systematic Literature Review

New citation alert added.

This alert has been successfully added and will be sent to:

You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below.

New Citation Alert!

Please log in to your account

Information & Contributors

Bibliometrics & citations, view options, 1 introduction, 2 information quality, 3 methodology.

3.1 Selection through Database Querying

3.1.1 identification..

3.1.2 Screening.

3.1.3 eligibility..

3.1.4 Inclusion.

3.2 selection through citation tracking.

3.3 Data Collection

4 overview of included articles, 4.1 methods.

4.1.1 Actionable Models and Visualization Tools.

4.1.2 multi-language assessment..

4.2 Year of Publication

4.3 Publication Venues

4.4 Publication Influence—References and Citations

4.5 Abstract and Keyword Analysis

4.6 Authors and Affiliations

4.7 Datasets, Source Code, and External Tools

5 Machine Learning Approaches

5.1 Classical Learning

5.2 Deep Learning

6 Article Features and Quality Metrics

6.1 article features.

6.1.1 Content Features.

6.1.2 Style Features.

6.1.3 Readability Features.

6.1.4 History Features.

6.1.5 Network Features.

6.1.6 Popularity Features.

6.1.7 Summary.

6.2 Quality Metrics

6.2.1 stvilia’s iq metrics., 6.2.2 text survival., 7 discussion, 7.1 rq1. what are the most commonly used methods for the automatic quality assessment of wikipedia articles, 7.2 rq2. how can machine learning be best applied to predict article quality, and how do different approaches compare, 7.3 rq3. what are the most common article features and quality metrics used to evaluate article quality in wikipedia how do these features compare, and how do they affect the performance of automatic assessment methods, 7.4 rq4. which common themes and gaps are there in the literature concerning this topic, and how can existing studies be improved to increase the adoption of automatic methods for the quality assessment of wikipedia.

8 Conclusions and Future Work

Index terms.

Applied computing

Document management and text processing

Computing methodologies

Artificial intelligence

Natural language processing

Machine learning

Information systems

Information systems applications

Collaborative and social computing systems and tools

Recommendations

Assessing the quality of wikipedia articles.

Wikipedia is a very important information reference source for the Internet users. Due to the fact that the content of Wikipedia is the collaborative result from a massive number of participants all over the world, the quality of Wikipedia might be ...

Mutual evaluation of editors and texts for assessing quality of Wikipedia articles

In this paper, we propose a method to identify good quality Wikipedia articles by mutually evaluating editors and texts. A major approach for assessing article quality is a text survival ratio based approach. In this approach, when a text survives beyond ...

Indicator of quality for environmental articles on Wikipedia at the higher education level

Wikipedia is important in higher education because students and scholars often use it. Nevertheless, the issue of Wikipedia’s quality is an obstacle for its use at the higher education level. In order to contribute to this discussion, we have proposed ‘...

Information

Published in.

University of Sydney, Australia

Association for Computing Machinery

New York, NY, United States

Publication History

Permissions, check for updates, author tags.

• quality assessment
• information quality

Contributors

Other metrics, bibliometrics, article metrics.

• 0 Total Citations
• 2,112 Total Downloads
• Downloads (Last 12 months) 2,112
• Downloads (Last 6 weeks) 301

View options

View or Download as a PDF file.

View online with eReader .

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Share this publication link.

Copying failed.

Share on social media

Affiliations, export citations.

• Please download or close your previous search result export first before starting a new bulk export. Preview is not available. By clicking download, a status dialog will open to start the export process. The process may take a few minutes but once it finishes a file will be downloadable from your browser. You may continue to browse the DL while the export process is in progress. Download
• Download citation
• Copy citation

We are preparing your search results for download ...

We will inform you here when the file is ready.

Your file of search results citations is now ready.

Your search export query has expired. Please try again.

Reference management. Clean and simple.

How to write a systematic literature review [9 steps]

What is a systematic literature review?

Where are systematic literature reviews used, what types of systematic literature reviews are there, how to write a systematic literature review, 1. decide on your team, 2. formulate your question, 3. plan your research protocol, 4. search for the literature, 5. screen the literature, 6. assess the quality of the studies, 7. extract the data, 8. analyze the results, 9. interpret and present the results, registering your systematic literature review, frequently asked questions about writing a systematic literature review, related articles.

A systematic literature review is a summary, analysis, and evaluation of all the existing research on a well-formulated and specific question.

Put simply, a systematic review is a study of studies that is popular in medical and healthcare research. In this guide, we will cover:

• the definition of a systematic literature review
• the purpose of a systematic literature review
• the different types of systematic reviews
• how to write a systematic literature review

➡️ Visit our guide to the best research databases for medicine and health to find resources for your systematic review.

Systematic literature reviews can be utilized in various contexts, but they’re often relied on in clinical or healthcare settings.

Medical professionals read systematic literature reviews to stay up-to-date in their field, and granting agencies sometimes need them to make sure there’s justification for further research in an area. They can even be used as the starting point for developing clinical practice guidelines.

A classic systematic literature review can take different approaches:

• Effectiveness reviews assess the extent to which a medical intervention or therapy achieves its intended effect. They’re the most common type of systematic literature review.
• Diagnostic test accuracy reviews produce a summary of diagnostic test performance so that their accuracy can be determined before use by healthcare professionals.
• Experiential (qualitative) reviews analyze human experiences in a cultural or social context. They can be used to assess the effectiveness of an intervention from a person-centric perspective.
• Costs/economics evaluation reviews look at the cost implications of an intervention or procedure, to assess the resources needed to implement it.
• Etiology/risk reviews usually try to determine to what degree a relationship exists between an exposure and a health outcome. This can be used to better inform healthcare planning and resource allocation.
• Psychometric reviews assess the quality of health measurement tools so that the best instrument can be selected for use.
• Prevalence/incidence reviews measure both the proportion of a population who have a disease, and how often the disease occurs.
• Prognostic reviews examine the course of a disease and its potential outcomes.
• Expert opinion/policy reviews are based around expert narrative or policy. They’re often used to complement, or in the absence of, quantitative data.
• Methodology systematic reviews can be carried out to analyze any methodological issues in the design, conduct, or review of research studies.

Writing a systematic literature review can feel like an overwhelming undertaking. After all, they can often take 6 to 18 months to complete. Below we’ve prepared a step-by-step guide on how to write a systematic literature review.

• Decide on your team.
• Formulate your question.
• Plan your research protocol.
• Search for the literature.
• Screen the literature.
• Assess the quality of the studies.
• Extract the data.
• Analyze the results.
• Interpret and present the results.

When carrying out a systematic literature review, you should employ multiple reviewers in order to minimize bias and strengthen analysis. A minimum of two is a good rule of thumb, with a third to serve as a tiebreaker if needed.

You may also need to team up with a librarian to help with the search, literature screeners, a statistician to analyze the data, and the relevant subject experts.

Define your answerable question. Then ask yourself, “has someone written a systematic literature review on my question already?” If so, yours may not be needed. A librarian can help you answer this.

You should formulate a “well-built clinical question.” This is the process of generating a good search question. To do this, run through PICO:

• Patient or Population or Problem/Disease : who or what is the question about? Are there factors about them (e.g. age, race) that could be relevant to the question you’re trying to answer?
• Intervention : which main intervention or treatment are you considering for assessment?
• Comparison(s) or Control : is there an alternative intervention or treatment you’re considering? Your systematic literature review doesn’t have to contain a comparison, but you’ll want to stipulate at this stage, either way.
• Outcome(s) : what are you trying to measure or achieve? What’s the wider goal for the work you’ll be doing?

Now you need a detailed strategy for how you’re going to search for and evaluate the studies relating to your question.

The protocol for your systematic literature review should include:

• the objectives of your project
• the specific methods and processes that you’ll use
• the eligibility criteria of the individual studies
• how you plan to extract data from individual studies
• which analyses you’re going to carry out

For a full guide on how to systematically develop your protocol, take a look at the PRISMA checklist . PRISMA has been designed primarily to improve the reporting of systematic literature reviews and meta-analyses.

When writing a systematic literature review, your goal is to find all of the relevant studies relating to your question, so you need to search thoroughly .

This is where your librarian will come in handy again. They should be able to help you formulate a detailed search strategy, and point you to all of the best databases for your topic.

➡️ Read more on on how to efficiently search research databases .

The places to consider in your search are electronic scientific databases (the most popular are PubMed , MEDLINE , and Embase ), controlled clinical trial registers, non-English literature, raw data from published trials, references listed in primary sources, and unpublished sources known to experts in the field.

➡️ Take a look at our list of the top academic research databases .

Tip: Don’t miss out on “gray literature.” You’ll improve the reliability of your findings by including it.

Don’t miss out on “gray literature” sources: those sources outside of the usual academic publishing environment. They include:

• non-peer-reviewed journals
• pharmaceutical industry files
• conference proceedings
• pharmaceutical company websites
• internal reports

Gray literature sources are more likely to contain negative conclusions, so you’ll improve the reliability of your findings by including it. You should document details such as:

• The databases you search and which years they cover
• The dates you first run the searches, and when they’re updated
• Which strategies you use, including search terms
• The numbers of results obtained

➡️ Read more about gray literature .

This should be performed by your two reviewers, using the criteria documented in your research protocol. The screening is done in two phases:

• Pre-screening of all titles and abstracts, and selecting those appropriate
• Screening of the full-text articles of the selected studies

Make sure reviewers keep a log of which studies they exclude, with reasons why.

➡️ Visit our guide on what is an abstract?

Your reviewers should evaluate the methodological quality of your chosen full-text articles. Make an assessment checklist that closely aligns with your research protocol, including a consistent scoring system, calculations of the quality of each study, and sensitivity analysis.

The kinds of questions you'll come up with are:

• Were the participants really randomly allocated to their groups?
• Were the groups similar in terms of prognostic factors?
• Could the conclusions of the study have been influenced by bias?

Every step of the data extraction must be documented for transparency and replicability. Create a data extraction form and set your reviewers to work extracting data from the qualified studies.

Here’s a free detailed template for recording data extraction, from Dalhousie University. It should be adapted to your specific question.

Establish a standard measure of outcome which can be applied to each study on the basis of its effect size.

Measures of outcome for studies with:

• Binary outcomes (e.g. cured/not cured) are odds ratio and risk ratio
• Continuous outcomes (e.g. blood pressure) are means, difference in means, and standardized difference in means
• Survival or time-to-event data are hazard ratios

Design a table and populate it with your data results. Draw this out into a forest plot , which provides a simple visual representation of variation between the studies.

Then analyze the data for issues. These can include heterogeneity, which is when studies’ lines within the forest plot don’t overlap with any other studies. Again, record any excluded studies here for reference.

Consider different factors when interpreting your results. These include limitations, strength of evidence, biases, applicability, economic effects, and implications for future practice or research.

Apply appropriate grading of your evidence and consider the strength of your recommendations.

It’s best to formulate a detailed plan for how you’ll present your systematic review results. Take a look at these guidelines for interpreting results from the Cochrane Institute.

Before writing your systematic literature review, you can register it with OSF for additional guidance along the way. You could also register your completed work with PROSPERO .

Systematic literature reviews are often found in clinical or healthcare settings. Medical professionals read systematic literature reviews to stay up-to-date in their field and granting agencies sometimes need them to make sure there’s justification for further research in an area.

The first stage in carrying out a systematic literature review is to put together your team. You should employ multiple reviewers in order to minimize bias and strengthen analysis. A minimum of two is a good rule of thumb, with a third to serve as a tiebreaker if needed.

Your systematic review should include the following details:

A literature review simply provides a summary of the literature available on a topic. A systematic review, on the other hand, is more than just a summary. It also includes an analysis and evaluation of existing research. Put simply, it's a study of studies.

The final stage of conducting a systematic literature review is interpreting and presenting the results. It’s best to formulate a detailed plan for how you’ll present your systematic review results, guidelines can be found for example from the Cochrane institute .

Introduction to Systematic Reviews

In this guide.

• Introduction
• Lane Research Services
• Types of Reviews
• Systematic Review Process
• Protocols & Guidelines
• Data Extraction and Screening
• Resources & Tools
• Systematic Review Online Course

What is a Systematic Review?

Knowledge synthesis is a term used to describe the method of synthesizing results from individual studies and interpreting these results within the larger body of knowledge on the topic. It requires highly structured, transparent and reproducible methods using quantitative and/or qualitative evidence. Systematic reviews, meta-analyses, scoping reviews, rapid reviews, narrative syntheses, practice guidelines, among others, are all forms of knowledge syntheses. For more information on types of reviews, visit the "Types of Reviews" tab on the left.

A systematic review varies from an ordinary literature review in that it uses a comprehensive, methodical, transparent and reproducible search strategy to ensure conclusions are as unbiased and closer to the truth as possible. The Cochrane Handbook for Systematic Reviews of Interventions  defines a systematic review as:

"A systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question. Researchers conducting systematic reviews use explicit methods aimed at minimizing bias, in order to produce more reliable findings that can be used to inform decision making [...] This involves: the a priori specification of a research question; clarity on the scope of the review and which studies are eligible for inclusion; making every effort to find all relevant research and to ensure that issues of bias in included studies are accounted for; and analysing the included studies in order to draw conclusions based on all the identified research in an impartial and objective way." ( Chapter 1: Starting a review )

What are systematic reviews? from Cochrane on Youtube .

• Next: Lane Research Services >>
• Last Updated: Jun 21, 2024 2:50 PM
• URL: https://laneguides.stanford.edu/systematicreviews

Systematic Literature Reviews: Home

• Systematic Reviews - what are they?
• Meta-ethnography
• Systematic Reviews - recommended reading

Introduction

A  systematic review , or  systematic literature review , is a type of literature review that uses systematic methods to collect secondary data, critically appraise research studies, and synthesize findings qualitatively or quantitatively. Systematic reviews formulate research questions that are broad or narrow in scope, and identify and synthesize studies that directly relate to the systematic review question. They are designed to provide a complete, exhaustive summary of current evidence, published and unpublished, that is "methodical, comprehensive, transparent, and replicable."

An understanding of systematic reviews and how to implement them in practice is highly recommended for professionals involved in the delivery of health care, public health, and public policy. Systematic reviews of randomized controlled trials are key to the practice of evidence-based medicine ,  and a review of existing studies is often quicker and cheaper than embarking on a new study. Contrastingly, systematic reviews of observational studies rank lower in the evidence-based hierarchy. However, another important factor that impacts the quality of the evidence is the accuracy of the methodological design and execution of the systematic review carried out by the authors.

While systematic reviews are often applied in the biomedical or healthcare context, they can be used in other areas where an assessment of a precisely defined subject would be helpful. For example, systematic reviews are becoming increasingly common in management, accounting and finance .  Systematic reviews may examine clinical tests, public health interventions, environmental interventions, social interventions, adverse effects and economic evaluations.

'Systematic review' (2020) Wikipedia.  Available at: https://en.wikipedia.org/wiki/Systematic_review (Accessed: 20 October 2020). 

Assistance offered by the Library

The Library staff are available to assist with your systematic reviews: we can advise on the best databases to use for your particular topic, we provide guidance on searching techniques, we can show you how to set up search alerts so that you keep up to date with the latest research in your field, and we provide assistance in the use of reference management tools. The staff to contact are:

Science & Pharmacy Principal Faculty Services Librarian: Thandiwe Menze Email:  [email protected]

Commerce & Law Principal Faculty Services Librarian: Jill Otto Email:  [email protected]

Principal Librarian: Faculty Liaison Services (Science & Pharmacy)

• Next: Systematic Reviews - what are they? >>
• Last Updated: Feb 11, 2024 10:22 PM
• URL: https://ru.za.libguides.com/c.php?g=1086489

• Boston University Libraries

Systematic Reviews in the Social Sciences

What is a systematic review, difference between a systematic review and a literature review.

• Finding Systematic Reviews
• Conducting Systematic Reviews
• Saving Search Results
• Systematic Review Management Tools
• Citing Your Sources

"A systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question.  It  uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made   (Antman 1992, Oxman 1993) . The key characteristics of a systematic review are:

a clearly stated set of objectives with pre-defined eligibility criteria for studies;

an explicit, reproducible methodology;

a systematic search that attempts to identify all studies that would meet the eligibility criteria;

an assessment of the validity of the findings of the included studies, for example through the assessment of risk of bias; and

a systematic presentation, and synthesis, of the characteristics and findings of the included studies".

Cochrane Handbook for Systematic Reviews of Interventions . (March 2011)

(Original author, Meredith Kirkpatrick, 2021)

Kysh, Lynn (2013): Difference between a systematic review and a literature review . Figshare.https://doi.org/10.6084/m9.figshare.766364.v1  

Related Guides

• Social Work
• Social Work Policy Resources
• Literature Reviews in Social Work
• Systematic Reviews in the Health Sciences
• Next: Finding Systematic Reviews >>
• Last Updated: Jun 27, 2024 3:54 PM
• URL: https://library.bu.edu/systematic-reviews-social-sciences

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

An overview of systematic review

• PMID: 25458624
• DOI: 10.1016/j.jopan.2014.07.002

Systematic review is an invaluable tool for the practicing clinician. A well-designed systematic review represents the latest and most complete information available on a particular topic or intervention. This article highlights the key elements of systematic review, what it is and is not, and provides an overview of several reputable organizations supporting the methodological development and conduct of systematic review. Important aspects for evaluating the quality of a systematic review are also included.

Keywords: systematic review.

Copyright © 2014 American Society of PeriAnesthesia Nurses. Published by Elsevier Inc. All rights reserved.

PubMed Disclaimer

• The value of systematic review: guest editorial. Baker KA. Baker KA. J Perianesth Nurs. 2014 Dec;29(6):453. doi: 10.1016/j.jopan.2014.10.002. Epub 2014 Nov 20. J Perianesth Nurs. 2014. PMID: 25458623 No abstract available.

Similar articles

• What guidance is available for researchers conducting overviews of reviews of healthcare interventions? A scoping review and qualitative metasummary. Pollock M, Fernandes RM, Becker LA, Featherstone R, Hartling L. Pollock M, et al. Syst Rev. 2016 Nov 14;5(1):190. doi: 10.1186/s13643-016-0367-5. Syst Rev. 2016. PMID: 27842604 Free PMC article. Review.
• The methodological quality assessment tools for preclinical and clinical studies, systematic review and meta-analysis, and clinical practice guideline: a systematic review. Zeng X, Zhang Y, Kwong JS, Zhang C, Li S, Sun F, Niu Y, Du L. Zeng X, et al. J Evid Based Med. 2015 Feb;8(1):2-10. doi: 10.1111/jebm.12141. J Evid Based Med. 2015. PMID: 25594108 Review.
• [Volume and health outcomes: evidence from systematic reviews and from evaluation of Italian hospital data]. Amato L, Colais P, Davoli M, Ferroni E, Fusco D, Minozzi S, Moirano F, Sciattella P, Vecchi S, Ventura M, Perucci CA. Amato L, et al. Epidemiol Prev. 2013 Mar-Jun;37(2-3 Suppl 2):1-100. Epidemiol Prev. 2013. PMID: 23851286 Review. Italian.
• [Volume and health outcomes: an overview of systematic reviews]. Davoli M, Amato L, Minozzi S, Bargagli AM, Vecchi S, Perucci CA. Davoli M, et al. Epidemiol Prev. 2005 May-Aug;29(3-4 Suppl):3-63. Epidemiol Prev. 2005. PMID: 16529350 Italian.
• Systematic reviews in laboratory medicine: principles, processes and practical considerations. Horvath AR, Pewsner D. Horvath AR, et al. Clin Chim Acta. 2004 Apr;342(1-2):23-39. doi: 10.1016/j.cccn.2003.12.015. Clin Chim Acta. 2004. PMID: 15026264 Review.
• The effectiveness of patient-centered care vs. usual care in type 2 diabetes self-management: A systematic review and meta-analysis. Asmat K, Dhamani K, Gul R, Froelicher ES. Asmat K, et al. Front Public Health. 2022 Oct 28;10:994766. doi: 10.3389/fpubh.2022.994766. eCollection 2022. Front Public Health. 2022. PMID: 36388341 Free PMC article.
• Potential limitations in systematic review studies assessing the effect of the main intervention for treatment/therapy of COVID-19 patients: An overview. Mohseni M, Ameri H, Arab-Zozani M. Mohseni M, et al. Front Med (Lausanne). 2022 Sep 15;9:966632. doi: 10.3389/fmed.2022.966632. eCollection 2022. Front Med (Lausanne). 2022. PMID: 36203750 Free PMC article.
• Writing a review article - Are you making these mistakes? Daldrup-Link HE. Daldrup-Link HE. Nanotheranostics. 2018 Mar 19;2(2):197-200. doi: 10.7150/ntno.24793. eCollection 2018. Nanotheranostics. 2018. PMID: 29577022 Free PMC article.

Publication types

• Search in MeSH

LinkOut - more resources

Full text sources.

• Elsevier Science
• Ovid Technologies, Inc.

Other Literature Sources

• scite Smart Citations
• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

• Research Process
• Manuscript Preparation
• Manuscript Review
• Publication Process
• Publication Recognition
• Language Editing Services
• Translation Services

Systematic Literature Review or Literature Review?

• 3 minute read
• 51.4K views

Table of Contents

As a researcher, you may be required to conduct a literature review. But what kind of review do you need to complete? Is it a systematic literature review or a standard literature review? In this article, we’ll outline the purpose of a systematic literature review, the difference between literature review and systematic review, and other important aspects of systematic literature reviews.

What is a Systematic Literature Review?

The purpose of systematic literature reviews is simple. Essentially, it is to provide a high-level of a particular research question. This question, in and of itself, is highly focused to match the review of the literature related to the topic at hand. For example, a focused question related to medical or clinical outcomes.

The components of a systematic literature review are quite different from the standard literature review research theses that most of us are used to (more on this below). And because of the specificity of the research question, typically a systematic literature review involves more than one primary author. There’s more work related to a systematic literature review, so it makes sense to divide the work among two or three (or even more) researchers.

Your systematic literature review will follow very clear and defined protocols that are decided on prior to any review. This involves extensive planning, and a deliberately designed search strategy that is in tune with the specific research question. Every aspect of a systematic literature review, including the research protocols, which databases are used, and dates of each search, must be transparent so that other researchers can be assured that the systematic literature review is comprehensive and focused.

Most systematic literature reviews originated in the world of medicine science. Now, they also include any evidence-based research questions. In addition to the focus and transparency of these types of reviews, additional aspects of a quality systematic literature review includes:

• Clear and concise review and summary
• Comprehensive coverage of the topic
• Accessibility and equality of the research reviewed

Systematic Review vs Literature Review

The difference between literature review and systematic review comes back to the initial research question. Whereas the systematic review is very specific and focused, the standard literature review is much more general. The components of a literature review, for example, are similar to any other research paper. That is, it includes an introduction, description of the methods used, a discussion and conclusion, as well as a reference list or bibliography.

A systematic review, however, includes entirely different components that reflect the specificity of its research question, and the requirement for transparency and inclusion. For instance, the systematic review will include:

• Eligibility criteria for included research
• A description of the systematic research search strategy
• An assessment of the validity of reviewed research
• Interpretations of the results of research included in the review

As you can see, contrary to the general overview or summary of a topic, the systematic literature review includes much more detail and work to compile than a standard literature review. Indeed, it can take years to conduct and write a systematic literature review. But the information that practitioners and other researchers can glean from a systematic literature review is, by its very nature, exceptionally valuable.

This is not to diminish the value of the standard literature review. The importance of literature reviews in research writing is discussed in this article . It’s just that the two types of research reviews answer different questions, and, therefore, have different purposes and roles in the world of research and evidence-based writing.

Systematic Literature Review vs Meta Analysis

It would be understandable to think that a systematic literature review is similar to a meta analysis. But, whereas a systematic review can include several research studies to answer a specific question, typically a meta analysis includes a comparison of different studies to suss out any inconsistencies or discrepancies. For more about this topic, check out Systematic Review VS Meta-Analysis article.

Language Editing Plus

With Elsevier’s Language Editing Plus services , you can relax with our complete language review of your systematic literature review or literature review, or any other type of manuscript or scientific presentation. Our editors are PhD or PhD candidates, who are native-English speakers. Language Editing Plus includes checking the logic and flow of your manuscript, reference checks, formatting in accordance to your chosen journal and even a custom cover letter. Our most comprehensive editing package, Language Editing Plus also includes any English-editing needs for up to 180 days.

How to Make a PowerPoint Presentation of Your Research Paper

What is and How to Write a Good Hypothesis in Research?

You may also like.

Descriptive Research Design and Its Myriad Uses

Five Common Mistakes to Avoid When Writing a Biomedical Research Paper

Making Technical Writing in Environmental Engineering Accessible

To Err is Not Human: The Dangers of AI-assisted Academic Writing

When Data Speak, Listen: Importance of Data Collection and Analysis Methods

Choosing the Right Research Methodology: A Guide for Researchers

Why is data validation important in research?

Writing a good review article

Input your search keywords and press Enter.

• Literature review
• 2 Primary and secondary research
• 5 References

A literature review is a synthesis of past academic discussion and research findings.

Primary and secondary research

Literature reviews can be considered secondary research, which may then lead on to original research (primary research). Sometimes the boundary between primary and secondary research is not clear (e.g., a critical review of previously published ideas and data can lead to the discovery of new ideas).

Literature reviews can be [1] :

• Scoping (Munn et al., 2018)
• Traditional or narrative
• Systematic (Munn et al., 2018)
• Meta-analytic
• Meta-synthesis
• Formulating literature reviews (UC-Pharmacy-Research Wikiversity)
• Literature review (Wikipedia)
• Literature review (UC-Pharmacy-Research Wikiversity)

Navigation menu