Systematic Reviews and Meta-Analyses: Get Started
To "Get Started" let's cover:
Note: these are not necessarily presented in order. For example, the reporting guideline / methodological guidance you use and those on your team may depend on the research question and scope, which may be impacted by the presence of existing reviews.
A. Identify Existing Reviews & In-progress Reviews
Don't Duplicate a Review Effort
Consider the contribution this review will make to your field. If a systematic review and/or meta-analysis has already been done (or is in progress), it is not appropriate to pursue your own systematic review addressing the same question.
B. Methodological Guidance
Methodological guidance, often found in the form of Handbooks or Manuals, describe how to conduct the review. These handbooks are developed by teams of experienced researchers and synthesizers, usually supported by a collaboration or group.
To conduct a proper systematic review and/or meta-analysis, you'll need to follow accepted methodological guidance. Much of the guidance currently available is rooted in health and medicine, but there are existing adaptations for social, animal/food, and environmental sciences. As evidence synthesis expands to other fields, formal guidance for other disciplines will continue to be developed. If you can't find guidance specific to your field, use a handbook or manual that is most applicable.
Note that this library guide is not methodological guidance - rather, we have curated resources to help you through the process, and linked to methodological guidance (and other helpful material) throughout.
Cochrane Collaboration | Cochrane Handbook for Systematic Reviews (health & medicine)
Cochrane is a leader in the field of evidence synthesis, learn more on the next tab!
Guidance from Other Collaborations
- Agency for Healthcare Research and Quality (AHRQ) | Methods Guide for Effectiveness and Comparative Effectiveness Reviews
- Campbell Collaboration | MERCCIR Conduct Standards (social sciences)
- Collaboration for Environmental Evidence | Guidelines and Standards for Evidence synthesis in Environmental Management
- Institute of Medicine (IOM) | Finding What Works in Healthcare: Standards for Systematic Reviews
- Joanna Briggs Institute (JBI) | Manual for Evidence Synthesis (health)
- Systematic Reviews of Qualitative Evidence
- Systematic Reviews of Effectiveness
- Systematic Reviews of Text and Opinion
- Systematic Reviews of Prevalence and Incidence
- Systematic Reviews of Economic Evidence
- Systematic Reviews of Etiology and Risk
- Mixed Methods Systematic Reviews
- Diagnostic Test Accuracy Systematic Reviews
- Systematic Reviews of Measurement Properties
- Systematic Reviews for Animal & Food (SYREAF) | Series of articles: "Conducting systematic reviews of intervention questions" and more
These publications are not full manuals or handbooks, but can supplement the guidance above. This material is especially useful if there is not yet formal guidance available in your discipline or field.
- Systematic literature reviews in software engineering - A systematic literature review (2009)
- Procedures for Conducting Systematic Reviews (computer science; 2004)
- Guidelines for conducting systematic mapping studies in software engineering: An update (2015)
- Systematic Literature Reviews in Engineering Education and Other Developing Interdisciplinary Fields (2014)
- Viewing systematic reviews and meta-analysis in social research through different lenses (2014)
The Cochrane Collaboration gets its namesake from Archie Cochrane who "highlighted the need for evidence in medicine". Today, the Cochrane Collaboration continues to be a leader in the field of evidence synthesis and as such, methodological guidance for other fields tend to use the Cochrane Handbook as a foundation.
The Cochrane Handbook includes 4 parts:
- Part 1: About Cochrane Reviews
- Part 2: Core methods
- Part 3: Specific perspectives in reviews
- Part 4: Other topics
Part 1 addresses various organizational and procedural considerations when undertaking systematic reviews within Cochrane.
Part 2 provides the core methodology for undertaking systematic reviews on the effects of health interventions, with a particular emphasis on reviewing randomized trials.
Part 3 provides considerations for tackling systematic reviews from different perspectives, such as when thinking about specific populations, or complex interventions, or particular types of outcomes. It comprises the following chapters:
Part 4 covers a range of further topics, including reviewing evidence other than straightforward randomized trials. It comprises the following chapters:
Chapter 22: Prospective approaches
Chapter 23: Variants on randomized trials
Chapter 24: Including non-randomized studies
Chapter 25: Risk of bias in non-randomized studies
Chapter 26: Individual participant data
C. Reporting Guidelines
Even though reporting is often thought of as one of the last phases of research, it is best to choose and become familiar with reporting guidelines early. This makes the reporting process easier, as you will already know and have thoroughly documented everything you need to report in the final manuscript (or other output). For proper reporting of a systematic review and/or meta-analysis, you must follow a reporting guideline.
This is the most common, and several PRISMA extensions have been developed, see the next tab for more!
- COASTER | Conduct of Systematic Reviews in Toxicology and Environmental Health Research
- ENTREQ | Enhancing Transparency in Reporting the Synthesis of Qualitative Research
- MECIR | Methodological Expectation of Cochrane Intervention Reviews
- MERCCIR | Methodological Expectations of Campbell Collaboration Intervention Reviews
- MOOSE | Meta-Analysis of Observational Studies in Epidemiology
- NIRO | Non-Interventional, Reproducible, and Open Systematic Review
- ROSES | RepOrting standards for Systematic Evidence Syntheses in environmental research
- SWiM | Synthesis Without Meta-Analyses (Intended to compliment PRISMA)
The Preferred Reporting Items for Systematic Reviews and Meta-Analyses or PRISMA is one of the most well-known, commonly used, and adaptable reporting guidelines currently available for systematic reviews and/or meta-analyses.
PRISMA includes 4 parts:
Provides context for developing the PRISMA Guideline, overview of how it was developed, and how it should be used.
Provides a detailed explanation of each required reporting item that is outlined in the checklist (see below). We recommend reviewing this document in full prior to starting. This library guide is informed by the PRISMA EE.
Outlines 27-items intended to improve transparency in systematic reviews for the team. These items cover all aspects of the manuscript, including title, abstract, introduction, methods, results, discussion, and funding.
Illustrates the flow of references during eligibility screening and review. It is intended to help authors transparently report of how, why, and at what point authors decided to keep or remove references.
In contrast to a traditional literature review, systematic reviews require a team. A minimum of three (3) individual members is recommended to fill the following roles:
2 Reviewers | There must be at least two (2) individuals who will each review every reference for every level of review
1 Tie-Breaker | There must be one (1) individual who rounds out the consensus process or acts as a tie-breaker when resolving disagreements between the two reviewers
Member Roles and Description
There are several other roles and responsibilities that should be considered, including those that follow. Note that several roles/responsibilities may be filled by a single individual with corresponding skills.
Subject Expert | Strong familiarity and comfort with the field(s) related to your research question(s); not all team members need to be an expert in the subject matter
Methodological Expert | Strong familiarity and comfort with the systematic review methodology; ideally having past experience on a different systematic review team
Information Retrieval Expert | Skilled at designing, troubleshooting, executing, and documenting a comprehensive search strategy; familiarity and comfort with relevant databases; often has a background in library and information sciences
Project Manager | Ensures the project stays on track in terms of timeline, budget, and meeting methodological expectations; ideally having experience with systematic review management in particular including methodological requirements as well as software and tools
Statistician | Familiarity with statistical synthesis methods including meta-analyses; ideally having experience running meta-analyses and/or as a member of a different systematic review team
Stakeholder and Community Engagement
Systematic reviews are designed to provide the "evidence-base" in "evidence-based" decision making, policy, practice guidelines, etc. Therefore, you should consider the communities and stakeholders that might be interested in the outcomes of your review, and engage with them throughout the review, especially during the planning phases.
Check out the Cochrane Collaboration's Involving People resource, Dr. Neal Haddaway's Stakeholder and Synthesis repository, and the EPPI Centre's Engaging Stakeholders Evidence and Uncertainty.
- III.3 Reporting of new Cochrane Reviews
An introduction to systematic reviews in animal health, animal welfare, and food safety. O’Connor AM, Sargeant JM. Anim Health Res Rev. 2014 Jun;15(1):3-13. doi: 10.1017/S146625231400005X. PMID: 25605276
Introduction to systematic reviews in animal agriculture and veterinary medicine. Sargeant JM, O’Connor AM. Zoonoses Public Health. 2014 Jun;61 Suppl 1:3-9. doi: 10.1111/zph.12128. PMID: 24905991
Application of systematic review methodology to food and feed safety assessments to support decision making. EFSA Journal 2010; 8(6):1637. [90 pp.]. doi:10.2903/j.efsa.2010.1637.
Finally, the opportunity to publish systematic review protocols, systemic reviews and guidelines in animal health, animal welfare, and food safety.O’Connor AM, Sargeant JM. Anim Health Res Rev. 2014 Jun;15(1):1-2. doi: 10.1017/S1466252314000097. PMID: 25605275
Applicability and feasibility of systematic review for performing evidence-based risk assessment in food and feed safety. Aiassa E, Higgins JP, Frampton GK, Greiner M, Afonso A, Amzal B, Deeks J, Dorne JL, Glanville J, Lövei GL, Nienstedt K, O’connor AM, Pullin AS, Rajić A, Verloo D. Crit Rev Food Sci Nutr. 2015 Jun 7;55(7):1026-34. doi: 10.1080/10408398.2013.769933. PMID: 25191830
Sargeant JM, Rajic A, Read S, Ohlsson A. The process of systematic review and its application in agri-food public health. Prev. Vet. Med. 2006; 75: 141-151. PMID: 16725217
C1. Formulating review questions - "The needs of consumers play a central role in Campbell Reviews and they should play an important role in defining the review question" (title registration & protocol)
C15. Choosing outcomes - "Choose outcomes that are relevant to stakeholders such as consumers, practitioners, and policy makers." (protocol)
Section 2. Identifying the need for evidence, determining the Evidence Synthesis type, and establishing a Review Team
2.1 Determining the need for evidence
2.2 Getting people involved
2.4 Systematic Review or Systematic Map?
2.5 Establishing a Review Team
2.6 Involving stakeholders
2.7 Advisory Groups or Panels
Reporting in Protocol and Final Manuscript
In the Protocol | PRISMA-P
Contact Information (Item 3a)
Provide name, institutional affiliation, and email address of all protocol authors; providing physical mailing address of corresponding author
Contributions (Item 3b)
Describe contributions of protocol authors and identify the guarantor of the review
Amendment (Item 4)
If the report represents an amendment of a previously completed or published protocol, identify as such and indicate what changes were made; otherwise state plan for documenting important protocol amendments
Rationale (Item 6)
Describe the rationale for the review in the context of what is already known
In the Final Manuscript | PRISMA
Support (Item 25)
- Describe sources of financial or non-financial support for the review, specifying relevant grant ID numbers for each funder. If no specific financial or non-financial support was received, this should be stated.
- Describe the role of the funders or sponsors (or both) in the review. If funders or sponsors had no role in the review, this should be declared—for example, by stating, “The funders had no role in the design of the review, data collection and analysis, decision to publish, or preparation of the manuscript.”
Competing Interests (Item 26)
- Disclose any of the authors’ relationships or activities that readers could consider pertinent or to have influenced the review.
- If any authors had competing interests, report how they were managed for particular review processes.