Skip to Main Content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

Systematic Reviews and Meta-Analyses: Define Scope

This guide is designed to help novice and experienced review teams navigate the systematic review and/or meta-analysis process

The scope of a systematic review is defined through the A. research question and B. eligibility criteria.

The scope will inform every aspect of your systematic review.

 Decorative

The exploratory search is also relevant to scope development and discussed in more detail in the subtab of this page.


A. Research Question

Research QuestionDecorative

As with any project, the first step is to define the goal - in research, the goal is usually defined as a research question

A systematic review must have a well-defined and specific research question(s). To support development, your team may run exploratory searches, (2) speak with others in your field, and/or (3) gather input from stakeholders such as policy makers and community members who may be interested in the findings of your review. 


Research Question Development Tools (e.g., PICO)

Research question development tools, also called 'research question formulation frameworks' or 'concept developers' are simply acronyms that help you consider the details behind your research question. 

The intervention-focused tool PICO, which stands for Population, Intervention, Comparator, and Outcome, is a common example used in systematic reviews. However, you have many more options including the 5W's and H, SPIDER, and CIMO. More about choosing a concept developer!

Example

Your team would like to know if/how socioeconomic factors that lead to health disparities lead to a risk of late diagnosis of colorectal cancer. An intervention your team may consider is holding screening events and follow ups with patients. The outcome of interest is raising awareness about colorectal cancer.

PICO

  • Population: groups with socioeconomic factors leading to health disparities in the United States
  • Intervention: screening events and follow ups with patients
  • Comparator: compared to no intervention
  • Outcome: awareness of colorectal cancer; likelihood of early diagnosis in order to improve likelihood of successful treatment

5W’s and H:

  • Who: groups with socioeconomic factors leading to health disparities
  • What: awareness and diagnosis of colorectal cancer
  • When: studies must include a measurement of the frequency of contact with patients through the intervention
  • Where: studies must be conducted in the United States
  • How: screening events and follow ups with patients

Refined Research Question: Do screening events and follow ups with patients (How/Intervention) increase early diagnosis of colorectal cancer (What/Outcome) for individuals with socioeconomic factors/health disparities in the United States (Who/Population) as compared to no intervention (Comparator)?

B. Eligibility Criteria

Eligibility Criteria

Eligibility criteria, also called inclusion and exclusion criteria, is simply a more detailed definition of scope. Think of it as an extension of your Decorativeresearch question that clearly identifies:

Inclusion Criteria or the set of characteristics that a reference must have to be included in the final synthesis

Exclusion Criteria or the set of characteristics that, if present, would make the reference outside of your scope

Eligibility criteria must be defined before starting the review. By reducing the likelihood of having to make ad hoc decisions during the review, eligibility criteria defined ahead of contributes directly to the cornerstone of reducing likelihood of bias.


Tips for Developing Eligibility Criteria

Having clear, unambiguous, and easy to apply criteria is vital for the review, as all reviewers will need to apply the same criteria in the same way during the eligibility screening.

Try looking at your criteria from the perspective of someone who is unfamiliar with your topic; would these criteria still be clear? Is your criteria relying on assumptions based on your personal background (e.g., discipline, culture, education, age)? Does the criteria make sense given the context of your problem?

ExampleIt is common to only include studies examining a specific 'age-cohort'. For example your eligibility criteria may state "only studies that examine adult populations will be included".

In the U.S., it might be taken for granted that "adult" refers to anyone over "18 years of age or older”. However, the age of legal adulthood varies across nations - if the age in years is the important characteristic here, a reframing the criteria as "...18 years of age or olderwould be more clear.

However, if you were interested in the legal definition of adulthood, your criteria may be more appropriately framed as “...populations considered at the age of maturation or legal adulthood in the country where study was performed”

Also consider more nuanced decisions that the reviewers might need to make when applying the criteria during the eligibility screening.

ExampleWith the 'age' example above, you may also consider whether studies that examines both adult and adolescent / children populations are within your scope. What conditions (e.g., grouped analyses are available so that you can pull out just the results for adult populations) might an article need to meet to be considered?

Methodological Guidance

Cochrane Handbook - Part 2: Core Methods

Chapter 2: Determining Scope and Questions provides guidance for scope development

  • 2.1 Rationale for well-formulated questions
  • 2.2 Aims of reviews of interventions
  • 2.3 Defining the scope of a review question
  • 2.4 Ensuring the review addresses the right questions
  • 2.5 Methods and tools for structuring the review

Chapter 3: Defining the criteria for including studies and how they will be grouped for the synthesis 

  • 3.2 Articulating the review and comparison PICO
  • 3.3 Determining which study designs to include
  • 3.4 Eligibility based on publication status and language

SYREAF Tutorials

Step 1: Developing a protocol

Conducting systematic reviews of intervention questions I: Writing the review protocol, formulating the question and searching the literature. O’Connor AM, Anderson KM, Goodell CK, Sargeant JM. Zoonoses Public Health. 2014 Jun;61 Suppl 1:28-38. doi: 10.1111/zph.12125. PMID: 24905994

Campbell - MECCIR

C1. Formulating review questions (title registration & protocol)

C2. Predefining objectives (title registration & protocol)

C3. Considering potential adverse effects (protocol)

C4. Considering equity and specific populations (protocol)

C5. Predefining unambiguous criteria for participants (protocol)

C6. Predefining a strategy for studies with a subset of eligible participants (protocol)

C7. Predefining unambiguous criteria for interventions and comparators (protocol)

C8. Clarifying role of outcomes (protocol & review / final manuscript)

C9. Predefining study designs (protocol)

C10. Including randomized trials (protocol - effectiveness reviews only)

C11. Justifying choice of study design (protocol)

C12. Including studies regardless of publication status (protocol & review / final manuscript)

C13. Changing eligibility criteria (review / final manuscript

C14. Predefining [primary and secondary] outcomes (protocol)

C15. Choosing outcomes (protocol)

C16. Predefining outcome details (protocol)

C17. Predefining choices from multiple outcome measures (protocol)

C18. Predefining time points of interest (protocol)

CEE Guidelines and Standards for Evidence synthesis in Environmental Management

Section 2. Identifying the need for evidence, determining the Evidence Synthesis type, and establishing a Review Team

2.3 From a problem to a reviewable question: Question generation and formulation

Reporting in Protocol and Final Manuscript

In the Protocol | PRISMA-P

Objectives (Item 7)

Provide an explicit statement of the question(s)the review will address with reference to participants, interventions, comparators, and outcomes (PICO)


Eligibility Criteria (Item 8)

Specify the study characteristics (such as PICO, study design, setting, time frame) and report characteristics (such as years considered, language, publication status) to be used as criteria for eligibility for the review

In the Final Manuscript | PRISMA

Objectives (Item 4)

Essential Items
  • Provide an explicit statement of all objective(s) or question(s) the review addresses, expressed in terms of a relevant question formulation framework (see Booth et al and Munn et al for various frameworks).
  • If the purpose is to evaluate the effects of interventions, use the Population, Intervention, Comparator, Outcome (PICO) framework or one of its variants to state the comparisons that will be made.

Eligibility Criteria (Item 5)

Essential Items
  • Specify all study characteristics used to decide whether a study was eligible for inclusion in the review, that is, components described in the PICO framework or one of its variants, and other characteristics, such as eligible study design(s) and setting(s) and minimum duration of follow-up.
  • Specify eligibility criteria with regard to report characteristics, such as year of dissemination, language, and report status (for example, whether reports such as unpublished manuscripts and conference abstracts were eligible for inclusion).
  • Clearly indicate if studies were ineligible because the outcomes of interest were not measured, or ineligible because the results for the outcome of interest were not reported. Reporting that studies were excluded because they had “no relevant outcome data” is ambiguous and should be avoided.
  • Specify any groups used in the synthesis (such as intervention, outcome, and population groups) and link these to the comparisons specified in the objectives (item #4)
Additional Items

Consider providing rationales for any notable restrictions to study eligibility. For example, authors might explain that the review was restricted to studies published from 2000 onward because that was the year the device was first available.