Research Guides: Systematic Reviews and Meta-Analyses: Define Scope

The scope of a systematic review is defined through the A. research question and B. eligibility criteria.

The scope will inform every aspect of your systematic review.

The exploratory search is also relevant to scope development and discussed in more detail in the subtab of this page.

A. Research Question

Research Question

As with any project, the first step is to define the goal - in research, the goal is usually defined as a research question.

A systematic review must have a well-defined and specific research question(s). To support development, your team may run exploratory searches, (2) speak with others in your field, and/or (3) gather input from stakeholders such as policy makers and community members who may be interested in the findings of your review.

Research Question Development Tools (e.g., PICO)

Research question development tools, also called 'research question formulation frameworks' or 'concept developers' are simply acronyms that help you consider the details behind your research question.

The intervention-focused tool PICO, which stands for Population, Intervention, Comparator, and Outcome, is a common example used in systematic reviews. However, you have many more options including the 5W's and H, SPIDER, and CIMO. More about choosing a concept developer!

Your team would like to know if/how socioeconomic factors that lead to health disparities lead to a risk of late diagnosis of colorectal cancer. An intervention your team may consider is holding screening events and follow ups with patients. The outcome of interest is raising awareness about colorectal cancer.

PICO:

Population: groups with socioeconomic factors leading to health disparities in the United States
Intervention: screening events and follow ups with patients
Comparator: compared to no intervention
Outcome: awareness of colorectal cancer; likelihood of early diagnosis in order to improve likelihood of successful treatment

5W’s and H:

Who: groups with socioeconomic factors leading to health disparities
What: awareness and diagnosis of colorectal cancer
When: studies must include a measurement of the frequency of contact with patients through the intervention
Where: studies must be conducted in the United States
How: screening events and follow ups with patients

Refined Research Question: Do screening events and follow ups with patients (How/Intervention) increase early diagnosis of colorectal cancer (What/Outcome) for individuals with socioeconomic factors/health disparities in the United States (Who/Population) as compared to no intervention (Comparator)?

B. Eligibility Criteria

Eligibility Criteria

Eligibility criteria, also called inclusion and exclusion criteria, is simply a more detailed definition of scope. Think of it as an extension of your research question that clearly identifies:

Inclusion Criteria or the set of characteristics that a reference must have to be included in the final synthesis

Exclusion Criteria or the set of characteristics that, if present, would make the reference outside of your scope

Eligibility criteria must be defined before starting the review. By reducing the likelihood of having to make ad hoc decisions during the review, eligibility criteria defined ahead of contributes directly to the cornerstone of reducing likelihood of bias.

Tips for Developing Eligibility Criteria

Having clear, unambiguous, and easy to apply criteria is vital for the review, as all reviewers will need to apply the same criteria in the same way during the eligibility screening.

Try looking at your criteria from the perspective of someone who is unfamiliar with your topic; would these criteria still be clear? Is your criteria relying on assumptions based on your personal background (e.g., discipline, culture, education, age)? Does the criteria make sense given the context of your problem?

It is common to only include studies examining a specific 'age-cohort'. For example your eligibility criteria may state "only studies that examine adult populations will be included".

In the U.S., it might be taken for granted that "adult" refers to anyone over "18 years of age or older”. However, the age of legal adulthood varies across nations - if the age in years is the important characteristic here, a reframing the criteria as "...18 years of age or older" would be more clear.

However, if you were interested in the legal definition of adulthood, your criteria may be more appropriately framed as “...populations considered at the age of maturation or legal adulthood in the country where study was performed”

Also consider more nuanced decisions that the reviewers might need to make when applying the criteria during the eligibility screening.

With the 'age' example above, you may also consider whether studies that examines both adult and adolescent / children populations are within your scope. What conditions (e.g., grouped analyses are available so that you can pull out just the results for adult populations) might an article need to meet to be considered?

Note about Language Limits

It is common to find eligibility criteria in systematic reviews that exclude articles outside of the languages of those in the team. In the US, this often means "English only" articles are included. However, this exclusion poses a serious risk of bias toward research produced by countries where English is a dominant language. Therefore, it is best to avoid excluding based on language when you can. Check out our resources at the bottom of the "Find Full Text" box on the Eligibility Screening tab.

Methodological Guidance

Cochrane Handbook - Part 2: Core Methods

Chapter 2: Determining Scope and Questions provides guidance for scope development

2.1 Rationale for well-formulated questions
2.2 Aims of reviews of interventions
2.3 Defining the scope of a review question
2.4 Ensuring the review addresses the right questions
2.5 Methods and tools for structuring the review

Chapter 3: Defining the criteria for including studies and how they will be grouped for the synthesis

3.2 Articulating the review and comparison PICO
3.3 Determining which study designs to include
3.4 Eligibility based on publication status and language

SYREAF Tutorials

Step 1: Developing a protocol

Conducting systematic reviews of intervention questions I: Writing the review protocol, formulating the question and searching the literature. O’Connor AM, Anderson KM, Goodell CK, Sargeant JM. Zoonoses Public Health. 2014 Jun;61 Suppl 1:28-38. doi: 10.1111/zph.12125. PMID: 24905994

Campbell - MECCIR

The Campbell Collaboration has combined their reporting and conduct (methodological) guidelines.

Section 1 - Scope of Review (p.3)

Additional guidance regarding scope

Define a clear review question (Item 1a)
Address relevant issues (Item 1b)
Define clear outcomes of interest (Item 1e)
Align synthesis approach with research question (Item 1f)
If multiple interventions, describe plan for analysis
Indicate relevance to interest holders' needs (Item 1b)
Discuss how issues related to marginalized populations (Item 1c) and adverse effects (Item 1d) will be considered

Section 2 - Eligibility Criteria (p.3)

Establish unambiguous, justified selection criteria; define the relevant
- population (Item g)
- intervention or predictor (Item h)
- comparator (Item i)
- primary and secondary outcomes (Item j)
- study designs (Item k)
- time frame, geographic boundaries, etc. (Item l)
Do not exclude due to missing data
If excluding due to risk of bias, justify

CEE - Guidelines and Standards for Evidence synthesis in Environmental Management

Section 2. Identifying the need for evidence, determining the Evidence Synthesis type, and establishing a Review Team

2.3 From a problem to a reviewable question: Question generation and formulation

Reporting in Protocol and Final Manuscript

Protocol
Final Manuscript

In the Protocol | PRISMA-P

Objectives (Item 7)

Provide an explicit statement of the question(s)the review will address with reference to participants, interventions, comparators, and outcomes (PICO)

Eligibility Criteria (Item 8)

Specify the study characteristics (such as PICO, study design, setting, time frame) and report characteristics (such as years considered, language, publication status) to be used as criteria for eligibility for the review

In the Final Manuscript | PRISMA

Objectives (Item 4)

Essential Items

Provide an explicit statement of all objective(s) or question(s) the review addresses, expressed in terms of a relevant question formulation framework (see Booth et al and Munn et al for various frameworks).
If the purpose is to evaluate the effects of interventions, use the Population, Intervention, Comparator, Outcome (PICO) framework or one of its variants to state the comparisons that will be made.

Eligibility Criteria (Item 5)

Essential Items

Specify all study characteristics used to decide whether a study was eligible for inclusion in the review, that is, components described in the PICO framework or one of its variants, and other characteristics, such as eligible study design(s) and setting(s) and minimum duration of follow-up.
Specify eligibility criteria with regard to report characteristics, such as year of dissemination, language, and report status (for example, whether reports such as unpublished manuscripts and conference abstracts were eligible for inclusion).
Clearly indicate if studies were ineligible because the outcomes of interest were not measured, or ineligible because the results for the outcome of interest were not reported. Reporting that studies were excluded because they had “no relevant outcome data” is ambiguous and should be avoided.
Specify any groups used in the synthesis (such as intervention, outcome, and population groups) and link these to the comparisons specified in the objectives (item #4)

Additional Items

Consider providing rationales for any notable restrictions to study eligibility. For example, authors might explain that the review was restricted to studies published from 2000 onward because that was the year the device was first available.