Skip to Main Content

Systematic Reviews and Meta-Analyses: Comprehensive Search

This guide is designed to help novice and experienced review teams navigate the systematic review and/or meta-analysis process

A comprehensive search is a systematic effort to find all available evidence to answer your specific question(s).

The validity and usefulness of a synthesis hinges, in part, on a high-quality comprehensive search. Like all the other stages of a systematic review and/or meta-analysis, the process itself should be replicable.

We cover this content across four subtabs:

(A) Where to Search | (B) How to Search | (C) Grey Literature | (D) What about errata and retractions?


Reporting Guideline for SearchingDecorative - PRISMA Logo

PRISMA-S is a reporting guideline for the search strategy. It should be used in conjunction with a systematic review and/or meta-analysis reporting guideline (e.g., PRISMA). Using this guideline will help you ensure "each component of a search is completely reported and...reproducible".

This section of our Library Guide is informed by the PRISMA-S Reporting Guideline.


Overview of Comprehensive Searching

Decorative

What are you searching for?

First identify the type of material that can answer your question - this may already be part of your eligibility criteria. For most research questions, you will likely need at least peer-reviewed empirical research.


Peer-Reviewed Empirical Research

In some cases, it may make sense to only include peer-reviewed research, or even a specific type of research like randomized controlled trials. Peer-reviewed research should be located systematically so that the search is replicable and comprehensiveness can be reasonably justified. Therefore, a comprehensive search for peer-reviewed literature takes place primarily in academic journal databases. The where and how to search sections of this guide are primarily focused on searching in academic databases to find peer-reviewed research.


Grey Literature

In other cases, grey literature may be required to properly answer a question. Grey literature is a broad term that varies across discipline. Some common examples of grey literature include unpublished research, conference proceedings, government publications, social media content, blogs, newspapers, datasets, etc. Grey literature can rudimentarily be defined as anything that is not peer-reviewed, empirical research

Because of this variation, finding grey lit in a systematic, transparent, and replicable manner can be challenging. Where you search will vary based on what kind of grey lit you're looking for - how you search will vary based on the options available within the interface or database. However, it is important to document your search terms and process to be as systematic, transparent, and replicable as possible.


Where to search?

Once you've identified what kind of material you're looking for, you can identify where to search. This will include academic journal databases at a minimum. Check out the Where to search tab for more!


How to search?

The design of your search strategy will depend on what you're looking for and where you're looking. Check out the How to search tab for more!

Methodological Guidance

Cochrane Handbook - Part 2: Core Methods

Chapter 4: Searching and Selecting Studies provides guidance for both the search and screening/review (link)

  • 4.2 General issues
    • 4.2.1 Role of information specialist/librarian
    • 4.2.2 Minimizing bias
    • 4.2.3 Studies versus reports of studies
  • 4.3 Sources to search
    • 4.3.1 Bibliographic databases
    • 4.3.2 Ongoing studies and unpublished data sources
    • 4.3.3 Trials registers and trials results registers 
    • 4.3.4 Regulatory agency sources and clinical study reports
    • 4.3.5 Other sources
  • 4.4 Designing search strategies
    • 4.4.1 Introduction to search strategies
    • 4.4.2 Structure of a search strategy
    • 4.4.3 Sensitivity versus precision
    • 4.4.4 Controlled vocabulary and text words
    • 4.4.5 Language, date, and document format
    • 4.4.7 Search filters
    • 4.4.6 Identifying fraudulent studies, other retracted publications, errata, and comments 
    • 4.4.8 Peer review of search strategies
    • 4.4.9 Alerts
    • 4.4.10 Timing of searches
    • 4.4.11 When to stop searching 
  • 4.5 Documenting and reporting the search process

SYREAF Protocols 

Step 2: Conducting a search

Conducting systematic reviews of intervention questions I: Writing the review protocol, formulating the question and searching the literature. O’Connor AM, Anderson KM, Goodell CK, Sargeant JM. Zoonoses Public Health. 2014 Jun;61 Suppl 1:28-38. doi: 10.1111/zph.12125. PMID: 24905994

Technical Manual for Performing Electronic Literature Searches in Food and Feed Safety.

Campbell - MECCIR

C19 + C24. Planning the search (protocol)

C25. Searching specialist bibliographic databases (protocol)

C26. Searching for different types of evidence (protocol)

C27. Searching trials registers (protocol)

C28. Searching for grey literature (protocol)

C29. Searching within other reviews (protocol)

C30. Searching reference lists (protocol)

C31. Searching by contacting relevant individuals and organizations (protocol)

C32 Structuring search strategies for bibliographic databases (review / final manuscript)

C33. Developing search strategies for bibliographic databases (review / final manuscript)

C34. Using search filters (review / final manuscript)

C35. Restricting database searches (protocol & review / final manuscript)

C36. Documenting the search process (review / final manuscript)

C37. Rerunning searches (review / final manuscript)

C38. Incorporating findings from rerun searches (review / final manuscript)

C48. Obtaining unpublished data (protocol & review / final manuscript)

"Searching for Studies", the Campbell information retrieval guide

CEE Guidelines and Standards for Evidence synthesis in Environmental Management

Section 5. Conducting a Search

Key CEE Standards for Conduct and Reporting

5.2 Conducting the Search

5.3 Managing References and Recording the Search

5.4 Updating and Amending Searches

Reporting in Protocol and Final Manuscript

In the Protocol | PRISMA-P

Describe all intended information sources (Item 9)

...such as electronic databases, contact with study authors, trial registers or other grey literature sources...with planned dates of coverage


Present draft of search strategy (Item 10)

Having a search strategy peer reviewed may help to increase its comprehensiveness or decrease yield where search terminology is unnecessarily broad.

In the Final Manuscript | PRISMA

Information Sources (Item 6)

Essential Items:
  • Specify the date when each source (such as database, register, website, organisation) was last searched or consulted.
  • If bibliographic databases were searched, specify for each database its name (such as MEDLINE, CINAHL), the interface or platform through which the database was searched (such as Ovid, EBSCOhost), and the dates of coverage (where this information is provided).
  • If study registers (such as ClinicalTrials.gov), regulatory databases (such as Drugs@FDA), and other online repositories (such as SIDER Side Effect Resource) were searched, specify the name of each source and any date restrictions that were applied.
  • If websites, search engines, or other online sources were browsed or searched, specify the name and URL (uniform resource locator) of each source.
  • If organisations or manufacturers were contacted to identify studies, specify the name of each source.
  • If individuals were contacted to identify studies, specify the types of individuals contacted (such as authors of studies included in the review or researchers with expertise in the area).
  • If reference lists were examined, specify the types of references examined (such as references cited in study reports included in the systematic review, or references cited in systematic review reports on the same or a similar topic).
  • If cited or citing reference searched (also called backwards and forward citation searching) were conducted, specify the bibliographic details of the reports to which citation searching was applied, the citation index or platform used (such as Web of Science), and the date the citation searching was done.
  • If journals or conference proceedings were consulted, specify the names of each source, the dates covered and how they were searched (such as handsearching or browsing online).

Present the full search strategies (Item 7)

Essential Items:
  • Provide the full line by line search strategy as run in each database with a sophisticated interface (such as Ovid), or the sequence of terms that were used to search simpler interfaces, such as search engines or websites.
  • Describe any limits applied to the search strategy (such as date or language) and justify these by linking back to the review’s eligibility criteria.
  • If published approaches such as search filters designed to retrieve specific types of records (for example, filter for randomised trials) or search strategies from other systematic reviews, were used, cite them. If published approaches were adapted—for example, if existing search filters were amended—note the changes made.
  • If natural language processing or text frequency analysis tools were used to identify or refine keywords, synonyms, or subject indexing terms to use in the search strategy, specify the tool(s) used.
  • If a tool was used to automatically translate search strings for one database to another, specify the tool used.
  • If the search strategy was validated—for example, by evaluating whether it could identify a set of clearly eligible studies—report the validation process used and specify which studies were included in the validation set.
  • If the search strategy was peer reviewed, report the peer review process used and specify any tool used, such as the Peer Review of Electronic Search Strategies (PRESS) checklist.
  • If the search strategy structure adopted was not based on a PICO-style approach, describe the final conceptual structure and any explorations that were undertaken to achieve it (for example, use of a multi-faceted approach that uses a series of searches, with different combinations of concepts, to capture a complex research question, or use of a variety of different search approaches to compensate for when a specific concept is difficult to define).

PRISMA-S

Database Name (Item 1)

Name each individual database searched, stating the platform for each...There is no single database that is able to provide a complete and accurate list of all studies...


Multi-Database Searching (Item 2)

If databases were searched simultaneously on a single platform, state the name of the platform, listing all of the databases searched...

Study Registries (Item 3)

List any study registries searched...study registries allow researchers to locate ongoing clinical trials and studies that may have gone unpublished


Online Resources and Browsing (Item 4)

Describe any online or print source purposefully searched or browsed (e.g., tables of contents, print conference proceedings, web sites), and how this was done...

Web search engines and specific websites

"...list all websites searched, along with their corresponding web address...if authors used a general search engine, authors should declare whether steps were taken to reduce personalization bias...if review teams choose to review a limited set of results, it should be noted in the text, along with the rationale..."

Conference proceedings:

"...authors must specify the conference names, the dates of conferences included, and the method used to search the proceedings (i.e., browsing print abstract books or using an online source)..."

General browsing:

"When purposefully browsing, describe any method used, the name of the journal or other source, and the time frame covered by the search, if applicable..."


Citation Searching (Item 5)

...can be complicated to describe, but the explanation should clearly state the database used...and describe any other methods used. Authors also must cite the “base” article(s) that citation searching was performed upon, either for examining cited or citing articles...


Personal Contact (Item 6)

Contact methods may vary widely...may include personal contact, web forms, email mailing lists, mailed letters, social media contacts, or other methods...[which are] inherently difficult to reproduce, [so] researchers should attempt to give as much detail as possible...


Other Methods (Item 7)

...declare that the method was used, even if it may not be fully replicable...[include] other additional information sources or search methods used in the methods section and in any supplementary materials...


Full Search Strategies (Item 8)

It is important to document and report the search strategy exactly as run, typically by copying and pasting the search strategy directly as entered into the search platform...repeat the database or resource name, database platform or web address, and other details necessary to clearly describe the resource....Report the full search strategy in supplementary materials as described above. Describe and link to the location of the supplementary materials in the methods section.


Limits and Restrictions (Item 9)

...report any limits or restrictions used or that no limits were used in the abstract, methods section, and in any supplementary materials, including the full search strategies (Item 8)...[and] the justification for any limits used...


Search Filters (Item 10)

...cite any search filter used in the methods section and describe adaptations made to any filter. Include the copied and pasted details of any search filter used or adapted for use as part of the full search strategy (Item 8)...


Prior Work (Item 11)

Sometimes, authors adapt or reuse [previously published search strategies] for different systematic reviews...it is appropriate to cite the original publication(s) consulted.


Updates (Item 12)

If there are no changes in information sources and/or search syntax (Table 2), it is sufficient to indicate the date the last search was run in the methods section and in the supplementary materials.

If there are any changes in information sources and/or search syntax, the changes should be indicated (e.g., different set of databases, changes in search syntax, date restrictions) in the methods section...explain why these changes were made...

If authors use email alerts or other methods to update searches, these methods can be briefly described by indicating the method used, the frequency of any updates, the name of the database(s) used...Report the methods used to update the searches in the methods section and the supplementary materials, as described above.


Dates of Searches (Item 13)

...date of the last search of the primary information sources used...the time frame during which searches were conducted...the initial and/or last update search date with each complete search strategy in the supplementary materials...


Peer Review (Item 14)

Describe the use of peer review in the methods section.


Total Records (Item 15)

...report the total number of references retrieved from all sources, including updates...[such that] if a reader tries to duplicate a search from a systematic review, one would expect to retrieve nearly the same results when limiting to the timeframe in the original review...


Deduplication (Item 16)

...describe [the method] and cite any software or technique used...if duplicates were removed manually, authors should include a description...


In the PRISMA Flowchart

 


Search Summary Table

In addition to the items required by PRISMA and PRISMA-S, Bethel, Rogers, and Abbot (2021) recommend including a search summary table "containing the details of which databases were searched, which supplementary search methods were used, and where the included articles were found."

[Bethel, Rogers, and Abbot (2021) Search Summary Table Template

 


Decorative - Recording(s) available on this topic!

We host two workshops each fall on advanced and comprehensive searching approaches, check out our latest recordings!