Skip to Main Content

Systematic Reviews and Meta-Analyses: Where to Search

This guide is designed to help novice and experienced review teams navigate the systematic review and/or meta-analysis process

This page provides more information about where to search.

The short answer: Search anywhere that may contain information that can answer your research question(s)

Where to search

Databases

Because most research questions suited for a systematic review will be answerable with primary peer-reviewed empirical research, academic journal databases will likely be your team's primary resource.

Identifying Relevant Databases

A great place to start is your institutions' list of databases - VT's A-Z List of Databases - and subject guides - VT's Subject Guides. For a systematic review, it is important to search in several databases relevant to your research questions, which may mean searching in databases outside of your primary discipline.

Subject librarians can help you to find databases appropriate for your topic! Find VT subject librarian contact information in the accompanying Subject Guide..

Searching Databases

The exact search 'string' you'll use in each database will depend on the database, because the syntax required by each database will vary. However, the search terms you use and any limitations (e.g., language, geographic location) should remain consistent. Before running your search in each database, it is best to develop a base search strategy, troubleshooting in one or two databases.

For more about designing a search string, go to the How to search tab.

Downloading Results from Databases

In a systematic review, you do not determine the relevance of references in the database itself. Instead, download all references yielded by your search into a citation manager. This process allows for more transparency as the exact yield (count and content) from the comprehensive search is documented and the process for determining relevance is systematic and, in theory, replicable. 

Download LimitsDecorative

Please note that the University Libraries and our vendors implement download limits that may be reached when attempting to perform a comprehensive search via EZProxy. We know these limitations can be frustrating, but they are here to protect services for all users. Learn more by visiting our Collections Management Policy Library Guide.

If you encounter problems with these limits, please contact eataccessproblems@vt.edu

Resource

Gusenbauer M, Haddaway NR. Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Res Synth Methods. 2020 Mar;11(2):181-217. doi: 10.1002/jrsm.1378. Epub 2020 Jan 28. PMID: 31614060; PMCID: PMC7079055

UX Caucus Database Search Tips

Study Registries & Preprint Archives

For a comprehensive search, it's important to locate and include unpublished to the extent possible in order to reduce the risk of missing important emerging information and to avoid risk of publication bias.

Study registries contain unpublished research, which may (or may not) be considered a type of grey literature in your field. Research may be unpublished for several reasons - for example, the research is still in progress or not ready to publish. Sometimes research is unpublished for other reasons, such as negative or null results, or the study was not considered novel (e.g., replications). 

Some health-based study (and clinical trial) registeres include:

Preprints are studies that have not yet been peer-reviewed and/or published by a journal. Preprints exist to get research into the community more quickly; much of this material ends up going through the formal peer review and publishing process. Note: it is important to match preprints with their subsequent publication(s) and to not duplicate findings in your synthesis. 

Some archives to explore include:

Online Resources

It is best practice to supplement primary searching (done in academic databases) by searching other online resources and web browsing. This kind of searching may result in additional primary research and/or grey literature.

Web search engines and specific websites

While general search engines like Google should never be the primary resource for your search, it is a great way to supplement the results of academic database searches. These searches may yield primary research and grey literature.

Companies like Google tailor search results to individual users, which increase the risk of introducing bias, but there are ways to reduce this risk. For example, log out of your account, use 'incognito' mode (or similar), and/or opt for a search engine that doesn't track your data like DuckDuckGo or StartPage.

In contrast to searches in academic databases, it is more often appropriate to apply artificial limits in web searches, as yields tend to be unmanageable. For example, a team may only review the first 200 records (or 20 pages). Search strings themselves may also need to be refined to accommodate character limits in both general search engines and specific websites. Exactly how to search specific websites will vary. Note that you can also use general search engines to search within specific websites or even types of websites (e.g., government sites by specifying a .gov url).

Resources for web browsing

Citation Searching

Citation searching, also referred to as pearl growing, snowballing, and citation chasing, chaining, or tracking, describes two processes: 

Backward citation chasing, where reviewers scan work cited by included references and other relevant reviews, and

Forward citation chasing, where reviewers search for references that cite the references included in your review and other relevant reviews.

In a comprehensive search, we often find existing reviews (e.g., traditional reviews, scoping reviews, restricted/rapid reviews) on the same topic and/or other systematic reviews on similar topics. Though it is not appropriate to synthesize the reviews themselves in your review, be sure to use them in your citation search.

Citation searching applied to included references (references your team has already identified as relevant).  As such, this step does not take place until after reviewing the full-Text of included references.

How to citation search?

Citation searching can be done manually (e.g., reviewing full text) and by using bibliographic databases that index both citations and standard bibliographic data. For example, Web of Science Core Collection provided by Clarivate Analytics, Scopus provided by Elsevier, and Google Scholar (for forward citation searching only).

There are also semi-automated tools available to support this stage, such as Citation Chaser and SpiderCite.

There is now guidance available for determining when and how to conduct and report the citation chasing phase of evidence synthesis reviews:

Hirt, J., Nordhausen, T., Fuerst, T., Ewald, H., & Appenzeller-Herzog, C. (2024). Guidance on terminology, application, and reporting of citation searching: the TARCiS statement. BMJ, e078384. https://doi.org/10.1136/bmj-2023-078384 

Personal Contacts

Contacting relevant authors and other stakeholders can occur during the:

Comprehensive searching phase to collect unpublished research or other grey literature, and

Eligibility screening, critical appraisal, and data extraction phases as follow-up when there is not enough information provided in an original publication to determine relevance, appraise quality, or extract data properly.

Also consider contacting groups, organizations, committees, listservs, etc. 

Example

Example from: Curry, O. S., Rowland, L., Van Lissa, C. J., Zlotowitz, S., McAlaney, J., PhD, & Whitehouse, H. (2019, October 1). Happy to Help? A systematic review and meta-analysis of the effects of performing acts of kindness on the well-being of the actor. Retrieved from osf.io/sey6x, Appendix https://osf.io/6pgzc/ 

Predatory Journals

"Predatory journals-also called fraudulent, deceptive, or pseudo-journals-are publications that claim to be legitimate scholarly journals but misrepresent their publishing practices. Some common forms of predatory publishing practices include falsely claiming to provide peer review, hiding information about article processing charges, misrepresenting members of the journal's editorial board, and other violations of copyright or scholarly ethics." (Elmore & Wetson, 2020) 

For more about predatory journals, check out the GRAD 5124 topic 11.4 Issues in Academic Publishing.

Predatory Journals in Systematic Reviews 

Given the description above, it's no surprise that whether or not to include and how to include publications from predatory publishers in systematic reviews and/or meta-analyses. However, the poor practices of a publisher does not necessarily mean the research published there was done poorly. There have been a few recommendations set forth over the years.

Poliani, et al. (2020) recommend to:

  1. determine whether the journal or publisher is part of COPE, DOAJ, OASP, INASP Journals Online Platform, STM (global voice of scholarly publishing), and
  2. critically appraise reports from uncertain journals or publishers using a tool like Think, Check, Submit

According to Munn, et al., (2021), "Options for systematic reviewers could include excluding all studies from suspected predatory journals, applying additional strategies to forensically examine the results of studies published in suspected predatory journals, setting stringent search limits, and applying analytical techniques (such as subgroup or sensitivity analyses) to investigate the impact of suspected predatory journals in a synthesis."

And Rice, et al. (2021) suggest to include "(1) detail methods for addressing predatory journal articles a priori in a study protocol, (2) determine whether included studies are published in open access journals and if they are listed in the directory of open access journals, and (3) conduct a sensitivity analysis with predatory papers excluded from the synthesis."

Resources

Elmore SA, Weston EH. Predatory Journals: What They Are and How to Avoid Them. Toxicol Pathol. 2020 Jun;48(4):607-610. doi: 10.1177/0192623320920209. Epub 2020 Apr 22. PMID: 32319351; PMCID: PMC7237319.

John D, Polani Chandrasekar R, Lohmann J, Dazy A. Identifying predatory journals in systematic reviews. In: Advances in Evidence Synthesis: special issue Cochrane Database of Systematic Reviews 2020;(9 Suppl 1). https://doi.org/10.1002/14651858.CD202001

Munn Z, Barker T, Stern C, Pollock D, Ross-White A, Klugar M, Wiechula R, Aromataris E, Shamseer L. Should I include studies from "predatory" journals in a systematic review? Interim guidance for systematic reviewers. JBI Evid Synth. 2021 Jun 28;19(8):1915-1923. doi: 10.11124/JBIES-21-00138. PMID: 34171895.

Rice, D.B., Skidmore, B. & Cobey, K.D. Dealing with predatory journal articles captured in systematic reviews. Syst Rev 10, 175 (2021). https://doi-org.ezproxy.lib.vt.edu/10.1186/s13643-021-01733-2

ExampleIt is important to report where you search with sufficient detail for replication. Include information like the database and platform (or vendor that hosts each database) and the coverage of each database that you have access to (usually through your institution, like Virginia Tech). Don't forget to reference reporting guideline (e.g., PRISMA and PRISMA-S) requirements!

 For example, in A Systematic Review of Brainstem Contributions to Autism Spectrum Disorder, the databases searched (or where they searched) and the platform through which they searched those databases is clearly reported in the final manuscript (see below). 

       

 

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License unless otherwise noted on a specific visual or other content item.