Systematic Reviews and Meta-Analyses: How to Search
This page provides more information about how to design a search strategy.
The short answer: Start by creating a base search strategy, then translate across all your sources
Designing a Comprehensive Search
Overview of How to Search
In a systematic review and/or meta-analysis, the search should collect all potentially relevant material that is available. In other words, the search must be comprehensive.
Though exact strategies will vary according to where you search, you can start by designing a base search strategy by following these steps:
- Identify important concepts
- Create exhaustive list of synonyms
- Test out natural language terms to confirm usefulness and find controlled vocabulary
- Join synonyms within a concept using the OR operator
- Join concepts using the AND operator
Check out the content below for more about designing a comprehensive search strategy.
Resource
Bramer, W. M., De Jonge, G. B., Rethlefsen, M. L., Mast, F., & Kleijnen, J. (2018). A systematic approach to searching: An efficient and complete method to develop literature searches. Journal of the Medical Library Association, 106(4). https://doi.org/10.5195/JMLA.2018.283
More detail
Choose concepts to search for
Concepts come directly from your research question and eligibility criteria - they are the basic ideas underlying your scope. However, not all concepts will be appropriate to include in the search strategy.
Bramer, et. al., (2018) suggest mapping concepts on a scale of specificity and importance (see image to the left). Ideally, a search will only include specific, important terms. In some cases, it may be necessary to use more general terms. However, unimportant terms should never be included in the search strategy.
Guiding Consideration
When considering whether a concept should be included or not, ask yourself: Is it possible for an article to be relevant and not contain terms related to this concept? If yes, the concept should not be included. If no, the concept would be appropriate to include in the search. This logic can also be used when determining which terms should be included in your search strategy.
Exhaustive Synonyms
Some concepts are well-defined, meaning a fairly uniform set of terms related to that concept are used within a field. But this is not always the case, and identifying an exhaustive list of synonyms can be difficult.
Process
Start by identifying synonyms based on your team's preexisting knowledge - but never stop here!
Look at the titles and abstracts of seminal articles and other relevant reviews to find terms, spellings, etc. that you may not have considered. You may also look to collaborations and experts in your field who may be able to provide feedback on the comprehensiveness of your terms. LitsearchR is a tool that can partially automate this process.
Grames, EM, AN Stillman, MW Tingley, and CS Elphick (2019). An automated approach to identifying search terms for systematic reviews using keyword co-occurrence networks. Methods in Ecology and Evolution 10: 1645-1654. https://doi.org/10.1111/2041-210X.13268
Finally, controlled vocabulary descriptions often include a list of relevant terms to consider adding to your list of natural language terms. See the next tab for more details about controlled vocabulary. Hedges can also be helpful in identifying synonyms!
Considerations
Depending on your scope (e.g., geographic region, timeframe) be sure to consider other languages, alternate spellings (e.g., US v. UK English), and changes in terminology over time.
Video: Developing Keywords
If you're new to developing keywords, this video is a great place to start:
Controlled Vocabulary
Controlled vocabulary, also known as subject headings, thesaurus terms, or indexed terms describes predefined terms established by the host of a database that are manually applied to articles related to that term. This is a normalizing tool used to curate all related material, despite the unique terms authors use. In effect, searching for a controlled vocabulary term will result in any article categorized under that term, even if the original authors didn't use that exact term. This way, you are less likely to miss relevant results that don't use your exact terminology.
Unfortunately, because controlled vocabulary is manually applied to articles, if you rely solely on controlled vocabulary, you may miss material that has not yet been processed (e.g., new publications, older backlogged material). Therefore, always use controlled vocabulary with natural language terms.
|
Fold relevant MeSH terms into your search strategy, combining them with natural language terms using the OR operator. Or automatically search for MeSH terms here! Another example is EBSCO's Comprehensive Subject Index (CSI). The subject Thesaurus located at the top of EBSCOHost database advanced search feature and are database-specific subsets of the CSI vocabulary, as seen below. Note, you have to select your databases first for the thesaurus for that database to be accessible.
Start by searching the relevant thesaurus or thesauri to find relevant CSI terms. Fold relevant CSI terms into your search strategy by combining them with the natural language terms in each respective concept using OR operators. |
In health and medicine, the National Library of Medicine's (NLM) 
If you're searching a database(s) that uses Medical Subject Heading (MeSH) terms, quickly identify terms relevant to your topic using the MeSH On Demand tool from the National Library of Medicine (NLM).
Build the Search Statement
Once you've identified all of the relevant terms in both natural language and controlled vocabulary*, it's time to structure the search string. This is done using boolean operators, parentheses, quotations, and potentially many more syntax operators. Below, we describe some important details related to search string development.
*remember, controlled vocabulary varies across databases
|
Component |
Description |
|
Fields to search |
The field in which you search for each concept (and/or term) will vary according to the scope and goal of your review. In most cases, it is appropriate to search only in the title, abstract, and keywords. This practice assumes that if the term is not located in one of these spaces, it is unlikely that the reference will be relevant. |
|
Boolean Operators |
Boolean operators are standard logic operators. They are inserted between two terms or groups of terms to indicate a relationship: AND - return only references that contain both terms OR - return references that contain at least one of the terms, but not necessarily both NOT - return references that contain the preceding term, but not the term following the NOT operator Important Note: NOT operators can be tricky to use, and you can unintentionally omit relevant references. Therefore, NOT shouldn't be used in a final comprehensive search strategy. However, the NOT operator can be useful when testing out terms and concepts. For example, if you want to see the impact of using or removing a term, you can run (1) the original search, with the term in question, (2) NOT, (3) The new search, without the term. The result of this query are articles you would miss by not including that term. Reviewing these can help your team determine whether the term is worth including. |
|
Parentheses |
Parenthesis tell the search engine how to group terms. Use parenthesis to join similar terms joined by OR operators. In more complex searches, parenthesis allows for nesting. |
|
Quotations |
Quotations tell the search engine to search for an exact set of terms. This is particularly useful when searching for two or more term words. If you don't use quotations, the search engine will search for each word on its own, effectively joining them with an OR operator. For example, if you search for systematic review without quotations, you'll retrieve references that use the term "systematic" OR "review". If you join the terms with quotations "systematic review" you'll only retrieve references that use that exact phrase. |
|
Truncation and Wildcards |
Often symbolized as an asterisk (*), truncation tells the search engine to search for all references that contain the root term accompanied by any variation of term endings. For example, if we want to find references with the term review, reviews, reviewing, and reviewed, we could achieve this using a single search term review*. Sometimes the root of a word will many irrelevant terms. For example, searching for cat* to capture both cat and cats, will also bring back articles that use terms like catastrophic, cataracts, catapult, etc. so it is important to place the asterisk strategically. Wildcards, often symbolized as a question mark (?), function the same way as truncation, but can be used in the middle of a term. For example, behavio?r would return both behavior and behaviour and gr?y would return both grey and gray. Some databases offer both wildcards and truncation, some only offer one, and sometimes this feature is not available at all. |
|
Automatic Term Mapping / Smart Search |
Some databases (e.g., PubMed) have a features meant to increase the efficiency of novice searches by automatically mapping the terms you use to synonyms not included in your search. As the name suggests, this feature automatically tells the search engine to search for other variations of the term(s) searched. Note that using truncation may turn off ATM. There is no hard rule about when to use ATM or truncation. Instead, it is important that you understand how the ATM functions in that database and with your particular search terms. |
Video: From Keywords to Database
If this is your first time building a search string, check out this video.
Limiting & Using Hedges
Limiting the search (e.g., by year of publication, publication type) is discouraged, as the goal is to be as comprehensive as possible. However, sometimes it is necessary to limit the search in a way that aligns with your scope. When it comes to limiting, there are two basic approaches - using (1) the filters built-into the databases, and (2) hedges. 
The built-in filters (like those highlighted in this image from PubMed) are manually applied to the content in a database. Therefore, they are inappropriate to use for systematic reviews, as older and newer material that has not yet been processed will be missed.
What is a hedge?
Hedges are validated sets of terms that aim to retrieve specific content, often including natural language terms and controlled vocabulary. These search strings are tailored to a specific database and intended for use beyond a single project.
Hedges, which can also be called filters, are a more systematic, comprehensive counterpart to the filters/limiters built into databases often found on the side of a results page, as seen in the example from PubMed. For an example of a hedge, check out the Canadian Health Libraries Association hedge for studies focused on adolescent and young adult populations.
Both built-in filters and hedges can also be helpful when identifying synonyms!
Finding Hedges
Though there won't be hedges available for every discipline or topic, you can search in a web browser for each of your concepts accompanied by the term "hedge" or "filter". Here are some (mostly health) collections of to get you started!
Collections of Hedges
- InterTASC Information Specialists'' Sub-Group (ISSG) - ISSG Search Filter Resource
- MD Anderson Cancer Center Library Guide - Literature Search Hedges & Filters Guide
- University of Alberta - Health Science Search Filters Library Guide
- American University of Beirut - Systematic Reviews: Health Sciences - Search Filters/Hedges
- Library of Search Strategy Resources (LSSR) from the European Association for Health Information and Libraries
- Health Improvement Scotland - SIGN Search Filters
- McMaster - Health Knowledge Refinery
Translating across databases
Translating in this case is mostly about syntax, or how the database interprets your search query. Databases may share common search syntax features like boolean operators , parentheses, and quotations, however many features vary in terms of availability and syntax.
For example, you may be able to truncate in most academic journal databases. In some databases the symbol for truncation may be an asterisk (*) and in others, a question mark (?). In another example, Proximity operators, which allow you to search for terms that are within a certain number of words apart, are only available in some databases, but not all.
Controlled vocabulary will be unique to both the scope of your review and each database.
|
Adjusting field codesPossibly the most common syntax feature that will need to be adjusted for each database in a comprehensive search is the 'field code' for where to search. PubMed allows for both title and abstracts to be searched by applying a single identifier - [tiab]. A single-word search in titles and abstracts in PubMed might look like this: influenza[tiab] In another database, for example, EBSCOHost, you may no longer have the option to search both titles and abstracts with the same identifier. To run the same search, it would like like this, with TI indicating a search for 'influenza' in titles, and AB for 'influenza' in abstracts: (TI influenza OR AB influenza) |
Using Word Macros for Translation
As you might imagine, making these adjustments for long search strings across several databases can be time consuming and tedious. Wichor Bramer, et al., (2018) outlined the differences in syntax across the top 5 most used databases for biomedical literature (Table 1) and describe a translation process (Item 14) aided by macros in MS Word. If you will be using these databases, you can set up macros by following these instructions!
Using Polyglot from the SR-Accelerator for Translation across Health Sciences Databases
Polyglot is a semi-automated tool available through the SR-Accelerator that allows you to translate your search strategy to databases commonly used in health science systematic reviews. However, there are more general purpose databases Scopus and platforms like Web of Science options that you could use for topics outside of health sciences!
Download All Results
After executing the search in each database, you'll download all of the results into a citation manager. Note, there may be limits to how many results can be downloaded at once. Downloading all of the results instead of cherry picking seemingly relevant articles from the database is an important mechanism for reducing bias. However, this step is also helpful in terms of project management, as it is much easier to keep track of all results in one central library or even a single RIS file.
In the next stages of the systematic review, you'll systematically process all of these articles through the eligibility screening, critical appraisal, and data extraction.
|
|
More about the Process
Peer Review
According to Cochrane, Part 2: Chapter 4, Section 4.8:
Peer review of search strategies is increasingly recognized as a necessary step in designing and executing high-quality search strategies to identify studies for possible inclusion in systematic reviews.
Get your search peer-reviewed!
Library and information science professionals are a great resource for peer-reviewing a completed search strategy. Peer reviewers may rely on guidelines such as the PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. Check out the Evidence Synthesis Services (ESS) current support offerings.
Updates
According to the Cochrane Handbook 4.4.10 Timing of Searches:
Searches for all the relevant databases should be rerun prior to publication, if the initial search date is more than 12 months (preferably six months) from the intended publication date (see MECIR Box 4.4.g). This is also good practice for searches of non-database sources. The results should also be screened to identify potentially eligible studies. Ideally, the studies should be incorporated fully in the review.
Additional Guidance
Bramer, W., & Bain, P. (2017). Updating search strategies for systematic reviews using EndNote. Journal of the Medical Library Association, 105(3). https://doi.org/10.5195/JMLA.2017.183
Documenting the Search
Documenting the search with enough detail that another team could feasibly replicate it is a requirement for systematic reviews and/or meta-analyses. PRISMA-S outlines information that is necessary to report in the final manuscript. It also helps to keep thorough documentation throughout the search development and execution.
Tools for documenting search development
While developing the search strategy, you should also keep a search journal to track sources searched, when a search is executed, the exact search statement used, limits, results, etc. You can make a copy of this search strategy journal (template) from the Evidence Synthesis Services (ESS) team to track when, how, and where you searched! This is a great way to increase replicability and transparency of the final manuscript (e.g., by including as supplemental material), as well as keeping track of your team's progress mid-review.
Once you've finalized your search approach, you can share it publicly. For example the searchrxiv, hosted by CABI, is available as a repository to share search strings from any a review in any discipline.
|
It is best practice to report any information that is useful to determining the validity of your search strategy (e.g., peer review) and increasing the search replicability. For example, in A Systematic Review of Brainstem Contributions to Autism Spectrum Disorder, the initial (or exploratory) search is described and peer-reviewers are named. Don't forget to reference reporting guideline (e.g., PRISMA and PRISMA-S) requirements!
|
