A comprehensive search is a systematic effort to find all available evidence to answer your specific question(s).
The validity and usefulness of a synthesis hinges, in part, on a high-quality comprehensive search. Like all the other stages of a systematic review and/or meta-analysis, the process itself should be replicable.
We cover this content across four subtabs:
(A) Where to Search | (B) How to Search | (C) Grey Literature | (D) What about errata and retractions?
PRISMA-S is a reporting guideline for the search strategy. It should be used in conjunction with a systematic review and/or meta-analysis reporting guideline (e.g., PRISMA). Using this guideline will help you ensure "each component of a search is completely reported and...reproducible".
This section of our Library Guide is informed by the PRISMA-S Reporting Guideline.
First identify the type of material that can answer your question - this may already be part of your eligibility criteria. For most research questions, you will likely need at least peer-reviewed empirical research.
In some cases, it may make sense to only include peer-reviewed research, or even a specific type of research like randomized controlled trials. Peer-reviewed research should be located systematically so that the search is replicable and comprehensiveness can be reasonably justified. Therefore, a comprehensive search for peer-reviewed literature takes place primarily in academic journal databases. The where and how to search sections of this guide are primarily focused on searching in academic databases to find peer-reviewed research.
In other cases, grey literature may be required to properly answer a question. Grey literature is a broad term that varies across discipline. Some common examples of grey literature include unpublished research, conference proceedings, government publications, social media content, blogs, newspapers, datasets, etc. Grey literature can rudimentarily be defined as anything that is not peer-reviewed, empirical research.
Because of this variation, finding grey lit in a systematic, transparent, and replicable manner can be challenging. Where you search will vary based on what kind of grey lit you're looking for - how you search will vary based on the options available within the interface or database. However, it is important to document your search terms and process to be as systematic, transparent, and replicable as possible.
Once you've identified what kind of material you're looking for, you can identify where to search. This will include academic journal databases at a minimum. Check out the Where to search tab for more!
The design of your search strategy will depend on what you're looking for and where you're looking. Check out the How to search tab for more!
Chapter 4: Searching and Selecting Studies provides guidance for both the search and screening/review (link)
Conducting systematic reviews of intervention questions I: Writing the review protocol, formulating the question and searching the literature. O’Connor AM, Anderson KM, Goodell CK, Sargeant JM. Zoonoses Public Health. 2014 Jun;61 Suppl 1:28-38. doi: 10.1111/zph.12125. PMID: 24905994
Technical Manual for Performing Electronic Literature Searches in Food and Feed Safety.
C19 + C24. Planning the search (protocol)
C25. Searching specialist bibliographic databases (protocol)
C26. Searching for different types of evidence (protocol)
C27. Searching trials registers (protocol)
C28. Searching for grey literature (protocol)
C29. Searching within other reviews (protocol)
C30. Searching reference lists (protocol)
C31. Searching by contacting relevant individuals and organizations (protocol)
C32 Structuring search strategies for bibliographic databases (review / final manuscript)
C33. Developing search strategies for bibliographic databases (review / final manuscript)
C34. Using search filters (review / final manuscript)
C35. Restricting database searches (protocol & review / final manuscript)
C36. Documenting the search process (review / final manuscript)
C37. Rerunning searches (review / final manuscript)
C38. Incorporating findings from rerun searches (review / final manuscript)
C48. Obtaining unpublished data (protocol & review / final manuscript)
"Searching for Studies", the Campbell information retrieval guide
Key CEE Standards for Conduct and Reporting
5.2 Conducting the Search
5.3 Managing References and Recording the Search
5.4 Updating and Amending Searches
...such as electronic databases, contact with study authors, trial registers or other grey literature sources...with planned dates of coverage
Having a search strategy peer reviewed may help to increase its comprehensiveness or decrease yield where search terminology is unnecessarily broad.
Name each individual database searched, stating the platform for each...There is no single database that is able to provide a complete and accurate list of all studies...
If databases were searched simultaneously on a single platform, state the name of the platform, listing all of the databases searched...
List any study registries searched...study registries allow researchers to locate ongoing clinical trials and studies that may have gone unpublished
Describe any online or print source purposefully searched or browsed (e.g., tables of contents, print conference proceedings, web sites), and how this was done...
"...list all websites searched, along with their corresponding web address...if authors used a general search engine, authors should declare whether steps were taken to reduce personalization bias...if review teams choose to review a limited set of results, it should be noted in the text, along with the rationale..."
"...authors must specify the conference names, the dates of conferences included, and the method used to search the proceedings (i.e., browsing print abstract books or using an online source)..."
"When purposefully browsing, describe any method used, the name of the journal or other source, and the time frame covered by the search, if applicable..."
...can be complicated to describe, but the explanation should clearly state the database used...and describe any other methods used. Authors also must cite the “base” article(s) that citation searching was performed upon, either for examining cited or citing articles...
Contact methods may vary widely...may include personal contact, web forms, email mailing lists, mailed letters, social media contacts, or other methods...[which are] inherently difficult to reproduce, [so] researchers should attempt to give as much detail as possible...
...declare that the method was used, even if it may not be fully replicable...[include] other additional information sources or search methods used in the methods section and in any supplementary materials...
It is important to document and report the search strategy exactly as run, typically by copying and pasting the search strategy directly as entered into the search platform...repeat the database or resource name, database platform or web address, and other details necessary to clearly describe the resource....Report the full search strategy in supplementary materials as described above. Describe and link to the location of the supplementary materials in the methods section.
...report any limits or restrictions used or that no limits were used in the abstract, methods section, and in any supplementary materials, including the full search strategies (Item 8)...[and] the justification for any limits used...
...cite any search filter used in the methods section and describe adaptations made to any filter. Include the copied and pasted details of any search filter used or adapted for use as part of the full search strategy (Item 8)...
Sometimes, authors adapt or reuse [previously published search strategies] for different systematic reviews...it is appropriate to cite the original publication(s) consulted.
If there are no changes in information sources and/or search syntax (Table 2), it is sufficient to indicate the date the last search was run in the methods section and in the supplementary materials.
If there are any changes in information sources and/or search syntax, the changes should be indicated (e.g., different set of databases, changes in search syntax, date restrictions) in the methods section...explain why these changes were made...
If authors use email alerts or other methods to update searches, these methods can be briefly described by indicating the method used, the frequency of any updates, the name of the database(s) used...Report the methods used to update the searches in the methods section and the supplementary materials, as described above.
...date of the last search of the primary information sources used...the time frame during which searches were conducted...the initial and/or last update search date with each complete search strategy in the supplementary materials...
Describe the use of peer review in the methods section.
...report the total number of references retrieved from all sources, including updates...[such that] if a reader tries to duplicate a search from a systematic review, one would expect to retrieve nearly the same results when limiting to the timeframe in the original review...
...describe [the method] and cite any software or technique used...if duplicates were removed manually, authors should include a description...
In addition to the items required by PRISMA and PRISMA-S, Bethel, Rogers, and Abbot (2021) recommend including a search summary table "containing the details of which databases were searched, which supplementary search methods were used, and where the included articles were found."
[Bethel, Rogers, and Abbot (2021) Search Summary Table Template]
We host two workshops each fall on advanced and comprehensive searching approaches, check out our latest recordings!