History: Text & data mining: Sources

Can I Mine This Source?

Library Databases:

Most of the libraries' databases do not allow text or data mining due to license agreements. We will continue to work with database vendors to include TDM in future license agreements. See the list below for vendors that allow TDM.

Project Support: For TDM projects using content from a library database, or other content sources where you are uncertain of rights or permissions, please contact us and we can investigate options with you.

Other Sources:

There are a variety of content sources that allow text and data mining openly or within stated restrictions. Resources listed in these sections below are selected examples. For assistance in identifying additional sources, please contact us.

Open Access Publishers and TDM

Subscribed sources for TDM

Source Content Fee Access Details Help Guides
Adam Matthew Any Adam Matthew primary source database we subscribe to No Contact your college librarian Adam Matthew guide
Brill Subscribed and open access journals and ebooks   Contact your college librarian  
Elsevier, ScienceDirect, and Scopus Subscribed and open access content on ScienceDirect, bibliographic citations in Scopus No Request an API key, then programmatically access the data
Gale Primary source documents: historical news, books, journals and magazines Yes, if not through Artemis Content delivered in hard drive or through Gale Artemis: Primary Sources Gale guide
HathiTrust Complete set of digitized books and journals No HathiTrust guide
IEEE All online content No Contact your College Librarian; requires notifying IEEE  
JSTOR Licensed and open access journals and books, up to 25,000 articles at a time No Data for Research Data for Research guide
PubMed Bibliographic citations in XML No FTP baseline citations and incremental updates PubMed info
SAGE Subscribed and open access journals No Restrictions on speed of downloads SAGE guide
Springer Nature Subscribed and open access journals and books Sometimes Substantial downloads should be made through their API, which has a cost involved Springer Nature guide
Wiley Subscribed and open access journals and books No Use CrossRef Text and Data Mining Service Wiley info


Need Assistance?


Content sources: Subject Specialists

Analysis and Tools: dataservices@vt.edu