Text and Data Mining: TDM: Tools & Technology

Policies, practices, tools, and current issues related to TDM.

Considerations for Mining

Contact your Subject Specialist librarian for data source options, or try the Finding Data library guide. Contact dataservices@vt.edu for additional recommendations on processes and tools for manipulating, analyzing, and visualizing text or data. Some general considerations for pulling large amounts of data:

  • Use an official API or bulk download service if provided.
  • Web scraping is an option, but can have drawbacks. Consider the amount and speed with which you are pulling data, and put forth a good faith effort to identify terms of use from your sources.
  • If accessing subscription or licensed content, be sure to check the policies or terms.

Tool Lists:

Tutorials, Toolsets, and Resource Lists

Data Processing: Getting, Scraping, Extracting, Transcribing, and Cleaning Data from websites, documents

Analyzing and Visualizing Data

Python Resources

R Resources

Need Assistance?


Content sources: Subject Specialists

Analysis and Tools: dataservices@vt.edu