Research Impact Metrics
A guide for those wanting to use research impact metrics for evaluation, analytics, and reviews, e.g., promotion & tenure.
Research Impact Librarian
Responsible Research Assessment Overview
- Responsible Use of Rankings: 10 PrinciplesTen principles developed by CWTS that are intended to guide the responsible use of university rankings. These principles apply to university rankings in general.
- Bibliometrics: The Leiden Manifesto for research metricsA set of ten principles for best practice in metrics-based research assessment so that researchers can hold evaluators to account, and evaluators can hold their indicators to account.
- San Francisco Declaration on Research Assessment (DORA)The Declaration on Research Assessment (DORA) recognizes the need to improve the ways in which the outputs of scholarly research are evaluated.
- Harnessing the Metric Tide: indicators, infrastructures & priorities for UK responsible research assessmentThis review was commissioned by the joint UK higher education (HE) funding bodies as part of the Future Research Assessment Programme (FRAP). It revisits the findings of the 2015 review The Metric Tide to take a fresh look at the use of indicators in research management and assessment.
- The Hong Kong Principles for assessing researchers: Fostering research integrity[Abstract]: For knowledge to benefit research and society, it must be trustworthy. Trustworthy research is robust, rigorous, and transparent at all stages of design, execution, and reporting.
Image credit: David Parkins (from the Leiden Manifesto article), used under the fair use provision of U.S. copyright law.
- Using InCites responsibly: a guide to interpretation and good practiceThis guide has been created by bibliometric practitioners to support other users of InCites and promote a community of informed and responsible use.
- Using SciVal responsibly: a guide to interpretation and good practiceThis guide is designed to help those who use SciVal to source and apply bibliometrics in academic institutions.
- Using Altmetric Data Responsibly: A Guide to Interpretation and Good PracticeThis guide focuses specifically on data from the data provider and company, Altmetric, but other types of altmetrics are mentioned and occasionally used as a comparison in this guide, such as the Open Syllabus database to find the educational engagement with scholarly outputs.
- Responsible metrics: One size doesn't fit allBlog post from Ludo Waltman summary: Differentiates between micro-level and macro-level research evaluation (i.e., difference between evaluation of institutions and evaluation of individuals and research groups).
- Metrics ToolkitA resource for researchers and evaluators that provides guidance for demonstrating and evaluating claims of research impact. With the Toolkit you can quickly understand what a metric means, how it is calculated, and if it’s good match for your impact question.
- Developing Metrics Literacies: Competencies, dispositions, and knowledge for the critical assessment and ethical use of scholarly metrics [blog post]Authors discuss their groundbreaking work on the Metrics Literacies Project. The Metrics Literacies project is informed by this concept: ‘an integrated set of competencies, dispositions and knowledge that empowers individuals to recognize, interpret, critically assess and effectively and ethically use scholarly metrics,’ which is guided by the overarching question, ‘How can the understanding and use of scholarly metrics in academia be improved?’
Review, Promotion, and Tenure Project
Review, promotion, and tenure (RPT) guidelines play a key role in university workplace advancement, shaping how academic success is defined and viewed. To understand the nature of these powerful documents, our lab conducted a multi-year research project analyzing the contents of more than 850 RPT guidelines from across the US and Canada. Here, we summarize some of the key takeaways from that research, as well as a vision for a more open academic reward system.
- Review, Promotion, & Tenure ProjectHow can research institutions incentivize openness and accessibility?
- RPT Project Infographics(Example below) These infographics breakdown the results of the studies that analyzed the contents of >850 RPT guidelines from across the US and Canada.
Image credit: ScholCommLab, CC BY-NC-SA. Information from infographic can be found at 10.7554/eLife.42254
- SCOPE Framework for Research EvaluationThe SCOPE framework for research evaluation is a five-stage model for evaluating responsibly.
- Protocol for Research Assessments in the NetherlandsThe Standard Evaluation Protocol (SEP) describes the methods used to assess research conducted at Dutch universities and Netherlands Organisation for Scientific Research (NWO) and Academy institutes every six years, as well as the aims of such assessments.
- How academia is exploring new approaches for evaluating researchers
An overview of how research assessment reform is being approached across the globe, but most especially in Europe, and how we can learn lessons from such attempts at reform. - The Agreement on Reforming Research Assessment
The Agreement on Reforming Research Assessment, from the Coalition for Advancing Research Assessment (COARA), sets a shared direction for changes in assessment practices for research, researchers and research performing organizations, with the overarching goal to maximize the quality and impact of research. - Rethinking Global University Rankings
In an effort to highlight the disconnect between the approaches taken by some of the global university rankings and community-agreed best practice in responsible research evaluation, the Rankings sub-group of the INORMS Research Evaluation Working Group developed a mechanism for rating global University Rankers. - AGORRA: A global observatory of responsible research assessment
The Research on Research Institute's (RoRI’s) AGORRA project aims to generate comparative data, evidence and analysis to support and accelerate the transformation of research assessment reform across national assessment systems. - Tools to Advance Research Assessment (TARA)
Tools to Advance Research Assessment (TARA) is a project to facilitate the development of new policies and practices for academic career assessment. - Virginia Tech Faculty Statement on Responsible Use of Research Metrics
This statement was endorsed and approved by the Virginia Tech Faculty Senate on April 21, 2023. - List of Statements of Responsible Metrics
Lists universities' statements of responsible research metrics and case studies from countries and universities in which they have implemented responsible research assessment.
- 2021 Competency Model for Bibliometric WorkThe 2021 bibliometric competencies can help identify skills gaps, support progression through career stages for practitioners in the field of bibliometrics, and prepare job descriptions.
- LIS-Bibliometrics Conferences & EventsA roundup of major conferences and events in bibliometrics.
- Leiden University CWTS course programThe Centre for Science and Technology Studies (CWTS) is proud to introduce its newly developed course program.
Upcoming Conferences (chronological order)
- BRIC 2025 (Bibliometrics and Research Impact Community)BRIC is a Canadian community, and BRIC 2025 will take place in Montreal, Canada, June 4-5.