Library:Scholarly Communications/Citation Analysis/Criticism

From UBC Wiki
Jump to: navigation, search

Criticism of Impact Factors

Dora-logo-big.png

Criticism of research assessment methods has been aimed at the practice of using the Journal Impact Factor (JIF) as a measure of the impact of individual research publications. This criticism has most notably been voiced in the San Francisco Declaration on Research Assessment (DORA), which was initiated in December 2012 by the American Society for Cell Biology (ASCB) and a group of scholarly journal editors.

The JIF was originally introduced as a metric to aid libraries in collection development: by dividing the total number of citations received by the number of articles that have been published in the previous two years, the JIF provides the average of citations received per article. However, this metric has also come to be used as a tool in assessing the impact of individual research papers; a use for which this tool was never intended. Thomson Reuters also addressed this, releasing this video of David Pendlebury on why the Journal Impact Factor was originally created and how it should be used, as well as a statement in response to the San Franscisco Declaration on Research Assesment.

As has been pointed out in a variety of studies, the journal impact factor judges only how much a journal is cited on average, a number which bears little relation to the individual papers within a journal. According to a 2012 study, the strength of the relationship between citation rates and the IF has been weakening for the last 40 years[1].

The San Francisco Declaration on Research Assessment states that “There is a pressing need to improve the ways in which the output of scientific research is evaluated by funding agencies, academic institutions, and other parties”.

It also provides recommendations for funding agencies, institutions, publishers, organizations that supply metrics, and researchers, including:

  • To avoid using journal-based metrics to assess an individual scientist in hiring, promotion, and funding decisions.
  • To consider the value and importance of all research outputs, including datasets and software, in addition to scholarly publications.
  • To consider a broad range of impact measures, including qualitative indicators of research impact, such as influence on policy and practice.
  • To challenge research assessment practices that rely inappropriately on Journal Impact Factors and promote and teach best practice that focuses on the value and influence of specific research outputs.

To see the full list of recommendations, find out more, or to add an individual or organizational signature to the list of DORA supporters, visit San Francisco Declaration on Research Assessment page.



  1. Lozano, 2012. More information about this study appears on the London School of Economics blog