A team of researchers at the National Institute of Health led by George Santangelo has developed a new article-level metric: the Relative Citation Ratio.
The Relative Citation Ratio is article level and field independent and provides an alternative to the invalid practice of using journal impact factors to identify influential papers. Image credit: Unsplash.
The Relative Citation Ratio (RCR) will allow scientists and funders to quantify and compare the influence of an article, regardless of publication and scientific field. While RCR cannot replace expert review, it does overcome many of the issues faced by previous metrics.
Currently, metrics are determined at the journal level, and the influence of an article is based on the journal in which it was published.
Individual articles and researcher performance are evaluated based on the assumption that all articles published in high impact journals are uniformly of high impact, and that high impact science is not published in lower impact factor journals.
Scientists also recognize the importance of each other’s work based on citations; however, citation practices vary between fields.
To address these issues, RCR uses a co-citation network that is formed from the reference lists of papers that cite the article in question, defining a unique field for each article.
For example, if Article X is cited by Article A, Article B, and Article C, then the co-citation network of Article X would contain all the articles from the reference lists of Articles A, B, and C.
Comparing the citation rate of Article X to the citation rate in the co-citation network allows each article to create its own individualized field.
In addition, the new metric is benchmarked to a peer comparison group to determine the relative influence of an article, assuming citations are a measure of influence, and to allow comparisons between similar types of articles, or output from similar institutions.
“This unique benchmarking step is particularly important as it allows ‘apple to apples’ comparisons in comparing groups of papers, e.g. comparing research output between similar types of institutions or between developing nations,” Dr. Santangelo and co-authors said.
The authors demonstrate that their quantitative RCR values correlate well with the qualitative opinions of subject experts. Many in the scientific community already support the use of the new metric.
“RCR evaluates science by putting discoveries into a meaningful context. I believe that RCR is a road out of the Journal Impact Factor swamp,” commented Dr. Stefano Bertuzzi, Executive Director of the American Society for Microbiology.
Dr. Santangelo and his colleagues described their new metric in the journal PLoS Biology.
“A beta version of iCite, our web tool for calculating Relative Citation Ratios of articles listed in PubMed, is available at https://icite.od.nih.gov,” the authors said.
While RCR represents a major advance, the team acknowledges that it should not be used as a substitute for expert opinion.
While it does measure an article’s influence, it does not measure impact, importance, or intellectual rigor. It is also too early to apply RCR in assessing individual researchers.
“No number can fully represent the impact of an individual work or investigator,” Dr. Santangelo said.
“Neither RCR nor any other metric can quantitate the underlying value of a study, nor measure the importance of making progress in solving a particular problem.”
“While the gold-standard of assessment will continue to be qualitative review by experts, RCR can assist in the dissemination of a dynamic way to measure the influence of articles on their respective fields,” he said.
_____
Hutchins B.I. et al. 2016. Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level. PLoS Biol 14 (9): e1002541; doi: 10.1371/journal.pbio.1002541
