Bibliometrics is unquestionably a research policy minefield, but is citation counting really ¡°killing¡± academic dissent?

According to an opinion piece published in Times ÖйúAƬ?last week, the use of citations as a measure of scholarly impact is having a variety of negative effects, potentially including the stifling of academic debate. But I would argue that the position taken by Jonathan Goodman manages to both put the cart before the horse and suggest that we throw the baby out with the bathwater.

Goodman discusses coercive citation practices ¨C the trading of citations for publication ¨C as if they were a symptom of peer-reviewers or editors trying to maximise their own citations. There is no doubt that some of this goes on, but the most systematic and damaging of coercive citation are the result of editors and publishers attempting to increase the impact factor of their journals.

This points to the central problem in the academic incentive system: the relentless focus on publishing in ¡°prestigious¡± journals ¨C identified according to dubious journal-level metrics such as the ubiquitous impact factor. Even anti-metrics initiatives like the limit themselves to criticism of journal-level metrics; they do not discount out of hand the responsible use of article-level bibliometrics.

ÖйúAƬ

Goodman is concerned about self-citation, but this is a perfectly normal part of science, especially if you are a leader in your field or work in a niche area. It would be ridiculous not to cite your previous work if it is relevant to your current work. The difficult question is where to draw the line. At what point does self-citation become pathological? Thanks to the very of to which Goodman refers, we now have a much better idea about this. And if you are particularly concerned about distortions caused by self-citation, you can usually exclude them from your analysis.

In my adopted homeland of Norway, a recent proposal to use bibliometrics in evaluation was criticised because than women do, introducing the potential for bias. But how do we know this? By counting citations. If bibliometricians hadn¡¯t crunched the numbers, we wouldn¡¯t know that there was a problem to fix.

ÖйúAƬ

Goodman¡¯s chief objection to citation counting is that it may discourage junior scholars from criticising their seniors, out of concern for their careers. Having a paper published in early in my own career, I share that concern. However, the potential for torpedoing your career by criticising the wrong person would still exist even if citations were not counted at all. Thanks to the humble bibliography, a senior researcher will find out eventually that some young upstart has criticised their work, regardless of whether a bibliometrician or automated citation database has been performing mathematical acrobatics on the citation counts of their articles in the meantime.

Article-level citation indicators are obviously not perfect, for some of the reasons that Goodman outlines. But the perfect should never be the enemy of the good, and when scholarly output is , they are as good a method as any by which to filter the juicy plankton out of the tsunami. As indicators of an article¡¯s influence, they are a lot better than the impact factor of the journal in which it was published, or any number of derivatives or proxies of that.

More widespread use of article-level citation metrics may even help to break the oligopoly of certain journals and publishers ¨C which would correspondingly help to reduce the very coercive editorial practices that Goodman mentions. It would also mean a lot less time wasted in the rinse-and-repeat cycle of submission and rejection as ambitious researchers work their way down the hierarchy of ¡°top-tier¡± journals.

Being published in certain journals has understandably become an absolute raison d'¨º³Ù°ù±ð for some, fetishised to the point that a work¡¯s readership, citation count and societal impact have become entirely secondary. But while publications in highly ranked journals hold the potential for impact, citation counts actually demonstrate it. And I like to think that we scientists care about evidence.

ÖйúAƬ

Craig Aaen-Stockdale is a senior adviser in the research administration at BI Norwegian Business School.