Previously, we have talked about a journal level metric, the journal impact factor (JIF), and an article level metric, the citation count. We have seen how both of these metrics might be related to, but are not direct measures of some quality of the author or authors – perhaps productivity.
In this blog entry, we want to talk about an early attempt to turn a bibliometric, citation count, into a scientometric called the H-index. The H-index, a calculation based on the citation counts of each of an author’s papers, was first proposed by the physicist Jorge Hirsch in 2005.1 The metric is calculated by plotting the citation count of each paper against the ranking of all of the papers. The H-index is the point at the number of papers with a certain citation count is at or exceeds that citation count. Sorry, that’s a bit confusing and it is easier to explain by example. No matter how many papers an author has published, if only 5 of them have a citation count of 5 or more, then the author’s H-index is 5. If the author has 20 papers with 20 or more citations, then the H-index is 20. In either case, the H-index ignores or fails to reflect those papers with lower numbers of citations.
The H-index has a couple of virtues. First, it is based on the researcher’s entire career output, and so it gives something of an overview while discounting outliers. Secondly, it is influenced more by the highly cited papers, which are the ones likely to reflect important achievements in research. It therefore strikes a nice balance between total citation count and total publication count.
Of course, there are some disadvantages as well. One is that the H-index can never exceed the total number of publications. Therefore, someone with a long career and many papers with relatively low citation rates, for example, one hundred papers in all but only 5 with more than 5 citations, would have an H-index of 5. Another brilliant young scientist who at the beginning of their career had only 5 papers each with more than 100 citations would have that same lowly H-index of 5. That doesn’t seem right, does it? That lack of discrimination is can be a big drawback.
Another challenge is that the H-index treats all authors on a paper equally, regardless of how large or small their role was in the research. There are solutions to this challenge, but the raw H-index doesn’t take advantage of them.
Finally, there is that same old problem that we have with all metrics in science: people learn to ‘game’ the system, for example through self-citation or even coercion – as a reviewer, I might not approve of you manuscript until you cite my paper – it happens!
So what can we do to salvage the good in the H-index? Well, part of the answer is, as we have said elsewhere, to use the metric to compare like with like. For example, we can use it to compare researchers in the same field who have been in research for the same period of time. That will reduce anomalous rankings. We also need to remember that each author level metric only provides a one-dimensional measure of a person, and so we need to use as many perspectives as possible to get a comprehensive picture of how good a researcher is at their job.
References:
- Hirsch JE. An index to quantify an individual’s scientific research output. PNAS 2005;102(46):16569-16572.