The metrification of the research enterprise is inescapable but, used properly, brings a measure of fairness to the evaluation of researchers and research producing organizations. However, a drawback that we have discussed previously is that individual metrics provide only a pinhole view of the researcher or organization. We therefore need to appreciate the limitations of each metric and we need to bring a selection of metrics to bear, essentially triangulating the object of our attention and placing them in the context of their field.
This is the approach of beam plots, a relatively new tool that combines metrics into a comprehensible visualization of research output.(1) The example presented here shows the citation counts for an imaginary author plotted over a 10-year period.
The vertical scale, time, is fixed in this illustration, but the horizontal scale ‘floats’ because each article’s absolute number of citations can normalized against all citations for a particular field or genre within each year. Hence, for example, the rankings for RCTs might be normalized against the average citation count for all RCTs published in a particular year (or even all RCTs within one domain, such as low back pain). Rankings for case studies, by our same imaginary author, would be normalized against average citation count for all case studies in their specific year of publication.
This tool is therefore an excellent way for a researcher to see how they are performing relative to their peers. It can also suggest where the researcher’s strengths and weaknesses are when it comes to success in publication. Our example used two axes, time and citation counts, but there is also the opportunity to pack information into the colours and sizes of the nodes which we have plotted. For example, we can use node size to represent impact factor of the journal of publication, and colour to represent genre or topic area. We could also change the axes to reflect other metrics, such as amounts of funding or years in research.(2) There are many possibilities.
Web of Science recently introduced beam plots as one of the utilities available in their suite of analyses. There are also several free utilities floating about, and one can always sit down and draw these things by hand. A challenge for researchers is chiropractic, however, is that we do not have clear benchmarks for our discipline. We do not yet know what the norms are for journals focused on chiropractic, and we do not yet know want the norms are for chiropractor researchers publishing across the breadth of biomedicine. This could, of course, be one of the goals for the Global Chiropractic Research Enterprise Initiative as more individual researchers and research producing organizations contribute their publications data.
References:
- Bornmann L, Werner M. Distributions instead of single numbers: percentiles and beam plots for the assessment of single researchers. JAIST 2014;65(1):206-208.
- Bornmann L, Haunschild R. Plots for visualizing paper impact and journal impact of single researchers in a single graph. Scientometrics 2018;115:385-394.