In our recent study (Liu et al., 2022), we dissected a reference list with five distinct dimensions, namely the number of references, the number of citations of references, the age of references, the number of nodes in a reference cascade, and the density of bibliographic coupling networks. It is worth noticing that the latter two indicators are network- instead of count-based, which is quite different from previous scientometric analyses methodologically.

Based on over one million articles published in 2005 covered by Microsoft Academic Graph, we found either inverted-L or -U shapes (non-linearity) for the relations between these characteristics of references and citations of scientific publications with the universal existence of a critical point, i.e., a position indicating phase transitions for each distribution. We named it a “lucky point” in the study and proposed a mathematical definition to calculate the specific occurrence of this critical point. Here, “lucky” hints that, when the value of the indicator is around a certain value, the publication has more chances of obtaining a relatively higher citation.

These non-linear trends are consistent with common sense, i.e., there is a set of “suitable” references relating to a higher citation impact for each publication. For example, for a given research topic, the candidates appropriate as references are relatively fixed. Hence, a too limited number of references would lower the reliability and credibility, while redundant references might counter the normative theory of citations as well. The findings emphasize the importance of looking for “suitable” references. Besides the theories, methodologies, and empirical results, references themselves play a significant role in improving framing, narratives, and storytelling.

The definition of “lucky points” allows us to calculate this indicator for each discipline. We found that “lucky points” in different disciplines vary greatly. By mapping those five “lucky points” to discipline-level indicators, we further explored how the discipline-level differences relate to the characteristics of the discipline itself. We observed a ubiquitous effect of disciplinary academic “environments” on where a lucky point appears. For instance, a larger “lucky point” for the number of references or the number of citations of references is more likely to be observed in disciplines where scientists tend to cite more references (e.g., Biology) or higher-cited references (e.g., Psychology). This finding indicates the phenomenon of “peer effect” in academic circle, i.e., scientists’ performances shape the referencing preference of their discipline.

The original article on which this essay is based is: Liu, J., Chen, H., Liu, Z., Bu, Y., & Gu, W. (2022). Non-linearity between referencing behavior and citation impact: A large-scale, discipline-level analysis. Journal of Informetrics, 16(3), 101318. DOI: https://doi.org/10.1016/j.joi.2022.101318.

Authors

Jialin Liu is an undergraduate student in Data Science at the Department of Information Management, Peking University. He is doing research in scientometrics and science policy. Particularly, his research focuses on the mobility of scientists and its dynamics.

Hongkan Chen is an undergraduate student in Data Science at the Department of Information Management, Peking University. He is studying science paradigm using quantitative methods with massive datasets.

Zhibo Liu is an undergraduate student in Data Science at the Department of Information Management, Peking University. Zhibo’s research aims to understand the social dimensions of the global scientific ecosystem with computational techniques.

Yi Bu is an Assistant Professor in Data Science at the Department of Information Management, Peking University. Yi is particularly focusing on scholarly data mining and computational social sciences.