Quantifying Scientific Breakthroughs by a Novel Disruption Indicator Based on Knowledge Entities
Quantifying Scientific Breakthroughs by a Novel Disruption Indicator Based on Knowledge Entities
Shiyun Wang, Yaxue Ma, Jin Mao, Yun Bai, Zhentao Liang, Gang Li
Center for Studies of Information Resources, Wuhan University, Wuhan, Hubei, China
School of Information Management, Wuhan University, Wuhan, Hubei, China
School of Information Management, Nanjing University, Nanjing, Jiangsu, China
Measuring scientific breakthroughs enables tracking and forecasting of emerging areas in science and assists stakeholders in making more informed decisions on funding allocation, employment, and scientific awards. Therefore, it has garnered widespread interest from a variety of disciplines to probe into scientific breakthroughs.
It is difficult to precisely define a scientific breakthrough. Previous studies have examined scientific breakthroughs from a variety of angles. For example, operationalizing scientific breakthroughs as highly cited papers or characterizing them as transformative research. Recently, Wu et al. (2019) applied a previously defined network-based disruption indicator (Funk & Owen-Smith, 2017) to quantify the extent to which an article disrupts or develops current research streams. This disruption indicator sheds new light on the detection of scientific breakthroughs. Subsequent research has proposed several variant indicators to modify the original disruption indicator. Nonetheless, most of the previous disruption indicators are concerned more with the citation patterns of publications than with their body of knowledge. We argue that in comparison to citations, text demonstrates an article’s contribution to the knowledge system more directly.
—in comparison to citations, text demonstrates an article’s contribution to the knowledge system more directly.—
In our recent study (Wang et al., 2022), we aim to quantify scientific breakthroughs by examining the characteristics of fine-grained knowledge embodied in scientific texts. We propose a novel disruption indicator that differs from prior ones by incorporating both citation patterns and knowledge elements. Following the fundamental idea of the original disruption indicator, the ego citation network of a focal paper, comprising both its references and subsequent papers that cite the paper, is constructed and employed as the proxy for the evolutionary trajectory of the focal paper. Then, two components are designed in our indicator to independently quantify the change of knowledge directly created by a focal paper and that inspired by the focal paper in its evolutionary trajectories. Two groups of knowledge elements, including MeSH terms and their co-occurrences, are employed independently by the indicator to measure the change of knowledge.
The proposed disruption indicator was applied to 6,112,372 PubMed articles published between 1991 and 2014. The effectiveness of the proposed indicators was evaluated against four standard scientific breakthrough datasets linked to the PubMed dataset and was compared to some baseline citation-based disruption indicators, i.e., the mCD index proposed by Funk & Owen-Smith (2017), the disruption indicator DI1 applied by Wu et al. (2019), and the best-performed variant indicator DI5 evaluated by Bornmann et al. (2020). Three methods were applied for the validation, including the ranking analysis of the standard scientific breakthroughs among all PubMed articles, comparative analysis between scientific breakthroughs and control papers, and regression analysis between disruption scores and scientific breakthroughs.
Our results show that the proposed disruption indicator based on MeSH co-occurrences outperforms that based on MeSH terms and the baseline disruption indicators. Moreover, the component used to measure the change of knowledge inspired by a focal paper in its evolutionary trajectory is a more important contributor for our indicator in detecting scientific breakthroughs.
Our study not only offers empirical insights into conceptual understanding of scientific breakthroughs but also provides practical disruption indicator for scientists and science management agencies searching for valuable research.
The Original Article
Wang, S., Ma, Y., Mao, J., Bai, Y., Liang, Z., & Li, G. (2022). Quantifying scientific breakthroughs by a novel disruption indicator based on knowledge entities. Journal of the Association for Information Science and Technology. https://doi.org/10.1002/asi.24719
References
Bornmann, L., Devarakonda, S., Tekles, A., & Chacko, G. (2020). Are disruption index indicators convergently valid? The comparison of several indicator variants with assessments by peers. Quantitative Science Studies, 1(3), 1242–1259. https://doi.org/10.1162/qss_a_00068
Funk, R. J., & Owen-Smith, J. (2017). A dynamic network measure of technological change. Management Science, 63(3), 791–817. https://doi.org/10.1287/mnsc.2015.2366
Wu, L., Wang, D., & Evans, J. A. (2019). Large teams develop and small teams disrupt science and technology. Nature, 566(7744), 378–382. https://doi.org/10.1038/s41586-019-0941-9
Cite this article in APA as: Bai, Y., Li, G., Liang, Z., Ma, Y., Mao, J., & Wang, S. (2022, December 14). Quantifying scientific breakthroughs by a novel disruption indicator based on knowledge entities. Information Matters, Vol. 2, Issue 12. https://informationmatters.org/2022/12/quantifying-scientific-breakthroughs-by-a-novel-disruption-indicator-based-on-knowledge-entities/
