Beyond citation counts: Understanding the concept of pathways

Figure 1 illustrates two scenarios in which research articles are cited by policy documents. In the first case, Article 1 is directly cited by two policies issued in the same year by the same agency. In the second, Article 2 is also cited by two policies; however, one of these policies also cites Article 3, which itself had cited Article 2. Furthermore, these policies are interlinked and referenced by another policy, and they were released by different institutions over a longer time span.

While both Article 1 and Article 2 are cited by policy documents twice, Article 2 clearly exerts broader, more sustained, and more complex policy impact. This highlights the limitations of focusing solely on whether and how often an article is cited in policy. It is crucial to examine the pathways through which research articles are cited.

Cross-domain citation analysis: Four types of pathways from research to policy

Table 1 presents a model of four citation pathways based on a two-layer binary classification. The first layer distinguishes direct from indirect impact – where a focal policy directly cites an article versus citing a policy that had cited the article. The second layer distinguishes whether reinforcement effects exist – i.e., whether the focal policy is connected to the article through multiple citation links, which often indicates a stronger and more interactive influence.

Using a dataset of 9,963 policy documents related to artificial intelligence (AI) and over 1.6 million citation links to research articles, the proportions of the four pathway types (A1, A2, B1, B2) were found to be 3.41%, 2.81%, 44.95%, and 48.83%, respectively. This suggests that indirect impact is more prevalent than direct impact, and that reinforcement effects are relatively evenly distributed.

Disparity analysis: The role of policy-issuing institutions and problem domains

Figure 2 shows the distribution of citation pathway types across policies issued by different institutions. EU institutions and international intergovernmental organizations are clustered in the lower-left quadrant, indicating that their policies tend to cite research articles indirectly and through multiple pathways. In contrast, policies issued by national governments such as those of Spain and Singapore are more likely to cite articles directly, with weaker reinforcement effects.

Figure 3 presents the distribution of citation pathway types across policy domains. Policies in domains such as education, economic policy and business, and sustainability and the environment tend to cite research articles indirectly and through multiple pathways. These results indicate that policy areas with high social relevance tend to have more intensive interactions with scientific knowledge.

In-depth analysis: Multidimensional features of policy citations of research articles

From an actor perspective, Figure 4 shows the geographical flow of citations from research articles to citing policies. USA-produced articles have influenced policies across numerous countries and regions. Meanwhile, articles from Germany, the Netherlands, and Italy are more likely to support EU or international organization policies. Intergovernmental organizations, in particular, play a prominent intermediary role in indirect citation pathways. Table 2 further shows that policies indirectly citing an article are often issued by institutions within the same region or of the same type as those issuing the intermediary policies.

From a thematic perspective, Figure 5 illustrates the disciplinary flow from cited articles to citing policies. Indirect and reinforced citation pathways more frequently integrate knowledge from the social sciences. Nonetheless, policy documents broadly incorporate multidisciplinary knowledge. When combined with a temporal perspective (Figure 6), the data reveal that natural sciences are playing an increasingly significant role in policy development.

Case illustration: Scientific knowledge in the “Blueprint for an AI Bill of Rights”

Taking the Blueprint for an AI Bill of Rights, issued by the White House Office of Science and Technology Policy, as an illustrative case, Figure 7 presents a visualized network of “policy-article” citation relationships. A detailed analysis of the citation pathways within this policy is available in the original text.

Conclusion

This study proposes a citation pathway model linking policy documents to research articles. It extends traditional citation analysis into the cross-domain “science–policy” interface and provides a new lens to understand the complex processes by which scientific research influences policy. Through an empirical analysis of AI-related policy documents, the study reveals key patterns – such as the mediating role of international organizations and the growing importance of natural sciences in policymaking – that offer evidence-based insights into enhancing the policy relevance of scientific research.

As evidence-based policymaking gains increasing prominence, such research can help scientists, policymakers, and institutions better understand and build effective modes of interaction. Ultimately, enhancing the societal impact of research and promoting the real-world application of scientific knowledge will benefit us all by enabling science to better serve society.