To explore how common this problem might be, we examined more than 3,200 peer-reviewed papers published between 2023 and 2025 in fields related to AI in education and related areas. We looked for references that could not be independently verified and then manually examined selected cases to understand what was happening.

The results were surprising. We identified 69 papers—about 2.16% of the papers examined—that contained at least one reference we could not verify. Most papers did not contain such references, and an unverifiable citation does not automatically mean that a paper is incorrect or that it was produced using AI. There are many possible explanations, including incomplete information, unusual publications, or simple mistakes.

However, these findings show that the problem is not merely theoretical. References that cannot be traced back to a real source are already appearing in parts of the published scholarly record.

The most unexpected finding came from looking at how these references appear in information systems used by researchers every day.

Many researchers assume that if a paper appears in Google Scholar, it must exist. However, search systems do not simply contain a list of verified publications. They also collect information about citations that appear inside other papers.

This creates a surprising possibility: a paper that does not exist can still leave traces behind.

Imagine a non-existent article being cited by several other papers. Those citations become visible in scholarly search systems. A researcher searching for the title may find evidence that appears to confirm the article’s existence, even though the original publication cannot be located.

In other words, citations can create the appearance of legitimacy. A reference may become discoverable not because the original work exists, but because other publications have repeated the same citation.

An unverifiable reference can become visible in scholarly search systems after being cited by other publications. Discoverability does not always guarantee authenticity.

This distinction between being discoverable and being authentic is important. Information systems help us find information, but finding information is not the same as verifying it. Trust depends on knowing where information came from and whether that path can be followed back to a reliable source.

This is why unverifiable references matter. They do not prove that research findings are wrong, but they may reveal weaknesses in the processes surrounding research creation and evaluation.

Producing a scholarly paper requires many forms of care: locating reliable sources, checking evidence, understanding previous work, and ensuring that references accurately represent the literature. If one of the easiest parts of a paper to verify—the reference list—contains problems, it is reasonable to ask whether other parts of the process deserve closer attention.

The challenge is not only technological. AI tools may make it easier to create convincing but inaccurate references, but maintaining a trustworthy scholarly record requires attention from everyone involved: authors, reviewers, editors, publishers, and information systems that organize and distribute knowledge.

Automated tools can help identify references that need checking, but technology alone cannot replace careful scholarly practice. The solution lies in combining better tools with stronger habits of verification and a renewed emphasis on research quality.

References often receive little attention because they appear at the end of a paper. Yet they form the connective tissue of scholarship, linking discoveries across disciplines and generations.

A reference is more than a bibliographic detail. It is a promise that evidence can be found, checked, and built upon. As AI becomes a routine part of research and writing, keeping that promise may become one of the most important challenges facing scholarly communication.

This article is based on a recent study published in the Journal of the Association for Information Science and Technology.