Our study reveals that the potential for bias within AI-driven language models, such as GPT-3, poses a significant threat to the integrity of science. These models are trained on vast amounts of data, primarily from the internet, which can lead to a bias in the data. For example, if the data source is biased or incomplete, this bias will be reflected in the model’s output. Furthermore, the coding process used to train these models may also introduce biases. If the data used to train the models is not diverse enough, the resulting model may be biased towards certain groups or viewpoints. These biases can have a far-reaching impact on research, as the use of biased language models can perpetuate existing biases and misconceptions.

The issue of copyright ownership in AI-generated content is a complex and multifaceted one that requires careful consideration by legal experts and policymakers alike. As AI-driven language models such as GPT-3 become increasingly complex and capable of generating content that is indistinguishable from human writing, the question of who owns the resulting intellectual property becomes more pressing. Currently, the lack of transparency on data sources makes it difficult to know the extent to which content may be plagiarized.

As for citation practice, ChatGPT has been found to produce academic essays with missing references (see Appendix 1 of the original article for an example). The lack of references in early versions of ChatGPT could lead to disorder in scholarly publishing, as the integrity and credibility of the research may be called into question without proper citations. Indeed, when ChatGPT is asked to write a paper with citations, it often provides phantom citations to articles that do not actually exist.

Additionally, academic publishing platforms such as ChatGPT, which may use factors like citation counts in determining which publications to cite, may exacerbate the Matthew effect, which refers to the tendency for successful researchers with high citation counts to continue to be successful and frequently cited, while lesser-known researchers struggle to gain recognition and citations. In the context of academic writing, ChatGPT generates new papers based on existing papers and researchers’ needs. This could potentially exacerbate the lack of innovation and disconnection from practice in academic research.

In order to address these concerns, it may be necessary to develop new legal frameworks and guidelines that consider the unique challenges posed by AI-generated content. This process could involve clarifying existing copyright laws to account for the use of machine learning algorithms and training data, or establishing new regulations to ensure that AI-generated content is properly sourced and attributed.

ChatGPT and related technologies have the potential to significantly impact academia and scholarly research and publishing. However, it is important to consider the ethical implications of these technologies, particularly regarding the use of ChatGPT by academics and researchers. Collaboration between researchers, publishers, and AI-driven language model developers could be helpful for establishing guidelines and protocols that ensure the ethical, transparent, and accountable use of these technologies. Failure to develop such policies may undermine public trust in the scientific process, leading to far-reaching consequences for future research and innovation.

References

Hu, K. (2023). ChatGPT sets record for fastest-growing user base. Reuters, February 2, 2023.