Indian Journal of Medical Ethics

EDITORIAL


ChatGPT: Impact of an artificial author on bibliometrics

Shubhada Nagarkar

DOI:10.20529/IJME.2023.029

The use of Artificial Intelligence (AI) in medical science has been widely discussed and debated. Topol foresaw that AI, particularly deep learning, would be used in a variety of applications, with users ranging from specialty doctors to paramedics [1]. He discussed how deep neural networks (DNNs) of AI can help interpret medical scans, pathology slides, skin lesions, retinal images, electrocardiograms, endoscopy, faces, and vital signs. He has described its application in radiology, pathology, dermatology, ophthalmology, cardiology, mental health, and other fields [1]. Among several other AI applications used in our daily life, the next generation breakthrough, AI model ChatGPT-3 (https://chat.openai.com/) was launched on November 30, 2022 by OpenAI, California, which is well-known for its innovations in automated text generation. ChatGPT converses with the user, ascertains the user’s needs, and responds accordingly. It can write a poem, a diet plan, recipes, letters, computer programmes, a eulogy, do copy editing, and so on.

There has been a lot of discussion on the uses and misuses of ChatGPT. It could help non-English-speaking authors to improve efficiency and accuracy in writing; or assist in planning and structuring of various types of writing, including research communications. The rising concern is its potential ability to generate a research paper without actually conducting research. In addition, there have been published research papers in which ChatGPT has been listed as an author. This may have several consequences and may alter scientific facts, spread misinformation, and give rise to unethical practices in scholarly communication [2, 3, 4]. The text generated by this artificial author may even be plagiarised, giving rising to doubts about the author’s credibility. Researchers have already discussed the fabricated and fake references generated by ChatGPT [5]. Moreover, it will challenge the nature of errors, biases, research integrity and ethics [4]. If ChatGPT is one of the artificial authors in research papers published in journals indexed in citation databases, it will change the entire approach to bibliometric studies. This editorial focuses on the consequences of an artificial author being present in citation databases.

Citations are a dynamic relationship between the cited (original paper) and citing (containing references) documents. If either is generated by an AI bot like ChatGPT, it will challenge existing authorship norms, quantitative assessment of research contributions, intellectual property rights, and ownership of data [6]. In addition, citation-based research metrics like the Impact Factor of a journal, h-index, i10 index and CiteScore will be misleading and questionable. For example, consider an article published in a journal that is indexed in Web of Science and has a 2.5 impact factor for the year 2021. If ChatGPT is a co-author with the most citations, the impact factor in 2023 will be much higher.

Artificial authorship will cause serious problems in citation analysis. It will produce deceptive metrics. The Scopus database was searched for the term “ChatGPT” (All fields) to investigate this. It is not surprising that a total of 335 published papers were identified (upto April 21, 2023). Scopus considers ChatGPT as an individual author because it appears as a co-author in three papers. Moreover, these articles have received citations. Because ChatGPT is treated as an individual author, it has its own Scopus profile [https://drive.google.com/file/d/1hh53gyb4g9ejt2muJbvuQO13cCZfAUUY/view], which includes citations, h-index, and ORCID iD. This will have an impact on the ranking of individual authors, journals, and countries. Although ChatGPT is a co-author, its profile shows that, as of April 21, 2023, it has been cited 32 times in 29 documents, which is a matter of great concern because it suggests that an AI can possess domain specialisations and knowledge, like a human researcher.

Another important parameter of bibliometric studies is the ranking of authors based on the number of papers they have published and the number of citations their works have received. ChatGPT is ranked fourth in this data set [https://drive.google.com/file/d/15DejBUrv-U5xw-zgYJZ6WZh4E43Q-I0p/view]. Ranking an artificial author would create further problems, especially with regard to the affiliation of such an author. For instance, if any one outside the USA uses ChatGPT as a co-author, the USA would automatically be listed as a collaborative country since ChatGPT’s address is OpenAI, USA. In case of citation-based rankings, the paper, in which ChatGPT is one of the authors, is ranked fourth. In future, if citations of these papers increase, then the total metrics of papers in disciplines may be misleading because, while the metrics show large numbers of papers, many may not be human-authored or solely-human authored.

Since an author is a legal, institutional and societal entity in the publication/communication, accountabilty for the creation and its impact on society remain with the author [7]. Potts explains that the AI or digital scripter generates communications by imitating language (or data) from its “immense dictionary” or database, then mixing and blending that data into newly generated work [8]. Therefore, the proof of concept of ChatGPT to generate original and novel textual content is yet to be established, as novelty and originality are two aspects of any new creation. Subsequently, the question of accountability is probably the biggest hurdle that papers with ChatGPT as a co-author will encounter. To date, OpenAI cannot be held responsibile for the text generated by ChatGPT and ownership of intellectual property rights [9].

It is crucial to sensitise the scientific community, researchers, publishers of journals and pre-print archives and holders of citation databases about the consequences, advantages and disadvantages of AI tools in scholarly communication. The World Association of Medical Editors (WAME) has recommended that Chatbots cannot be authors in any type of publication, thus altering their publication policies to counter the ChatGTP invasion. Conversely, it could be made mandatory for authors to disclose usage of tools like ChatGPT along with the assurance of non-plagiarised text [10]. Recently, the publishers — Elsevier and Cambridge University Press & Assessment — have announced the acceptance of use of ChatGPT in writing research papers but not accepted it as a co-author [11].

Recognising ChatGPT as an author will corrupt the research processes and bibliometric studies and wipe out the efforts of pioneers in this field. Plagiarism, predatory journals, paper retraction, duplicate submission, data fabrication, and paper mills derived from the “publish or perish” pressure on researchers have already assumed epidemic proportions. ChatGPT has, in addition, posed fresh challenges that will change the landscape of research communications [12] and, ultimately, impact bibliometric studies.

Finally, each discourse has its own genre and its own concerns documented after observations and experimentation, and this is where ChatGPT falls short. Consequently, ChatGPT with its inherent advantages and disadvantages should only be considered a tool. The knowledge created by Generative AI far exceeds its comprehension by human beings and it offers a difficult set of challenges to science.

References

  1. Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019 Jan;25(1):44-56. https://doi.org/10.1038/s41591-018-0300-7
  2. Hosseini M, Rasmussen LM, Resnik DB. Using AI to write scholarly publications. Account Res. 2023 Jan; (25):1-9. https://doi.org/10.1080/08989621.2023.2168535
  3. van Dis EAM, Bollen J, Zuidema W, van Rooij R, Bockting CL. ChatGPT: five priorities for research. Nature. 2023 Feb;614(7947):224-226. https://doi.org/10.1038/d41586-023-00288-7
  4. Liebrenz M, Schleifer R, Buadze A, Bhugra D, Smith A. Generating scholarly content with ChatGPT: ethical challenges for medical publishing. Lancet Digit Health. 2023 Feb; 6:(5):E105-E106. https://doi.org/10.1016/S2589-7500(23)00019-5
  5. Stokel-Walker C. ChatGPT listed as author on research papers: many scientists disapprove. Nature. 2023 Feb;613(7945):620-621. https://doi.org/10.1038/d41586-023-00107-z
  6. Lund BD, Wang T, Mannuru NR, Nie B, Shimray S, Wang Z. ChatGPT and a new academic reality: Artificial Intelligence-written research papers and the ethics of the large language models in scholarly publishing. JASIST. 2023 Mar; 1– 12. https://doi.org/10.1002/asi.24750
  7. Smith-Laing T. An Analysis of Michel Foucault’s what is an Author? Taylor & Francis; 2018
  8. Potts J. The Near-death of the Author: Creativity in the Internet Age. University of Toronto Press, 2022.
  9. Chomsky, N., Roberts, I., Watumull, J., & Chomsky, N. The false promise of ChatGPT. The New York Times. 2023 [Cited 2023 January 28]. Available from: https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chatgpt-ai.html
  10. Zielinski C, Winker M, Aggarwal R, et al; WAME Board. Chatbots, ChatGPT, and scholarly manuscripts: WAME recommendations on ChatGPT and chatbots in relation to scholarly publications. WAME. 2023 January 20 [Cited 2023 January 28]. Available from: https://wame.org/page3.php?id=106
  11. Jain Alka. Elsevier, Cambridge University Press allow use of ChatGPT for academic writing. Mint. 2023 [Cited 2023 January 28]. Available from: https://www.livemint.com/news/world/elsevier-cambridge-university-press-allow-use-of-chatgpt-for-academic-writing-11679201026288.html
  12. The AI writing on the wall. Nature Machine Intelligence. January; 2023,5:1. https://doi.org/10.1038/s42256-023-00613-9