Researchers from Charles University, Czech Republic used ChatGPT-3 to generate a completely fabricated scientific article in the field of neurosurgery. Questions and prompts were refined as ChatGPT generated responses, allowing the quality of the output to be iteratively improved.
The results of this proof-of-concept study were striking—the AI language model successfully produced a fraudulent article that closely resembled a genuine scientific paper in terms of word usage, sentence structure, and overall composition. The article included standard sections such as an abstract, introduction, methods, results, and discussion, as well as tables and other data. The entire process of article creation took just one hour without any special training of the human user.
However, examination expert readers were able to identify semantic inaccuracies and errors particularly in the references—some references were incorrect, while others were non-existent.
Tracking
As AI continues to advance, it becomes crucial for the scientific community to verify the accuracy and authenticity of content generated by these tools and implement mechanisms for detecting and preventing fraud and misconduct, says the researchers. How this could be achieved is less clear.
“We should at least declare the extent to which AI has assisted the writing and analysis of a paper as a starting point,” suggests author Dr. Pedro L. Ballester. Another possible solution proposed by colleagues is making the submission of data sets mandatory.
Citation: Pedro L. Ballester. Open Science and Software Assistance: Commentary on “Artificial Intelligence Can Generate Fraudulent but Authentic-Looking Scientific Medical Articles: Pandora’s Box Has Been Opened." https://www.jmir.org/2023/1/e46924/ doi: 10.2196/46924