Hello, my name is Dr. Katerina Matersky, and I am an assistant professor of nursing at the Daphne Koko School of Nursing at Toronto Metropolitan University in Toronto, Canada. In terms of the background for the study, the exponential growth in the popularity of Chad GPT has sparked concerns and academic integrity violations, including plagiarism, as students are now able to almost instantaneously complete assignments without much critical thought. Specifically in nursing education, the use of Chad GPT is concerning as the development of critical thinking skills is essential for nursing students within the decision-making process to implement patient safe care interventions. Therefore, the use of the software cannot replace the knowledge, skill, and judgment to be developed by students and demonstrated in their evolving nursing practice.
Although this is a recent concern within the academic community, there are no studies that evaluate whether educators could detect if an assignment was submitted by a student or written through Chad GPT. Thus, the purpose of this research is to learn whether educators can distinguish between AI-generated and student-crafted assignments while evaluating student work in undergraduate nursing theory courses.
In terms of the methods, four students were hand-picked from a second-year undergraduate nursing course that represented a wide variety of backgrounds and grades in that particular course. They were asked to submit two versions of a scholarly paper assignment: one that they wrote by themselves and one that they wrote through Chad GPT technology. Both versions were blinded and used the same prompts when writing the assignment outline. Then, they were submitted to me as the educator and evaluator, and I reflected on my experience of evaluating both versions of the same scholarly assignment.
In terms of the findings, I present detailed findings on the slide, but I will highlight some of the key findings for you as well. When looking at the AI-generated papers in comparison to student-written papers, the AI papers were written better grammatically. Looking at the references, AI papers generally relied more on the use of outdated literature and lacked in providing conceptual definitions of key concepts that students were required to use in their papers. As well, the references were largely beyond the 10-year currency requirement for this particular assignment, and when I clicked on these references or tried to locate them, they did not exist, which is what Chad GPT terms as being hallucinations.
In student-generated papers, there were a lot more scholarly references that were included, as well as conceptual depth and definitions provided. Overall, the AI papers lacked critical thinking, depth, and discussion. The paragraphs were very short, and the structure of the paper did not go into the level of depth that I had talked about and requested students to include as part of a lecture for this course. Whereas the student-generated papers really included that intricate connection, critical thinking, and application to their evolving nursing practice.
In conclusion, AI-written papers and student-written papers can be distinguished, but prompts can reduce the differences that were noted. As students are getting better at introducing and including the various prompts into Chad GPT, this distinguishable difference will become smaller and smaller. Finally, when we look at course assignments, they must take AI technology into consideration, particularly as we discussed, it is very important for nursing students to develop their critical thinking skills. So when putting together course assignments, we really need to think as educators on how we can still continue to evolve critical thinking skills in our nursing students in the event that nursing students do go to Chad GPT to generate those course assignments.
On this slide, I present my references via QR code, and my contact information can be found at the bottom. Thank you for listening.
