ChatGPT will not be replacing human teachers anytime soon, with a study into the technology’s capabilities finding it can’t help students critically understand academic assessment solutions.
The study from Charles Darwin University (CDU) explored whether ChatGPT could help students understand the answers in two financial accounting course unit assessments.
Study author and CDU Professor of Accounting Indra Abeysekera said ChatGPT and later version ChatGPT-4 were fed a series of multiple-choice questions from assessments from an introductory and advanced financial accounting course unit respectively.
ChatGPT correctly answered eight out of 10 questions from the introductory test, and scored 5 out of 10 in the advanced test. ChatGPT-4 correctly answered nine out of 10 questions, and scored seven out of 10 in the advanced test.
Professor Abeysekera said while the technology had the ability to correctly answer questions, it couldn’t support beginner learners to develop the knowledge and skills to find solutions.
“The solutions provided by ChatGPT showed that it is a solution provider rather than a teacher or instructor,” Professor Abeysekera said.
“ChatGPT can be constructive to a competent learner who has reached the competency level to further develop critical understanding. As research has indicated, high achievers can have a fear of numbers, and they can benefit by using ChatGPT solutions as validation checks for their learning.
“ChatGPT does not provide scaffolding for novice learners to take over their learning and develop individual competencies to be less or not reliant on it. It can be destructive to an incompetent learner and can serve as a platform to simply find the solution or as a channel to ease their fear of numbers.
“Furthermore, the findings showed that ChatGPT is not a foolproof solution provider, especially when questions have discipline-specific underlying assumptions and increased technical and task complexity.”
Professor Abeysekera said the study was inspired by the opportunities and risks ChatGPT poses to academia. He found while ChatGPT had many capabilities it could not surpass the critical thinking benefits of academia.
“ChatGPT can process unstructured data, convert it into information, and share it as knowledge,” Professor Abeysekera said.
“It can meet some criteria that meet originality, such as synthesising information differently and providing a new interpretation using known information. However, there are originality tasks that ChatGPT is incapable of meeting, at least for now. These include testing someone else’s idea in a different context or developing a research tool. Hence, academic focus on learning and assessment should shift toward original knowledge.”
ChatGPT and academia on accounting assessments was published in the Journal of Open Innovation: Technology, Market, and Complexity.
