ASSESSMENT OF THE ACCURACY OF VARIOUS ARTIFICIAL INTELLIGENCE CHATBOTS IN ANSWERING QUESTIONS ON FLUORIDE USE

Ataş E., Alkaç Ekici B. E.

29th Congress of the Balkan Stomatological Society – BaSS 2025, Belgrade, Serbia, 24 - 26 April 2025, pp.407, (Summary Text)

  • Publication Type: Conference Paper / Summary Text
  • City: Belgrade
  • Country: Serbia
  • Page Numbers: pp.407
  • Kütahya Health Sciences University Affiliated: Yes

Abstract

ASSESSMENT OF THE ACCURACY OF VARIOUS ARTIFICIAL INTELLIGENCE CHATBOTS IN ANSWERING QUESTIONS ON FLUORIDE USE

Beyza Ecem Alkaç Ekici, Esra Ataş Kütahya Health Sciences University, Faculty of Dentistry, Department of Pediatric Dentistry, Kütahya, Turkey

Objective: The aim of this study is to compare the accuracy of the answers provided by various artificial intelligence chatbots to questions related to fluoride use in pediatric dentistry.

Methods: In the study, ChatGPT 4.0, Claude and Google Gemini, were used. A total of 32 questions, which are frequently asked in the pedodontic clinic, whose answers to which are clearly stated in the AAPD and EAPD guidelines directed to chatbots twice one-week intervals. Their answers accuracy evaluated blindly according to the guidelines by a researcher and recorded as “true” or “false”. The data were analyzed with IBM SPSS V23 and Jamovi V2.3.28. The relevance between the answers over time was analyzed with Kappa and McNemar's tests. Cochran Q test was used to compare the accuracy rates of chatbot's (p<0.05).

Results: When the consistency of the chatbots was assessed between the first day and the first week, ChatGPT 4.0 and Claude demonstrated a moderate level of relevance, whereas GEMINI exhibited a very good level of relevance. Statistically significant higher accuracy was obtained with ChatGPT 4.0 and Claude compared to the Gemini. On first day, the difference between the chatbots was significant, but the multiple comparisons were not significant. In first week, the difference between the chatbots was not statistically significant.

Conclusions: Within the limitations of this study, it is thought that all chatbots showed a high level of accuracy and will not make a clinical difference. Further development through advanced studies is needed to improve their accuracy and reliability.

Keywords: Artificial intelligence, chatbots, fluoride, pediatric dentistry