Investigating the Use of Chatbots as an Educational Tool

Abstract

With the rise of online asynchronous learning and low levels of instructor presence, students have become self-regulated learners who must monitor their performance and adapt their learning strategies as necessary. Previous studies have shown that chatbots are a promising alternative to traditional study tools such as flashcards. This study examined the effects of a chatbot’s embodiment (Humanoid, Animated) and conversational style (Formal, Informal) on learning performance and behavioral engagement. Participants were asked to watch a lecture video and interact with a chatbot to review the material. After studying with the chatbot, participants completed a quiz that was evenly split according to difficulty (easy versus hard questions), as well as study type (questions that were and were not studied with the chatbot). Participants’ ratings of usability, usefulness, ease of use, and affective engagement were obtained. Results showed that participants performed better on easy questions than hard questions. Additionally, participants performed better on studied questions than non-studied questions. However, for the informal conversational style, participants scored higher on hard questions than easy questions amongst studied questions. Embodiment and conversational style had no impact on behavioral engagement. Overall, participants rated the chatbot above average in terms of its usability, usefulness, ease of use, and affective engagement. We conclude that chatbots are an effective study tool, but they may be better suited for learning easy, surface-level knowledge. Additionally, an informal conversational style may be preferred since it matches the linguistic features used by human tutors. Limitations and future directions for research are discussed.

Keywords: conversational agent, pedagogical agent, online learning, instructional design, computer assisted instruction

DOI: 10.54941/ahfe1005752