Assessing Engagement of the Elderly in Active Listening from Body Movement

Abstract: This study propose s a method for estimating the conversational state in active listening using voice and body movement data to facilitate for participants to exchange their words. In recent years, the problem of apathy among the elderly has run into a serious problem as the population ages. Active listening, a type of counseling technique, is useful to address the problem. In active listening, to activate the elderly person, a listener should do nothing other than listen to what the elderly person talks. However, it is difficult for the elderly person to willingly talk because they are not familiar enough with each other to make conversation in a frank mode. In the study, the body movement of the elderly person is recorded as well as the voice of both participants. Hidden Markov models , to which those data are fed, estimate the latent conversational state during active listening. A random forest models are constructed to examine the importance of each variables fed to the hidden Markov model.To test the usefulness of the proposed method, a one-on-one listening experiment is conducted between a listener and a speaker. The difference in the body movement derives two personas, for each of which hidden states are estimated. The body movement data with a small variance estimates three explicit states of the listener's speech, the speaker's speech, and silence, as well as two implicit states of the speaker's thinking and the speaker's laughter. On the other hand, the persona of large body movement variation indicates the same three explicit states, as well as the implicit state of the speaker's giving responses and an uninterpretable state. The result above indicates that it is possible to estimate almost all of the conversational states. Labeling manually some of the voice data, random forest models are constructed to know the importance of the variables. It turns out the mean of the body movements has the highest importance for the body movement with a small variance, while the maximum value of the voice per interval has the highest importance for that with a large variance.An initial prediction using only voice data presents the accuracy of 0.61 and 0.59 for body movement with small variance and for that with large variance, respectively. On the contrary, the prediction using body movement greatly improves the accuracy to 0.96 and 0.99 for body movement with small variance and large variance, respectively. This suggests that body movement is useful for estimating the conversational state.The method for state estimation enables us to automate the labeling of conversational states that previously had to be done manually.Furthermore, the method to find hidden conversational states the analyst has not assumed can provide a stepping stone to facilitating listening. The uninterpretable state may come up because of the shortage of information to be fed to the model.The need for biometric data other than body movements and video data during listening is indicated to interpret the uninterpretable state.

Keywords: body movements, listening carefully, elderly people

DOI: 10.54941/ahfe1004022

Cite this paper:

​