A protocol for collecting speech data with varying degrees of trust

Abstract

This paper describes a novel experimental setup for collecting speech data from subjects induced to have different degrees of trust in the skills of a conversational agent. The protocol consists of an interactive session where the subject is asked to respond to a series of factual questions with the help of a virtual assistant. In order to induce subjects to either trust or distrust the agent?s skills, they are first informed that the agent was previously rated by other users as being either good or bad; subsequently, the agent answers the subjects? questions consistently to its alleged abilities. These interactions will be speech-based, with subjects and agents communicating verbally, which will allow for the recording of speech produced under different trust conditions. Ultimately, the resulting dataset will be usedto study the feasibility of automatically predicting the degree of trust from speech. This paper describes a preliminary experiment using a text-only version of the protocol in Argentine Spanish. The results show that the protocol effectively succeeds in influencing subjects into the desired mental state of either trusting or distrusting the agent?s skills. We are currently beginning the collection of the speech dataset, which will be made publicly available once ready.