Can AI trust you? Cultivating perceptions of trustworthiness in online CAs

by Eirine Ntaligkari
Thesis supervisor: dr. Theo Araujo

The aim of this research was to explore possible pathways that would enable the cultivation of trustworthiness perceptions towards disembodied conversational agents (DCAs; and more specifically, chatbots).

The main rationale behind the model lies in the idea that interpersonal communication findings could be applied in the online context, based on Computers Are Social Actors (CASA) paradigm by Nass & Reeves (1996). Taking that as a starting point, this study examined the effects of the Similarity-Attraction hypothesis (Berscheid & Walster, 1969; Byrne, 1970) of interpersonal communication, which posits that similarity between two subjects can impact attraction, and thus trustworthiness perceptions. The hypotheses were tested via an online experiment, through which users interacted with a DCA (Mis)matching of personality and anthropomorphism were tested as possible homophily cues, and possible paths between homophily, attraction and trustworthiness perceptions.

My experience using DigiComLab’s CART
My research design was enhanced by the Conversational Agent Research Toolkit developed by the University of Amsterdam’s Digital Communication Methods Lab (Araujo, 2019). The process was quite challenging, but also very interesting, and resulted into an exciting academic experience. The innovativeness of the toolkit assisted in designing proper manipulations for the experiment, as it allowed respondents to have a realistic conversation that simulated that of an existing chatbot (the look and feel of the created agents can be seen in Figure 1). During the process, we were able to make sure that responses and chat logs were kept anonymized and respondents’ data were kept safe. Apart from using the toolkit, the DigiCom Lab Grant enabled me to recruit respondents through the Lab, as well as through Amazon’s Mechanical Turk.

Results
The experimental results indicated that there is indeed a significant positive relationship between homophily and attraction between user and chatbot, as well as a significant positive relationship between attraction and trustworthiness perceptions, meaning that the Similarity-Attraction Hypothesis could be mirrored in the online world. What is more, trustworthiness perceptions towards the agent were found to have a positive impact on desired organizational outcomes, like recommendation acceptance and perceived trustworthiness of the organization. Nonetheless, (mis)matching personality and anthropomorphism were not sufficient to impact perceptions of homophily, and thus attraction and trustworthiness (which could be explained by the failure of some manipulations).

Contributions
This study adds to the existing DCA literature in the sense that it shed light in the area of trustworthiness perceptions towards agents, and more specifically, it combined existing findings related to the Similarity-Attraction hypothesis, matching personality and anthropomorphism. From a social perspective, the study aimed to help in understanding the dynamics of behaviors happening online, the trust relationships formations between human users and automated machines, as well as the connection of these relationships/dynamics with organizational outcomes.

Overall it was a really rewarding and fulfilling procedure that broadened my horizons and allowed me to explore the complex environment of online communication. The DigiCom Lab initiative helped me explore a new research methods and I am looking forward to see how future students will come up to using these tools.

References

Araujo, T. (2019, May). Going beyond the wizard: Using computational methods for conversational agent communication research. Presented at the International Communication Association (ICA) Conference, Washington DC, USA.

Berscheid, E. & Walster, E. (1974). Physical Attractiveness. Advances in Experimental Social Psychology (Vol. 7, pp. 157-215)

Reeves, B. & Nass, C. (1996). The media equation : how people treat computers, television, and new media like real people and places. 1996, Stanford, Calif.; New York: CSLI Publications ; Cambridge University Press.