[en] Virtual reality has shown great potential in many fields, especially in business and psychology. By immersing someone in a new computer-generated reality, it is possible to create realistic, safe, and controllable simulations for research and training, as well as new three-dimensional-enriched consumer experiences and services. Most of these environments, especially in the metaverse, rely on virtual representations of people called avatars. The design and non-verbal behaviors of these avatars must be carefully crafted to provide a realistic and truly immersive experience. This paper aims to understand how avatar nonverbal behaviors (i.e., body posture, facial expression, and head movement) are perceived by users immersed in a virtual reality context, a very common situation encountered in many simulations and especially during training. Therefore, the first objective of this study is to validate, through an experiment with 125 participants, how the audience's levels of emotional valence and arousal are perceived in virtual reality. Based on these results, a library of audience non-verbal behaviors corresponding to different arousal and valence levels is now available for future applications. The experiment also examines the benefits of using low-end versus high-end virtual reality headsets, and photo-realistic versus cartoon avatars. The results have implications for the design of realistic, challenging, and interactive virtual audiences.
Disciplines :
Quantitative methods in economics & management Treatment & clinical psychology
Leclercq, Anne-Lise ; Université de Liège - ULiège > Département de Logopédie ; Université de Liège - ULiège > Unités de recherche interfacultaires > Research Unit for a life-Course perspective on Health and Education (RUCHE)
Remacle, Angélique ; Université de Liège - ULiège > Département de Logopédie > Logopédie des troubles de la voix ; Université de Liège - ULiège > Unités de recherche interfacultaires > Research Unit for a life-Course perspective on Health and Education (RUCHE)
Amin, A., Gromala, D., Tong, X., & Shaw, C. (2016). Immersion in cardboard VR compared to a traditional head-mounted display, In International conference on virtual, augmented and mixed reality (pp. 269–276). Springer.
Argyle, M., & Ingham, R. (1972). Gaze, mutual gaze, and proximity. Semiotica, 6(1), 32–49.
Babin, B. J., & Attaway, J. S. (2000). Atmospheric affect as a tool for creating value and gaining share of customer. Journal of Business Research, 49(2), 91–99.
Batrinca, L., Stratou, G., Shapiro, A., Morency, L.-P., & Scherer, S. (2013). Cicero-towards a multimodal virtual audience platform for public speaking training. In International Workshop on Intelligent Virtual Agents (pp. 116–128). Springer.
Beck, J., Rainoldi, M., & Egger, R. (2019). Virtual reality in tourism: A state-of-the-art review. Tourism Review, 74(3), 586–612.
Belanche, D., Casaló, L. V., Schepers, J., & Flavián, C. (2021). Examining the effects of robots' physical appearance, warmth, and competence in frontline services: The humanness-value-loyalty model. Psychology & Marketing, 38(12), 2357–2376.
Benbasat, I., & Wang, W. (2005). Trust in and adoption of online recommendation agents. Journal of the Association for Information Systems, 6(3), 72–101.
Bonetti, F., Warnaby, G., & Quinn, L. (2018). Augmented reality and virtual reality in physical and online retailing: A review, synthesis and research agenda. Augmented Reality and Virtual Reality, 119–132.
Bouchard, S., Dumoulin, S., Robillard, G., Guitard, T., Klinger, É., Forget, H., Loranger, C., & Roucaut, F.-X. (2017). Virtual reality compared with in vivo exposure in the treatment of social anxiety disorder: A three-arm randomised controlled trial. British Journal of Psychiatry, 210(4), 276–283.
Bouchard, S., & Robillard, G. (2019). Validation canadienne-française du Gatineau Presence Questionnaire auprès d'adultes immergés en réalité virtuelle. 87e Congrès de l'ACFAS, Québec, mai 2019.
Boyd, D. E., & Koles, B. (2019). An introduction to the special issue “virtual reality in marketing”: Definition, theory and practice. Journal of Business Research, 100, 441–444.
Brady, M. K., Robertson, C. J., & Cronin, J. J. (2001). Managing behavioral intentions in diverse cultural environments: an investigation of service quality, service value, and satisfaction for American and Ecuadorian fast-food customers. Journal of International Management, 7(2), 129–149.
Butt, A. H., Ahmad, H., Goraya, M. A., Akram, M. S., & Shafique, M. N. (2021). Let's play: Me and my ai-powered avatar as one team. Psychology & Marketing, 38(6), 1014–1025.
Cafaro, A., Vilhjálmsson, H. H., & Bickmore, T. (2016). First impressions in human–agent virtual encounters. ACM Transactions on Computer-Human Interaction, 23(4), 1–40.
Chollet, M., Marsella, S., & Scherer, S. (2021). Training public speaking with virtual social interactions: Effectiveness of real-time feedback and delayed feedback. Journal on Multimodal User Interfaces, 16, 17–29.
Chollet, M., Massachi, T., & Scherer, S. (2016a). Investigating the physiological responses to virtual audience behavioral changes a stress-aware audience for public speaking training, In IVA 2017 Workshop on Physiologically-Aware Virtual Agents.
Chollet, M., & Scherer, S. (2017). Perception of virtual audiences. IEEE computer graphics and applications, 37(4), 50–59.
Chollet, M., Wörtwein, T., Morency, L.-P., & Scherer, S. (2016b). A multimodal corpus for the assessment of public speaking ability and anxiety, In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16) (pp. 488–495).
Chollet, M., Wörtwein, T., Morency, L.-P., Shapiro, A., & Scherer, S. (2015). Exploring feedback strategies to improve public speaking: An interactive virtual audience framework, In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (pp. 1143–1154).
Craske, M. G., Treanor, M., Conway, C. C., Zbozinek, T., & Vervliet, B. (2014). Maximizing exposure therapy: An inhibitory learning approach. Behaviour Research and Therapy, 58, 10–23.
De Keyser, A., Köcher, S., Alkire (née Nasr), L., Verbeeck, C., & Kandampully, J. (2019). Frontline service technology infusion: Conceptual archetypes and future research directions. Journal of Service Management, 30(1), 156–183.
Deng, X., Unnava, H. R., & Lee, H. (2019). “too true to be good?” when virtual reality decreases interest in actual reality. Journal of Business Research, 100, 561–570.
Di Natale, A. F., Simonetti, M. E., La Rocca, S., & Bricolo, E. (2023). Uncanny valley effect: A qualitative synthesis of empirical research to assess the suitability of using virtual faces in psychological research. Computers in Human Behavior Reports, 10, 100288.
Dwivedi, Y. K., Hughes, L., Wang, Y., Alalwan, A. A., Ahn, S. J., Balakrishnan, J., Barta, S., Belk, R., Buhalis, D., & Dutot, V. (2023). Metaverse marketing: How the metaverse will shape the future of consumer research and practice. Psychology & Marketing, 40(4), 750–776.
Eckert, D., & Mower, A. (2020). The effectiveness of virtual reality soft skills training in the enterprise: A study. PwC Public Report.
Farrell, A. M., Souchon, A. L., & Durden, G. R. (2001). Service encounter conceptualisation: Employees' service behaviours and customers' service quality perceptions. Journal of Marketing Management, 17(5–6), 577–593.
Flavián, C., Ibáñez-Sánchez, S., & Orús, C. (2019). The impact of virtual, augmented and mixed reality technologies on the customer experience. Journal of Business Research, 100, 547–560.
Fraune, M. R., Oisted, B. C., Sembrowski, C. E., Gates, K. A., Krupp, M. M., & Šabanović, S. (2020). Effects of robot-human versus robot-robot behavior and entitativity on anthropomorphism and willingness to interact. Computers in Human Behavior, 105, 106220.
Freeman, D., Reeve, S., Robinson, A., Ehlers, A., Clark, D., Spanlang, B., & Slater, M. (2017). Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychological Medicine, 47(14), 2393–2400.
Glémarec, Y., Bosser, A.-G., Buche, C., Lugrin, J.-L., Landeck, M., Latoschik, M., & Chollet, M. (2019). A scalability benchmark for a virtual audience perception model in virtual reality. In: 25th ACM Symposium on Virtual Reality Software and Technology, (1–1).
Goberman, A. M., Hughes, S., & Haydock, T. (2011). Acoustic characteristics of public speaking: Anxiety and practice effects. Speech Communication, 53(6), 867–876.
Hainey, T., Connolly, T. M., Boyle, E. A., Wilson, A., & Razak, A. (2016). A systematic literature review of games-based learning empirical evidence in primary education. Computers & Education, 102, 202–223.
Hennig-Thurau, T., Aliman, D. N., Herting, A. M., Cziehso, G. P., Linder, M., & Kübler, R. V. (2022). Social interactions in the metaverse: Framework, initial evidence, and research roadmap. Journal of the Academy of Marketing Science, 1–25.
Hilken, T., Chylinski, M., de Ruyter, K., Heller, J., & Keeling, D. I. (2022). Exploring the frontiers in reality-enhanced service communication: From augmented and virtual reality to neuro-enhanced reality. Journal of Service Management, 33(4/5), 657–674.
Howard, M. C. (2017). A meta-analysis and systematic literature review of virtual reality rehabilitation programs. Computers in Human Behavior, 70, 317–327.
Howard, M. C., Gutworth, M. B., & Jacobs, R. R. (2021). A meta-analysis of virtual reality training programs. Computers in Human Behavior, 121, 106808.
Jerald, J. (2015). The VR book: Human-centered design for virtual reality. Morgan & Claypool.
Jingen Liang, L., & Elliot, S. (2021). A systematic review of augmented reality tourism research: What is now and what is next? Tourism and Hospitality Research, 21(1), 15–30.
Kang, H., & Kim, H. K. (2020). My avatar and the affirmed self: Psychological and persuasive implications of avatar customization. Computers in Human Behavior, 112, 106446.
Kang, N., Brinkman, W.-P., Birna van Riemsdijk, M., & Neerincx, M. (2016). The design of virtual audiences: Noticeable and recognizable behavioral styles. Computers in Human Behavior, 55, 680–694.
Khandelwal, K., & Upadhyay, A. K. (2021). Virtual reality interventions in developing and managing human resources. Human Resource Development International, 24(2), 219–233.
Kim, D. Y., Lee, H. K., & Chung, K. (2023). Avatar-mediated experience in the metaverse: The impact of avatar realism on user-avatar relationship. Journal of Retailing and Consumer Services, 73, 103382.
Kim, S. Y., Schmitt, B. H., & Thalmann, N. M. (2019). Eliza in the uncanny valley: Anthropomorphizing consumer robots increases their perceived warmth but decreases liking. Marketing Letters, 30(1), 1–12.
Kleinsmith, A., Rivera-Gutierrez, D., Finney, G., Cendan, J., & Lok, B. (2015). Understanding empathy training with virtual patients. Computers in Human Behavior, 52, 151–158.
Knapp, M. L., Hall, J. A., & Horgan, T. G. (2013). Nonverbal communication in human interaction. Cengage Learning.
Koles, B., & Nagy, P. (2021). Digital object attachment. Current Opinion in Psychology, 39, 60–65.
Krumhuber, E., Manstead, A. S. R., Cosker, D., Marshall, D., & Rosin, P. L. (2009). Effects of dynamic attributes of smiles in human and synthetic faces: A simulated job interview setting. Journal of Nonverbal Behavior, 33(1), 1–15.
Laforest, M., Bouchard, S., Crétu, A.-M., & Mesly, O. (2016). Inducing an anxiety response using a contaminated virtual environment: Validation of a therapeutic tool for obsessive–compulsive disorder. Frontiers in ICT, 3, 18.
Larivière, B., Bowen, D., Andreassen, T. W., Kunz, W., Sirianni, N. J., Voss, C., Wünderlich, N. V., & De Keyser, A. (2017). “service encounter 2.0”: An investigation into the roles of technology, employees and customers. Journal of Business Research, 79, 238–246.
Letheren, K., Jetten, J., Roberts, J., & Donovan, J. (2021). Robots should be seen and not heard. sometimes: Anthropomorphism and ai service robot interactions. Psychology & Marketing, 38(12), 2393–2406.
Leveau, P.-H., & Camus, S. (2023). Embodiment, immersion, and enjoyment in virtual reality marketing experiences. Psychology & Marketing, 40(7), 1329–1343.
Liao, H., & Chuang, A. (2004). A multilevel investigation of factors influencing employee service performance and customer outcomes. Academy of Management Journal, 47(1), 41–58.
Lloyd, A. E., & Luk, S. T. K. (2011). Interaction behaviors leading to comfort in the service encounter. Journal of Services Marketing, 25(3), 176–189.
Loureiro, S. M. C., Guerreiro, J., Eloy, S., Langaro, D., & Panchapakesan, P. (2019). Understanding the use of virtual reality in marketing: A text mining-based review. Journal of Business Research, 100, 514–530.
Lugrin, J.-L., Latoschik, M. E., Habel, M., Roth, D., Seufert, C., & Grafe, S. (2016). Breaking bad behaviors: A new tool for learning classroom management using virtual reality. Frontiers in ICT, 3, 26.
Maria Stock, R., Jong, A. d., & Zacharias, N. A. (2017). Frontline employees' innovative service behavior as key to customer loyalty: Insights into fles' resource gain spiral. Journal of Product Innovation Management, 34(2), 223–245.
Miao, F., Kozlenkova, I. V., Wang, H., Xie, T., & Palmatier, R. W. (2022). An emerging theory of avatar marketing. Journal of Marketing, 86(1), 67–90.
Murphy, J., Gretzel, U., & Pesonen, J. (2019). Marketing robot services in hospitality and tourism: The role of anthropomorphism. Journal of Travel & Tourism Marketing, 36(7), 784–795.
Nash, E. B., Edwards, G. W., Thompson, J. A., & Barfield, W. (2000). A review of presence and performance in virtual environments. International Journal of Human-Computer Interaction, 12(1), 1–41.
Orús, C., Ibáñez-Sánchez, S., & Flavián, C. (2021). Enhancing the customer experience with virtual and augmented reality: The impact of content and device type. International Journal of Hospitality Management, 98, 103019.
Pertaub, D.-P., Slater, M., & Barker, C. (2002). An experiment on public speaking anxiety in response to three different types of virtual audience. Presence: Teleoperators and Virtual Environments, 11(1), 68–78.
Poeschl, S. (2017). Virtual reality training for public speaking - A QUEST-VR framework validation. Frontiers in ICT, 4, 13.
Rafaeli, A., Altman, D., Gremler, D. D., Huang, M.-H., Grewal, D., Iyer, B., Parasuraman, A., & de Ruyter, K. (2017). The future of frontline research: Invited commentaries. Journal of Service Research, 20(1), 91–99.
Remacle, A., Bouchard, S., Etienne, A.-M., Rivard, M.-C., & Morsomme, D. (2021). A virtual classroom can elicit teachers’ speech characteristics: Evidence from acoustic measurements during in vivo and in virtuo lessons, compared to a free speech control situation. Virtual Reality, 25, 935–944.
Remacle, A., Bouchard, S., & Morsomme, D. (2023). Can teaching simulations in a virtual classroom help trainee teachers to develop oral communication skills and self-efficacy? a randomized controlled trial. Computers & Education, 200, 104808.
Robinson, S., Orsingher, C., Alkire, L., De Keyser, A., Giebelhausen, M., Papamichail, K. N., Shams, P., & Temerak, M. S. (2020). Frontline encounters of the ai kind: An evolved service encounter framework. Journal of Business Research, 116, 366–376.
Scheveneels, S., Boddez, Y., Van Daele, T., & Hermans, D. (2019). Virtually unexpected: No role for expectancy violation in virtual reality exposure for public speaking anxiety. Frontiers in Psychology, 10, 2849.
Schrader, C. (2019). Creating avatars for technology usage: Context matters. Computers in Human Behavior, 93, 219–225.
Schwind, V., Knierim, P., Haas, N., & Henze, N. (2019). Using presence questionnaires in virtual reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1–12).
Selzer, M. N., Gazcon, N. F., & Larrea, M. L. (2019). Effects of virtual presence and learning outcome using low-end virtual reality systems. Displays, 59, 9–15.
Seufert, C., Oberdörfer, S., Roth, A., Grafe, S., Lugrin, J.-L., & Latoschik, M. E. (2022). Classroom management competency enhancement for student teachers using a fully immersive virtual classroom. Computers & Education, 179, 104410.
Seymour, M., Riemer, K., & Kay, J. (2018). Actors, avatars and agents: Potentials and implications of natural face technology for the creation of realistic visual presence. Journal of the Association for Information Systems, 19(10), 953–981.
Seymour, M., Yuan, L., Dennis, A. R., & Riemer, K. (2021). Have we crossed the uncanny valley? understanding affinity, trustworthiness, and preference for realistic digital humans in immersive environments. Journal of the Association for Information Systems, 22(3), 591–617.
Sibilla, F., & Mancini, T. (2018). I am (not) my avatar: A review of the user-avatar relationships in massively multiplayer online worlds. Cyberpsychology: Journal of Psychosocial Research on Cyberspace, 12(3):Article 4.
Slater, M. (2003). A note on presence terminology. Presence Connect, 3(3), 1–5.
Slater, M., & Steed, A. (2000). A virtual presence counter. Presence: Teleoperators and Virtual Environments, 9(5), 413–434.
Smith, S., & Wheeler, J. (2002). Managing the customer experience: Turning customers into advocates. Pearson Education.
Solomon, M. R., Surprenant, C., Czepiel, J. A., & Gutman, E. G. (1985). A role theory perspective on dyadic interactions: The service encounter. Journal of Marketing, 49(1), 99–111.
Sundaram, D. S., & Webster, C. (2000). The role of nonverbal communication in service encounters. Journal of Services Marketing, 14(5), 378–391.
van Bennekom, M. J., de Koning, P. P., & Denys, D. (2017). Virtual reality objectifies the diagnosis of psychiatric disorders: A literature review. Frontiers in Psychiatry, 8, 163.
Van Pinxteren, M. M. E., Wetzels, R. W. H., Rüger, J., Pluymaekers, M., & Wetzels, M. (2019). Trust in humanoid robots: Implications for services marketing. Journal of Services Marketing, 33(4), 507–518.
Vlachopoulos, D., & Makri, A. (2017). The effect of games and simulations on higher education: A systematic literature review. International Journal of Educational Technology in Higher Education, 14(1), 22.
Weech, S., Kenny, S., & Barnett-Cowan, M. (2019). Presence and cybersickness in virtual reality are negatively related: A review. Frontiers in Psychology, 10, 158.
Wörtwein, T., Chollet, M., Schauerte, B., Morency, L.-P., Stiefelhagen, R., & Scherer, S. (2015). Multimodal public speaking performance assessment. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (pp. 43–50).
Yao, Q., Kuai, L., & Jiang, L. (2023). Effects of the anthropomorphic image of intelligent customer service avatars on consumers' willingness to interact after service failures. Journal of Research in Interactive Marketing. Advance published online.
