NOTICE: this is the author’s version of a work that was accepted for publication in the International Journal of Human-Computer Studies. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in International Journal of Human - Computer Studies 72/1 (2014), pp. 23-32, DOI:10.1016/j.ijhcs.2013.09.004
All documents in ORBi are protected by a user license.
[en] While 3D cinema is becoming increasingly established, little effort has focused on the general problem of producing a 3D sound scene spatially coherent with the visual content of a stereoscopic-3D (s-3D) movie. The perceptual relevance of such spatial audiovisual coherence is of significant interest. In this paper, a subjective experiment is carried out where an angular error between an s-3D video and a spatially accurate sound reproduced through Wave Field Synthesis (WFS) is simulated. The psychometric curve is measured with the method of constant stimuli, and the threshold for bimodal integration is estimated. The impact of the presence of background noise is also investigated. A comparison is made between the case without any background noise and the case with an SNR of 4 dB(A). Estimates of the thresholds and the slopes, as well as their confidence intervals, are obtained for each level of background noise. When background noise was present, the point of subjective equality (PSE) was higher (19.4° instead of 18.3°) and the slope was steeper (-0.077 instead of -0.062 per degree). Because of the overlap between the confidence intervals, however, it was not possible to statistically differentiate between the two levels of noise. The implications for the sound reproduction in a cinema theater are discussed.
Disciplines :
Electrical & electronics engineering
Author, co-author :
André, Cédric ; Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Dép. d'électric., électron. et informat. (Inst.Montefiore)
Corteel, Etienne; sonic emotion labs
Embrechts, Jean-Jacques ; Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Techniques du son et de l'image
Verly, Jacques ; Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Exploitation des signaux et images
Katz, Brian F.G.; LIMSI-CNRS > Communication Homme-Machine > Groupe Audio & Acoustique
Language :
English
Title :
Subjective Evaluation of the Audiovisual Spatial Congruence in the Case of Stereoscopic-3D Video and Wave Field Synthesis
D. Alais, and D. Burr The ventriloquist effect results from near-optimal bimodal integration Current Biology 14 2004 257 262 10.1016/j.cub.2004.01.029
André, C.R.; Embrechts, J.J.; Verly, J.G.; 2010. Adding 3D sound to 3D cinema: identification and evaluation of different reproduction techniques. In: Proceedings of the Second International Conference on Audio Language and Image Processing (ICALIP 2010), pp. 130-137. http://dx.doi.org/10. 1109/ICALIP.2010.5684993.
André, C.R.; Rébillat, M.; Embrechts, J.J.; Verly, J.G.; Katz, B.F.G.; 2012. Sound for 3D cinema and the sense of presence. In: Proceedings of the 18th International Conference on Auditory Display (ICAD 2012), Atlanta, GA, pp. 14-21.
P.W. Battaglia, R.A. Jacobs, and R.N. Aslin Bayesian integration of visual and auditory signals for spatial localization Journal of the Optical Society of America 20 2003 1391 1397 10.1364/JOSAA.20.001391
A.J. Berkhout A holographic approach to acoustic control Journal of the Audio Engineering Society 36 1988 977 995
J. Braasch, and K. Hartung Localization in the presence of a distracter and reverberation in the frontal horizontal plane. I. Psychoacoustical data Acta Acustica united with Acustica 88 2002 942 955
de Bruijn, W.P.J.; Boone, M.M.; 2003. Application of Wave Field Synthesis in life-size videoconferencing. In: Audio Engineering Society Convention 114.
Corteel, É.; 2006. On the use of irregularly spaced loudspeaker arrays for Wave Field Synthesis, potential impact on spatial aliasing frequency, in: Proceedings of the Ninth International Conference on Digital Audio Effects (DAFx'06), Montréal, Canada.
Corteel, É.; Rohr, L.; Falourd, X.; NGuyen, K.V.; Lissek, H.; 2012. Practical 3-dimensional sound reproduction using Wave Field Synthesis, theory and perceptual validation. In: Proceedings of the 11th French Congress of Acoustics and 2012 Annual IOA Meeting, Nantes, France, pp. 895-900.
M. Courgeon, and C. Clavel MARC a framework that features emotion models for facial animation during human-computer interaction Journal of Multimodal User Interfaces 2013 2013 1 9 10.1007/s12193-013-0124-1
Courgeon, M.; Rébillat, M.; Katz, B.F.; Clavel, C.; Martin, J.C.; 2010. Life-sized audiovisual spatial social scenes with multiple characters: MARC & SMART-I2. In: Proceedings of the 5èmes Journées de l'AFRV, Orsay, France.
Dodgson, N.A.; 2004. Variation and extrema of human interpupillary distance. In: Proceedings of SPIE 5291, San Jose, CA, pp. 36-46, http://dx.doi.org/10.1117/12.529999.
Doukhan, D.; Rilliard, A.; Rosset, S.; Adda-Decker, M.; d'Alessandro, C.; 2011. Prosodic analysis of a corpus of tales. In: INTERSPEECH - 2011, pp. 3129-3132.
M.O. Ernst, and M.S. Banks Humans integrate visual and haptic information in a statistically optimal fashion Nature 415 2002 429 433 10.1038/415429a
Évrard, M.; André, C.R.; Verly, J.G.; Embrechts, J.J.; Katz, B.F.G.; 2011. Object-based sound re-mix for spatially coherent audio rendering of an existing stereoscopic-3D animation movie. In: Audio Engineering Society Convention 131.
M.D. Good, and R.H. Gilkey Sound localization in noise the effect of signal-to-noise ratio Journal of the Acoustical Society of America 99 1996 1108 1117 10.1121/1.415233
ITU, 2003. Recommendation BS.1284. General Methods for the Subjective Assessment of Sound Quality. ITU-R.
C.V. Jackson Visual factors in auditory localization Quarterly Journal of Experimental Psychology 5 1953 52 65 10.1080/17470215308416626
S. Komiyama Subjective evaluation of angular displacement between picture and sound directions for HDTV sound systems Journal of the Audio Engineering Society 37 1989 210 214
C.F. Lam, J.R. Dubno, and J.H. Mills Determination of optimal data placement for psychometric function estimation a computer simulation Journal of the Acoustical Society of America 106 1999 1969 10.1121/1.427944
J. Lewald, W.H. Ehrenstein, and R. Guski Spatio-temporal constraints for auditory-visual integration Behavioural Brain Research 121 2001 69 79 10.1016/S0166-4328(00)00386-7
J. Lewald, and R. Guski Cross-modal perceptual integration of spatially and temporally disparate auditory and visual stimuli Cognitive Brain Research 16 2003 468 478 10.1016/S0926-6410(03)00074-0
C. Leys, C. Ley, O. Klein, P. Bernard, and L. Licata Detecting outliers do not use standard deviation around the mean, use absolute deviation around the median Journal of Experimental Social Psychology 49 2013 764 766 10.1016/j.jesp.2013.03.013
C. Lorenzi, S. Gatehouse, and C. Lever Sound localization in noise in normal-hearing listeners Journal of the Acoustical Society of America 105 1999 1810 1820 10.1121/1.426719
Melchior, F.; Brix, S.; Sporer, T.; Roder, T.; Klehs, B.; 2003. Wave Field Synthesis in combination with 2D video projection. In: AES 24th International Conference.
Melchior, F.; Fischer, J.; de Vries, D.; 2006. Audiovisual perception using Wave Field Synthesis in combination with augmented reality systems: horizontal positioning. In: AES 28th International Conference.
Perrott, D.R.; 1993. Auditory and visual localization: two modalities, one world. In: AES 12th International Conference, pp. 221-231.
M. Rébillat, X. Boutillon, É. Corteel, and B.F.G. Katz Audio, visual, and audio-visual egocentric distance perception by moving subjects in virtual environments ACM Transactions on Applied Perception 9 2012 19:1 19:17 10.1145/2355598.2355602
Rébillat, M.; Corteel, É.; Katz, B.F.G.; 2008. SMART-I 2: spatial multi-user audio-visual real time interactive interface. In: Audio Engineering Society Convention 125.
Rébillat, M.; Katz, B.F.G.; Corteel, É.; 2009. SMART-I 2: "spatial Multi-user Audio-visual Real-time Interactive Interface", A Broadcast Application Context. In: Proceedings of the 3DTV Conference, Potsdam, Germany, http://dx.doi.org/10.1109/3DTV.2009.5069682.
D.A. Slutsky, and G.H. Recanzone Temporal and spatial dependency of the ventriloquism effect Neuroreport 12 2001 7 10 11201094
Start, E.W.; 1997. Direct Sound Enhancement by Wave Field Synthesis. Ph.D. Thesis. TU Delft, The Netherlands.
Theile, G.; Wittek, H.; Reisinger, M.; 2003. Potential wavefield synthesis applications in the multichannel stereophonic world. In: Proceedings of the AES 24th International Conference.
W.R. Thurlow, and C.E. Jack Certain determinants of the "ventriloquism effect" Perceptual and Motor Skill 36 1973 1171 1184 10.2466/pms.1973.36.3c.1171
B. Treutwein Adaptive psychophysical procedures Vision Research 35 1995 2503 2522 8594817
Verheijen, E.N.G.; 1998. Sound Reproduction by Wave Field Synthesis. Ph.D. Thesis. Delft University of Technology.
V. van Wassenhove, K.W. Grant, and D. Poeppel Temporal window of integration in auditory-visual speech perception Neuropsychologia 45 2007 598 607 10.1016/j.neuropsychologia.2006.01.001
Welch, R.B.; 1999. Meaning, attention, and the "unity assumption" in the intersensory bias of spatial and temporal perceptions. In: Cognitive Contributions to the Perception of Spatial and Temporal Events. Elsevier Science, pp. 371-387.
F.A. Wichmann, and N.J. Hill The psychometric function I. Fitting, sampling, and goodness of fit Perception & Psychophysics 63 2001 1293 1313 11800458
F.A. Wichmann, and N.J. Hill The psychometric function II. Bootstrap-based confidence intervals and sampling Perception & Psychophysics 63 2001 1314 1329 11800459
P. Zahorik, D.S. Brungart, and A.W. Bronkhorst Auditory distance perception in humans a summary of past and present research Acta Acustica united with Acustica 91 2005 409 420 (12)
