Virtual Sound

Until recently, reproduced sound typically lacked the accurate 3-dimensional character of real-world sound. Modern digital signal processing techniques, however, have made it possible to fully reproduce the spatial qualities of sound present in the real world. These techniques, known as virtual auditory space (VAS) techniques, rely on an exquisitely accurate reproduction of the sound waveform present at both left and right eardrums. To achieve this accuracy, the acoustics of the ears and head, as well as the intended environment must all be taken into account, something that is not done with standard reproduction systems. The VAS technique uses measurements of head and ear acoustics, known as head-related transfer functions (HRTFs), as well as measurements of environmental acoustics to construct a set of spatially dependant digital filter pairs -- one for each ear at every spatial location to be simulated. The process of "spatializing" a given sound becomes nothing more than applying the digital filter pair that corresponds to the intended spatial location. Scientists are viewing this technique as a valuable research tool because the realistic control of acoustical effects in VAS is as simple as changing parameters in the digital filters.

An example of microphone placement for head-related transfer function (HRTF) measurement, an integral part of realistic virtual auditory space displays.

Current Projects: Current Projects:

Jack Loomis and his colleagues are currently using the VAS technique in this laboratory to examine the perceptual processes involved in determining sound source distance. Of particular interest is the influence vision has on the perceived distance of a sound source. Though limited past research has suggested that the perceived distance of a plausible visual target tends to "capture" the apparent distance of a sound source, the precise nature of this relationship is largely unknown. We are attempting to expanding scientific knowledge on this topic in two ways: (1) by measuring the explicit regions of visual "capture" to plausible sound sources over varying conditions of sound source distance and stimulus type, (2) by determining the relative perceptual saliency of auditory and visual distance cues under conditions of concurrent stimulation.

VAS Display with Changing
Sound Source Distance (1.8MB)

Note: The file (.wav) requires a CD-quality (16-bit, 44.1 kHz) sound-card to be reproduced properly. Listen through a pair of high-quality headphones. The sounds should appear to increase in distance away from your left ear.

Recent advances in both psychophysical methods (weighting procedures) and stimulus control (virtual display technology), make these problems tractable and, as a result, will lend to a fuller and more complete understanding of visual capture phenomena. Insight into the perceptual processes that subserve such phenomena will be gained by testing related models of sensory cue combination with the psychophysical data generated from experiments being conducted currently.

For literature on sound source research, see:

Zahorik, P. (1997). Scaling perceived distance of virtual sound sources. Journal of the Acoustical Society of America, 101, 3105-3106.

Navigation Systems for the Blind

Since 1985, a multidisciplinary team of researchers (Jack Loomis, Department of Psychology, UCSB; Reginald Golledge, Department of Geography, UCSB; Roberta Klatzky, Department of Psychology, Carnegie-Mellon University) has been developing a prototype navigation system for the visually impaired that uses virtual environment technology. The system uses a portable computer and consists of three functional modules. The first determines the position of the user using differential GPS and orientation of the user using an electronic compass. The second module is the Geographic Information System (GIS) which contains the database of the local environment and software for providing the desired functionality. The third module is the user interface. For display of information to the user, we are using virtual sound. In the current design, the visually impaired person is guided through the environment using virtual sound beacons which appear in front of the person and can be approached as the person walks in their direction. In addition, off-route features (e.g., buildings, landmarks, distant street intersections) are displayed using synthetic speech that is spatialized by the virtual sound board. From the user's standpoint, it is as if the environmental features were calling to the user, much as it would be if loudspeakers were attached to the environmental features and the names of the features were being constantly displayed through these speakers. The above picture shows Reg using the system in 1997. Our current work involves miniaturizing the system and doing research to improve the user interface.