In the dark

At last year’s Tokyo Game Show the prototype for a rather unusual video game was unveiled by Tokyo Communication Arts school. While most game studios use expos to wow players with the latest in 3D graphics rendering, TCA’s Blind Braver was built almost entirely around sound. Designed for the blind, the Xbox game put players in the shoes of a partially sighted character, forcing them to navigate through an auditory rather than visual world.

Although a more complete version of Blind Braver has yet to materialise, another title currently in development in the UK will bring the basic idea to a new audience. Created for the iPhone by production company Somethin’ Else, Papa Sangre is an adventure game that takes place entirely in darkness – the phone’s screen remains impenetrably blank throughout play. Trapped in an eerie, beast-inhabited underworld, the player must travel through Papa Sangre’s various palaces using only sounds for orientation.

’It was inspired by a theatre game called Sangre y Patatas, in which players are blindfolded and made to walk on different materials,’ says Ben Cave, Papa Sangre’s producer at Somethin’ Else. In the dark, each person is trying to avoid Sangre, the player who is the killer.

What makes Papa Sangre special is its complex 3D sound design. The game would not work without a delicately balanced sound field in which ’sound objects’ – surface textures, instruments, creatures and the like – are located precisely all around the player using ordinary stereo headphones.

Funded by Channel 4’s Innovation for the Public fund, 4iP, and developed by a multidisciplinary team, including sound designer and composer Nick Ryan, Papa Sangre uses binaural sound placement – special recordings that mimic the way the ears and brain perceive the location of sound in the real world. Binaural recording of 3D sound is not new, but unlike other binaural iPhone games such as Soultrapper by Real Time Audio Adventures, in Papa Sangre the spatialised sound effects are computed and triggered in response to the player’s movement through the environment. In other words, the binaural sound is dynamic, not pre-recorded and fixed.

Computational 3D audio on a phone is the latest breakthrough in a long history of 3D sound research and implementation, demonstrating what can now be achieved with the processor in a tiny handheld device.

’We have been working with 3D soundscapes for ten years, so it doesn’t surprise me that people are building a binaural game for the iPhone,’ says Martyn Ware, director of Illustrious Company, a venture he co-founded with fellow musician and producer Vince Clarke. But unlike the audio world of Papa Sangre, which is created specifically for headphones, the majority of Illustrious projects have been designed for physical spaces.

’The implications of 3D sound for exhibition and public spaces is very interesting. It’s about creating a totally immersive experience. The better the technology is at creating a sense of reality, the less apparent it is and the more affecting the experience becomes,’ says Ware.

The Dark, for example, was a touring installation produced in 2004 by Braunarts to tell the story of Britain’s role in the slave trade during the 18th century. Visitors were taken to a pitch black room containing a 3D audio environment designed by Illustrious. Left only with sound and imagination, the audience could explore ’ghost’ voices from, for example, the decks of a slave ship.

The development of a complex, three-dimensional sound world presents particular problems for sound designers and software developers, says Cave. Even with the algorithms and processing power to handle a dynamic binaural environment, some sounds are inherently easier for people to locate spatially than others. And in a system without visuals, some sounds require context, while others don’t.

But when these challenges are overcome, responsive 3D sound environments may provide a new component in interaction design. ’Could you use a 3D sound interface as a menu structure, or to browse collections of data, such as a music catalogue?’ asks Cave. In fact, Mark Sandler, a Professor at the School of

Electronic Engineering & Computer Science at Queen Mary, University of London, is researching this very idea.

As an alternative to graphic user interfaces on portable audio players, Sandler is developing ’a purely audio means of interacting with a playlist’ of songs in a music catalogue. In this system, four songs are played simultaneously through headphones, but they are separated spatially to different points in the 3D sound environment. This allows the user to listen to all four distinctly and simultaneously, navigating to the one they want either through buttons (real or virtual) or by titling the device in the appropriate direction.

The accessibility implications for the blind or partially sighted are obvious. With more powerful processors available each year and with gesture control emerging as a digital interaction technique, the possibilities for exploring three-dimensional sound worlds are tantalising.