Nth Sense
Aural AR for Hybrid Space
Nth Sense grants intrinsic spatial awareness to people and media, allowing one to feel their relationship to places and individuals, and enabling reactive media to ally or compete with the ruling genius loci.
PROJECT TYPE
Academic Thesis, 2022
TOOLS
Javascript|Bluetooth Low-Energy
PROBLEM
Augmented reality is poised to mutate the evolutionary bond of human to environment. Information overlaid rather than integrated risks further obfuscation of surroundings already forsaken in favor of distal data. Nth Sense orients toward a future in which hybrid spaces serve to reforge, not fray our bond to the physical—one where media dance with experience rather than supplant it.
Nth Sense endows new senses.
Perceptually low-level affordances of localized sound are utilized to endow a preternatural sense of the user’s three-dimensional relationship to a destination or dynamic points in space. This constellation’s rate of change in turn produces a real-time measure of environmental flux used to adapt sonic intensity and character to its physical context.
Binaural stereo signals situate audio playback, calling one to a point as the siren calls a sailor—yet while of certain bearing, one maintains agency, and freedom to wander, discover, and attune. The variable strength of radio signals emanating from mobile electronics furnishes a platform-agnostic, permissionless method for localizing others, whose number and choreography constitute the escapement of public experience.
Delivered via smartphone though standard headphones, Nth Sense requires no specialized hardware.
The user’s position and orientation relative to a destination is used to spatialize the audio leaving your phone such that it feels like it is coming from the direction you need to go, no matter which way you turn. It’s subtle—easy to forget it’s happening. Like following a friend down the street, it enables confident wandering.
Put in an address or lat/long coordinates, wander purposefully to your heart’s content, and afterwards admire the digital artifact you created representing your path. Distance from your destination is communicated the same way nature does it - as things grow in distance, they grow quieter - certain frequencies more than others. Here, visually, the white circle gets closer to the center as you grow closer to your destination and rotates to indicate its relative position.
Orcas echolocate to identify and track objects in cold, black waters. The second Nth sense allows users to feel the relative position of people around them in real time.
This is a phone, but it’s also a cicada. Everyone’s phone is incessantly shouting "hey!!!! I’m here! Wanna connect??" FindMy and coronavirus tracking apps operate on this principle.
Bluetooth Low-Energy signals are used to localize and characterize people moving through space. This approach is platform-agnostic and permissionless.
The first two senses of direction and space lay the third's foundation, constituting a measure of dynamism used to drive reactive media intentionally apposite, or opposite to the user's location.
This is the same room, but it is not the same place. The flow of people, whether a trickle, or a churning mass comprises the dynamism of public space.
https://youtu.be/l5yzi6e6LB8 sonar demo
https://youtu.be/9DLePk2HCBY song demo