Microsoft Research has unveiled a groundbreaking system called AdHocProx, aimed at improving co-located collaboration across multiple devices. This technology, demonstrated at CHI 2023, eliminates the need for externally-anchored beacons or even WiFi connectivity. Instead, AdHocProx uses device-relative, inside-out sensing to augment collaborative interactions in dynamic, ad-hoc arrangements.
The system incorporates dual ultra-wideband (UWB) radios for sensing distance and angle to other devices. Capacitive grip sensors are also used to identify where the user’s hands are holding the device and to partially correct for any resulting UWB signal attenuation. Spatial sensing and communication occur through the side-channel capability of the UWB radios, making AdHocProx suitable for small-group collaboration across up to four devices (eight UWB radios).
These sensors work together to detect proximity and natural, socially meaningful device movements, enabling contextual interaction techniques. In an offline evaluation, AdHocProx achieved a 95% accuracy rate in recognizing various ad-hoc device arrangements. Participants particularly appreciated interaction techniques that automatically leveraged proximity-awareness and relative orientation among multiple devices.
AdHocProx’s inside-out tracking system relies on ultra-wideband radios and a peer-to-peer networking protocol for communication. Devices use pairs of tags mounted on each device and a machine learning model to sense proximity between devices in a round-robin fashion. Each device can independently infer its proximity and orientation to other devices without any external infrastructure. The peer-to-peer nature of this protocol allows devices to dynamically enter and leave a formation.
Inspired by the increasing role of computing devices in our social lives, AdHocProx aims to address the challenges of incorporating and configuring cross-device interactions. Through a formative study involving participants engaging in a collaborative task with paper props, the researchers developed a set of interaction techniques that support various scenarios. These techniques include opening a mutual portal on each device for limited shared space, enabling content dragging, panning across devices, and offering contextually aware tools like sticky notes.
Preliminary feedback revealed positive attitudes toward the system and its designed interaction techniques. AdHocProx has the potential to revolutionize the way devices are used in ad-hoc social settings, paving the way for more seamless and intuitive collaborative experiences.
See Microsoft’s demo of the technology below:
