Dance Floor AR-DJ

Use a large screen or projector to create an interactive magic mirror for the dance floor. 

Dancers trigger different loops, effects, and visuals depending on where and how hard they dance. Get more people on the dance floor to remix and develop a dance song in real time. Get at least 3 people dancing to unlock the whole track. 

About This App

This app was created for the Niantic November 2022 Mini-Jam with the theme "People." I was excited to work on this particular theme because "people" is something I think a lot about when developing for AR.  

One current challenge of AR is that it is necessarily a somewhat isolating experience. It is an experience between the user and their device. Even when you have a shared AR session, it's still private among those people, and those people are still interacting with their devices primarily and not each other. 

How can we remove the wall of the physical device and create engaging AR experiences for groups of people? How can we encourage group play in the built environment? How can we create a spectacle? 

I explored a handful of approaches and settled on - instead of each person having one device - we will have one giant device for all the people to look at together. Plug an AR capable device into a TV or projector and you have a gigantic AR capable device! 

What I love about this approach is that it can draw anyone in. There is no app to install or technology to understand. It is a completely discoverable experience and since the full track can't be heard without at least 3 people, others are naturally encouraged and rewarded for jumping in. 

In order to accomplish this I needed to explore an unconventional approach to the ARDK tools. The ARDK is designed to be used on a device that is in someone's hand and is in motion. What are the implications of placing the device on a tripod in a fixed position? How do the various capabilities of the ARDK change and behave in this scenario? Read on for the technical nitty gritty.

But first... 

Instructions For Setup

Tools needed: 

• An iPhone or iPad running DanceFloorDJ (This app was built and tested with an iPhone 13 Pro.)
• A tripod or stable surface on which to put iDevice
• An HDMI adapter
• Some kind of display or projector
• A dance floor with ~4m of depth 
• A bangin sound system

Set up the camera in a stationary location facing the dance floor. Leave ~1m of space between the camera and the very front of the dance floor. The device should be in landscape and ~5' off the ground but feel free to play around. 

Set up the display or projector to display behind the camera. Set the display to flip horizontally to create a magic mirror effect. 

Before plugging in the display, start the app and check that no samples are playing. If samples are playing make sure the dance floor is clear all the way to the back. The app scans a line across the horizon line to detect depths, so the horizon specifically should be clear to ~4m. 

When everything is working and set up plug in your display and sound system and get some people on the Dance Floor! 

Technical Info 

This app makes use of the ARDK's depth manager. A row of depth probes are sent out in a horizontal row across the middle of the screen. Think of shooting 50 lasers out of the front of the phone. Wherever each laser hits, a tiny 3D sphere is placed. 

The dance floor is then made of of several invisible 3D boxes, each assigned to muting or unmuting a given looping sample. If one of the depth probe spheres ends up within one of those 3d boxes (because someone is standing there), the box is triggered.  

I initially started with a more complicated design in mind with the camera up high and facing down, but this presented some problems and made for less interactive output. For the purposes of the jam I decided to keep it simple and go head on. 

The collision boxes above correspond to different loops. The center track has overlapping drums that can go together while the other two instruments are separated.  

Get The Game

Join the beta on testflight: https://testflight.apple.com/join/Y5Ddl7IC 

Credits

Engineering by Sam Garfield

Music by Daniel Foster Smith

Merry Thieves, BLLC