COVID-19 AR Elixir

Two days after I was accepted into the Adobe Augmented Reality Residency, the COVID-19 quarantine began. To say that it colored my time at the residency would be a significant understatement. It redrew the lines, then colored outside of them. It’s not surprising that when it came time to create my final residency project, I found that I couldn’t think about anything other than the unfurling catastrophes around me. Rather than sticking to the plan that I had been developing for weeks, I decided to follow my heart and make a piece about the current situation, how COVID-19 ripped open pre-existing schisms in the United States’ systems.

Each plant carries a specific message. Eglantine roses are a European variety that were spread around the world by colonialists. Unfortunately, it’s a highly invasive species and once planted it begins to take over whatever area surrounds it, suffocating indigenous plants and irrevocably changing native landscapes. It is currently illegal to sell in most of the world, but still remains available for purchase in the United States.

Participants interact with the environment through proximity and tapping. By using active interactions (direct participant choices) and passive interactions (optional elements) I was able to add depth and authenticity to the “world”. Subtle ambient animations and sounds (like the breathing lungs) also contribute to the sense of place. 

Audio is an incredibly important interactive element. Since I’m a musician, I felt quite lucky that Aero added audio functionality right before my final project was due. I created the composition using a combination of field recordings, piano, and my voice. The foundation for the entire piece is a field record of an actual hospital lobby (Courtesy of Tim Kahn. ) The main percussive element is a heart rate monitor. I played the piano and sang the choral elements. When you tap the lungs, they play a recording of me breathing. The candles have a proximity sensor that plays individual choral tones, creating a “candle choir”. 

When the pandemic hit, I began searching for ways that I could help. I believe deeply in the cathartic power of art, so I attempted to create a medicine of sorts, an experiential elixir. It’s a place to deeply feel and consider how and why we ended up here. Sometimes, catastrophe leads to the sort of fundamental change that wouldn’t have been possible otherwise. It’s my hope that we can take this opportunity to build a more equitable world where everyone is treated fairly despite the color of their skin or economic status.

Public spaces like churches, community organizations, and schools are typically places where people find support during emotionally challenging events, but in the midst of a pandemic, these aren’t available.  I hope that this piece can act as an “anywhere altar” in their absence. People can experience it wherever they are, whenever they need it. I shot this video in my own accessible space, my backyard.

Above all, this piece reflects this exact moment. It’s not only a snapshot of the global pandemic, but also of the emerging AR medium and the software used to create it.  I hope you enjoy it.

EXPERIENCE it yourself. | LEARN how I made it.