Who Am I?—an augmented reality mobile app

Two friends and I have been working on a collaborative passion project, an augmented reality app that is intended to foster empathy.

Who Am I? invites users to listen responsively to stories that are mostly difficult.

While a recorded story is narrated in first person, listeners are given a choice of artifacts—some related to the story and some more abstract—with which to build a “monument” in response. At key points, follow-up resources are provided for further reflection or action.

A little background

The app started out as an art project. As we explored possible approaches, it didn’t take long to see how augmented reality might help an audience transcend the passive consumption of artwork and become active co-creators.

We were well into development of our first experimental art space when the BLM protests began.

We are a multiracial team, with a synergy of perspectives on how to respond to the whirlwind of chaos, pain/numbness and frustration around and within us. With user involvement already in mind, we quickly refocused, and Who Am I? was born.

I was sensitive to the fact that this was not anyone’s day job, so my application of UX frameworks had been light and casual. Now, with a more acute intent, the process became more focused. I began to scaffold and document it with more rigor.

Another challenge, and an opportunity

Shortly after we released our beta version of the app (Android only for now), I saw an open call for submissions from the Design Futures Initiative (DFI) in partnership with the Innovation Cell of the UN Department of Political and Peacebuilding Affairs (DPPA).

In celebration of the UN’s 75th anniversary this year, they asked “designers and futurists to create speculative artifacts that evoke novel futures of how to better sustain peace.”

The deadline for the Futuring Peace initiative was August 31! We jumped at the chance to present the concept and got our submission in just a day early.

It’s hard work translating an intuitively conceived idea into a systematically developed and logically defended one. Regardless of the outcome (we’ll know in a couple of weeks), it was a valuable process, and helped us more thoroughly examine:

• the priorities of the project
• the feedback (and types of feedback) we were collecting
• the supporting research for our design strategy

What we learned

This post is not intended to be an academic defense, so I won’t include all the research. Instead, I’ll list the four defining research questions and some key takeaways:

1. How does augmented reality affect the brain?
Using brain-imaging technology to measure cognitive functions like attention, personal relevance, emotional response and memory encoding, a Mindshare Futures and Zappar market research study showed almost double engagement (1.9x) when tasks were done using AR versus without. This was consistent across a broad variety of tasks. 

2. Does augmented reality positively impact empathy?
Ketaki Shriram, co-founder of Krikey, an Augmented Reality gaming app, studied the effects of immersive experiences on empathy towards animals and natural ecosystems as part of her PhD from Stanford University’s Virtual Human Interaction Lab.

In her VentureBeat article, she notes that “when people took on the identity of a piece of acidifying coral in virtual reality, they were more likely to show empathy towards the ecosystem after the experience.”

In essence, their behavior changed.

One key appears to be the ability of virtual reality—and for similar reasons, augmented reality—to make outside experiences less abstract and more personal, getting us closer to “walking a mile in someone else’s shoes.”

3. What is known about the active listening technique (called “third things”) we are utilizing?
The standard techniques of active listening—nodding and paraphrasing, for example—have been proven effective in crisis situations, such as a hostage or suicide intervention. Something different is demanded of a listener in longer term relationship healing and building.

The term, “third things”, coined by educator and activist Parker Palmer, refers to the introduction of some external third thing, like a topic or activity, to the two people talking. While I didn’t find comparative studies on this technique in time for the submission, there were plenty of compelling anecdotes that show how participants were able to work through upsetting, even traumatic, exchanges.

The monument building activity in Who Am I? is our “third thing”.

Can it still be true if one of the speakers isn’t actually present? We think so. The listener is given space to respond freely, incorrectly, repeatedly, as desired. And more importantly, there is no need to ask anyone to keep resurrecting a traumatic experience or to once again repeat a story they have, through years or even generations, grown tired of hoping others would finally hear.

4. How can we make sure the monument building experience truly supports active listening?
This will take testing and iterative improvements on our part, keeping in mind that our target audience is people who have some desire to hear. The beta version of the app provides only abstract artifacts, like gems and flowers, for building monuments. But a common request was for artifacts that more literally reflected the story and the person telling it.

I also looked at research on a likely parallel: doodling, apparently an aspect of fidgeting. After years of trying to minimize these behaviors, they are being reexamined and found, in certain cases, to help listeners remain focused and hear better.

Listening as an essential skill—a value

Kate Murphy, author of You’re Not Listening: What You’re Missing and Why It Matters, is a highly accomplished interviewer—a professional listener. In her book, she observes that rhetoric, debating, and persuasion skills are taught and valued in our Western cultures while little attention is given to listening.

She describes a study in which parents’ improved listening skills resulted in reductions in children’s problem behavior.

Murphy also notes her own and others’ experiences with the power of listening to have a profound impact on both the listener and the speaker. In my own life and career as a designer, mentor and parent, I would agree.

Christian Picciolini’s TED talk is a great example of how listening changed not only the speaker’s life, but also the lives of the people around him. He doesn’t describe what happened as active listening, but I trust that you’ll notice how he became an active listener, how connections grew, and how change happened.

About the team

Who Am I? was co-conceived and is being produced by Breona (Bree) Jenkins, Sarvagya (Survy) Vaish and myself. While Bree is focused on curating and recording the stories, Survy is coding the Android app, and I concentrate on art direction and steering the user experience, everyone’s role is fluid and shared.

While our project focuses on the stories of Black Americans, the potential to build bridges across cultural and racial divides is applicable globally. 

Who Am I? beta demo video