Collaborative Augmented Reality Note-Taking with AiRNote

The team behind AirNote wanted to bring a fun and novel way to collaborate around note-taking. I sat down with Peter Zhang, Shuntian Liu, Sudarshan Sreeram, and Tom Zhao to ask them about their project.

This was a project of many 'firsts' for the team, who made the decision to try out technologies they had not built with before.

Sudarshan tells me that their interest in Augmented Reality goes back to attending the Apple Worldwide Developer Conference in 2018, where they had a table-based AR game with iPads. "AR was an area we wanted to experiment with and often gets overlooked in the wider set of iOS features." he says.

Tom continues, "We explored a few demos from the Apple Developer site. They have an AR game project demo, so we explored building a game first, and that naturally evolved into a shared AR workspace."

Having seen our Deepgram 5-minute demo, they had the idea for AiRNote: an interactive, collaborative note-taking application using Augmented Reality.

Building AirNote

Shuntian was in charge of building the app's integration with Deepgram's Speech Recognition API and used our Live Transcriptions with iOS blog post as a starting point. Thankfully, many of the snippets from our iOS post could be copied and pasted, meaning less time learning to use Deepgram and more time focusing on complex parts of their project.

Because there were so many new things for the AiRNote team to learn, they had to overcome a number of challenges, including learning how to write a native iOS app with ARKit and RealityKit, how to use Blender to create 3D models of sticky notes and pins, and how to collaborating on an XCode project - which, I'm told, was not the easiest thing to do while using git version control.

I only know the most basic concepts in iOS development, but Sudarshan also spoke about an experience I have encountered first-hand - for anything but the most simple apps, the Apple documentation forces developers to go down many rabbit holes to learn the pre-requisite skills needed to be successful.

The team managed to overcome their challenges and build a compelling demo that I had the pleasure to try out. For further development, the team wants to add more ways of interacting with other users, such as supporting drawings, file sharing, or even the ability to share any AR object in the session.

You can see the code behind AiRNote on GitHub.