Fueled by the adrenaline rush from Treehacks 2021, my team (Table1) decided to tackle Tartanhacks, Carnegie Mellon’s annual hackathon hosted by ScottyLabs. This time, we created an augmented-reality geospatial social media platform to connect users to their local communities. We won the Scott Krulcik Grand Prize and “Best Throwback Hack,” and somehow qualified for Pinnacle for the second time among 52 submissions and 300+ hackers. This is our roller coaster🎢 of a journey —
Background
I attended Treehacks mid-February and met my wonderful teammates Mitchell and Ryan through a mixing event one week prior to the event. We clicked immediately, had a really great time hacking together, and kept in touch through Discord almost everyday afterward (read about Treehacks here).
The greatest prize from Treehacks was not the iPad Pros, but the amazing teammates — Mitchell and Ryan — that I met along the way 💖
While at Treehacks last month, I was looking for other students who were participating from CMU, as I was curious to how CMU was represented. Through Linkedin and the Treehacks #introduction slack channel, I found Keren, a freshman studying Electrical and Computer Engineering at CMU. We exchanged a couple Linkedin messages, and he encouraged me to participate in ScottyLab’s Tartanhacks in April (if you’re reading this thank you so much, Keren!!). I took note, and kept an eye out for Tartanhacks registration to open in the coming weeks after Treehacks.
Following the rush of Treehacks, I also began to become interested in the behind the scenes role in hosting a hackathon — I felt that attending the hackathon hosted by my own university would give me a good sense if I decide to be part of the organizing team next year.
Pre-Tartanhacks
Hardware
Unlike Treehacks with VR, we decided early on that we would instead opt to creating something in AR this time around to explore a different medium. This lessoned the stress of each having access to a headset, and made it easy for all of us to test on our smartphones for whatever project we decided to do.
Platform
Tartanhacks was hosted on Hopin, a conferencing platform that I had previously seen while hopping into a couple online career fairs. I was excited to try it out for the first time and experience a change from Zoom.
Tartanhacks Day 1
Friday, March 5th
I had a six different meetings the day Tartanhacks started, so by the time opening ceremony came around, I was exhausted. Balancing hackathons with a course load had been a stressor on all of us as students, but I’m so happy to have a team that’s so dedicated to spend the time despite it being midterm season. I tried my best to finish as many of my obligations for the next week as early as possible, as I knew I would not work on any homework during the hackathon. Nevertheless, I was excited to finally get to dive into another project and reunite with my teammates!
Hacking begins — Brainstorming
7:00pm EST
At the beginning of Tartanhacks, we wondered if the Ohyay expo rooms from Treehacks were still open, and to our surprise it was! We happily began our brainstorm there, as we found the rooms really allowed for the natural flow of idea flow.
I began by sharing an idea of using AR as a method to digitally donate to hotspots in the real world — hovering a phone over a donate box or a landmark to be able to process a digital transaction. However, we quickly realized that this had many safety concerns with making a transaction in open space, and also had little experience with working with payment, so that idea quickly fell off.
Afterwards, Mitchell brought up an idea of using AR to get students familiar with a college campus, and allow students to leave anonymous notes at various locations for others to see (ex: leaving a note at a cafe, recommending a specific drink). His idea was inspired by a really wholesome game called Kind Words where players leave nice notes for each other.
Mitchell’s idea instantly sparked vivid memories of my confusions when first finding classes on campus last fall as a freshman navigating between the weird stairwells of Wean, Gates, and Doherty. I was fortunate enough to have an orientation counselor guide me through the nooks and crannies of the mysterious CMU hallways, but knew that much of the incoming class would never get the luxury of an opportunity I had. I instantly saw how useful this app would be to any student first stepping foot on the CMU campus, and we all loved the idea, and decided to go forth with it.
For task delegation, we decided that Ryan has the most experience working in AR, so she would be tasked with that and focus on figuring out spatial anchors. Since Ryan would be focusing more in that aspect, I would be responsible for more of the 3D modeling in the space for creating a Scotty dog mascot to help with CMU map navigation as well as filming some sample clips of CMU halls to use as test footage as I’m the only one in Pittsburgh and near CMU campus. Mitchell would focus on working in Unity and development side of the functionality of the application.
The “Design for America” category focusing on creating a campus life hack was perfect for our project. However, we realized afterward that this category was only limited to CMU students. In fact, to proceed to the next round for this prize for a CMU showcase, I would be the only one on the team representing our project as Mitchell and Ryan are not CMU students.
I didn’t want to leave my teammates behind, so I decided that we should drop the campus navigation theme, and pivot to creating a more general social media AR application, in which users would be able to leave notes in places that they would like to for others to build off of.
At about the same time, I realized that this would be the perfect opportunity to tackle a retro theme. I’ve recently become fascinated with old-looking GUIs (graphical user interfaces) after Professor Jason Hong showed us several examples in my Programmable User Interfaces class and really wanted to try my hand at making some skeuomorphic UI elements.
I seized the opportunity and began looking for references. Thanks to Mitchell’s knowledge of the indie game design space, he knew two games that were perfect to reference these styles from — Hypnospace Outlaw and Broken Reality. I created a moodboard quickly before diving into creating graphics.
As for software, I decided to try one that specializes in creating pixel art — Aseprite (previous I had used Photoshop to create pixel art). I quickly watched a tutorial and scribbled down a list of shortcuts and familiarized myself with them before diving straight into it.
To get on the same page with Mitchell while he developed the interface in Unity with placeholder buttons, we constantly updated each other by drawing on a Google Jamboard. I’m still in the early stages of learning Unity to implement UI components, so really wanted to make sure that what I was making would be usable and able to be implemented as easily as possible. Since this was the first time I’ve designed a user interface in AR, I also tried to keep UX principles in mind as much as possible (not blocking a user’s view, making sure the cognitive load on each page was not too much, etc.)
I began by thinking about the crucial elements that were necessary in the final design and the different states of the buttons that needed to be made (pressed or unpressed). For the rectangle (non-rounded) shape of the buttons, I was inspired by the Windows 3.1 start menu with the shadows that indicated whether a button was “pressed in” or “sticking out.”
As for the icons on the screens on the bottom for adding 3D model, drawing, and text, I opted for a 3D cube model icon for the 3D model, a pencil icon for adding drawings, and a chat icon for text.
A couple hours into the hackathon, Ryan was having difficulties with Azure, and conveniently used the handy mentor ‘ping’ feature in the Tartanhacks Discord to summon a mentor. To our luck, a mentor named Ron happily helped us out. I thought Ron looked really familiar, and quickly realized that he was in one of my HCI classes. It was so nice getting to meet a fellow classmate through the hackathon, and we all exchanged contact information to keep in touch afterward. Throughout the hackathon, Ron was such a big help and supported us every step of our journey (if you’re reading this, thank you so much Ron!)
I was able to get the buttons ready on the first day before getting some much-needed rest to prepare for a full-day grind the next day.
Tartanhacks Day Two
Saturday, March 6th
As I got ready for the new day of hacking, I began thinking about names for our application so I could start on brainstorming possible themes for a logo idea. My brain was set on any word that had the letters ‘A’ and ‘R’ next to each other to form AR.
tARtan… internet… int-er-net… intARnet!
IntARnet embodied what we were making — a new form of connecting a network of people together but through augmented reality.
After settling on our project name, I began thinking more about the icons on my buttons, and how I would perceive them as a user. Something felt off — I realized that I didn’t like how the chat icon reminded me too much of Twitch’s logo. It seemed less like an option to add text than a way to socialize and chat with others.
I decided to switch it to a ‘T’ for a text icon instead to make it more standard like adding text in any other design software. This seemed to fit better and make our interface more intuitive, so I decided to stick with it. I hope my intuitions were right, as I’ve basically done user testing on two versions of myself from two different days instead of actually getting feedback from users (note to self: leave some time for basic user testing in future hackathons).
I also didn’t like how the 3D model icon fit in. It resembled the option to add generic cubes into 3D modeling softwares and as a user, I felt that I would surely be confused seeing this icon as it would remind me too much of adding a primitive shape into a space.
I knew there had to be a better way to embody the wide variety of 3D model selections that we had lined up for users to use. I thought about what it meant to put 3D models in the real world, and decided that these almost resembled “stickers” on 2D surfaces. I then realized that these models would be the 3D depiction of stickers as a way to decorate the real world. This thought led to to opt for a sticker icon, and I quickly remade the icon.
Finally, as I finished up creating the icons, menus, and other buttons in the interface, Mitchell was finalizing his work and starting with app testing. The early versions of our app was starting to come together, and the older-pixelated shader Mitchell added really brought together our theme.
Logo Design
As he was testing out the app and refining features, I began focusing on creating a logo. I wanted to create something that embodied our aesthetic and meaning behind our app. I looked at adding particles and textures to create a faded effect, but I think I was still stuck in a vaporwave mindset from the previous hackathon so my first logo idea didn’t quite fit the theme we wanted.
I got some feedback from Mitchell and Ryan, and they felt the same way. I decided to go back to pixel designs and began by looking at pixel fonts. I came across one set on Spriter’s Resource that I really thought would match our theme well, so I pieced them together to create a simple logo and added some simple shading and lighting to fix it up a little and add depth.
I then decided to add a wave effect, as I really liked the aesthetic of 3D wavy text microsoft text-art style. I also added a slight glow to the ‘AR’ to emphasize the fact that we are an AR application.
The logo did look a bit empty, so I looked at creating some more assets to add in. I played around with adding some pixel clouds, some noise effects, a cursor, binary numbers, and pixel computers until I made something I felt satisfied with. I thought it would be cool to add the computers with clouds, representing the modern age of cloud computing juxtaposed next to older Macintosh computers and some binary numbers.
Sometime in between hacking, I found the time to take a short breather break and take a walk outside. As I was walking, I reflected on our application and the features we had created. I realized that what we had made was not very different or advanced — it simply combined what already existed in allowing a user to adding stickers, text, and drawings in one application just in 3D.
I realized that it would be critical for us to focus on pitching the use cases that are possible with our application.
Unlike our previous hackathon where we created a game, here we were creating a platform for others to create and unleash their imagination. I knew that people were not going to imagine the possibilities themselves immediately. Take YouTube as an example. If someone told me “it’s a platform to show your videos to others”, I would’ve been unimpressed, as compared to showing off the community boasting a wide selection of diverse creators known as YouTubers.
I pinged my teammates and we did a quick brain-dump of possible ideas we could say are use cases, and had a lot of fun thinking about possibilities. I later distilled the ideas down to six that I thought would be valuable to go over in the short 4 minutes we had for our pitch.
Slidedeck Design
For the slide deck, I began by creating a design in Google Slides like in previous projects. However, I realized that Slides really limited my ability to add texturing to the slides as well as control for layer blending, which I needed to add the dust and grain effects to fit in with the theme.
A couple hours of struggling in Slides later, I pivoted to creating the our slidedeck on Figma. As this was my first time creating a presentation in Figma (I’ve previously used it only for prototyping), I initially found it pretty difficult to manage all of my frames and making sure the slides were laid out in the right order and items were positioned in uniform places on the slides. I took much longer than I anticipated, but ended up flushing out a first draft of the slides before the day ended.
I went for a research-backed pitch style at first, beginning with a definition of social media in general, some statistics, drawbacks, an introduction of AR and the problem space of social media today. I planned for us to introduce our application after defining the problem space.
However, after getting some feedback from my teammates, I realized that this approach took a lot of time away in the short 4 minutes we had from talking about the most important part — the use cases.
Since everyone already knows what social media is, we decided it would make more sense to start off with our pitch video, explain our application, use cases, and then talk how we made our platform and how it compared with existing social media with the time left.
We flipped our presentation structure around, starting with the solution and then backtracking to then talk about the rationale behind our application and how we created it.
I got to work with the slidedeck to make these changes, continuing through the night while also polishing and creating some of the last UI elements that we needed to add.
During some hour deep into the night, we also tried out building the app to iOS so I could try it out and film some footage to help with the video. This led to some troubles in downloading 10gb XCode and resulting in me having to delete a lot of files (including Roblox oops) on my computer to make 30gb room for it to run properly. Luckily, I did get the app to work on my phone eventually (although with a couple bugs) and had some fun finally getting a chance to play around with the interface.
Day two was slowly melting into day three as we continued working and polishing and re-polishing…
Tartanhacks Day Three
Sunday, March 7th
At around 6am, the sun rose. Mitchell took this as the sign to go outside and film some footage to use for our pitch video around the UMich campus. As he was doing that, I finalized the slides and added the examples of use cases for our application in the slidedeck, leaving room for some gifs we could use to show these use cases in action.
After Mitchell finished filming some videos, I converted snippets of the videos into short gifs to fit on each of the use case examples. At the same time, I sent him the graphics I had used so he could use it to create the pitch video. He made everything fit together and made a really amazing video with super on-point music (check it out below)!
As it was nearing the submission date, so we submitted the presentation on the dashboard and took a short break before coming back and running through our slides a couple of times, hoping that our collective sleep was enough to keep us going through the 2.5 hours.
Presentation Expo
1:00–4:00pm EST
Each team had a designated room to present in to judges as they hopped in and out of the rooms. After seeing no judges for a couple of minutes, we decided to stay awake by entertaining ourselves with various Snap Lens filters (we were careful to turn it off before judges came in).
Afterwards, judges started coming in and we began our presentations! It was so exciting finally getting other eyes on our project and jump out of our siloed workspace and hear all the valuable feedback.
Some highlights:
- Ron bringing in the Microsoft judges to watch us despite us not entering for that category from failing to get Azure to work
- Getting feedback from leaders in the VR space those at Facebook Reality Labs themselves —thank you so much Zitong and Lifei!
- Realizing that we should’ve cycled our presentation roles and learned each others parts so we were able to take breaks in between
- Using our feedback to edit our presentation and live iterate on it — many judges brought up the concern for content overloading an area or managing of content so we decided to edit our “next steps slide” to include that later on
- Figma’s failure to allow for embedding videos… and Hopin’s failture to let me share sound and switch tabs leading to me making awkward screenshare pauses and Ryan having to supplement the silence with small talk
Before we knew it, it all came to an end and the wait began.
Closing Ceremony
5:00–6:30pm EST
Before closing ceremony began, we were ~mysteriously~ added to a channel named “table-1-presenting” …
… before getting confirmation from an organizer that we would be presenting to the final panel of judges at the awards ceremony!
Five teams were selected to present, and we were one of them! I knew one of the judges on the panel — Professor Kelly Rivers — as I had taken Principles of Computing with her last spring and loved it so much. I was so excited to get the chance to show off our project and get her insights.
At the start of the closing ceremony, we heard from Guillermo Ruiz from Oracle, and he gave really great advice on taking risks in experimenting with emerging tech like AR, which I found really relevant and valuable, as we definitely did struggle a lot with working in AR for the first time as a team.
We gave our presentation for the final judging round in the beginning of the ceremony, so judges had time to deliberate before winners were announced at the end. We were the first to present, and went “on stage” through the “back stage” option to give our pitch.
After our pitch, we were able to view some of the other finalists, and it was so nice to see so many other XR related hacks and hear what some of the other hackers had created over the last few days.
Finally, they moved into announcing the winners for each category, and we were so happy to hear that we won the “Best Throwback Hack” category!
As the ceremony drew to a close, we grew more and more anxious. We had not won any of the sponsor prizes we had hoped for — Facebook’s “Best Community Hack,” Sandia’s “Best Safe and Healthy Lifestyle Hack”, or the Design for America Award.
Finally, it was time.
One of the final panel judges — Professor Pat Virtue — came on and announced us as the final winners of the Scott Krulcik Grand Prize!
Shaking and excited, we couldn’t believe it.
We just won the top prize and qualified for Pinnacle again?!
Dwindling on one hour of sleep from Sunday, I crashed and slept 12 hours on Monday. On Tuesday, I slept another 12 hours… and then had a midterm on Wednesday :(
It really is a struggle, if you’re reading this and struggling with balancing school work alongside hackathons, my heart goes out to you 💞. Though it has not been easy, doing Treehacks and Tartanhacks have really pushed me to find time and prioritize my activities, and I really learned so much so quickly in the past month than I ever have before. Though in previous semesters I would have told myself it was impossible to do hackathons alongside school, I realized that it just comes down to prioritizing and making the most out of each day.
Links
Our Project
Devpost | Pitch Video | Slidedeck
Other Cool Projects we loved from Tartanhacks
- TPTracker — Ever take a dump and go “Oh SHIT! I’m out of toilet paper!”? Well, never again. Our project will upgrade CMU’s hygiene, happiness, and efficiency by notifying custodial staff when toilet paper is low.
- Healthy Habits — An application that encourages you to workout!
Special Shoutouts
- All the Tartanhacks judges — for taking the time to listen to our project presentation and giving us all the valuable feedback! 📝
- Mitchell — for once again being the best teammate I could ever ask for, and giving me so many great references for retro art to feed my mind with 🧠
- Ryan — for finding us Ron, and for all your perseverance in Azure throughout the weekend 💪
- Ron — for being such a great mentor and supporting us every step of our journey 👣
- Professor Caspar — for providing the mentorship and support and believing in our team 🙏
- My roommate Annie — for once again dealing with me as a mess pulling another all nighter 💖
- You — for making it through my long chronicle of our experience! 🥳
