This past weekend, I attended Treehacks, Stanford’s premier hackathon with 2,000+ attendees from across the world. It was held virtually this year through Ohyay! Our team created a gladiator cat-themed interactive virtual reality game that allows you to learn and improve your American Sign Language skills. We won the Moonshot Prize and qualified for Pinnacle’s first annual “Olympic of the Hackathons” out of 200+ submissions. Here’s our story —
Background
Last year, I searched for hackathons that provided free travel because I was looking for excuses to escape the freezing Pittsburgh weather during the semester, and through a friend I learned about Treehacks. Though travel was out of the water after COVID began, I decided to give the virtual hackathon a shot and applied to Treehacks as an out-of-state applicant. With their sub-10% acceptance rate from previous years for out-of-state hackers, I didn’t expect much. But on January 17, I was delighted to receive an email from Treehacks:
You’re In! Welcome to TreeHacks 2021! We loved reading through your application and seeing your passion for building and creating! We’re so excited to invite you to TreeHacks and can’t wait to see you in February ❤️
I was in disbelief. As a non-CS major at Carnegie Mellon who had only participated in one hackathon before, I really didn’t know what to expect. I guess they liked my response for one of my essay prompts roasting Cal —
As an applicant applying for treehacks, I’m trying not to blend in with the rest. I know I’m not stanford-quirky, but at least I’m not a cal bear :D
Treehacks was set for February 12–14, on the second week of my classes. Though I had already agreed to do the Deloitte Undergraduate Case Competition with my friends for February 10–12, I couldn’t pass up this offer. I decided to do a case competition and Treehacks back-to-back (rip me).
Online Format
The hackathon was set to happen virtually through Ohyay, one of the many online socializing platforms that spurred from COVID, recently started by a small team. Initially, I was skeptical — it was different from everything I had seen before:
You’re able to physically “move around” by dragging yourself to sit in virtual chairs, which I thought was just plain weird. However, over the course of the hackathon, it slowly grew on me…
and eventually I started wishing Zoom never existed and Ohyay just took over everything.
Team Formation
Saturday, February 6th 8:00pm EST
Treehacks hosted a team-formation event the weekend before the hackathon officially started. Though everyone introduced themselves in the Slack channels and digitally shook hands through LinkedIn, this event was really helpful in meeting and talking through ideas.
At the team formation event, one thing that blew me away the most was how receptive many of the developers were for UX or graphic designers. I came into the event feeling like an imposter as design isn’t a coveted skill in hackathons I’ve dabbled in at CMU. Treehacks was quite the opposite.
I remember virtually stepping in a room in which a guy asked, “Is there any graphic designers or UI/UX Designers here?” to which I replied and introduced myself. He straight up then said, “the best UI usually wins.” I was also DM’d by multiple groups looking for a UX designer, and it felt so exotic feeling valued as a designer.
I spent my time at the team formation event jumping through rooms and trying to find affinity to proposed project ideas. The virtual map indicated the number of people at each table, and I migrated around to the tables with more people. Eventually, I joined Table 1, and stuck around for the rest of the event (even after the team formation event closed). There, I met Mitchell, who I eventually formed a team with for our project! I had spoken with Mitchell a bit on LinkedIn and Slack about a couple ideas as I really loved some of his past VR work that he had done in Game Jams (go play his games here). I thought his work was super cool, but did not think we would actually work together, as most of my skills lie closer to graphic design and my only previous game creation experience had only been my 3-hour Roblox Studio training that I did for iD Tech Camps.
We bounced our ideas around and to my surprise, this had been the closest to a class discussion setting I have had since the past year. We talked about everything from DDOS for protesting online, using screaming echolocation in VR for gaining sight in a horror game, using social media to increase accessibility to education, and more. The Ohyay room made it truly feel like a conversation rather than talking in a circle like in Zoom.
Some of our other conversation highlights include:
- Us joking about letting others “take a seat” by moving into the “parking lot” slots on the top with overflowed people who don’t get to “sit in a chair”
- Making fun of how some of the requests for team members resembled job role descriptions
- Spamming the applause emoji whenever someone new joined our room and adding them to our #table1 slack channel
Overall, it was a super fun time and by the end, we had all stayed over an hour past the event just chatting about our ideas. Before departing on our own ways, we swore to all reference the orange table1 in each of our projects.
Pre-Hackathon
Schedule
I was blown away by how organized Treehacks was. Every event was laid out on a calendar to show where events started, finished, and even overlapped. Kudos to the Treehacks team for making everything so intuitive and easy to use!
Our Team
A couple days after the event, Mitchell reached out asking if I would like to form a team with Ryan and Jieying. I knew that this would be a fun team to work with, and I happily agreed. We quickly exchanged Discord contacts to begin coordinating hardware prior to the event to make sure we had all the materials we needed to dive right in. We knew that we would create something having to do with finger tracking, but didn’t have a clear direction yet. I own a rather outdated HTC Vive, which does not have those capabilities, so I spend the rest of the week before the hackathon asking around to see if I could lend an Oculus Quest from CMU. I was really lucky to have found one and picked it up the day Treehacks started from a professor I had for a virtual reality storytelling course I took this past summer (If you’re reading this, thank you so much Professor Caspar!!)
Trying the Oculus Quest for the first time was such a surreal experience. I had not realized how fast virtual reality headsets have evolved to have built-in finger tracking to replace controllers, be completely wireless, and turn on without needing to connect to a PC. I was instantly hooked.
Treehacks Day One
Friday, February 12th
Opening Ceremony
Friday, February 12th 7–8pm EST
During the opening ceremony, we learned about all the different sponsor prizes and also got to hear from PayPal mafia Max Levchin in a fireside chat. I had recently been reading Elon Musk’s biography and learned about the PayPal mafia so it was a wonderful surprise getting to hear from him!
Hacking Begins — Brainstorming
10:00pm EST
Once hacking officially began, we began fleshing out our ideas and prototyping. We were not the typical team, consisting of four different types of designers — Game Design, 3D Modeling/Animation, UX/Graphic Design, and Interaction Design, which made for a really interesting dynamic. We began with an idea to create a curriculum for American Sign Language learners, discussing the types of lessons, lesson content, and learning styles. However, at some point Mitchell inquired,
“What if this we made a game to battle monsters through using sign language to summon powers?”
As fans of Typeracer and other similar typing games, we all loved the idea and were instantly on board.
As we conversed, we quickly realized our ideas were too complicated to visualize, so we hopped on a Google Jamboard (gotta love online collaboration) and started sketching our ideas. We began with an idea that involved players summoning powers to then use as projectiles to shoot toward the upcoming monsters and asked ourselves how we might display relevant health, sign language symbols, and power options.
Drawing the figure out was helpful, but still didn’t solve for thinking about how elements might be displayed (following the player’s view or staying still in the environment). I’ve never prototyped for VR before, and it was new for many of us to think about the surroundings and displays within the player screen and the environment simultaneously. We wondered if displaying too much information in the player’s view might distract them from the game, so we then tried placing the points and ASL quiz style questions out of the view.
However, I quickly identified a drawback to this design. I recently attend a UX in AR prototyping workshop with Sarah Tan (thanks to Students of UXD), and I remember hearing that it’s important to keep the objects within 60 degrees from one’s view to avoid turning and dizziness within the 3D environments. We began thinking of alternatives for addressing this issue of balancing information display.
In elementary and the beginning of middle school, I was obsessed with playing a MMORPG game called Fantage. There were mini-games within Fantage, one of which was called “Typo Boo,” where you type words to kill ghosts as they appear and it was one of my favorite mini-games. This game inspired me to propose attaching corresponding letters to the monsters that will be appearing in the screen to “destroy” them. This addressed the issue of information display, as the letters would be attached to the monsters as they appear.
Finally, Ryan suggested we have cats instead of monsters to make the game more widely accessible. As plushie owners and cat lovers, we were only fueled to bring our own cat dreams to life. Rather than ghost hunter games which often cater toward a young male audience, cat characters allowed our game to appeal to any age group. Mitchell then suggested we have the cats come in from a stadium themed arena. With that, Catiator (Cat + Gladiator) was born and we began creating!
Roadblocks
12:00am EST
One (of the many) difficulties of hacking online was our differences in hardware. Mitchell didn’t have a graphics card so wasn’t able to live debug and test the scenes in Unity. I didn’t have much Unity experience but did have a PC with a graphics card, so by getting my Quest connected with Unity and sharing my screen, I could conveniently be a virtual gaming computer.
However, I couldn’t even get that working. The first realization I had was that I didn’t own a USB-C to USB-A cable to be able to connect the headset because my PC is old and doesn’t have a USB-C port. Luckily, thanks to GoPuff, I was able to order the cable and get it delivered in under 30 minutes. Cable in hand, I really thought all my problems would finally be fixed. Wrong.
First of all, the cable was extremely short, so I spent much of the time getting out of my chair to sit on the ground next to my PC each time I ran the scene. Then, I ran into issue after issue viewing scenes in the headset, sometimes facing an issue of having a black screen, other times not being able to view a scene in 360 view. I spent my time downloading packages after packages, uninstalling and reinstalling, downgrading and updating versions, toggling options in the setting, endlessly scrolling through forums, restarting my computer, hoping something, anything, would remedy the problem.
This lasted until about 3am, when my patience reached its limit and I finally called it quits.
Treehacks Day Two
Saturday, February 13th, 10:00am EST
New day, new me. As I was getting ready for the day before going back to tedious debugging, it hit me.
I’m an online instructor at iD Tech Camps a STEM camp for children ages 7–19, in which I teach courses such as Photoshop, video editing, Python, Wacom Tablet, and some other courses. A perk of being an instructor was that I had access to all the curriculum for all courses, including the more recent Oculus Quest 2 with Unity course. Clutching my half-finished Beijing yogurt, I scrambled back to my desk.
I scrolled through the iD Tech course material, making sure all my settings were the same as the ones that were suggested for instruction. I then scrolled through the #vr-oculusquest channel. Bingo.
A Unity C# instructor, Charles, had saved me:
Hey! A quick updated on the black screen issues I was having:
After downloading the student’s project, I think that the main thing causing the issue was that the student’s computer ran out of space halfway through installing some of the packages. The reason I think this was upon trying to reimport the oculus integration package, I found that a lot of files were actually missing from the package — and I know we had an issue building the project due to a lack of space as well. After re-importing both the Oculus Integration, the XR Plug-in Manager and the Oculus XR packages, it finally worked! So in the end, can’t say for sure what the issue was, but I am 95% sure that the issue was his computer ran out of space while installing the packages, and Unity just failed to tell us that it ran out of space.
Charles, please accept my greatest gratitude. You are blessed.
Apparently, my computer was just low on storage (8 GB left). I quickly went to delete all my beefy applications, and hilariously found that my sister’s 18 GB Chinese gatcha game, Genshin Impact, was hogging all my storage. After clearing adequate storage, I could finally view my scenes in the headset! Whew.
I felt super bad, I had contributed basically nothing to the team up until this point. The rest of my team had progressed smoothly, while I had spent a day just connecting the Oculus headset to my PC. I hoped to make up for all the time I had wasted and contribute in a different way — I pivoted and designed the Catiator logo. I recently started learning text effects in Photoshop, and tried my hand at making one for memes. It actually turned out to look decent, so we also decided this would be our logo and color scheme!
After finishing the logo, I started helping create gestures for the ASL letters to use in the game. Since gestures had to be performed while running the game and wearing the headset, it is it difficult to reference letters as I signed them in the game. As a result, I decided to memorize the ASL alphabet to make the process smoother. I had learned the ASL alphabet before, but had forgot most of it. I went online and found diagrams to start.
I quickly found how hard these diagrams were to use. It was so hard to see the details of the gestures, so I tried resolving this by looking at videos. This was more effective, but also tedious, as I would have to skip to the part of the video with the right letter and see someone rotate their hand in multiple directions. Some of the letters that were really similar were not able to be recognized due to the limitations of the hand recognition detection, so we just left those out (these include letters like M and N). In addition, letters like J and Z that require movement were also not recognizable due to the limitations of our static letter recognition.
The difficulty to learn the letters reinforced how helpful a 3D recreation of ASL learning would be in VR, to allow learners to see how a sign would look from all directions.
After learning the ASL language, I was able to actually survive pretty long in our game, and as the first user tester and I’m happy to say I did have had a lot of fun learning ASL in the process! Learning the letters in virtual reality definitely made it easier to view them from multiple angles as well.
After saving the finger positions of the ASL alphabet, I went back to designing some more graphics. I also helped Ryan with texturing the 3D models of the Catiators and added faces to them.
Countdown
Some wee hour of the morning EST
After some time, our game was basically finished, so we started focusing more on the aesthetics. Coincidently, all our colors and aesthetic seemed to be converging toward Vaporwave style, so we decided to go all out and add the Vaporwave sun, music, the Parthenon, and even The Thinker statue.
Finally, we made a demo video to capture the highlights of our game. The video from Oculus came out as a square, which coincidently fit the older aesthetic we were going for.
Treehacks Day Three
Sunday, February 14th
Presentation day!
I hopped into the Ohyay presentation room and decorated our virtual wall with our logo. Driven by one hour of sleep, it was time to start to start on the slide deck. I threw together some slides based on our Devpost and hoped for the best.
Presentations
2:30pm-5:00pm EST
The presentations were in an Expo-style format, taking place over the span of 2.5 hours as judges hopped in and out the presentation rooms. We were all really exhausted from the previous day of hacking, so the thought of sitting in a room for 2.5 hours did not sound appealing. But as the presentations went on, we found that it was actually really enjoyable getting to know the judges and hearing their feedback for our project! It was so nice getting some reflection after putting our heads down for so long.
After presentations, we started getting some visitors from other Treehacks projects and it was really nice meeting everyone and talking about our project! As this concluded, we got a Slack message from one of the Treehacks organizers, Abi:
Would you be able to demo at the end of closing? As long as you’re in the zoom webinar we’ll bring you up to the stage when your prize :))) is announced!
We made it. We were so excited! To see each other’s reactions, we decided to remain in the Ohyay room and join the separate zoom call for the awards ceremony.
Closing Ceremony
7:00pm EST
It was finally time. Closing ceremony began with a flash announcement of all the sponsor prizes. This breezed by super quick, but we caught a glimpse of our prize — [Pinnacle] Best Hackathon Project.
Each winning team member will receive tickets to Pinnacle, which invites the world’s top 50 collegiate hackathons for an epic finale event in fall 2021. More information can be found here: https://pinnacle.us.org
“We won a prize to another hackathon?!” we exclaimed.
After the sponsor prizes, they began to announce the larger prizes, and team after team was brought to the virtual “stage” to give demos. As this was happening, I frantically tested our my Unity and Oculus setup to make sure it was running smoothly. As the closing ceremony neared an end, we held our breath.
“Theres no way we won the Moonshot prize right?”
As the prizes were coming to a close, we were the last team left and the Moonshot Prize was the only one left on the table.
“Congratulations to Catiator for winning the Moonshot Prize!”
No way. We came on screen and gave our demo from the presentations, but this time live in Unity. As the presentation drew to a close, tired and in shock, we celebrated.
We are so excited that Pinnacle has given us the opportunity to work again together in September! We truly enjoyed every moment of working in our team, and we know our team spirit is one to last. I had never thought of having the chance to participate in Treehacks, let alone win the largest prize. I’m so thankful for everything Treehacks had to offer, and would highly recommend anyone from any skill level apply next year.
Links
Our Project
Devpost // Pitch Video // Fanart Page
Download Catiator (password: Treehacks)
Other Cool Projects I Really Liked at Treehacks
Ugly.video — Less serious video conferencing.
Wonderly — Interact and learn from the world’s greatest minds using Augmented Reality. Creating access to high intelligence with low barriers of entry for everyone.
Maskus — Currently, there are not enough N95s to supply all healthcare workers. We aim to provide a low-cost, custom-fitted, 3D printable facemask which offers superior protection to healthcare workers.
Special Shoutouts
- Treehacks team — for such a well-organized event
- Mitchell — for dealing with my lack of Unity understanding and being so patient with my Oculus not connecting to Unity
- Ryan — for all the wholesome cat content, Maya mentorship, and for always making us all laugh
- Jieying — for creating aesthetic buttons and more Catiator fanart to add to our collection
- Khoi — for opening the door to VR for me at Stanford Splash 2016
- Charles — for saving me with the message in the iD Tech Slack channel
- Joey — for believing in our team
- My Mom — for shipping over my gaming PC from California a couple weeks back when I said I needed it to run Maya and play with VR
- Professor Caspar — for letting me borrow the Oculus Quest 2 and going through the hoops to get it to me through CMU despite not being on campus
- My Roommate Annie — for dealing with me not sleeping for a day and talking the whole night, and for forgiving all the dishes I kept delaying washing
- iD Tech Camps — for providing me with the resources through prepared curriculum to help with my Unity debugging process
- You — for making it so far in my article, thank you so much for reading such a long reflection!
