Tribeca – 2016



With the launch of the Oculus Rift Kickstarter campaign in 2012, virtual reality (VR) has been quickly gaining popularity. Content creators from movies, documentaries and video-games are all clamoring on board a crowded band-wagon. On the week of April 15th, the Tribeca International Film Festival hosted its first ever space dedicated to virtual reality. The TIFF titled the space ‘Virtual Arcade’, and it included 13 VR experiences from independent game designers from various parts the globe. Some pieces were fully interactive while others offered a more passive experience, but all were fully immersive virtual reality experiences. A floor above the Virtual Arcade housed the Games For Change – Games and Media Summit. These interactive experiences were more experimental and focused on how VR can change the future of education. The booth I was a part of, BrainVR, was an educational interactive that taught users how the eye picks up patterns of light, and how the corresponding synapses in the brain fire as a result. BrainVR was created by the people behind Eyewire (known for the gamification of the otherwise impossible task of reconstructing neural structures of the brain based on 2-dimensional scans into 3D models), some folks from MIT, and myself. I was asked to assist in presenting BrainVR to the general public along with the rest of the group – here’s what happened.

On Sunday, April 17th I left for New York. I drank too much the night before – gravity was oppressive that day so I took escalators instead of stairs. By the time I was on the Chinatown bus my head was pounding and my pants were sprinkled with urine (the urinal’s splash guard acted opposite to its intended purpose). After spending five hours in fear that I’d have to puke in the Lucky Star bus toilet, I made it to NYC with my breakfast still in my stomach. I hopped on the subway and met up with my old college buddy Tim, who agreed to house me for the night. We played Dungeon and Dragons until about 1:00am, and then I restlessly slept with Tim’s cat Garfield on the couch. In the morning Tim and I made coffee, bid our farewells, and I made my way to Tribeca.

Good ol’ trashy NYC

A thousand years ago I worked in the Tribeca area for a mural painting company, so I knew my way around. I arrived at the Tribeca building an hour and thirty minutes late. The place was buzzing with people. I saw maybe a hundred college students employed as orderlies in red shirts. I saw smart guys with square glasses and crisp jackets, and women who must have been from another planet. I told them who I’m with but not who I was, and before I knew it I had a badge and a wristband, ‘6th floor, please’. I took the elevator and bounced around until I saw Scott, my main connection to the MIT Media Lab. He was next to a handsome fellow I did not know – I later learned his name was Matt. Nice guy. Indeed I was late but managed to arrive before the rest of the BrainVR group.

The exhibition space was vast, much bigger than I had anticipated. The room had a high ceiling adorned with crisscrossing track lighting, bathing everything in a deep futuristic blue. Once in a while you heard the hiss of a machine that spewed smoke smelling of cotton candy. Everyone was setting up, all were hurried but relaxed.

The red shirt orderlies proved to be incredibly helpful. They acted on our demands without question, eagerly awaiting orders like a dog who loves to show off his tricks. One of them, named Matt, had bright youthful eyes and a big mess of light-brown hair. He asked me if he could take my bag for safe keeping. I said yes, but I wasn’t very comfortable telling a guy not much younger than me what to do. By the end of the show I was sending Matt for coffee and sandwiches like I was King Louis XVI.

About an hour later, the team arrived and we finished setting up our space. We were covered in sweat after correcting a couple snafus with the electricity, but we had everything up and running. At noon, people begin to trickle into the space, their heads on a swivel looking for what to view first. A few approached our booth and we smiled and we said things like:

“Have you ever tried VR before?”

“We’re part of the MIT Media Lab!”

“We’re using VR to explore ways to educate through communicative storytelling!”

We gave this speech about seven hundred times throughout the day.

Daniel, our programmer, in way over his head

We were elated. Pure euphoria. We were experts of our own product. We made something, and we’re showing it to strange people whom we’ve never met. TIFF visitors tried it and loved it, we asked them how they felt and whether or not they learned anything. Some nodded, some looked perplexed and some said no. Over time we honed the process of pressing palms, asking questions, managing the line and dunking the Vive onto people’s heads.

At around 2:00pm we began to loose steam – I could see it on our faces and feel it in my knees. The sweat had dried and I was cold (air conditioning had been blasting all day). We grabbed one of our assigned orderlies and taught her how to set people up in the Vive, and like an orderly cast down from red-shirt Heaven she picked it up immediately, no problem. We left her and the exhibit hall and made our way downstairs to the Virtual Arcade (Presented by AT&T!).

The Virtual Arcade

The elevator doors opened onto a wide room with a similar vibe as our own exhibit hall, but more pronounced. The blue lighting seemed to press down on us, as if gravity were stronger in this room. The smoke was so thick that they might as well have crammed cotton candy in your mouth upon entering. The room’s length began at our left extending 80 yards to our right and was filled with monitors, fashionistas, and snazzy white swivel chairs running single file down the length of the room, occupied by people wearing Gear VRs. Dance music was blaring. Put it all together and it was VR Studio 54. Instead of people snorting coke off bare asses and/or providing sexual favors, people were sitting around with big boxes on their face. This was New York, so they managed to hold dignified postures while in complete oblivion to the real world.

Well, kind of dignified

To our right, a row of white booths ran from the entrance to the end of the room – these were the advertised VR experiences. There were 13 different experiences to try, and I’ve been overhearing people raving about a game where you fly on the back of a dragon, so I had no trouble choosing what I wanted to try first. I made way to the booth owned by Blackthorn Media, the creators of said dragon flying game. I talked to the founders, Michael and Lyndon, both warm and smiling guys who could not have been more giddy of their enterprise. They were once involved in the Visual Effects industry for movies and TV shows, but recently took their skills of creating virtual worlds and brought it into the dynamic realms of gaming. When you play their games, it shows.

Blackthorn’s assigned orderlies who were running an app that allowed people to queue up without having to stand in line. Michael told me that the queue was running a bit long, and that the wait could be as long as an hour and a half. Apparently everyone wanted to ride on the back of a dragon as well. However, to my luck, their line queue app failed and they had to renew the waiting process. As to who was next, it came down to whomever happened to be nearest the booth. That person happened to be yours truly. Michael whispered behind his hand if I wanted try something they were working on but were not expecting to show that day. I spat an emphatic ‘yes’ and made my way into the booth, where I met Keith who introduced me to Blackthorn’s games.

This is what being oblivious looks like

There’s nothing quite like being in a virtual world and reaching out and grabbing something, to physically interact with a digital environment. It’s as if the brain wants to be fooled, and feels no need to try and rationalize that everything you’re looking at is not ‘real’. In fact the brain splits the difference, and makes you feel the weight of an object, heat from fire, a knife to the stomach. However the dragon game lacked this type of immersion. You got onto the back of the dragon (you actually had to walk over and physically sit on a chair, which in the virtual world was the dragon’s back) and flew around and blasted fire out of the dragon’s mouth, aiming the flames with your controller. I flew past castles shooting flaming arrows, trees and valleys, and other dragons, but the immersion fell flat. My brain was not convinced. I wondered why everyone was foaming at the mouth about this game, and figured it was due to the recent onset of dragon fever. I flew past a another castle which shot more flaming arrows that didn’t seem to have any effect, and a rival dragon came into view and crashed into me, thus ending the demo.

I hid my disappointment behind some ‘ooo’s and ‘ahh’s, but I was miffed. This is exactly what people should not do in virtual reality. Another incomplete demo with boring mechanics built for hardware that has infinite potential. All of their effort, it seemed, was put into the dragon itself. I’ve been seeing a pattern of developers creating ‘demos’ with no perceived end in sight. These demos are either too big or not big enough. The latter always makes me ask myself ‘who cares?’, and the former makes me question their sanity. It frustrates me to think of all the hours developers put into projects lacking any foundation or long term planning. But of course, and I must remind myself of this constantly, not everyone can be a Steve Jobs – type visionary of video game development. With the sheer number of individuals creating video games you’re bound to get the bad with the good, most of it bad. One must be patient with such things.

Next up was the project they were not ‘meant to show’ – The Abbot’s Book. I was still wearing the headset and blind to the outside world, to which Keith asked me to stand so I did. I heard him sliding the chair away from the center of the booth to provide me space. Keith then gave me some context for the following experience: The Abbot’s Book takes place in 16th century Vicenza Italy, and centers around a family tormented by a diabolical book. He told me there’s a a lot of backstory that accompanies the game (a whole book, in fact), and this is only one chapter of what will eventually be a flowing episodic adventure. Intrigued, I gave him the go ahead and he threw me into the world.

It was dark. The moon was shining bright in the night sky, partly obscured by passing clouds. I noticed that I had a torch in my right hand and used it to light the area around me. I was standing in the middle of a ruined castle, the ground was overgrown with weeds that reached up to my knees, swaying with the wind. Old and worn gravestones stood tall around me, like a line of menacing soldiers alerted by my sudden presence. Owls hooted and frogs chirped, the weeds hissed as the wind blew. Off in the distance I could sense the immensity of the castle, and saw tree covered mountains disappearing into the horizon. All of Blackthorn Media’s hard work payed off. I was utterly immersed.

A big issue with VR is movement. When you’re moving through the virtual space, there’s a disconnect between what you’re seeing and the actual feeling of movement. The brain looks for the sensation of forward momentum and doesn’t find it, but sees it, so things get all queasy. This is known as simulation sickness. One trick that developers have discovered to avoid simulation sickness is by ‘Blinking’, or teleporting from place to place. With this method you don’t slide through the virtual space, but you teleport from your current position to an area you point at. This allows players to move through the virtual space without wanting to puke 10 minutes in. Blackthorn Media applied this method of movement in The Abbot’s Book and it worked like a charm.

As I moved down a hill, I bent down and inspected the land around me. I saw well rendered weeds and rocks but nothing else of interest. I looked up and saw a heavy iron gate down the slope, I made my way to it. With both hands I pushed the iron doors open and found a monk standing on the other side, waiting for me with an air of importance and regality. I used the light from the torch to get a better look. The monk was aged fifty years or so, his robes were simple and humble and swayed with his movements. Despite his grave face, he did not appear to hold any ill will towards me. He informed me that we were to head down to the catacombs, just below the castle, and find a lost colleague named Calidan.

I continued on the path and down a narrow staircase, and stopped at the entrance of the catacombs. The old abbot warned me of dangers and terrors awaiting behind the door, and advised me to ‘proceed with caution’. I moved through the door and into the cave. It was pitch black, the only source of light was my torch which illuminated the walls of the cave. Ornate stonework and busts of snarling beasts stood sentry at the cavern’s opening. I proceeded with caution, probably more than the abbot intended – I was pretty damn scared. I turned around and saw that he was accompanying me, which lent me some comfort. There were unlit lanterns handing from the walls, and I lit these with my torch as I moved down the hall. Light spread down the cavern, which opened onto a four-way intersection of hallways, all surrounding a tomb at the center. I made my way into this space, and I saw a hint of blue light at the end of the hall to my left. I made for it and moved down the hall until I reached the precipice of a cliff. I looked up and gasped.

The hallway opened into an enormous cave, filled with structures and churches, buildings, lit by small openings to the outside world that brought in blades of blue moonlight. The space of the cavern seemed to stretch on forever. The path ended on a sheer cliff, and as I went to peak over the edge a large chunk of rock fell off the narrow path and I jumped back, as to not go tumbling over the edge. I then entered a room to my right in which I found a Gollum-like creature feasting on a corpse. It looked up and saw me and I dug my feet into the ground, preparing to defend myself. It screamed, stood and charged me. I coward in fear. The demo ended.

My eyes watering with excitement, I took off the Vive headset and returned to reality. I tried to use my words and tell Michael, Lyndon and Keith how awesome that was, but my brain was on fire with so much inspiration and so many ideas that no doubt what I threw at them was unintelligible. Regardless, they seemed to appreciate the fandom; they busted their asses on this project after all.

Up until that point I had not been in a virtual experience as immersive or as polished as The Abbot’s Book. Within five minutes of being in the game, the real world fell away; I forgot I was at Tribeca, and I forgot Keith was standing there watching me fumble around the booth, blind deaf and dumb. For something barely in its infancy, when it works, the HTC Vive works pretty goddamn well. The Vive was created by HTC, a multinational manufacturer of smart phones and tablets, and Valve, the video game company who built Steam, a digital distribution platform for managing the purchase and downloads of thousands of video game titles. Together they hedged their bets on a concept called ‘room-scale VR’. The concept is simple and brilliant:

Room-Scale VR looks like this

Imagine, if you will, that you wanted to set up your living room for room-scale VR. First and foremost, you’ll want to clear a space. Move the coffee table, reorient the couch, do whatever you need to do to make a large open space, free of any obstacles (especially those below the knee). The Vive kit comes with two motion sensors, which you can mount on a wall or a mic stand. Think of the perimeter of virtual space as a rectangle. The sensors make up two opposite corners of the rectangle, and track the movements of the VR helmet and the pair of controllers. Once everything is all set up and calibrated, you can celebrate by doing cartwheels, cranking out jumping jacks or running in circles, the way room-scale VR was intended to be used.

I’ve been on this Earth for 29 years, and have been cognizant for about 26 of them, so I have some experience with hyped products that end up falling flat (the infamous 3-D-O comes to mind). Oculus Rift opened its Kickstarter in 2012, just four years ago, and here we are just four years later, giddy and teary eyed imagining the possibilities of future virtual entertainment. The hype has now reached dangerous levels, and fanboys are waiting for the dam to break. As for Oculus, they brought virtual reality back into the minds of the general public at the perfect time. It was as if they said ‘Hey guys, remember this thing from the mid 90’s that completely flopped? Maybe it wasn’t such a bad idea after all’, and when game developers realized the potential of combining modern day graphics with virtual reality, the entire video game industry must have muttered ‘holy shit’. Now, the term ‘video game’ is giving way to the to the high falutin term ‘interactive experience’. Almost everyone I talked to at TIFF is approaching VR as an interactive experience, and not from the established paradigms from the past 30 years of game dev. They also seemed to have the idea that they were the first to think about it. I wanted to inform them that ‘this guy over there told me the same thing’, but I bit my tongue because we’re all in the same boat; humans tend to come up with the same idea at the same time.

We spent the rest of the day showing our VR piece and encroaching upon important people. The NY times, CNN, and a couple of big-shot indy film makers made their way by and we flocked to them like beggars to an aristocrat. The show ended at six, and the crowd thinned, and I finally got to try our BrainVR demo ( I had not experienced the final version until TIFF). It was a cute demo, the robot avatar I made did its thing and was perfectly implemented by the team’s programmer. There was voice over and music, a beginning, middle and end. It was quiet participation, not much to it. The neurons looked great, my 3D eyeball looked okay. What astonished me was the speed at which we put it together. The tools to create this stuff couldn’t be more accessible and easier to use, and they are going to become more accessible in the coming years.

A Tribecian trying our interactive

What will the world look like when everyone is doing this? When we make assumptions, we tend to say things like ‘everybody’, or ‘everywhere’, which of course, 7 billion people aren’t going to be programming VR games. But of a lot of them will be, and what will the market for this look like?

Software tools are built layer upon layer. Common repeated patterns will eventually be streamlined, and the hard work of a single individual will pay off for someone else down the line. Programmers pass the baton to strangers to take up the task of simplifying the process of creation. Things that are complex and cumbersome to accomplish today will be a button press down the road. It will become easier and easier for the individual artist to act as programmer, director, sound designer and writer for any given project. And with distribution from creator to consumer being as straight forward as it is, publishing a video game could be like publishing a blog post, a digital painting, or an e-book. The most popular of these tools, Unity, is free until the creator(s) obtain revenue or funding up to $100,000 – at that point you’ll need to pay $1,500 for a full license. Other than that, it’s free to download and use. The Unreal Engine, which has a similar pricing model, not too long ago used to be priced somewhere around $50,000. In under a decade, these tools have become (mostly) free to use, and dreamers with big ideas now have the tools and means to turn those ideas into reality.

TIFF hosting an entire swath of its exhibition hall to virtual reality is a testament to popular culture viewing VR as a potential storytelling medium. My compatriots and I are delving down the educational route, but dollar signs are nowhere in sight on the horizon. This stuff is a barrel of fun to create, but one needs to consider funding at some point. We all want to make games with flying dragons and beautiful 16th century settings like Blackthorn Media, but they’ll tell you the same thing they told me: People are not willing to fund this stuff. Fortunately there are options like Kickstarter and Indiegogo, but I can’t help but see these crowdfunding sites as the Craig’s List for would-be entrepreneurs. It’s difficult to find a reason to trust the creators, even harder to gain the confidence of potential backers, as every campaign acts like they’re just a dollar away from curing cancer. Right now the VR space is dominated by small-time developers. Eventually video game giants such as EA and Ubisoft will come stomping through the VR space like robotic Godzillas, spewing money and drowning out all competition in their path. Big companies are just lying in wait, ready to pounce on the winning formulas the little guys create. It’s up to the public to decide who wins the battle.

At last, the show ended and we sent the orderlies for our personal items while we packed up. We had a drink (on the house) and some laughs, pleased by how the day went. I bid farewell to the BrainVR group, got some chinese food and hopped on the 2nd-to-last Chinatown bus to Boston.

The BrainVR team

After another arduous trip aboard the Lucky Star bus, I was covered in piss again (note: do not use a bus toilet while the bus is leaving the city) and I suspected I got food poisoning from the General Tso’s Chicken I nabbed before leaving NYC. At 12:30am, while walking back to my apartment, I thought about how much I hate the term ‘VR’. It’s now on the lips of everywhere I go. At work, at home, amongst my family and friends, I can’t escape it. Like when your mom joined Facebook, ‘VR’ no longer seems new and exciting. Every time I hear it in popular culture or some actor spouting about how excited he is, its sharp sleek edges rust and become a little bit more dull. Maybe it’s just the company I keep, but everywhere I look I see people digging their feet into the ground preparing themselves to swallow up the electric tidal wave of oncoming virtual reality based entertainment. No-one knows where this ride is going to take us. We can prophesize the end of face to face human interaction all we want, yet not a one of us can accurately predict the future of anything. All I know is this: Virtual Reality is not just hype, it works too well. It grabs our two most important senses – sight and sound – by the balls and we become slaves to the experience. And if the creators of these experiences keep moving at their current velocity, what we’ve seen thus far is child’s play compared to what we’ll witness a few years down the road. This is bandwagon I’d like to be on.