The race for Virtual Reality (VR) market share dominance just became even more interesting. This week marks the launch of PlayStation VR, the third and most reasonably priced entry into the mainstream VR hardware battle royale. PlayStation VR enters the consumer marketplace following in the wake of the Oculus Rift and HTC Vive, two high-end PC-based VR rigs at the forefront of the VR renaissance. While the Rift and Vive offer high-quality VR, they’ve priced themselves out of the mainstream’s price range. The Vive and Rift (when factoring in the Rift’s additional $200 motion controllers) both ring in at close to $1k and require powerful PCs to run optimally. PlayStation VR doesn’t produce the most high-quality VR, but it works with existing PlayStation 4 consoles ($299). On the other end of the spectrum sits Samsung’s Gear VR and Google’s upcoming Daydream View, two sub-$150 headsets that pair with smartphones and offer decent VR experiences.
With high-end VR gear having such a prohibitive cost of entry, 2016’s VR storm has come off more like a sad drizzle. As a result, movie studios and AAA game developers aren’t investing their resources in multimillion-dollar films and games. Why produce the next Avatar or Uncharted series when the number of early adopters is so small that you can’t earn a return on your investment? Fear not dear reader, VR’s indie movement is alive and well! There are legions of game developers and filmmakers out there plugging away at unique VR experiences right now. These virtual reality vanguards are chomping at the bit to utilize VR as their preferred form of artistic expression.
One such content creator is writer/director Alexander Oshmyansky. Oshmyansky is the mastermind behind Career Opportunities in Organized Crime, VR’s first feature-length cinematic experience. The film is about a Baltimore-based Russian crime boss who goes on a recruiting blitz — think of it as The Office meets Goodfellas. Career Opportunities in Organized Crime premiered earlier this year at SXSW and is currently available for digital download with all proceeds going towards the development of a non-profit pharmaceutical company.
I recently spoke to Oshmyansky about his film. During our discussion, he described his VR filmmaking process in great detail. The following conversation is essential reading for anyone considering diving into the world of VR filmmaking. Oshmyansky shares his thoughts on affordable cameras, his preferred editing software, and the diverse talent out there creating VR content.
How much did you know about virtual reality before you decided to make this film?
That was about a year and a half ago. Not a huge amount to be honest. I had been introduced to the technology and just kinda said, “Oh, I got to get involved in this”. I had to dive right in. I saw that no one had created a feature-length film in virtual reality yet and couldn’t resist the temptation to try to be the first.
After probably about three or four months of pre-production, we were shooting. We shot it in a couple of weeks, and post-production took a really long time. It took like nine months to a year, and then we released it.
What was your introduction to the technology?
Actually primarily reading about it online, and then I had gone to a conference in San Francisco, the Silicon Valley Virtual Reality conference, and played around with all the toys there and was like, “Yeah, this has a lot of potential”.
So you got bit by the bug at a tech show?
Yeah, pretty much.
Was there any place you consulted to bring you up to speed before going to work on your film?
There’s not really a lot out there right now. You can watch YouTube videos, that kind of thing. To put a bit of a shameless plug… right now, I’m about half way through writing a how-to guide intended as an e-book. Maybe we can get it published. A guide to how to do cinematic VR. Hopefully, it will be done by the end of November, October or November, somewhere around there.
I just saw there’s going to be a conference on the Paramount lot where there’s like 4,000 attendees already registered from the Hollywood motion picture bubble, so they’re beginning to pick up on it. I think the main thing keeping it from major adoption at the moment is, of course, the headset.
They’re not sexy.
They’re pretty goofy looking, and they’re super expensive. To play the high-end games, you need a high-end PC but I think in just ten days, 11 days, PlayStation VR is coming out. I think that’s going to be the first one that drives adoption if people can just plug that into their PlayStation and suddenly play the games.
As unattractive as these headsets are, once people try them on, they tend to be wowed more times than not.
Exactly. Generally, when you’re in the virtual reality, you’re not as self-conscious, and hopefully, some designers will make them look less ridiculous as time goes on and maybe more like a pair of glasses. At least I hope. You’re right, you do look kind of goofy — at least for my aesthetic sensibilities once you’re wearing them. But once you’re in the virtual reality itself, people tend to get blown away pretty quickly.
Movie making has always been a difficult and exclusive process, but it’s becoming democratized in the digital age. Making a VR film, let alone a full-length feature is extremely ambitious. What made you decide you had to make a movie this way?
I had always wanted to make a film, I think, like most people do or at least, I’m sure, many people in your readership do. I was caught up in the novelty of it. I think with the democratizing of filmmaking you’re speaking about, I think that extends even to virtual reality films. I think a lot of the tools for it are a lot more available than you might think otherwise, especially in the last year or so.
So when we shot it [Career Opportunities in Organized Crime], we did it on what’s called a Freedom 360 or essentially these guys who produce very sturdy 3D printed casings to put GoPro cameras in. And then after the fact, you go through and stitch it all together. As you know, technology moves very quickly.
The major manufacturer GoPro has introduced housing and casing that integrate with the actual GoPros themselves electronically. Their new Omni-Rig, which, you know. is not cheap to own, I think the whole thing including the GoPros is $5k. Just like a RED camera or an ARRI Alexa, you can rent it for fairly reasonable rates if you live around New York or Los Angeles. That I would say is probably the cheapest sort of cinematic quality camera for professional use. A lot of people I know in the VR space have really fallen in love with that one. Even lower down, you can get really, pretty good quality, consumer-grade virtual reality cameras.
I’ve been using recently Kodak’s PIXPRO a fair amount, which is essentially two back-to-back ultra-fisheye lens cameras. Its been producing really nice quality footage for me. It’s super portable and fairly rugged as far as these things go, not ultra-rugged. But, you know, they’re not unreasonable at all. We actually shipped one of those over to Iraq to catch footage of the city of Sinjar where the attempted ISIS Yazidi genocide of 2014 happened, and it turned out some really fantastic shots for us. Even the consumer-grade VR cameras which are on the market for less than $1k are really quite reasonable.
Samsung just released their 360 cameras with a relatively low price point.
I had recently purchased that one as well to play around with on a shoot. The Samsung Gear 360?
Yes!
It’s very cool looking. It’s very small. They did some great work with just the exterior design of it. Yeah, especially, I think that one retails for around $350 or something like that. It doesn’t have the same dynamic range or quite as much resolution as the PIXPRO does, but, you know, it really doesn’t produce half-bad footage for a very reasonable entry point.
In terms of not having the steepest learning curve, what would you recommend to people breaking into VR filmmaking and video production?
I think those two cameras are probably the best two on the market at the moment with sort of a short learning curve. Because, as you were saying, a lot of it is not so much capturing the footage that’s as much of a challenge. It can be because you’re using all these different cameras on the high-end rigs, and they’re kind of finicky and break down and such. It’s really the post-production, the technical aspect, where things get tricky because you got to take the footage from all these cameras and sort of piece them together.
Can you recommend a type of software?
Sure. I use Autopano 360. I don’t think it’s Autopano 360, just Autopano, which is probably the standard in the field right now in terms of being easier to use. The other video software package that is commonly used is VideoStitch. It’s a little more wonky. It’s a little bit more technical, so you have to play around with it a bit more to get stitches, but it allows you to work through some problem shooting areas. Both of those packages are a little pricey. Maybe the price point has changed since I did it. It’s about $1k for those programs.
The nice part about both the Gear and the PIXPROs, they both come with their automated stitching software. Now, that’s often times not ideal because what you’ll want to do is go in after the fact and sort the images to make sure that the stitch between the camera fits. Really, at the end of the day, for high-end VR, you’re probably using a new core compositing software package that depicts the fine details of things. That’s basically how the workflow goes. You create a preliminary stitch and one of these packages like Videostitch or Autopano (I use Autopano primarily), and then you go into your compositing software that depicts the things which you couldn’t quite get with the automated stitching software packages.
I’ve been exploring VR for quite a while. The first time I tried, it was 1992.
That was even before the Virtual Boy Era!
I think Virtual Boy was 1995 or ’96. I always thought of VR in terms of video games. It didn’t even cross my mind that we would be watching movies in it. The technology has had a bit of a renaissance. I’ve been attending various tech conferences and industry panels as well as reading up on how the technology can be applied. Everyone seems to agree that there’s no current cinematic language for making a VR movie: Where should the camera go, how should someone light a set, and how should directors block their actors? While watching your movie, it really dawned on me that VR films, at least for now, have to be staged like a play. You can’t cut between cameras because it’s too disorienting, and you have to run through long takes. What kind of obstacles did you run into that came out of nowhere?
We shot the film about a year and three months ago, and even since then, the language of VR has evolved very, very quickly. So, watch my film, and it almost seems even dated now.
I think every filmmaker feels that way.
I hope so (laughs). We did basically a proscenium in the round thing. There’s a lot of concern in VR about motion sickness, so I wanted to make sure that as many people could approach the film as possible. We kept the camera static for that reason. Now people are discovering ways to ease into avoiding the motion sickness issue. You just need a really good, steady steady-cam, an excellent steady-cam operator. What really gets people nauseous is the sense of acceleration. It’s not so much the motion itself. It’s the changes in the speed of the motion. So if you get a very smooth glide, you can do a nice pan without that sort of nauseous effect.
In terms of cutting, the standard is, you want to do your longer cuts in virtual reality. If you view some of the VR footage that’s out there, it’s generally two to three minutes between cuts as opposed to three to five seconds for conventional film. We experimented with that somewhat for the trailer for the movie. We just wanted to see what would happen, so I made a series of classic cinema-like cuts just to see how people would react. It’s basically how you would expect, they were fairly disoriented by it.
I think in virtual reality at the moment, you have this expectation that you’re physically in a space, and you don’t really have that in conventional film. When you’re watching a movie, you may project yourself upon the characters, but you don’t actually feel like you expect to have reality act like it does in real space. Suddenly, you can see one part of a room and then another without moving your head. In virtual reality, you have that expectation that the world around you should function the way it does in actual reality. Thus, teleportation in actual reality gets disorienting. I think as the medium evolves and the audience becomes accustomed to it or exposed to it for the first time really, it will sort of play upon these conventions. In the next few years, maybe even five to ten years, a lot of it will change dramatically. Right now, that’s kind of the storytelling framework, it’s these longer shots, sort of staged almost like a play.
A lot of the actors said it felt a lot like acting in the theater to them as opposed to acting in a film because they have to project more to the camera. The fine nuances of their faces aren’t being captured by the camera like it would in a close-up in a conventional film. All these little issues spring to light. In terms of lighting, sort of the conceit of our film was it was a documentary being shot. That was our little cheat for allowing lighting to be in place when it wouldn’t be otherwise. Natural light is used a lot, flat panel LEDs, which can be sort of inconspicuously placed behind a window or in some other way cheated.
A lot of what’s done actually is the shots aren’t actually composed simultaneously between multiple cameras. So, you’ll set up all your lighting equipment so that everything is caught on just one of say, six cameras (all the technical equipment, the lighting equipment). You’ll shoot your scene, and then you’ll shoot it again with the director in the view and the equipment in the view of another camera. Then, in post-processing, you’ll take different cuts.
Sounds like a nightmare.
It is a nightmare. As you might imagine, organizing all of this, it’s a big feat of logistics making sure it all happens right. Basically, at the end, there are ways to composite out technical equipment form your shot. But as you say, it’s a bit of a nightmare.
Were there any tricks you picked up towards the end that made you say: “Oh man, I wish I knew this a few weeks ago”.
A lot of it, to be honest, didn’t come immediately after the shoot, it came in post-production. Even though I had experimented with the technology before starting, just the sheer volume of footage we shot for the film (we shot maybe 30 to 40 hours of VR footage for the film, which is by VR standards, a fairly large amount to say the least), going through all that I’m like, “I should have repositioned the camera or the background differently in order to get the stitching correctly. I didn’t think that this person crosses the stitch line here, and that’s causing a nightmare of a post-production issue.” It was that kind of thing after the fact that really drove home the idea of how the camera should essentially “think” during production about post-production. I think that’s the biggest piece of advice I would have.
With the newer cameras, stitching issues are decreased. The new VR systems, the cameras are better synchronized. They’re placed closer to the focal point of the 360 cameras to minimize parallax effect. One of the big issues with VR is this thing called parallax, where if two people or two portions of the scene are different distances from the camera, the stitching has to be different for those two different parts of the scene.
Oh no…
What’s going on is because all the sensors of the cameras are kind of removed from each other, they’ll see things distorted in different ways. They’ll see some objects as bigger, some objects as smaller, and when you try and put that footage together, you can get everything that’s an identical distance from the camera distorted the same way, but once you distort that if there’s something else further from the camera, the distortion of that in your post-production fixing will be off. So, the new cameras essentially try to put all the different camera sensors as close together as possible to minimize that parallax effect. When you’re shooting for ease of post-production, it’s best to keep in mind exactly what distance from the camera the most important things are happening. You want to shoot things at different distances in extending concentric circles from the camera. Like, where are you going to put that so that when it comes together, in the end, you’ll be able to do that relatively smoothly.
That’s an art form in of itself.
You’re coordinating all of these things simultaneously, which are actor blocking and your production design. It is a logistical challenge, to say the least.
I’ve attended tech panels that presented VR cameras that stitch on the fly. If you have to frame shots in such particular ways, how do these cameras process all that info?
With cameras that stitch on the fly, especially those designed for live virtual reality, at live virtual reality events, the understanding is that your output is going to be a “Rough Stitch”, kind of like a rough cut. It will be a good first pass. It’s kind of understood it won’t be seamless, they’ll do their best basically.
Is that the equivalent of recording standard digital video in low resolution?
Yeah, exactly, exactly. Or like when you’re playing around in Premiere and you put it on 1/8 resolution so that it plays fast.
Are you ready for more VR filmmaking or have you had your fill?
Oh no, I think so. We’re planning on shooting a virtual reality documentary, not feature-length, but maybe like 10 five to 10 minute episodes in Iraq focusing on the re-taking of Mosul from ISIS. We’ve been putting that together quite intensely for the past several months, and hopefully that will come out at some point next year.
What made you decide to do a crime movie?
I just had the idea for many years. I always wanted to do it. When I was a kid, I had gotten into film school and chose not to do it. This was my opportunity to make the movie I always wanted to do so I went for it.
To your readers who are filmmakers, I would say it’s a great time to get involved in the medium just because the skill-set is so new and so specialized that people can come out of nowhere and produce things and have it be noticed very quickly. It’s a very fertile ground for new and rising talent. What’s really cool is that in the VR filmmaking community at the moment there’s actually a fair bit of diversity. It’s a big thing in Hollywood, the OscarsSoWhite thing last year.
The VR filmmaking community, because it’s so young and so robust, a large proportion of it is women, is minorities, which you wouldn’t expect at first. There was an article, I think today, about the female virtual reality mafia. It seems ultra-nerdy, and it would seem like one of these gamergate type exclusive things, but it’s really not. It’s so new that any creative person could sort of jump in without institutional roadblocks to doing it at pretty much the same level that everyone else is doing it at the moment. I would encourage your readers to just dive in. There’s really nothing stopping it.
Where can people look for you and your work?
We are on Reelhouse.org which is a really nice platform that allows you to upload your films without having to program your own app to play them.