The workspace of the future is in Elon Musk’s lab. Using a variety of virtual reality and gesture-sensing tools, Musk has set up a system that allows himself and his engineers to design and manipulate models of rocket parts using just their hands. He's compared it to the Iron Man laboratory, and in many ways, it looks like just that.
But you won't need the technological expertise of Tony Stark in order to make one: Musk employs a Leap Motion Controller, an Oculus Rift, and a projector — among other common tools — in order to make the setup work. Not all of those are necessarily being used at once though. Musk says that he began with the Leap Motion, and then expanded to more advanced setups, such as one that involved projecting 3D mockups onto a translucent pane of glass.
While Musk admits that it's partially just "a fun way to interact with a complex model," he thinks that this new setup could mean far more than that. "I believe we're on the verge of a major breakthrough in design and manufacturing." Musk demonstrates how using the Leap he can fully move and rotate the model by just swiping, opening, and closing his hands.
A video of a new virtual reality prototype that uses both Oculus Rift and Razer Hydra technology shows how users can use their mind to move around artificial environments.
Users require an Emotiv EPOC, a device capable of mapping certain thought patterns to actions, to read their brainwaves in order to interact with the prototype through thought. Naturally, this is still in its experimental phase; While virtual reality technology is becoming affordable, the Emotiv EPOC's capabilities are "still quite primitive," and not wholly user friendly, writes developer Chris Zaharia.
"With my experience in the education industry through my startup Zookal and keen interest in neuroscience, I had a thought around how these technologies could be used together to enhance education and at the same time, see how far can we go with using cognitive control in a virtual simulation," he writes.
Zaharia hopes to explore the possibility of using virtual reality for educational purposes ranging from engineering to biology. The demo offers a look at what is currently possible using virtual reality headsets, motion tracking through Razer Hydra and cognitive control in virtual simulations.
Tap for more stories
- Chris Zaharia (YouTube)
Automation has struck down on game development. Developing a game gets easier every year. Certain jobs aren't necessary anymore, game companies go bankrupt, business plans can not keep up with industry development speeds. In short the game industry requires a high shifting skill. Today I want to talk about a few of these speedy developments. Where is the industry heading? And what do I think are upcomming trends?
Oculus VR’s Palmer Luckey, left, and Nate Mitchell, right. At center, AllThingsD’s Lauren Goode tries out the Oculus Rift at CES 2013.
This is the second part of our two-part Q&A with Palmer Luckey and Nate Mitchell, the co-founders of virtual-reality gaming company Oculus VR. In Part One, Luckey and Mitchell discussed controlling expectations, what they want from developers, and the challenges of trying to make games do something radically different.
AllThingsD: What do you guys think about Google Glass? They’ve got their dev kits out right now, too, and –
Palmer Luckey: — What’s Google Glass? [laughs]
No, seriously, they’re doing something sort of similar with getting this wearable computing device to developers. Does the early buzz about Glass worry you?
Luckey: No. They’re not a gaming device, and they’re not a VR device, and they’re not an immersive device, and they’re five times more expensive than us.
Nate Mitchell: It’s just a completely different product. Wearable computing is super-interesting, and we’d love to see more wearable computing projects in the market. At Oculus, especially, we’re excited about the possibilities of Google Glass. We’ve seen it, we’ve checked it out, it’s very cool. But if you bring them together –
Luckey: Our image size is like 15 times larger than theirs. It’s like the difference between looking at a watch screen and a 60-inch monitor. It’s just an enormous difference.
Mitchell: With the Rift, you’re in there. You’re totally immersed in the world. I think one of the things people keep bringing up (with Glass) is the awkward, the social aspect. For the Rift, you strap into this thing, and you’re gone.
Luckey: It’s about being inside the virtual world, not caring about the real one.
Mitchell: You could put your Glass on in the virtual space.
Luckey: We could do that! We could simulate Glass. … It’s not that hard. You just have a tiny heads-up display floating there. A really tiny one.
Mitchell: I like it.
“Okay, Rift, take a picture. Okay, Rift, record a video …”
Luckey: There’s actually Second Life mods like that. People sell heads-up displays that you can buy.
Luckey: And they put information in there like distance to waypoints and stuff.
Mitchell: Oh, that’s cool!
Luckey: Yeah, they overlay it on the screen when your character’s wearing it.
I never really “got” Second Life. Minecraft, I can wrap my head around quickly. But Second Life …
Luckey: It’s very difficult to get into. There’s a steep learning curve. The last time I went into Second Life was to buy bitcoins from a crazy guy who was selling them below market value, but you had to go into Second Life to meet with him.
Mitchell: The underbelly of the Internet.
Luckey: They’re actually working on Oculus Rift support, though. The kind of people who make games like Second Life definitely see the potential for virtual reality — being able to step into your virtual life.
And if you’re completely immersed in the game, I guess that wherever you’re playing, you need to trust whoever’s around you.
Mitchell: Absolutely. There’s already some sneaking up on people happening in the office. Someone’s developing, they’re testing the latest integration, and then Palmer comes up and puts his hands on their shoulders: “Heyyyy, Andrew! What’s going on?” There’s a trust factor.
Luckey: Have you seen the Guillotine Simulator? (video below) Some people are showing that without even telling the person what it is: “Here, check this out!” “Whoa, what’s going on?” And then — [guillotine sound effect]
Mitchell: One thing that that does lead into is, we’re exploring ways to just improve the usability of the device. When you put on the Rift, especially with the dev kit, you’re shut off from the outside world. What we’re looking at doing is how can we make it easy to pull it off. Right now, you have to slip it over your head like ski goggles. The dev kit was designed to be this functional tool, not the perfect play-for-10-hours device. With the consumer version, we’re going for that polished user experience.
What about motion sickness? Is it possible to overcome the current need for people to only play for a short period of time on their first go?
Luckey: The better we make the hardware, the easier it’ll be for people to just pick up and play. Right now, the hardware isn’t perfect. That’s one of the innate problems of VR: You’re trying to make something that tricks your brain into thinking it’s real. Your brain is very sensitive at telling you things are wrong. The better you can make it, the more realistic you can make it, the more easily your brain’s gonna accept the illusion and not be throwing warning bells.
You mentioned in one of your recent speeches that the Scout in Team Fortress 2 –
Luckey: — he’s running at like 40 miles per hour. But it’s not just, “Oh, I’m running fast.” It’s the physics of the whole thing. In real life, if you are driving at 40mph, you can’t instantly start moving backward. You can’t instantly start strafing sideways. You have inertia. And that’s something that, right now, games are not designed to have. You’re reacting in these impossible ways.
Mitchell: In that same vein, just as Palmer’s saying the hardware’s not perfect yet, a huge part of it is the content.
Luckey: You could make perfect hardware. Pretend we have the Matrix. Now you take someone and put them in a fighter jet and have them spinning in circles. That’s going to make someone sick no matter how good it is, because that actually does make people sick. If you make perfect hardware, and then you do things that make people sick in real life, you’re gonna make them sick in VR, too. Right now, there’s lots of things going on in games that don’t make people sick only because they’re looking at them on a screen. Or, in so many games, they’ll have cutscenes where they take control of the camera and shake it around. You don’t want to do that in VR because you’re not actually shaking around in real life.
You’re changing the experience that you have previously established within VR.
Mitchell: It breaks the immersion.
Luckey: And that’s why it’s so hard to instantly transfer. In the original version of Half Life 2, when you’d go into a new space for the first time, the whole game would just freeze for a second while it loads. It’s just a short freeze, but players were running along or driving along and all of a sudden, jjt! Now it looks like the whole world’s dragging along with you, and a lot of people feel very queasy when that happens.
Mitchell: It comes back to content. My talk at GDC was very specifically about how developing for VR is different from a 2-D monitor. All those things like cutscenes, storytelling, scale of the world — if the player is at four feet on the 2-D monitor and you put them in there, they immediately notice. They look down and they have the stereo cues: “I’m a midget!” So you make them taller, and now they don’t fit through doors. We really do believe that, at first, you’re going to see these ports of existing games, but the best “killer app” experiences are going to come from those made-for-VR games.
Luckey: And that’s not even to say it has to be new franchises. It doesn’t have to be a new type of game. But you want the content to be designed specifically for the hardware.
Mitchell: It’s just like the iPhone. The best games come from developers pairing hardware and software.
Oculus VR CEO Brendan Iribe testing out the Rift at D: Dive Into Media.
And that’s the 10,000-foot view: Does VR change game design in a fundamental way?
Mitchell: Yes. Fundamentally. Absolutely. I think, right now, there’s this great renaissance in the indie community. Indie developers are doing awesome things. If you look at games like The Walking Dead, you’ve got the mainstream genres here. You’re going to have a lot of these indie games start to feel more natural in virtual reality, because that’s almost, like, the intended experience.
Luckey: And not to invent a whole new genre on the fly, but you don’t see many first-person card games or something. There’s a lot of card game videogames, but there’s not many that are first-person because it wouldn’t make any sense to do.
Like a poker game where you could look around the table and read people’s reactions?
Luckey: And you could have all kinds of things integrated into it. I guess that would fit into the first-person-shooter genre, but not really, because you’re not moving and you’re not shooting. You’re just playing cards.
Mitchell: And if you look at the research that’s been done on virtual characters, it’s the type of thing where, if you smile at me in VR, even if you’re an NPC (non-playable character), I’m much more likely to smile back. Your brain is tricked into believing you’re there.
Luckey: There’s also fascinating research on confidence levels in VR, even tiny things. There was a study where a bunch of people performed tasks in real life, in a control group, and then performed them in VR. And the only difference is that one group in VR was about six inches taller than the other group. So, one was shorter than the NPC they were interacting with, one was taller. Universally, all of the “taller” people exhibited better negotiation with the NPCs. Then, they took them out (of the VR simulation) and they redid the (real-world) study, putting everyone back in another trial with a physical person. The people who’d been tall in VR and negotiated as a taller person did better when they went back into the real negotiation as well. It’s ridiculous.
Mitchell: That’s the sort of thing we’re super-excited about. That’s the dream.
And do you have a timeline for when –
Mitchell: When the dream comes to fruition?
Luckey: It’s a dream, man! Come on! [laughs]
Not when it comes to fruition. Are there milestones for specific accomplishments along the way?
Luckey: Sure, we have them, internally. [laughs]
Mitchell: We have a road map, but like we keep saying, a huge part of this is content. Without the content, it’s just a pair of ski goggles.
Luckey: And we don’t even know, necessarily, what a road map needs to look like. We’re getting this feedback, and if a lot of people need a certain feature — well, that means it’s going to take a little longer.
Mitchell: But we have a rough road map planned, and a lot of exciting stuff planned that I think you’ll see over the course of the next year.
And is there a timeline for when the first consumer version comes out?
Mitchell: It’s TBD. But what we can say is, Microsoft and Sony release their dev kits years in advance before they get up onstage and say, “The Xbox One is coming.” We went for the same strategy, just open and publicly.
Luckey: And we don’t want to wait many years before doing it.
Mitchell: Right. So, right now, we’re giving developers the chance to build content, but they’re also co-developing the consumer version of the Rift with us. Once everyone’s really happy with it, that’s when you’ll see us come to market.
Luckey: And not sooner. We don’t want to announce something and then push for that date, even though we know we can make it better.
And what about the company, Oculus VR? Is this dream you’re talking about something you have to realize on your own? Do you want to someday get acquired?
Luckey: Our No. 1 goal is doing it on our own. We’re not looking to get acquired, we’re not looking to flip the company or anything. I mean, partnering with someone? Sure, we’re totally open to discussions. We’re not, like, we want to do this with no help.
But you wouldn’t want to be absorbed into a bigger company that’s doing more than just VR.
Mitchell: The goal has been to build great consumer VR, specifically for gaming. We all believe VR is going to be one of the most important technologies of –
Luckey: — ever!
Not to be too hyperbolic or anything.
Luckey: It’s hard not to be. It’s like every other technological advance could practically be moot if you could do all of it in the virtual world. Why would you even need to advance those things in the real world?
Mitchell: Sooo …
Mitchell: With that in mind, we have to figure out how we get there. But right now, we’re doing it on our own.
Luckey: And we think we can deliver a good consumer VR experience without having to partner with anyone. We’re open to partnering, but we don’t think we have to. We’re not banking on it.
And how does being based in southern California compare to being closer to a more conventional tech hub like Silicon Valley?
Mitchell: Recruiting is a little harder for us. But overall, we’ve been able to attract incredible talent.
Luckey: And if you’re in Silicon Valley, it’s probably one of the easiest places to start a company in terms of hiring people. But VR is such a tiny field, it’s not like all of a sudden we’re going to go to Silicon Valley and there’s, like, thousands of VR experts. Now, if I’m a Web company or a mobile company –
Mitchell: — that’s where I’d want to be.
Luckey: But in this case, these people aren’t necessarily all up in Silicon Valley. We’ve hired a bunch of people from Texas and Virginia and all these other places. It’s a niche industry. We actually have the biggest concentration of people working in consumer VR right now. And a lot of the top talent we get, they don’t care where we are, as long as it’s not, like, Alaska. They just really want to work on virtual reality, and there’s no one else doing it like we are.
Oculus VR’s Palmer Luckey, left, and Nate Mitchell, right. At center, AllThingsD’s Lauren Goode tries out the Oculus Rift at CES 2013.
There were plenty of great onstage interviews at D11 last week, but — as attendees doubtless know — the conversations that happen offstage are often just as engaging. Such was the case on the last day of the conference, when Oculus VR co-founders Palmer Luckey and Nate Mitchell drove up from their office in Irvine, Calif., for lunch and an hour-long chat.
Oculus is a 30-person startup focused on just one thing: Virtual-reality videogames, by way of a wearable headset that plugs into gamers’ PCs. Its much-anticipated VR headset, the Oculus Rift, was funded on Kickstarter last year to the tune of $2.4 million, and an early version is now in the hands of thousands of game developers. A consumer version is on the way — though the company has yet to announce a release date.
At conferences like this year’s GDC, Luckey has publicly acknowledged that the first versions of the headset won’t be perfect, because developers are still learning what game mechanics work (or don’t) in VR. In this wide-ranging Q&A, Luckey and Mitchell told AllThingsD about that learning process, the Rift’s limitations, its ballpark price point, what they want from developers, messing with coworkers wearing the Rift, and how it stacks up to other next-gen technology like the Xbox Kinect and Google Glass.
For easier reading, we’ve split the chat into two parts. Part One is below. And here’s Part Two.
(Before we begin, a sad note: This interview took place last Thursday, one day before Oculus VR’s lead engineer and fellow co-founder Andrew Reisse was struck and killed as a bystander during a high-speed car chase. The company memorialized Reisse as a “brilliant computer graphics engineer” on its blog on Saturday).
Asa Mathat / D: All Things Digital
AllThingsD: How do you control people’s high expectations for the Rift? At GDC, Palmer called virtual reality the “holy grail of gaming,” but was quick to clarify that the first version you release won’t completely fulfill that promise.
Palmer Luckey: The developer kit, especially, but yeah, even the first consumer Rift has a long way to go. People who research it tend to have good expectations, but there’s two other sets: You have people who think that VR tech is already super-advanced, that it’s like “The Matrix” already, and that we just happen to be cheaper. And then you have people who think that it’s completely broken and hopeless. The best way is to get them to look inside of a Rift, and usually they’re like, “Oh, I get it. It’s not the Matrix, but it’s also not terribly broken.”
Who’s the audience for the Rift? Who’s going to really appreciate it?
Luckey: I don’t think it’s just hardcore gamers. At GDC, Valve talked about how players who were very skilled at Team Fortress 2 felt like the Rift lowered their skill level. I play a ton of TF2: You’re jumping off things and spinning around and then instantly snapping back, constantly whipping back and forth as you walk along. But what they found with people who didn’t play games as much, who weren’t TF2 players — they reported that it increased their perceived skills. I think the Rift can open up the possibility, for a lot of games that have been “hardcore games,” for normal people to play them. They have the right muscle memory built up. Every day, they look around and they move their head to look around. It’s not a huge leap to do that inside of a video game when you have the proper tools.
Nate Mitchell: It also totally depends upon the content. We’ve already seen some people do Minecraft mods (unofficial modifications to the original game to support the Oculus Rift). We have the families in the office, they bring in their kids, and you’ve got 10 kids playing Minecraft in our conference room on the Rift, on the same server. That shows you that there is this huge audience of all sorts of people.
Luckey: In fact, we’ve done that some in the office, too. [laughs] It’s not just for the kids.
Mitchell: Right now, the audience is game developers, and the content is super-key to the whole user experience. Having content that appeals to those types of people, that’s what we want.
Do you need a killer app?
Mitchell: Definitely. We could use a couple killer apps. Ideally, we’d have a game for the niche market. You’d have Call of Duty 9 over here, and something like Minecraft over here, and a wide swath of games in between.
But what about a killer app that’s exclusively for the Rift? A lot of Wii owners only played Wii Sports. Do you need something like that to distinguish the game play?
Mitchell: I won’t say that we need it, but I will say that we want it. That’s something we are trying to figure out. Is it something someone else is going to develop? We’ve discussed — does it make sense to do something ourselves internally? We’re not sure yet. Right now, the focus has been, “Let’s build the tools, and help the developers get there.”
Luckey: It doesn’t make sense for our first focus to be to hire a bunch of game developers to sit and try and figure out what works best in VR, when there’s literally thousands of other people that are willing to figure it out for themselves. They want the privilege of being the first to work in this space.
Luckey: Right now, it’s just for PC games, because that’s the open platform. Mobile support’s also possible, but that’s just more of a technical problem — phones are not powerful enough to provide a good VR experience right now. There’s no technical reason that the Rift can’t work on consoles. It has standard input/outputs, it wouldn’t be a lot of work. It’s just a matter of console manufacturers deciding to license it as a peripheral. They’re the gatekeepers.
Have you talked to them about that?
Luckey: We can’t say.
Mitchell: I think when you look at this upcoming console generation, we are this black sheep, doing something completely different, but we like that. We’re aiming for what we consider to be next-generation gaming. Xbox and PlayStation, they’re doing awesome stuff. And we’re big fans. That said, the Rift is going to be something entirely different.
Luckey: And we’re focusing specifically on gaming. We’re not trying to make a multi-platform home media hub for the family.
How much is the consumer version of the Rift going to cost?
Luckey: The current developer kits are $300. We don’t know what the consumer version’s going to cost — it could be more, could be less. But we’re looking to stay in that same ballpark. We’re not going to be charging $800 or something. We have to be affordable. If you’re not affordable, you may as well not exist for a huge segment of the market.
I guess you would know, since you have the world’s largest private collection of VR headsets.
Luckey: I’m one of the few people where it’s different. I would spend whatever it was. Gamers are not known to be the most affluent population of people. If something’s even $600, it doesn’t matter how good it is, how great of an experience it is — if they just can’t afford it, then it really might as well not exist. We’re going for the mainstream, but time will tell what the market is.
Mitchell: A big part of it’s going to be the content. If it’s only Call of Duty 9, it’s only going to be the niche hardcore gamers. If we can get other stuff on there, which I think we’re already making exciting progress on, I think it’s going to be a lot broader. The three tenets for us are immersion, wearability and affordability. If we can nail those three things, that’s the killer combination that makes it a consumer VR device.
Luckey: The other thing is, it’s possible to make better hardware if you sell it at that lower price point. When you can sell thousands of something, or tens or hundreds or millions of something, you can afford to put better components into it than if you were only making a hundred of these things for $10,000 each. There are people who’ve said, “You should sell a version with better specs for $1,000,” but it’d be better to sell it for $200 and sell more of them.
What are the limitations of the Rift right now, beyond needing to be wired into a PC?
Mitchell: We don’t have positional tracking right now.
Luckey: [That means] you can’t track movement through space, you can only track rotation.
Mitchell: That’s a big one, something we’d love to solve for the consumer version. The only other “limitation,” I’d say right now — well, there’s things we want to improve, like weight. The more comfortable it is, the more immersive it is. So, there’s that. There’s resolution. We want to bring the resolution up for the consumer version.
And, for the foreseeable future, will players still need to use a handheld console-like controller?
Luckey: We don’t know yet.
Mitchell: Human-computer interaction and user input, especially for VR, is something that we’re constantly researching and evaluating.
Luckey: The reason we’re using gamepads (now) is that everyone knows how to use it, so we don’t need to teach a new [control] device while we’re demoing. But we do know that a keyboard, mouse or gamepad isn’t the best possible VR gaming interface.
Mitchell: It’s another abstraction. We’d love to — well, we’re exploring the possibilities.
Luckey: [waving hand] Use your imagination. [he and Mitchell both laugh]
Mitchell: Microsoft, with the new Kinect, is doing some really interesting stuff. Leap Motion is doing incredible stuff. This tech is out there. It’s a matter of packaging it just right for virtual reality, so that we’re putting players totally inside the game. We always joke, you want to look down in the game and go, ‘Yes, I’m Batman!’ And then you pull out your lightsaber or whatever it is — I know, I’m destroying canon here –
Luckey: — I, I’ll just leave that.
Luckey: One of the things I talked about at GDC is that other game consoles, it’s very abstract. You’re controlling something on a screen, using a controller that’s nothing like how you interact in real life. If you hand a person who doesn’t game a 360 controller, it’s like, “Here’s a 16-button, dual analog controller. Use it!” It’s very difficult for someone to pick it up.
And that was the brilliance of the Wiimote, right? If you want to bowl, here’s the controller, just move it like you’re bowling.
Luckey: Even then, it was an abstraction. But it’s clear you want a control interface so that people feel they’re inside the game. It’s clear that you want to take it to the level where they’re not just looking around in the game, but they’re interacting in the same way that they would interact with real life. On Kinect, no matter how great the tracking is, you’re still controlling something on a screen. You don’t feel like you’re inside of the game if you’re looking at a screen in your living room. It’s never going to feel good until you can feel like you’re actually that person.
In Part Two of this Q&A, Luckey and Mitchell discuss Google Glass, motion sickness, messing with coworkers, and their long-term plans for the company.
Three months ago, celebrated video game publisher Valve did something completely out of character: it fired up to 25 workers, in what one employee dubbed the "great cleansing." At the time, co-founder Gabe Newell quickly reassured gamers that the company wouldn't be canceling any projects, but it just so happens that one project managed to get away.
Valve was secretly working on a pair of augmented reality glasses... and those glasses are still being built by two Valve employees who lost their jobs that day.
"This is what I'm going to build come hell or high water."
It’s hard to pin down where John Francis Peters might be at any given time. Upstate New York, China, Mexico… and that was just last year. “Travel has been a big part of my life since childhood and engrained in my experience as a photographer,” recalls Peters. “Part of my focus on photography as a [...]
All the bits and pieces that go into a pair of virtual reality goggles.
iFixit posted a teardown of the Oculus Rift headset Wednesday to see what, exactly, the virtual reality headset is made of. The teardown reveals the types of screens and controllers the Oculus Rift uses, and though the score is preliminary, iFixit gave it a 9 out of 10 user repairability score—unusual in the glue, tape, and Torx screw times we now live in.
The Oculus Rift uses one 1280×800 LCD that is split down the middle to show one image each to the right and left eye to create a 3D image. The display is an Innolux HJ070IA-02D 7-inch LCD panel, provided by the same distributor rumored to be Apple’s source for replacement iPad mini screens. A custom-designed Oculus Tracker V2 board pings to track the headset's motion at a 1000Hz refresh rate.
The chips inside the device include an STMicroelectronics 32F103C8 Cortex-M3 microcontroller with a 72MHz CPU and an Invensense MPU-6000 six-axis motion tracking controller that has both a gyroscope and accelerometer. There is also a chip named A983 2206, which iFixit suspects is a “three-axis magnetometer, used in conjunction with the accelerometer to correct for gyroscope drift.”
The SXSW Gaming Expo is preposterously loud. At one side of the room, a Starcraft tournament is reaching its climax, but on the other side, one group of guys is yelling louder. They sound like a basement full of adolescents discussing the newest Electronic Gaming Monthly cover story, or like the NINTENDO SIXTY-FOUR kid unwrapping his Christmas present.
“Is the Oculus Rift, a virtual reality 3D headset, the future of gaming?” they ask. “Or, is it something bigger — the future of life on planet Earth?”
At a panel entitled “Virtual Reality: The Holy Grail of Gaming,” Gears of War designer Cliff Bleszinski joined Wing Commander creator Chris Roberts, Words with Friends co-creator Paul Bettner, and Oculus founder Palmer Luckey...