I think Sean Liu is a great ambassador for oculus. He not only knows his stuff but he's clearly enthusiastic about it all. I can see why Norm was so excited to meet him again. All this stuff is getting me so hyped for oculus
So glad he was openly speaking about pretty much all topics they threw at him. This gave a lot of insights about Oculus' considerations during past and future development. I just hope he won't get flak internally about revealing too much. As always great job, Norm and Jeremy!
Nothing controversial in there, and all very well pitched at a semi-specialist audience. I've just done my masters in VR and that gives me a lot of confidence in developing for the platform.
If somebody told me 5 years ago, we would have a standalone headset with hand tracking (albeit limited) and able to connect it to a PC I'd have laughed at them.
They really need a "learn sign language!" thing. Or even maybe a game designed for deaf people that recognizes sign language as a possible input to talk to npc's... or even a sign language to speech/text for community type things.
I love how VR has been properly bought into and is going places. I know it's not the best headset out there but i love my PSVR and cannot wait for the next version.
Sean is a superstar IMHO. answering every question with a great balance of company line and just enough of a tease. as usual, best VR content on the internet guys!
Grand Reality VIP = Technically: The leap motion has two 100% overlapping uv cameras (the quest cameras are in std color space). Back to action space: So the quest has three areas (two one camera only sections left and right, and one in the center where both cameras overlap). That limits the action space for the quest, even though they pull a lot of tricks out of the bag to meditate this. The biggest difference is the update speed: the leap motion runs much much faster and has „supposedly“ years of refining a mathematical model they stumbled over. What the quest does, is closer to deep learning. From a professionals points of view, I feel though that „normal people“ i.e. in business would not feel the difference
@@dinoscheidt I'd say the quest has technically 9 areas, but more likely 5. There are 4 cameras, one on each corner, which gives a space in the centre with all 4 overlapping, then 4 spaces where there is 100% overlap with two cameras either side and above and below. The other 4 to make up the technical 9 are the four diagonals which are only visible to one camera, but there would be absolutely no depth information there as the cameras are visible light only, now if they had used connect style sensors with time of flight instead those spaces could also be providing usable information, but the processing requirements would probably be higher which isn't good for a battery operated headset.
I've been sitting on the couch watching shows and wishing I had the single remote from the rift to interact with home, just to start a movie or something. Even if finger tracking isn't the end of controllers there are sooo many instances of quick UI interactions where this will be great.
I keep thinking about a game that uses one touch controller and hand tracking on whatever you choose to be your "dominant" hand. Seems like it would be comfortable for adventure games especially. Edit: nevermind, addressed in the video. not yet possible.
@@MrKlawUK yup. that would be fantastic. physical irl joystick for control, hand tracking for switches etc. same could go for racing games. wheel, gear shift, pedals etc and hand tracking for windscreen wipers, indicators, flicking switches in race cars for things such as abs, traction control, that formula one overtake, boost button. headlights, dip etc. or even playing games like gta, hold a virtual gun in one hand whilst holding a real wheel in the other, or a virtual phone, etc. with a game like gta, you could even just have the normal controller tracked as a gun etc. put it by your side while driving with a real wheel, look left and pick up off couch/ passenger seat and shoot while drving. etc. but then walking around.. i dont like walking in vr while sitting irl. so you would have to push your wheel stand out of the way while in vr when you get out of the car, lol. that would not work. especially not in my small space.
Oh, you fool. It is YOU who does not undersand a simple quote. Mixed reality means, the reality you are in is mixing with virtual reality. Your body and ass is in real world, but you have that thing on your face and the reality is mixed. 2 in 1. Where is your brain? On holiday?
@@SegginsProductions After oculus connect day 1 keynote i went online and ordered same day delivery on my quest. yeah its amazing. upgraded from a cv1 also
yea i got the rift because i wanted pc connectivity...upset doesn't begin to describe it for me and the rest of the people that lost on this little update. This is so poor on their part instead of waiting and coming out with a complete product.
@@mwhatnessasdf7054 I'd say it was actually quite a good move. If the Quest hadn't been the success it was, they could have redirected all the folk working on it to PCVR instead. That it was a phenomenal success without using all its hardware potential proved it was worth further investment.
I think one of the coolest things about this tech is that once controllers are no longer needed developers can start selling peripherals some imagine playing beat saber holding actual lightsabers.
that you cant even see... if its already tracking your hands, you could just hold a bland tube.. but then you might as well just use motion controllers for perfect tracking.. also.. i really really really dont understand why everyone loves beatsaber soo. much.. i enjoyed it vaguely for an hour. have tried it a few times since and find it immensely lame and boring. whereas i can play onward for hours. soo much fun. or even better dirt rally or project cars 2/assetto corsa. sim racing in vr with a wheel etc is literally as good as gaming gets. absolutely fantastic. but each to their own i guess..
Hand tracking sounds like a fantastic way to keep using the Quest to watch movies/browse the internet while you're in bed or on a plane or long car journey without having to hold a bulky controller in your hand, however I would assume unfortunately the same limitations that might interfere with being able to track your head movements whilst on a dimly lit airplane (for example) will probably also make hand tracking impossible in those situations. It's a small niggle, but I would love a way to interact with my Quest (at a very basic level) without having a controller in my hand, for example, to be able to launch Netflix and watch a movie on a virtual screen without the controller. Might request this feature on the Oculus feedback site...
Passive haptics with the arm is a great idea. They and Valve both should have thrown in extra cameras for future proofing. Oculus is bad at marketing, but they need to start using that mixed reality with both streamers and talk shows in order to show off VR.
The touch feature is thrilling to me thinking of how it can work in flight simulators. Being able to have a hotas setup, then reach out in a virtual flight deck to flip switch and turn knobs would be awesome.
Great interview and great episode. Sean Liu looks friendly and super excited about his (and his collegues) work. It's a contagious in a positive way! Now I'm a bit more excited too ^^
Harmonix needs to get on making a orchestral conductor sim/game with hand tracking. Each input would correspond to a section in the orchestra and could dynamically respond to hand gestures made by conductor. Oh man that would be amazing educational tool and could be crazy fun too.
Thanks for the great coverage guys. These recent announcements made me finally choose my first headset. After following the developments of VR tech over the years and waiting I went out and bought myself a Quest today. I'm not even that interested in gaming but mostly into 3D modelling/sculpting. I thought I'll have to grab a Rift S since Oculus Medium was not supported by Quest but if that link option is coming, surely Medium will be supported as well. And for these kinds of applications having a lower refresh rate isn't that much of a problem either.
the panel at oc6 on hand tracking was very interesting. I recomend checking it out if your a dev interested in building an app for oculus hand tracking!
I love new tech....but.....im not looking too much into hand tracking just yet. I remember the hype around the microsoft tech for kinect monitoring your whole body and showing it all on advertisements as perfect movement. We know where that all ended up.😶. I dont like being negative and pray it all works waaay better and....yes....its years ago since then. I still tread with caution though. 😉 Regardless, im more than happy with my vr and the physical controllers.
questions on hand tracking if you know or can find out. 1. how would they handle tracking if the player is partly or wholy missing fingers? will they remove the tracking points or just guess? 2. what will happen if while playing someone else walks up to you and inserts their hands in the playing area or even grabs the player?
alright i dont know why i was so worried people seem to get that for a lot of things hand tracking is worse. Haptics is the big frontier in vr, you can only do so much presence with visuals and audio, you need more feeling. You all are asking the right questions.
Haptics is a frontier like space. That is, we know it's there but we're barely fucking with it. Setting aside that the technology hasn't moved an inch in like a decade, haptic peripherals are associated with dreaded "friction."
Honestly, couldn't care less about hand tracking for gaming. For other stuff ok but for gaming ? Gloves or controllers like the Index ones/Touch controllers are the way to go !
@@butcherjsy8 And I just bought Vader Immortal Ep2. In that game, bracelet would be irreplaceable for high speed movement like when you force-grab object and throw them against the wall. Imagine that. And holding a lightsaber in the other hand. Damn...
I disagree with the dude on the left at 6:24. about the hand tracking not feeling right to developers. Microsoft already has hand tracking that detects hand gesture and spatial mapping touch interaction with their HoloLens and HoloLens 2 and developers already know how it feels. They even have virtual buttons you can interact with and holograms you can resize with your hands in the HoloLens. I gotta to be honest, no has ever complained about the using hand gesture or hand tracking with the HoloLens. In fact, its actually quite much like the scene from Minority Report where they use their hands to rewind memory footage. Putting hand tracking into the quest isn't going to have a disconnect with the immersion; its going to improve it. I feel like he's taking it for granted that the people are going hate it, when the truth is that gamers and even New Media Artist are going to find this really innovative considering that you get the similar technology in the Quest at 400 vs the HoloLens at 2000 or even HoloLens 2 at 10000. Honestly the only difference between the HoloLens and Quest right now is that ones VR and the other is Mixed Reality, aside from the tiny view screen on the HoloLens. Lets be real, I can put the same 3D model on both devices, accept now it will probably be easier the do now, since the quest is cheaper and easier to develop with unity dev kit than Microsoft is.
Interesting that Sean Liu seems to be a copy of Nate Mitchell. Both their personalities and expressions are similar. Sean also took over the same position Nate used to run: Product Management at Oculus.
I've been trying to think of how developers could make games with this and have the ability to walk and I finally figured out how they could do it. I first was thinking of some finger movements and poses and all but then I thought of how annoying it could get after a while of doing that. Then I thought of maybe moving your hands in front of you but that may start to hurt your upper arms after some time of doing that. Then I finally thought of something. Why not just swing your arms by your side forward and back like walking real life. Now, this could be used for future games that use this. Maybe in Vader immortal, you could have hand tracking or in Pavlov (soon to come to the quest). The possibilities are endless
WOW. Imagine when passive feedback can mean I bring in my own objects into VR. For example. For a shooter, I can bring in an airsoft gun (with CO2 cartridges for recoil). For a sword fight, I can bring in a nerf sword. That would be really game changing to bring objects into VR.
Well dry firing a airsoft gun isnt a smart move, unless its a cheap one. If you dry fire it its like beating itself to death. Oh yea if you are gonna use a gas airsoft gun (co2 or green gas) make sure you open the windows and make sure you dont breathe the gas in.
TL;DR Playlist for all VR related videos? I love the channel and LOVE Projections content!.. Unless I'm missing it, I can not find one place to view all the Tested VR videos? Searching for projections doesn't show all of them? Can we get a playlist or something of all the tested VR videos? Currently its hard to find them all. Since you have been doing this since oculus dev kit 1 ??? THANK YOU
To give haptics sensations with hands tracking, you can create phantom touch sensations using sounds and visual cues. Leap Motion is experimenting on this since years. It is not as real haptics, but it is a nice to have
I think Oculus needs to develop a glove. It would solve so many problems! In fact, I wouldn't be surprised if they've already considered it for the future.
They could use a elastic (like the elastics use on the vr head sets) "rings" with sensors on the tip of your fingers to make it easier and more accurate to track and for adding the haptic feedback on them in the future instead of a glove.
Maybe this defeats the purpose of not needing any sort of extra peripherals, but couldn't they put some of the technology that allows the controllers to be tracked, even when not directly seen by the cameras, into gloves. With gloves the whole skeletal mapping thing could still be done so that the immersion is there. But with added gyros, accelerometers, and normal IR stuff the hands could be accurately tracked even when the cameras can't directly see the hands. It seems like it would be an easy optional thing for people who want really accurate hand tracking with minimal occlusions.
Just a question--is there a reason they havent put as much of the electronics as possible into a separate device that you wear to make the headset lighter and more compact?
Bass Guitarist That what they should do right now while the tech is not there yet. Imagine a mibile GTX 1060 or higher can do. the headset gets lighter and compact maybe even just a see through lcd oled that auto dims. God, I wish I $$$money to develop one lol
@@JorgTheElder thanks Erik for the explanation, I appreciate it. Wouldn't it be the same as how they wire some vr sets to a pc? or do those headsets have components in them as well? I dont own VR gear so I'm not too sure. I wish I had the $$$, might get the quest, its very reasonable price.
@@kennl66 that sounds really cool Kennl, i have a day dream thats all i can afford right now and it got me hooked. I cant imagine how it would be to own a good vr system
@@bassguitarist2686 "Wouldn't it be the same as how they wire some vr sets to a pc?" Pretty much, yes. But a fair bit of processing power in wired VR is dedicated to predicting movement to cheat their way round latency, and the Quest is already pushing the hardware to the limit. Performance would be impacted pretty heavily.
@@nialltracey2599 Thanks Niall, yes it would be like wiring to pc but I'm mostly thinking of the bulk of a headset on my face, I get that snowboard sking mark around my eyes after a while. Not sure if the quest has cameras on outside to do AR, but i dont see much headsets like that. I bought one for like $15 where you put phone in and it has a door to open to expose phone camera so headset is on and AR can happen, but I dont see too many quality headsets like from Oculus doing this.
But how does it compare to Leap Motion? Because it's demos were also very interesting, and a VR plugin for Flight Simulator X was amazing. Is it any better, or it just made it more common, which means there will actually be apps for it?
This would be epic to use with Bigscreen. Just put the controllers down and use hands to pause the movie or quickly do stuff. I always have to take time to try and feel where I put my controllers when using bigscreen, would be nice to be able to use your hands.
i think hand controls would be great for games like skyrim something like you snap your fingers and the mic picks it up in game and it opens the menu and then you point/touch where you want to go in the menu and you could have one finger for equipet and one finger for unequipped in the menu and for sword and shield grab sticks from the yard for that realism factor though you won't need them and magic hold fingers in and close to making a fist and for releasing open up the hand and for talking to npc's just walk up to them and say hello to start them talking and pick their options with your hands and for picking up items you would have to have something like a hand jester if the hand tracking was perfect and you could overlay the real world bow and sticks and give them skins and you had a omnidirectional treadmill i think you would not want to go back to controls in games like the keyboard and mouse or gamepad mostly because the less you have on you that feels like it don't belong in that world the more immersive it is
When I first heard 5m USB-C Cable for $80 I thought that's a bit steep. But them using fiber optic transceivers on both ends of the cable to get to that length puts that price into perspective.
The newest kinect can use many kinect sensors together for full 360. If Microsoft does VR they could bring back kinect using those and have an amazing and upgradable system. Cost is too high for those new units out of mass production now.
guys remember. no buttons means high reliance on finger tracking.... there must be input, if you make an ok symbol or a thumbs up, it may mean one thing to the pc and another to geitrolls opoint is, if the occlusion causes loss of signal.. that also equates to lose of all button simulation.... if you want to drive with a streeting wheel and shift
Same here. I was hoping throughout the interview that this would be asked at some point. I suspect that they'll avoid doing so initially, as they don't want to split the Oculus PC dev chain, but that some of the larger players using VR for training will talk them into implementing it. At first it'll likely be a special build of the OS, but after an update or two, it'll no longer be worth Oculus's time maintaining two code branches and they'll mainstream it.
Have have a thought, wouldn't not be possible to use the Rift Cv1 sensors. If the sensors are the same tech on the Rift S as what they are on the Cv1 so maybe they could add external sensor hand tracking with out any blind spots. They must of thought about this
Hand tracking should be a supplement to controllers. It’s makes using touch controller, gun stocks, joysticks, steering wheels better. I don’t think it will shine by itself
It needs to be an combination, because sometimes it is hand movement only, sometimes an object e.g. gun. One hand gun other hand just your hand. I think oculus (facebook) also works on haptic wristbands or something like that.
I want to be able to switch from hand tracking to controller. Then you can use your hands for open word and then you can pull your controller out from your holster when it’s time to shoot everything up. Also need holsters lol.
What if in the future you could put on a glove that will push your hand out with the same amount of force you use to grab or hold something, making it actually feel like your holding something.
I'm a bit confused. It sounds like you'll need a native USB-C port on your gaming PC to enable the Quest to run games and charge at the same time. Am I wrong with that? My PC works fine with Virtual Desktop/SteamVR, but it doesn't have a USB-C port. I'm not very techie, does anyone know if (in theory) an A-to-C adapter will fully work with the link cable?
Chris Stacey Adapters could work. Technically you only need USB 2 speed, but the achieved speed depends a lot on cable length, and an adapter will make it worse. Ideally you'd use a high quality USB-A to USB-C cable. These aren't very common at lengths on 3m/10ft or more, and usually rather thick. These problems are the main reason why Facebook will sell a dedicated, very thin and light USD 79 USB-C fiber optic cable. So your PC will work, but you may be restricted regarding cable length.
@@faduci Thanks, that makes sense. I've already been perusing Amazon and it looks like there are a few 3.0 A-to-C cables of 4.5 and 5 meters (gotta love metric ;-) ) but wasn't sure if they could keep the Quest charged whilst also delivering the encoded data. And they're probably quite thick/less flexible than the link cable will be.
I think Sean Liu is a great ambassador for oculus. He not only knows his stuff but he's clearly enthusiastic about it all. I can see why Norm was so excited to meet him again. All this stuff is getting me so hyped for oculus
So glad he was openly speaking about pretty much all topics they threw at him. This gave a lot of insights about Oculus' considerations during past and future development. I just hope he won't get flak internally about revealing too much. As always great job, Norm and Jeremy!
It's all out in the open now. I'd like to know what legal agreement Oculus made with Lenovo when they outsourced the Rift S!
Nothing controversial in there, and all very well pitched at a semi-specialist audience. I've just done my masters in VR and that gives me a lot of confidence in developing for the platform.
10:03 yes your hands can run out of battery
it's called starving
Then your hands aren't running out of battery, They're running out of food i guess lol
@ร๏гг๏ฬ คςє๔ Still more importantly, oxygen ;).
big brain
Relict 805 yes it would make the least of your problems because you wouldn’t be able to take the headset off lmfao
Are 80's Ethiopian jokes still a thing?
Asking for a friend.
Man the interview at the end was golden ! He was so opened to answering all the questions
If somebody told me 5 years ago, we would have a standalone headset with hand tracking (albeit limited) and able to connect it to a PC I'd have laughed at them.
There is still much time for the tracking to be improved, so expect it to be much, much better than iits current state.
I'd have laughed at them like they asked for starwars holograms without glasses and screen.
@@minamoog as long as it's good for media apps I don't mind as you need a controller for serious games anyway
Eventually people will be like "what? You gotta use your eyes in a headset instead of hooking it to your brain?"
Me too!
3:01 the girl be like: "Yeah i am on cam...oh sorry :p"
That was Ana Ribeiro, creator of Pixel Ripped ;) She most likely knew who Jeremy and Norm were and did that on purpose!
Having fun too! Running with a beer in her hand 3:05 lol
She was the highlight of the video! Such an epic run
Ana is awesome and we love Pixel Ripped!
VR POWER HOUR! It's my favorite VR protagonist DOT! 🔥
They really need a "learn sign language!" thing. Or even maybe a game designed for deaf people that recognizes sign language as a possible input to talk to npc's... or even a sign language to speech/text for community type things.
Or even like a game that teaches u sign
Nathan Glass That’s what he said.
they kinda did it i think
I love how VR has been properly bought into and is going places. I know it's not the best headset out there but i love my PSVR and cannot wait for the next version.
NbAlIvEr100 the headset already has 360 degree tracking
@@JCMaldonado13 He means for everything. The controllers don't have 360 tracking.
Awesome! Hopefully Sony incorporates inside-out tracking for their next headset along with better controllers!
@@JCMaldonado13 Sony has a track record of abandoning good products, so don't get your hopes up.
They just need better hand controllers. A console doesn't need to be the best, just be good enough and cost effective.
That guy’s shirt says “fly me to the moon and let me play among the stars” and has silhouettes of every major starship of sci-fi history...dopeness
@Colin Leary Here you go - www.lastexittonowhere.com/catalogue/fly-me-to-the-moon-regular-black-t-shirt_121/
I was more amused by Norm's "introverted but willing to discuss VR" t-shirt!
@Colin Leary ......And may you have pleasure wearing it!
Chris Stacey that t shirt is a custom 1 of 1 for sure
And..... are you ready for this?.... it glows in the dark.
Sean is a superstar IMHO. answering every question with a great balance of company line and just enough of a tease. as usual, best VR content on the internet guys!
The big question I've not heard answered yet: how does Quest hand-tracking compare to Leap Motion?
Grand Reality VIP = Technically: The leap motion has two 100% overlapping uv cameras (the quest cameras are in std color space). Back to action space: So the quest has three areas (two one camera only sections left and right, and one in the center where both cameras overlap). That limits the action space for the quest, even though they pull a lot of tricks out of the bag to meditate this. The biggest difference is the update speed: the leap motion runs much much faster and has „supposedly“ years of refining a mathematical model they stumbled over. What the quest does, is closer to deep learning. From a professionals points of view, I feel though that „normal people“ i.e. in business would not feel the difference
@@dinoscheidt I'd say the quest has technically 9 areas, but more likely 5. There are 4 cameras, one on each corner, which gives a space in the centre with all 4 overlapping, then 4 spaces where there is 100% overlap with two cameras either side and above and below. The other 4 to make up the technical 9 are the four diagonals which are only visible to one camera, but there would be absolutely no depth information there as the cameras are visible light only, now if they had used connect style sensors with time of flight instead those spaces could also be providing usable information, but the processing requirements would probably be higher which isn't good for a battery operated headset.
Well I guess you could definitely say, this is a HANDS ON EXPERIENCE, HHAHAHAHAHAHAHAHAHA, KAREN WHY DID YOU LEAVE MEEEEEEEEEEEEE?!?!?!
@Rick Indergard because of that link, I'm now giving myself a hands on experience.
Because she was seeing the manager.
She didn't get enough hands on experiences.
LMAO
@@trejkaz Give this man a Medal!!!!! NOW!!!
Vive Index: I have the best controllers that track fingers.
Oculus Quest: Allow me to introduce myself.
Index: where’d you go? ... What’s occlusion?
I've been sitting on the couch watching shows and wishing I had the single remote from the rift to interact with home, just to start a movie or something. Even if finger tracking isn't the end of controllers there are sooo many instances of quick UI interactions where this will be great.
I keep thinking about a game that uses one touch controller and hand tracking on whatever you choose to be your "dominant" hand. Seems like it would be comfortable for adventure games especially.
Edit: nevermind, addressed in the video. not yet possible.
Looking forward to cockpit control for DCS World.
Perfect use case
Matt Wagner Absolutely! VR and hand tracking - the (hopefully near) future of all flight/motor simulation.
HOTAS + hand for ancillaries - sounds great
@@MrKlawUK yup. that would be fantastic. physical irl joystick for control, hand tracking for switches etc. same could go for racing games. wheel, gear shift, pedals etc and hand tracking for windscreen wipers, indicators, flicking switches in race cars for things such as abs, traction control, that formula one overtake, boost button. headlights, dip etc. or even playing games like gta, hold a virtual gun in one hand whilst holding a real wheel in the other, or a virtual phone, etc.
with a game like gta, you could even just have the normal controller tracked as a gun etc. put it by your side while driving with a real wheel, look left and pick up off couch/ passenger seat and shoot while drving. etc.
but then walking around.. i dont like walking in vr while sitting irl. so you would have to push your wheel stand out of the way while in vr when you get out of the car, lol. that would not work. especially not in my small space.
12:09 Microsoft, take note. This is how you do Mixed Reality!
Oh, you fool. It is YOU who does not undersand a simple quote. Mixed reality means, the reality you are in is mixing with virtual reality. Your body and ass is in real world, but you have that thing on your face and the reality is mixed. 2 in 1. Where is your brain? On holiday?
@@user-cw3nb8rc9e are you drunk?
Microsoft already has this for their HoloLens since last year.
@@user-cw3nb8rc9e I think he was comparing the holo lens to the tablet instead of talking about a quote.
As a Oculus Quest owner...
I adore it.
Daniel Sambar this and link might just get me to upgrade from my cv1!
@@SegginsProductions After oculus connect day 1 keynote i went online and ordered same day delivery on my quest. yeah its amazing. upgraded from a cv1 also
Swinny mines on the way for Friday!
Quest owners: Ooouuh hand tracking...
Quest Devs: It seems the data we've collected has been up and down gestures.... hmmmm.
Just sayin'
Awesome news. The development around the Quest is going to be amazing.So glad I went with the Quest over the Rift S.
yea i got the rift because i wanted pc connectivity...upset doesn't begin to describe it for me and the rest of the people that lost on this little update. This is so poor on their part instead of waiting and coming out with a complete product.
@@mwhatnessasdf7054 I'd say it was actually quite a good move.
If the Quest hadn't been the success it was, they could have redirected all the folk working on it to PCVR instead.
That it was a phenomenal success without using all its hardware potential proved it was worth further investment.
I think one of the coolest things about this tech is that once controllers are no longer needed developers can start selling peripherals some imagine playing beat saber holding actual lightsabers.
Thing with lightsabers is that I'm pretty sure you only feel the weight of the hilt, so as far as that goes we're good.
that you cant even see... if its already tracking your hands, you could just hold a bland tube.. but then you might as well just use motion controllers for perfect tracking..
also.. i really really really dont understand why everyone loves beatsaber soo. much.. i enjoyed it vaguely for an hour. have tried it a few times since and find it immensely lame and boring. whereas i can play onward for hours. soo much fun. or even better dirt rally or project cars 2/assetto corsa. sim racing in vr with a wheel etc is literally as good as gaming gets. absolutely fantastic.
but each to their own i guess..
Hand tracking sounds like a fantastic way to keep using the Quest to watch movies/browse the internet while you're in bed or on a plane or long car journey without having to hold a bulky controller in your hand, however I would assume unfortunately the same limitations that might interfere with being able to track your head movements whilst on a dimly lit airplane (for example) will probably also make hand tracking impossible in those situations. It's a small niggle, but I would love a way to interact with my Quest (at a very basic level) without having a controller in my hand, for example, to be able to launch Netflix and watch a movie on a virtual screen without the controller. Might request this feature on the Oculus feedback site...
There's no real reason they couldn't have the Go or Gear VR's small controller work with the Quest. They just haven't.
I wonder if some sort of highly trackable IR multi-haptic gloves would be a good next product now that handtracking will take off :)
2 main product lines. Mass market and enthusiast/business. It it can be done for mass market, there is a market for doing it better.
A very User Friendly OC6. And i love it
Great interview,it's so nice to see such enthusiastic and thoughtful responses from Sean.
This is why we need an Oculus Quest 2.0! They can improve it so much from all the feedback they got. I see potential in this Standalone VR.
Passive haptics with the arm is a great idea. They and Valve both should have thrown in extra cameras for future proofing. Oculus is bad at marketing, but they need to start using that mixed reality with both streamers and talk shows in order to show off VR.
this looks so cool! i can't wait to see where this goes in the future!
I'd love to see my hands tracked on a real racing wheel in a racesim and to be able to wave and gesture to other drivers.
berliner middle finger intensifies
Whenever you pass someone, look them in the eye and flip them off.
Your OC coverage is superb!
That bit at 3:01 when Ana Ribeiro video booms you is wonderfull!
The touch feature is thrilling to me thinking of how it can work in flight simulators. Being able to have a hotas setup, then reach out in a virtual flight deck to flip switch and turn knobs would be awesome.
These need to be more mainstream
Uhh its 399 or 499. I think its mainstream
By Christmas I predict there will be a massive VR boom.
@@Jenna_Talia yea and now with pewdipie posting content
@@Jenna_Talia thats getting good ammounts of views
You guys ask the best questions. Thanks for the excellent video!
Great interview and great episode. Sean Liu looks friendly and super excited about his (and his collegues) work. It's a contagious in a positive way! Now I'm a bit more excited too ^^
Harmonix needs to get on making a orchestral conductor sim/game with hand tracking. Each input would correspond to a section in the orchestra and could dynamically respond to hand gestures made by conductor. Oh man that would be amazing educational tool and could be crazy fun too.
3:01 yeet lmao
Thanks for the great coverage guys. These recent announcements made me finally choose my first headset. After following the developments of VR tech over the years and waiting I went out and bought myself a Quest today. I'm not even that interested in gaming but mostly into 3D modelling/sculpting. I thought I'll have to grab a Rift S since Oculus Medium was not supported by Quest but if that link option is coming, surely Medium will be supported as well. And for these kinds of applications having a lower refresh rate isn't that much of a problem either.
that part with the ipad was truly amazing
Uki Malefu mommy!!!
the panel at oc6 on hand tracking was very interesting. I recomend checking it out if your a dev interested in building an app for oculus hand tracking!
I love new tech....but.....im not looking too much into hand tracking just yet. I remember the hype around the microsoft tech for kinect monitoring your whole body and showing it all on advertisements as perfect movement. We know where that all ended up.😶. I dont like being negative and pray it all works waaay better and....yes....its years ago since then. I still tread with caution though. 😉
Regardless, im more than happy with my vr and the physical controllers.
questions on hand tracking if you know or can find out.
1. how would they handle tracking if the player is partly or wholy missing fingers? will they remove the tracking points or just guess?
2. what will happen if while playing someone else walks up to you and inserts their hands in the playing area or even grabs the player?
alright i dont know why i was so worried people seem to get that for a lot of things hand tracking is worse. Haptics is the big frontier in vr, you can only do so much presence with visuals and audio, you need more feeling. You all are asking the right questions.
Haptics is a frontier like space. That is, we know it's there but we're barely fucking with it. Setting aside that the technology hasn't moved an inch in like a decade, haptic peripherals are associated with dreaded "friction."
Nimble VR was an impressive handtracking demo!
Also, I am loving Norm's VR hair
Honestly, couldn't care less about hand tracking for gaming. For other stuff ok but for gaming ? Gloves or controllers like the Index ones/Touch controllers are the way to go !
First thing that came to me was that what if they had haptic wristbands. It could add another dimension to hand tracking if
possible.
Powerglove's smaller sexier relative
Same thought. I'm really used to haptic feedback. But it would be weird to have your wrist vibrating when it's your finger touching the object though.
@@remmahsroht i mean, couldnt they just make a glove?
@@remmahsroht How about haptic rings, or a haptic knuckle duster-esque device?
@@butcherjsy8 And I just bought Vader Immortal Ep2. In that game, bracelet would be irreplaceable for high speed movement like when you force-grab object and throw them against the wall. Imagine that. And holding a lightsaber in the other hand. Damn...
I disagree with the dude on the left at 6:24. about the hand tracking not feeling right to developers. Microsoft already has hand tracking that detects hand gesture and spatial mapping touch interaction with their HoloLens and HoloLens 2 and developers already know how it feels. They even have virtual buttons you can interact with and holograms you can resize with your hands in the HoloLens. I gotta to be honest, no has ever complained about the using hand gesture or hand tracking with the HoloLens. In fact, its actually quite much like the scene from Minority Report where they use their hands to rewind memory footage. Putting hand tracking into the quest isn't going to have a disconnect with the immersion; its going to improve it. I feel like he's taking it for granted that the people are going hate it, when the truth is that gamers and even New Media Artist are going to find this really innovative considering that you get the similar technology in the Quest at 400 vs the HoloLens at 2000 or even HoloLens 2 at 10000. Honestly the only difference between the HoloLens and Quest right now is that ones VR and the other is Mixed Reality, aside from the tiny view screen on the HoloLens. Lets be real, I can put the same 3D model on both devices, accept now it will probably be easier the do now, since the quest is cheaper and easier to develop with unity dev kit than Microsoft is.
Fantastic interview
I swear when AR really becomes a thing we'll be able to have inspector gadget hands
Interesting that Sean Liu seems to be a copy of Nate Mitchell. Both their personalities and expressions are similar. Sean also took over the same position Nate used to run: Product Management at Oculus.
Amasing shirts!
With ARM SOC's getting NVidia AI hardware, the future is looking bright for the Quest line going forward as most of the tech uses machine learning.
I've been trying to think of how developers could make games with this and have the ability to walk and I finally figured out how they could do it. I first was thinking of some finger movements and poses and all but then I thought of how annoying it could get after a while of doing that. Then I thought of maybe moving your hands in front of you but that may start to hurt your upper arms after some time of doing that. Then I finally thought of something. Why not just swing your arms by your side forward and back like walking real life. Now, this could be used for future games that use this. Maybe in Vader immortal, you could have hand tracking or in Pavlov (soon to come to the quest). The possibilities are endless
The most important thing is that this not only makes it better but cheaper depending of it's implementation
WOW. Imagine when passive feedback can mean I bring in my own objects into VR. For example. For a shooter, I can bring in an airsoft gun (with CO2 cartridges for recoil). For a sword fight, I can bring in a nerf sword. That would be really game changing to bring objects into VR.
Well dry firing a airsoft gun isnt a smart move, unless its a cheap one. If you dry fire it its like beating itself to death. Oh yea if you are gonna use a gas airsoft gun (co2 or green gas) make sure you open the windows and make sure you dont breathe the gas in.
TL;DR Playlist for all VR related videos?
I love the channel and LOVE Projections content!.. Unless I'm missing it, I can not find one place to view all the Tested VR videos? Searching for projections doesn't show all of them? Can we get a playlist or something of all the tested VR videos? Currently its hard to find them all. Since you have been doing this since oculus dev kit 1 ??? THANK YOU
To give haptics sensations with hands tracking, you can create phantom touch sensations using sounds and visual cues. Leap Motion is experimenting on this since years. It is not as real haptics, but it is a nice to have
I think Oculus needs to develop a glove. It would solve so many problems! In fact, I wouldn't be surprised if they've already considered it for the future.
Fantastic interview as always. Great job guys :)
IMHO: The coming of passive Haptic feedback is fascinating! \m/ I believe it can come to Quest within 18 months.
One issue is this removes force feedback. Unless there's a glove involved, you won't get rumble or anything.
They could use a elastic (like the elastics use on the vr head sets) "rings" with sensors on the tip of your fingers to make it easier and more accurate to track and for adding the haptic feedback on them in the future instead of a glove.
If they could map a keyboard or buttons that control different things on your arms, that would be really cool to use.
Maybe this defeats the purpose of not needing any sort of extra peripherals, but couldn't they put some of the technology that allows the controllers to be tracked, even when not directly seen by the cameras, into gloves. With gloves the whole skeletal mapping thing could still be done so that the immersion is there. But with added gyros, accelerometers, and normal IR stuff the hands could be accurately tracked even when the cameras can't directly see the hands. It seems like it would be an easy optional thing for people who want really accurate hand tracking with minimal occlusions.
Just a question--is there a reason they havent put as much of the electronics as possible into a separate device that you wear to make the headset lighter and more compact?
Bass Guitarist That what they should do right now while the tech is not there yet. Imagine a mibile GTX 1060 or higher can do. the headset gets lighter and compact maybe even just a see through lcd oled that auto dims. God, I wish I $$$money to develop one lol
@@JorgTheElder thanks Erik for the explanation, I appreciate it. Wouldn't it be the same as how they wire some vr sets to a pc? or do those headsets have components in them as well? I dont own VR gear so I'm not too sure. I wish I had the $$$, might get the quest, its very reasonable price.
@@kennl66 that sounds really cool Kennl, i have a day dream thats all i can afford right now and it got me hooked. I cant imagine how it would be to own a good vr system
@@bassguitarist2686 "Wouldn't it be the same as how they wire some vr sets to a pc?"
Pretty much, yes. But a fair bit of processing power in wired VR is dedicated to predicting movement to cheat their way round latency, and the Quest is already pushing the hardware to the limit. Performance would be impacted pretty heavily.
@@nialltracey2599 Thanks Niall, yes it would be like wiring to pc but I'm mostly thinking of the bulk of a headset on my face, I get that snowboard sking mark around my eyes after a while. Not sure if the quest has cameras on outside to do AR, but i dont see much headsets like that. I bought one for like $15 where you put phone in and it has a door to open to expose phone camera so headset is on and AR can happen, but I dont see too many quality headsets like from Oculus doing this.
But how does it compare to Leap Motion? Because it's demos were also very interesting, and a VR plugin for Flight Simulator X was amazing.
Is it any better, or it just made it more common, which means there will actually be apps for it?
If you did not go to it. It was better or yea.... right now you can do all hands and it is new so it will definitely ge5 be improved
Yay I love hearing this type of news. New tech is fun 👍🏻
This would be epic to use with Bigscreen. Just put the controllers down and use hands to pause the movie or quickly do stuff. I always have to take time to try and feel where I put my controllers when using bigscreen, would be nice to be able to use your hands.
i think hand controls would be great for games like skyrim something like you snap your fingers and the mic picks it up in game and it opens the menu and then you point/touch where you want to go in the menu and you could have one finger for equipet and one finger for unequipped in the menu and for sword and shield grab sticks from the yard for that realism factor though you won't need them and magic hold fingers in and close to making a fist and for releasing open up the hand and for talking to npc's just walk up to them and say hello to start them talking and pick their options with your hands and for picking up items you would have to have something like a hand jester if the hand tracking was perfect and you could overlay the real world bow and sticks and give them skins and you had a omnidirectional treadmill i think you would not want to go back to controls in games like the keyboard and mouse or gamepad mostly because the less you have on you that feels like it don't belong in that world the more immersive it is
Shout out to Ana Ribeiro! ^_^
When I first heard 5m USB-C Cable for $80 I thought that's a bit steep. But them using fiber optic transceivers on both ends of the cable to get to that length puts that price into perspective.
With they UI they could incorporate and build on so many techniques learnt both good and bad.. from Xbox kinect
definitly, kinect plus vr = cheap full body tracking
The newest kinect can use many kinect sensors together for full 360. If Microsoft does VR they could bring back kinect using those and have an amazing and upgradable system.
Cost is too high for those new units out of mass production now.
If they can get decent tracking, I’d be ok to buy Bluetooth haptic gloves just to give me touch sensations on surface/button pressing
Imagine combining the touch controllers and hand tracking. Using the tracking ring as a passive haptic thing
guys remember.
no buttons means high reliance on finger tracking....
there must be input, if you make an ok symbol or a thumbs up, it may mean one thing to the pc and another to geitrolls
opoint is, if the occlusion causes loss of signal.. that also equates to lose of all button simulation....
if you want to drive with a streeting wheel and shift
I need that shirt, Norm!
You can do a Vulcan salute? I guess it's settled then 🖖🖖🖖
12:30 pretty crazy having no green screen. Would be great for parties
I wonder if you can use hand tracking with the Quest through Oculus Link for PC games in the future.
Same here. I was hoping throughout the interview that this would be asked at some point.
I suspect that they'll avoid doing so initially, as they don't want to split the Oculus PC dev chain, but that some of the larger players using VR for training will talk them into implementing it. At first it'll likely be a special build of the OS, but after an update or two, it'll no longer be worth Oculus's time maintaining two code branches and they'll mainstream it.
Can't wait to try hand tracking in a VR cockpit ;)
I. Can't. Wait.
I'm really excited to play Facebook Horizon with finger tracking! VR is now!
Next thing may be haptic gloves that aren't like big but really minimalistic and don't get in the way
Have have a thought, wouldn't not be possible to use the Rift Cv1 sensors. If the sensors are the same tech on the Rift S as what they are on the Cv1 so maybe they could add external sensor hand tracking with out any blind spots. They must of thought about this
Hand tracking should be a supplement to controllers. It’s makes using touch controller, gun stocks, joysticks, steering wheels better. I don’t think it will shine by itself
It needs to be an combination, because sometimes it is hand movement only, sometimes an object e.g. gun. One hand gun other hand just your hand. I think oculus (facebook) also works on haptic wristbands or something like that.
I hope Oculus uses their prototype wristbands for haptic feedback etc..
I want to be able to switch from hand tracking to controller. Then you can use your hands for open word and then you can pull your controller out from your holster when it’s time to shoot everything up. Also need holsters lol.
So this obviously can't be used in the dark right? And they if they wanted could fix that in the future, right?
What if in the future you could put on a glove that will push your hand out with the same amount of force you use to grab or hold something, making it actually feel like your holding something.
i love you guys.
Does this mean we could get full body tracking on quest 🤔
I hope they decide to develop gloves in the future. That would solve a lot of problems with hand tracking.
0:05 I WANT THAT SHIRT. Norm! Where can I get it?? Just started watching, hope he mentions it.
Stellar T-Shhirt~!!!!!
Think of the Kinect for how well not having anything to hold works in terms of lack of feedback
This is more minority report than kinect
that would be so cool to like double tap your fore arm and have a menu appear on it
Jeremy has headset hair. Come on Norm. You could have warned him! Haha!
What happens if someone else gets in the camera view and it connects to their hands, would it just ruin your game 🧐
I'm a bit confused. It sounds like you'll need a native USB-C port on your gaming PC to enable the Quest to run games and charge at the same time. Am I wrong with that? My PC works fine with Virtual Desktop/SteamVR, but it doesn't have a USB-C port. I'm not very techie, does anyone know if (in theory) an A-to-C adapter will fully work with the link cable?
Chris Stacey Adapters could work. Technically you only need USB 2 speed, but the achieved speed depends a lot on cable length, and an adapter will make it worse. Ideally you'd use a high quality USB-A to USB-C cable. These aren't very common at lengths on 3m/10ft or more, and usually rather thick. These problems are the main reason why Facebook will sell a dedicated, very thin and light USD 79 USB-C fiber optic cable. So your PC will work, but you may be restricted regarding cable length.
@@faduci Thanks, that makes sense. I've already been perusing Amazon and it looks like there are a few 3.0 A-to-C cables of 4.5 and 5 meters (gotta love metric ;-) ) but wasn't sure if they could keep the Quest charged whilst also delivering the encoded data. And they're probably quite thick/less flexible than the link cable will be.