It’s incredibly helpful seeing companies starting at square 1 and seeing them take each step. Congratulations on getting your series made, looking forward to seeing more!
Brilliant video, you perfectly captured the frustations a lot of people had at the beggining of the lockdown and trying to learn virtual production on the fly, failing a lot on the way.
I used to be in this field then I quit for 4 years from my mental health. I was too scared to back to this field again. Now after your inspiration which you made things so possible... thanks dude
The RUclips gods sent me here. This is freakin amazing. I've been looking for alternatives to on-set filming for forever. I can't animate or draw, but I can definitely act like an idiot in front of a greenscreen.
And Jim Henson was revolutionizing the virtual production studio way back in 1987 for The Storyteller, and in 1989 for The Jim Henson Hour. Kind of amazing how far we've come in making the tech smaller-better-faster-cheaper, but the results from using chroma screen look exactly the same today, but in 4K. Volumetric studios are the best virtual ones, as rear projection work has always been better than chroma screens, and now we're taking it further with video walls and tech that tracks the camera in near real-time. Very exciting stuff! Thanks for this generous BTS video!
I recently did a very low-budget music video where I built two walls of LED-strips and used pixel-mapping to get lighting which matched a rear-projected background. It really sold the immersion. So I'm sure something similar might be able to work for them too.
Incredible achievement. I'm sure you've got lots of other things to do (that help you pay the bills), but a tutorial on how you did it would be fantastic. Or even a list of the links you found most hopeful.
I don't know how I came across this but you guys are amazing for having made such a detailed, comprehensive BTS on this process. I shot my first VP commercial last year and enjoyed it.
The jittering is absolutely not a limitation of the VR system. I don't know what caused this in your case, but you can definitely get a smooth image with the Vive tracker on your camera. Anyway, this was terrific! Virtual Production evolves so quickly, it's astounding! Last year we had to manually calibrate the camera, now we have a simple, almost automatic calibration system. Here's a few advice : - Virtual Production is even more interesting when you move the camera. For that, you need to make sure, in the UE scene, that the virtual screen (the rectangle on which you project the actor) keeps facing the camera. That way you can shoot scenes with a moving camera, instead of just fixed shots. - You can get shadows automatically with the proper material settings on the virtual screen. No need to put it on each frame manually! The limitation to that though is the shadow is projecting a 2D surface, so there's an angle at which point it doesn't feel like the natural shadow anymore ; but that's just a limitation to keep in mind when working on the lights! Aside from that, you get real-time accurate shadows.
Really fascinating. They say you should never let daylight in on magic, but I'm grateful! We use straight CSO / chromakey for our training programs but this is truly astonishing! Congratulations!
Tundra are making a big tracker that is designed to replace the Vive one for virtual production and Vive are working on some hardware to support this stuff too. Exciting time to be into VR tech!
Really good video. One very cheap option for camera matching is to control the real camera with the virtual camera. A pan and tilt on a tripod controlled with an arduino uno then mapped to the scene camera is simple but effective, certainly as a rig to "play with" as a learning exercise. I use Blender and OBS studio.
You should get some low res but bright LED Panels with diffusion on them, that are linked to live feeds of virtual cameras so that you get some interactive lighting. Ian Hubert talked about similar stuff on his Patreon a while ago.
What a time to be alive! Can't wait to be talking to some friends about a Tv show in 10 years, and i'm just like " Oh ya Them? I saw how they started " :D
Love this! I stream VR games in mixed reality, and do sketches too. The fact the tech is colliding in so many people's minds is brilliant. Glad production tools are catching up. Realising the 3080 Ti graphics card in my machine was powerful enough to do something like Mando, and in real time, was a game changer.
This was amazing and so insightful!! Thank you so much for sharing! I'm based up in the North East of England and love watching and seeing content like this. Hope to one day do EPK in film and TV and so seeing all of your BTS is really awesome and inspiring. Keep up the great work and will definitely check out your series on iplayer! :D
This is really excellent. I do a lot of work in After Effects and Cinema4D for Augmented Reality. But it’s all post production. Unreal Engine is something I’d love to get involved more in with and this is great video for starting with it. Thanks so much
This is fantastic! I'm about to make the jump back into live action and am really eager to get my head around Unreal 5 in an FX context. It would be great to be able to have comedy sketches having access to real-time FX in their scenes, hell even if it's as cheesy as South Park using live action explosions in the scenes there could be great comedy in it.
Wow, RUclips has just told me about this now so I'm a bit late to the par-tay! The mandalorian volume is actually new application of a very old VR space called a cave as opposed to the HMD we are now more used to. As you say technology is moving rapidly in this area, I saw a package from Quanta systems thar used its RGBWW tubes that to cast light in a similar manner to LCD panels but at much less cost. Thus getting the real world sense of the human interacting with the environment but without the cost. Its interesting that you didn't look at using rear/front projection especially to get the light scatter on the humans. More videos about what you learnt or want to achieve would be interesting But thanks for this I forgot to ask did you look at all the great work done on this by Robert Rodriguez especially with his use of green screen with Spy Kids & the Sin City series??
I'm sure that virtual production is the future. Technologies like The Volume as well as use of Unreal Engine like you've done so here. They're exciting times we live in!
Just a suggestion, but whilst robot arms are indeed expensive, there are still several good motorized sliders available (even a couple that support the Mini Pro) for reasonable pricing.
The Vive trackers have such weird issues. The first gen were great, but the second gen, while ostensibly better, had a lot more 'wiggle', or could be off by a solid 2CM. And these problems could be caused by anything from the USB port not being fast enough to something reflective on set confusing it. You can minimise it by using 4 lighthouses instead of 2, but that doesn't totally mitigate their issues. They're great bits of kit, and those original errors you show look more like update speed or rendering issues than tracking issues (long USB extender?), but they're so fiddly you're right to get away when you can.
You're so right about the Vives being fiddly and frustrating at times. Sometimes they would work brilliantly, giving very smooth movement on even a complex map, other times they would be really jittery, losing their orientation as well sometimes. We eventually put the wireless dongles on the camera, as close as we could to the actual trackers, by using long USB cables back to the PC. This did sort out a lot of the problems. I see the new Vive Mars system uses a box under the tracker to provide a hard-wired ethernet link back to the PC, so that should be better, but at a cost! That link can also include data from lens encoders. I've just been on a course to learn more about a different system, which has a similar but more powerful unit on the camera to send data back to the PC.
@@pogglemoose That sounds like a game-changer. Since I didn't want to paint my house green (like a coward) I started by using Kinect depth cameras to create the chroma. But the lasers the Kinect used interfered with the trackers so much they'd float away instantly. In the end I went tracker-less. The MR app 'LIV' uses some voodoo to get an excellent calibration in 30 seconds, so doing that after each camera move was easy enough. But I haven't got 4 actors staring at me between shots.
This is something Disney (and the team behind Lazy Town too!) has been doing for a few years now. Off the top of my head, the Muppet crew uses a mocap puppet rig in "Earth to Ned" that they use to drive a CG character in Unreal that gets comped into a scene with real actors. That way they can get all the eyecandy of CG animation with the spontaneity of their classic puppetry.
Absolutely love this - fantastic work and it's great to see how you learned and improved your techniques as you went along. Can't wait to see more from you :)
This was an awesome video! Great overview of the process: very informative! I definitely want to make a show like this, but I ditched greenscreen for mocap a few years ago.
Awesome job, it's great to see the creativity! About the 'choppy' camera movement with the tracker, I think that is a solvable problem. If I remember correctly the trackers track position at 200hz but the headset works at a 1000hz (to make the vision more smooth). So it should for sure be possible to smoothly track a camera with the vive system! You can perhaps even 'upgrade' the trackers to 1000hz.
I want more!!! Show me how I can start today. Just like you all started. Or make a video on what you think the next tech step you think would need to be.
It’s incredibly helpful seeing companies starting at square 1 and seeing them take each step. Congratulations on getting your series made, looking forward to seeing more!
Exactly. Really inspiring
Thanks!
This is really incredible, i'd love to hear more about the education behind pairing virtual production and small budget filming, it's pretty amazing
Brilliant video, you perfectly captured the frustations a lot of people had at the beggining of the lockdown and trying to learn virtual production on the fly, failing a lot on the way.
I used to be in this field then I quit for 4 years from my mental health. I was too scared to back to this field again. Now after your inspiration which you made things so possible... thanks dude
Thanks for also sharing some timings, which gives a better feeling for how long a start in virtual production really takes.
This is incredible Phillip thanks
The RUclips gods sent me here. This is freakin amazing. I've been looking for alternatives to on-set filming for forever. I can't animate or draw, but I can definitely act like an idiot in front of a greenscreen.
It all came together with the last gritty shot with the blue cast on the talents face - brilliant!
Thanks dear god, I struck gold today ! Huge love and respect to you guys for Sharing this !
And Jim Henson was revolutionizing the virtual production studio way back in 1987 for The Storyteller, and in 1989 for The Jim Henson Hour. Kind of amazing how far we've come in making the tech smaller-better-faster-cheaper, but the results from using chroma screen look exactly the same today, but in 4K. Volumetric studios are the best virtual ones, as rear projection work has always been better than chroma screens, and now we're taking it further with video walls and tech that tracks the camera in near real-time. Very exciting stuff! Thanks for this generous BTS video!
Perhaps the next step would be to invest in some hue-controllable lights, to make compositing easier!
I recently did a very low-budget music video where I built two walls of LED-strips and used pixel-mapping to get lighting which matched a rear-projected background. It really sold the immersion. So I'm sure something similar might be able to work for them too.
Yes but Hue Lights arent really the key here.
I imagine that soon the lights will communicate with the system and set the right hue automatically (if it's not already possible).
@@undergroundo that's what Martin Jones just said. Pixel mapping on the led lights.
Love it! Thanks for the shout-out!
Woa… ok I just clicked into it and was Shazaming the intro music and noticed only on the green fade over that this was already VP. Crazy feeling.
That's way more impressive than I initially thought before clicking on the video. Going to learn more about this stuff !!
Incredible achievement. I'm sure you've got lots of other things to do (that help you pay the bills), but a tutorial on how you did it would be fantastic. Or even a list of the links you found most hopeful.
This video just changed my life
Wonderful insight! Thanks for sharing!!!
I love this showing of the process!
Small and clever productions is an extremely accurate name 😂
I jump on my side to the terrific but exciting VR world in France. Great job guys
This is simply Great! Thank you for sharing your journey!!!
Inspiring and cool!
I don't know how I came across this but you guys are amazing for having made such a detailed, comprehensive BTS on this process. I shot my first VP commercial last year and enjoyed it.
Very inspiring stuff! Always great to see piers at the BBC pushing modern practises. I hope to shake your hand one day
So much problem solving. Wonderful storytelling. Thank you for inspiring and showing what’s possible
A brilliant overview of Virtual Production…. and interesting real world example.
Thanks for posting. Really generous of you 💗
thank you very much for sharing this. good luck with your show.
The jittering is absolutely not a limitation of the VR system. I don't know what caused this in your case, but you can definitely get a smooth image with the Vive tracker on your camera.
Anyway, this was terrific! Virtual Production evolves so quickly, it's astounding! Last year we had to manually calibrate the camera, now we have a simple, almost automatic calibration system.
Here's a few advice :
- Virtual Production is even more interesting when you move the camera. For that, you need to make sure, in the UE scene, that the virtual screen (the rectangle on which you project the actor) keeps facing the camera. That way you can shoot scenes with a moving camera, instead of just fixed shots.
- You can get shadows automatically with the proper material settings on the virtual screen. No need to put it on each frame manually! The limitation to that though is the shadow is projecting a 2D surface, so there's an angle at which point it doesn't feel like the natural shadow anymore ; but that's just a limitation to keep in mind when working on the lights! Aside from that, you get real-time accurate shadows.
Really fascinating. They say you should never let daylight in on magic, but I'm grateful! We use straight CSO / chromakey for our training programs but this is truly astonishing! Congratulations!
Really nice overview! Congrats on the BBC show.
Thanks for the mention in the description. :-)
Looking forward to seeing what you do next!
Tundra are making a big tracker that is designed to replace the Vive one for virtual production and Vive are working on some hardware to support this stuff too. Exciting time to be into VR tech!
Well done guys. Fun to watch and the tracker hint is nice too.
Cheers
this video explained omse concepts abot virtual production much better than others.
Thanks a bunch for the video, really amazing! I want to dabble in such productions myself, but it'll still be a while until I can do so
This is amazing! Must be a lot of fun once you got it all worked out.
Excellent stuff, was very interesting to see the BTS and see a step by step process. Cheers for sharing
Really good video. One very cheap option for camera matching is to control the real camera with the virtual camera. A pan and tilt on a tripod controlled with an arduino uno then mapped to the scene camera is simple but effective, certainly as a rig to "play with" as a learning exercise. I use Blender and OBS studio.
Very well done, and very useful information. Thanks for sharing
You should get some low res but bright LED Panels with diffusion on them, that are linked to live feeds of virtual cameras so that you get some interactive lighting. Ian Hubert talked about similar stuff on his Patreon a while ago.
Very interested to see more of this kind of content
Thanks for sharing your journey!
How is this trick with qr code and camera positioning called? Cant find it anywhere
Greg Corson has done a couple of tutorials on this now.
What a time to be alive! Can't wait to be talking to some friends about a Tv show in 10 years, and i'm just like " Oh ya Them? I saw how they started " :D
Love this! I stream VR games in mixed reality, and do sketches too. The fact the tech is colliding in so many people's minds is brilliant. Glad production tools are catching up.
Realising the 3080 Ti graphics card in my machine was powerful enough to do something like Mando, and in real time, was a game changer.
Awesome you have a team like this
That's really cool! Makes me want to play around with some stuff!
Very nice job. Thank you for sharing.
So awesome!! And inspiring. Bravo!! Hello from Melbourne, Australia 🤙
Excellent. I love seeing problems solved on a budget.
this is soo cool. Hats off to you gents. Love this
Many thanks for the explanations …
Brilliant video, excellent work, really interesting!
This was great guys! I want to do this myself (in Canada). Thanks for sharing your process.
awesome overview, thanks so much for this :)
This was amazing and so insightful!! Thank you so much for sharing! I'm based up in the North East of England and love watching and seeing content like this. Hope to one day do EPK in film and TV and so seeing all of your BTS is really awesome and inspiring. Keep up the great work and will definitely check out your series on iplayer! :D
This is an amazing use of virtual reality!
Thanks for sharing! Really inspiring.
Really cool and inspiring!
Thanks for sharing!
This is really excellent. I do a lot of work in After Effects and Cinema4D for Augmented Reality. But it’s all post production.
Unreal Engine is something I’d love to get involved more in with and this is great video for starting with it. Thanks so much
This is fantastic! I'm about to make the jump back into live action and am really eager to get my head around Unreal 5 in an FX context. It would be great to be able to have comedy sketches having access to real-time FX in their scenes, hell even if it's as cheesy as South Park using live action explosions in the scenes there could be great comedy in it.
that is pretty cool - thanks for sharing! Super interesting
Wow, RUclips has just told me about this now so I'm a bit late to the par-tay! The mandalorian volume is actually new application of a very old VR space called a cave as opposed to the HMD we are now more used to.
As you say technology is moving rapidly in this area, I saw a package from Quanta systems thar used its RGBWW tubes that to cast light in a similar manner to LCD panels but at much less cost. Thus getting the real world sense of the human interacting with the environment but without the cost. Its interesting that you didn't look at using rear/front projection especially to get the light scatter on the humans.
More videos about what you learnt or want to achieve would be interesting
But thanks for this
I forgot to ask did you look at all the great work done on this by Robert Rodriguez especially with his use of green screen with Spy Kids & the Sin City series??
Superb video! Thanks for sharing! Have been doing lots of VR and would love to dive into this area for some productions later on
Thank you very much for sharing!!!
Great video, thanks for sharing and congratulations on the results!
Wow. Incredible growth
Loved everything about this video, very informative and entertaining. Cheers.
VERY informative. Thanks for the insight ❤
This is great, thanks for sharing.
Very very inspiring. Thanks for sharing
Thanks for this video, this is really cool to see how you progressed!
Thanks for sharing! I've been trying to figure this whole thing out and there isn't much straight forward info out there.
Awesome video and I loved seeing how you worked things out on a small budget with a great result, thanks for sharing!
I'm sure that virtual production is the future. Technologies like The Volume as well as use of Unreal Engine like you've done so here. They're exciting times we live in!
Congrats guys!
Just a suggestion, but whilst robot arms are indeed expensive, there are still several good motorized sliders available (even a couple that support the Mini Pro) for reasonable pricing.
The Vive trackers have such weird issues. The first gen were great, but the second gen, while ostensibly better, had a lot more 'wiggle', or could be off by a solid 2CM. And these problems could be caused by anything from the USB port not being fast enough to something reflective on set confusing it. You can minimise it by using 4 lighthouses instead of 2, but that doesn't totally mitigate their issues.
They're great bits of kit, and those original errors you show look more like update speed or rendering issues than tracking issues (long USB extender?), but they're so fiddly you're right to get away when you can.
You're so right about the Vives being fiddly and frustrating at times. Sometimes they would work brilliantly, giving very smooth movement on even a complex map, other times they would be really jittery, losing their orientation as well sometimes. We eventually put the wireless dongles on the camera, as close as we could to the actual trackers, by using long USB cables back to the PC. This did sort out a lot of the problems. I see the new Vive Mars system uses a box under the tracker to provide a hard-wired ethernet link back to the PC, so that should be better, but at a cost! That link can also include data from lens encoders. I've just been on a course to learn more about a different system, which has a similar but more powerful unit on the camera to send data back to the PC.
@@pogglemoose That sounds like a game-changer. Since I didn't want to paint my house green (like a coward) I started by using Kinect depth cameras to create the chroma. But the lasers the Kinect used interfered with the trackers so much they'd float away instantly.
In the end I went tracker-less. The MR app 'LIV' uses some voodoo to get an excellent calibration in 30 seconds, so doing that after each camera move was easy enough. But I haven't got 4 actors staring at me between shots.
Great work guys!!!
it's really useful thank you , will have a lot to ask later
Great work guys 👍
Thanks for the video. Very informative
Amazing job, wouldn’t it be cool to have a character in animation in this interact with a real one like this?
This is something Disney (and the team behind Lazy Town too!) has been doing for a few years now. Off the top of my head, the Muppet crew uses a mocap puppet rig in "Earth to Ned" that they use to drive a CG character in Unreal that gets comped into a scene with real actors. That way they can get all the eyecandy of CG animation with the spontaneity of their classic puppetry.
This is truly amazing!
Absolutely love this - fantastic work and it's great to see how you learned and improved your techniques as you went along. Can't wait to see more from you :)
This is incredible !!
Amazing work. And you’re only going to get better from here!!!
This was an awesome video! Great overview of the process: very informative! I definitely want to make a show like this, but I ditched greenscreen for mocap a few years ago.
Thanks for this super video! There's plenty of advices (maybe all xD) to start working in virtual production from scratches eheh
Amazing.. Thanks for sharing this video
This is very nice! Its incredible
Awesome job, it's great to see the creativity! About the 'choppy' camera movement with the tracker, I think that is a solvable problem. If I remember correctly the trackers track position at 200hz but the headset works at a 1000hz (to make the vision more smooth). So it should for sure be possible to smoothly track a camera with the vive system! You can perhaps even 'upgrade' the trackers to 1000hz.
This is amazing I had no idea that a lot of this software that I've used is capable of this stuff
Great work I'v need the realsense to my studio :)
A really good and useful video, thank you
I want more!!! Show me how I can start today. Just like you all started. Or make a video on what you think the next tech step you think would need to be.
Amazing 👏. Just wow
Really impressive!
NICE! Love it! Good work!
Incredible!
Underrated