@Elite-Force Cienma I grew up loving the original unreal engine, v1. As much as I agree with the stockholder sentiment, this post isn't about politics. This is genuinely useful technology for everyone and well engineered. Worth hanging up the cynicism over where the cash injections come from for a minute to appreciate technical progress.
@@johnnyxp64 the logic is the same you just have to learn a different way to write. two weeks could be enough due to it being a little bit more complex
this is extremely cool! as a lighting artist I love how quick it looks to tweak your physical lighting based off the scene and not having to place actual real world lights and trying to match everything up. badass.
yep that was my first thought too, creating HDRIs for Keyshot has been such a mindblowing fun experience, I can't imagine the fun these people are having with this.
physical set designers in Hollywood... your days are numbered. Quixel and Epic just put you on notice. This is amazing for movie production, you can create any real world location you’d like, along with add lighting, reflections and ray tracing, in real time on a sound stage.
In virtual productin, set desiners are cruicial! They connect the background with the foreground. Props filling the empty spaces and with this tech they can spend more budget on these foreground props instead of expensive big low quality backgrounds. I think they are quite happy now.
Another small problem it's real subtle but people can notice is the bokeh problem. At narrow depth of field, if you have a foreground object at say 6 feet, a midground statue at 20 feet, and a giant pyramid on the horizon at infinity. With a real camera midground and backgrounds get progressively defocused with distance. If the midground to infinity was a LED screen, objects with great spatial disparity would have the same size out of focus bokeh. You could fix this if the projected images had suplimental depth of field, rendering sharp at the screen plane, and progressively defocused to infinity. This would be added to the depth of field defocus of the camera between the foreground subject and the screen plane.
I had the same exact thoughts. I don't think implementing this will be a big problem though, those folks don't half-ass things. This is a technology in its early stages. I guess you could even simulate anamorphic bokeh with this, although when you're using a rig like this, chances are you're using anamorphic anyway.
You're right, as a cinematographer i've noticed it straight away, How does UE plans to tackle this? perhaps they used more advanced features on the sets of Mandelorian
@@frosti7 UE4 has a pretty advanced depth of field implementation already, but yeah, I imagine it may be possible to fix the issue by augmenting real-life depth of field with virtual depth of field projected onto the screen. Or just going bigger - the screen used for the Mandelorian seams to be this massive, warehouse sized thing unlike the small set used in this demo, so the background will naturally be further away. Also, just a thought - since the camera and the computer system know exactly what's onscreen at each frame, I wonder if masking out the background would be a whole lot easier then even a green screen - after all, since the virtual camera being rendered by the screen is synced to the exact location of the real life camera, then presumably the system would be able to know exactly where the rendered image the LED screen ends and the real life, non-rendered actors and props begin, without the fringing issue that green screens have to contend with. I know they mention being able to activate a virtual green screen that follows the camera, but the larger you make the LED screen the larger that green square is going to be and the more it's going to effect the lighting of the shot.
This is truly the future happening now! What is the rig of the LED Video Walls? This video panel must have a higher than usual dot pitch so to avoid moire effect when shot with 4/8K cameras.
I don't think that. They probably come from this company. www.luxmc.com/ But with cinema cameras, I don't think the patterns would be that big of a concern, regardless.
Based on their claim that it's a 4k image and some set scale estimates, I put the screens at about 10 feet tall, which means the dot pitch is about 1.2-1.5mm/pixel, but if the image is kept out of focus and there's a diffusion sheet in front, I could see it being as low as 3.0mm/pixel, which is actually a fairly typical resolution for RGB LED panels.
Hey folks, what would you think about having a ‘WeWork’ of the virtual production ? Which basically means a space where anyone who wants to test content/games or anything related to virtual production can pop in and test their work? The space would be fully equipped with LED WALLS and all material for Virtual production at a AFFFORDABLE price.
They'll be shooting the next moonlanding with this thing lol Anyway there is something a bit too sterile about this environment, could use some straws moving in a light breeze.
I know you since Unreal Tournament 2001. Seeing you in movies creating wolds; thats unreal! You live up to your name, creating real worlds which are unreal.
i'm melting... if i wasn't working, at home behind unreal or outside i'd be watching unreal vids all the time I guess... if you think about how far you got, i loved the first unreal tournament so much already, but many years later and now this... amazing
And that my friends is how Disney will do the "Black Box" attraction. A large space that be changed easily for any event or ride with trackless vehicles moving about.
For all the recent Star Wars movies since Rogue One in the spaceship scenes they had actual projection screens on the outside of the cockpit displaying the hyperspace jump etc, so all of those VFX were basically done in-camera in real time. It's the future, really.
@@darkgreenguitarist Very cool! The first modern use of the technique that I'm aware of was the house set in Joseph Kosinski's Oblivion, and I think that one was a 360, but AFAIK they used regular rear projection screens instead of LEDs. You'd think that puts a bit of a limit on the dynamic range, but Oblivion had some amazingly naturally lit sunrise and sunset interior shots - no idea how they pumped that much light through that kind of setup.
KillahMate yes corridor crew recently featured Oblivion, I didn’t realise the footage was actually real that they were playing back, shooting those landscapes must have been an amazing experience.
KillahMate also the screen use I saw on Solo was mainly for lighting of actors and blaster shots that could be cued, as you said the laser projection screens were mainly in camera. Gravity used a screen system for lighting George C and Sandra B on a motion controlled rig.
Very cool demo thks. Is there a way to get a matte of what's being filmed? I mean a matte of the subject only, without the LED screens (or a matte of only the LED screens :) Regards.
A ton of attention to detail for achieving realism...right up until that Triumph started up and sounded like a V-Twin haha. But jokes aside, this is an amazing example of solid integration of available tools where its sum is far greater than its parts. Very awesome stuff. Kudos!
Question: do the virtual backdrops have to be static (e.g. rocks, sky, clouds), or can there be movement, like ocean waves, or even people (extras) moving in the distance, or tree branches in the wind?
We're doing this, it is not static, the HDRI can still be used for massive scenes but just like the ground rocks and close by objects clouds can also be physical assets that are moved and changed from nimbus to large cumulus ones for example. As for water that is a very costly object haven't tried it but I would wager it would need to be optimized to work for the scene.
So I guess the magic hour is anytime now, huh? I'm just a casual movie fan and even I understand how big this is. I remember hearing how The Revenant took forever to film because of the director's obsession with the lighting. I imagine this would have fixed those freezing months of misery with a few clicks. Viva la revolucion guys. Congrats and great job! :)
Sheesh!!! So are these the same led walls we see on big budget keynote presentation stages and large scale outdoor billboard advertisement panels we see hanging on buildings?!? 😱
Top cette technologie ! Dans les film récent, je ne supportais plus de voir des acteurs filmés devant un écran vert et qui étaient éclairés par une lumière différente de la lumière ambiante.
omg movies and video games are just... I can't even imagine how they will ... man just think about having live action actors integrated seamlessly into videogame mechanics in a generation or 2. movies and videogames will be....
If I were to build something like this what kind of screens would be needed and what else is required at minimum? Can you shoot with large screens, unreal, and a VR lighthouses/tracker or is there more required?
Impressive results, great work done! background music so much reminds me sugababes - stronger I feel a lack of light in front of model. I know that is fixable on pp but, glasses reflection is super clearly shows black room, such as absent front reflections of virtual open world on the model. maybe narrow line on top and on the bottom of 4th wall of scene will help a bit, maybe moving light array on camera itself to complete this lighting illusion
It's gonna make movies look awesome. But man, I'm also worried about how this kind of tech could be abused in a near future, along with deepfakes and voice synths able to mimic any voice. Remember "Running Man" and the Brubaker massacre? Or the scene with the fake death of Ben Richard?
this is such dope tech! whats the hobbyist level replication amateurs could make? using projectors? get these into the hands of young creators! the types of story telling they could do!!! on a much smaller budget with the quality of 'relocations' - its a new market place people will sell renders of all sorts. This is set production design to a whole new industry! I love the facts its simply millions on millons of leds on a panel, a very populated new lighting system. the music videos will come!
The basics of this technology are quite old actually. Kubrick used a somewhat comparable projection technology to shoot the ape scenes in the beginning of "2001" but he was limited to static shots as any camera movement would've messed up the immersion of the foreground into the background. The camera tracking is the real game changer, not to mention the insanely quick turnover when you switch location and the realistic lighting. Just marvelous. It'll be a while until this tech becomes realistically availibe for lower budgets, but it will be, it's too much of a money saver not to be used.
Does the virtual camera match the depth of field of the physical camera? Because it would look weird if the whole background was evenly blurred by the physical camera.
I guess one could implement a realistic DOF gradient depending on focal length and position. Should be easy enough, looking at what they have accoplished already .
Absolutely incredible tech. Hope we can get way higher fidelity movies for lower budgets!
well you are right now getting higher fidelity small screen production.
The guy sitting in the motorcycle doesn't even flinch. He's either the coolest man alive or blind
Blind? lmfao aha
He is the most interesting man in the world.
Nah, he is just a MetaHuman.
Ue4 is getting insanely powerful.
@Elite-Force Cienma i also prefer Unity because i know c#. i don't know c++ to use in Unreal... plus... they are expensive.
@Elite-Force Cienma I grew up loving the original unreal engine, v1. As much as I agree with the stockholder sentiment, this post isn't about politics. This is genuinely useful technology for everyone and well engineered. Worth hanging up the cynicism over where the cash injections come from for a minute to appreciate technical progress.
Elite-Force Cienma epic was the one who condemned blizzard for their actions with the chinese?
@@johnnyxp64 the logic is the same you just have to learn a different way to write. two weeks could be enough due to it being a little bit more complex
@@johnnyxp64 who's elite force cienma? is it just you got in the wrong comment?
This is amazing! And I believe The Mandorian is using this. Incredible stuff here!
Absolutely incredible. This is the future of filmmaking. Well done to everyone involved!
It's so rewarding coming back to a video you didn't understand at all a few months ago and being able to put the pieces together
this is extremely cool! as a lighting artist I love how quick it looks to tweak your physical lighting based off the scene and not having to place actual real world lights and trying to match everything up. badass.
Bill Burr said they used this tech in the Mandalorian.
Correct.
now go back 10 years and tell that to your self, see if that sentence makes sense
@@alex0589 You drunk or something?
@@alex0589 now go back 1 week and see if that comment makes sense
There is always that one person who's comment makes no sense whatsoever. This guy is one of those people.
So... It's basically HDRI in the physical world, COOL!!!
HDRI projected 3d scene, to be certain
Well.. Those LED screens aren't really as HDR as the real-world sky.
In CGI terminology, it's an image based lighting and backdrop of a real world subject.
yep that was my first thought too, creating HDRIs for Keyshot has been such a mindblowing fun experience, I can't imagine the fun these people are having with this.
But can you manipulate assets in real time with an HDRI?
physical set designers in Hollywood... your days are numbered. Quixel and Epic just put you on notice. This is amazing for movie production, you can create any real world location you’d like, along with add lighting, reflections and ray tracing, in real time on a sound stage.
In virtual productin, set desiners are cruicial! They connect the background with the foreground. Props filling the empty spaces and with this tech they can spend more budget on these foreground props instead of expensive big low quality backgrounds. I think they are quite happy now.
background music: My Skin Remembers
- The Sonic Hijackers
good boy
Pls, music link
@@MrSorgogbayar www.google.com
many thanks from China🎉❤❤
Another small problem it's real subtle but people can notice is the bokeh problem.
At narrow depth of field, if you have a foreground object at say 6 feet, a midground statue at 20 feet, and a giant pyramid on the horizon at infinity. With a real camera midground and backgrounds get progressively defocused with distance.
If the midground to infinity was a LED screen, objects with great spatial disparity would have the same size out of focus bokeh.
You could fix this if the projected images had suplimental depth of field, rendering sharp at the screen plane, and progressively defocused to infinity. This would be added to the depth of field defocus of the camera between the foreground subject and the screen plane.
I had the same exact thoughts. I don't think implementing this will be a big problem though, those folks don't half-ass things. This is a technology in its early stages. I guess you could even simulate anamorphic bokeh with this, although when you're using a rig like this, chances are you're using anamorphic anyway.
You're right, as a cinematographer i've noticed it straight away,
How does UE plans to tackle this? perhaps they used more advanced features on the sets of Mandelorian
@@frosti7 UE4 has a pretty advanced depth of field implementation already, but yeah, I imagine it may be possible to fix the issue by augmenting real-life depth of field with virtual depth of field projected onto the screen. Or just going bigger - the screen used for the Mandelorian seams to be this massive, warehouse sized thing unlike the small set used in this demo, so the background will naturally be further away.
Also, just a thought - since the camera and the computer system know exactly what's onscreen at each frame, I wonder if masking out the background would be a whole lot easier then even a green screen - after all, since the virtual camera being rendered by the screen is synced to the exact location of the real life camera, then presumably the system would be able to know exactly where the rendered image the LED screen ends and the real life, non-rendered actors and props begin, without the fringing issue that green screens have to contend with. I know they mention being able to activate a virtual green screen that follows the camera, but the larger you make the LED screen the larger that green square is going to be and the more it's going to effect the lighting of the shot.
This is truly the future happening now!
What is the rig of the LED Video Walls?
This video panel must have a higher than usual dot pitch so to avoid moire effect when shot with 4/8K cameras.
I'm pretty sure it's used in a way that keeps the screens out of full focus.
@@hornsolo_official So you believe it's off-the-shelf rather cheep LED panels you can even find on AliExpress? (they are priced relatively cheap).
I don't think that. They probably come from this company. www.luxmc.com/
But with cinema cameras, I don't think the patterns would be that big of a concern, regardless.
@@hornsolo_official thanks for this
Based on their claim that it's a 4k image and some set scale estimates, I put the screens at about 10 feet tall, which means the dot pitch is about 1.2-1.5mm/pixel, but if the image is kept out of focus and there's a diffusion sheet in front, I could see it being as low as 3.0mm/pixel, which is actually a fairly typical resolution for RGB LED panels.
How did the actor stay looking absolutely cool as a cucumber throughout the whole spot xD
Acting!
PRO
Twist, he's not really there, he's a 3D geometry asset.
It's Nicholas Cage... doesn't he always look cool?
probaply just a cucumber in a wig and the set makes him look awsome
THAT'S WHAT I CALL A TECH DEMO
Epic is pushing boundaries .. everyone else is just following or trying to..
The future of filmmaking is happening right now
I’m hearing this is the tech used for The mandalorian
yeah! I noticed Unreal Engine in the credits and wondered how it was being used.
Spagman so cool!
All the interior falcon shots from Solo were done this way. The actors literally felt like they were looking out into space.
Josef H that’s so interesting! I love this new tech their creating
This is totally insane ...! amazing stuff with UE & LED Walls.
I like how he went from Tatooine to Endor in a few seconds
Hyper Space
Holy... Im speechless. This is... Idk. I said I was speechless.
Hey folks, what would you think about having a ‘WeWork’ of the virtual production ?
Which basically means a space where anyone who wants to test content/games or anything related to virtual production can pop in and test their work? The space would be fully equipped with LED WALLS and all material for Virtual production at a AFFFORDABLE price.
They'll be shooting the next moonlanding with this thing lol
Anyway there is something a bit too sterile about this environment, could use some straws moving in a light breeze.
yes, but i think, you can put your own more vivid backrounds there and also use practical effects to merge everything better
A better example of the technology is probably the Mandalorian
Sweet to see this used in Mandalorian, stepping up the prod and post at the same time. Really cool stuff.
I know you since Unreal Tournament 2001.
Seeing you in movies creating wolds; thats unreal!
You live up to your name, creating real worlds which are unreal.
Kudos to all who created this entire technology
Wow, just like the CGI deer scene on TWD back in 2017! They must be so proud!
i'm melting... if i wasn't working, at home behind unreal or outside i'd be watching unreal vids all the time I guess... if you think about how far you got, i loved the first unreal tournament so much already, but many years later and now this... amazing
Absolutely amazing work
This is so amazing and fun to learn how new content will be made. I am excited to see what the future holds.
probably more global warming, i'm really stoked for it.
And that my friends is how Disney will do the "Black Box" attraction. A large space that be changed easily for any event or ride with trackless vehicles moving about.
For all the recent Star Wars movies since Rogue One in the spaceship scenes they had actual projection screens on the outside of the cockpit displaying the hyperspace jump etc, so all of those VFX were basically done in-camera in real time. It's the future, really.
It reminds me of front projection compositing, usedconvincingly in 2001 and Superman and other old movies
KillahMate they also used these exact screens to, I met Phill from LM when they were shooting some millennium falcon cockpit scenes on Solo.
@@darkgreenguitarist Very cool! The first modern use of the technique that I'm aware of was the house set in Joseph Kosinski's Oblivion, and I think that one was a 360, but AFAIK they used regular rear projection screens instead of LEDs. You'd think that puts a bit of a limit on the dynamic range, but Oblivion had some amazingly naturally lit sunrise and sunset interior shots - no idea how they pumped that much light through that kind of setup.
KillahMate yes corridor crew recently featured Oblivion, I didn’t realise the footage was actually real that they were playing back, shooting those landscapes must have been an amazing experience.
KillahMate also the screen use I saw on Solo was mainly for lighting of actors and blaster shots that could be cued, as you said the laser projection screens were mainly in camera. Gravity used a screen system for lighting George C and Sandra B on a motion controlled rig.
Matt and Matt will show a live matte!!!
Star Trek holosuites coming closer!
for a price :)
This is incredible technology right here
The guy on the bike at the beginning look different at the end!!!? Same guy??? That mind blown.
Future of filmmaking is here
Now, ... that is really exceptional.
thank you guys for nice feature in filmmaking
Can't wait for the making-of of the Mandalorian, to see how they used that fantastic Stagecraft tech.
just got uploaded
Very cool demo thks.
Is there a way to get a matte of what's being filmed? I mean a matte of the subject only, without the LED screens (or a matte of only the LED screens :)
Regards.
A ton of attention to detail for achieving realism...right up until that Triumph started up and sounded like a V-Twin haha. But jokes aside, this is an amazing example of solid integration of available tools where its sum is far greater than its parts. Very awesome stuff. Kudos!
cant win em all
Does anyone know what screens those are? Back Projection? TV?
i am damn sure they will be revolution in gaming industry then its only done by Epic games
Unreal tech is amazing.
This is Unreal!
why don't use to upload a 4k video on youtube rather than 1080p
Question: do the virtual backdrops have to be static (e.g. rocks, sky, clouds), or can there be movement, like ocean waves, or even people (extras) moving in the distance, or tree branches in the wind?
We're doing this, it is not static, the HDRI can still be used for massive scenes but just like the ground rocks and close by objects clouds can also be physical assets that are moved and changed from nimbus to large cumulus ones for example. As for water that is a very costly object haven't tried it but I would wager it would need to be optimized to work for the scene.
Squad.
very cool stuff here! I love #epicgame #iloveunrealengine4
epic shit ^.^
So I guess the magic hour is anytime now, huh? I'm just a casual movie fan and even I understand how big this is. I remember hearing how The Revenant took forever to film because of the director's obsession with the lighting. I imagine this would have fixed those freezing months of misery with a few clicks. Viva la revolucion guys. Congrats and great job! :)
This is phenomenal
what is the resolution? Would it also work for high resolution still photography?
That is just insane!
Wow, unreal in movies, that's really something
Im so hyped to work with this.
absolutely awesome, how does the virtual set know how to change perspective though?
It{s like if the man were really there. So cool
Stunning work guys
Brand new That room must smell greaaat
Amazing job guys!
Sheesh!!! So are these the same led walls we see on big budget keynote presentation stages and large scale outdoor billboard advertisement panels we see hanging on buildings?!? 😱
Top cette technologie ! Dans les film récent, je ne supportais plus de voir des acteurs filmés devant un écran vert et qui étaient éclairés par une lumière différente de la lumière ambiante.
If you render the background out, couldnt you get a key from that? Like calculate The footage minus the rendered background?
Mind Blowing thanks for the video !
I bet RED is feeling left out of this partnership. Whats the music they were using at the star and through out the video?
how does it handle DOF? Are the exposure settings and focus point sent to the engine?
It probably is. It already handles the camera tracking so I can't see why it couldn't do capture settings
Incredible
i'd like to do movies with this.. jaw dropping effects...
This is friggin insane!!!
omg movies and video games are just... I can't even imagine how they will ... man just think about having live action actors integrated seamlessly into videogame mechanics in a generation or 2. movies and videogames will be....
I wanted to know something, how much would a setup like this cost?
If I were to build something like this what kind of screens would be needed and what else is required at minimum? Can you shoot with large screens, unreal, and a VR lighthouses/tracker or is there more required?
Reminds me of Jean-Claude Van Damme and the Triumph in No Where To Run
What are you using to track camera movement and calculate the exact parallax and angles needed to project on the video walls?
Impressive results, great work done!
background music so much reminds me sugababes - stronger
I feel a lack of light in front of model.
I know that is fixable on pp but,
glasses reflection is super clearly shows black room,
such as absent front reflections of virtual open world on the model.
maybe narrow line on top and on the bottom of 4th wall of scene will help a bit,
maybe moving light array on camera itself to complete this lighting illusion
bruv
This is so amazing it's almost scary.
This is just wow!
What app he was using for light control anyone please tell me
Great, what's the studio setup here?
so awsome! great work, guys
It's gonna make movies look awesome. But man, I'm also worried about how this kind of tech could be abused in a near future, along with deepfakes and voice synths able to mimic any voice.
Remember "Running Man" and the Brubaker massacre? Or the scene with the fake death of Ben Richard?
LOL YESSSS Running Man ref FTW! but also agree, the fear is reel (see what I did there).
What are those screens behind the motorcycle called? Awesome!
Reminds me of the holodeck in Star Trek.
this is such dope tech! whats the hobbyist level replication amateurs could make? using projectors?
get these into the hands of young creators! the types of story telling they could do!!! on a much smaller budget with the quality of 'relocations' - its a new market place people will sell renders of all sorts.
This is set production design to a whole new industry!
I love the facts its simply millions on millons of leds on a panel, a very populated new lighting system.
the music videos will come!
The basics of this technology are quite old actually. Kubrick used a somewhat comparable projection technology to shoot the ape scenes in the beginning of "2001" but he was limited to static shots as any camera movement would've messed up the immersion of the foreground into the background. The camera tracking is the real game changer, not to mention the insanely quick turnover when you switch location and the realistic lighting. Just marvelous.
It'll be a while until this tech becomes realistically availibe for lower budgets, but it will be, it's too much of a money saver not to be used.
So Great Breakdown, thanks for your sharing
Moire not moray. Amazing tech!
This remind me of the scenarios of Tekken 3. The background moves a little artificial.
Holy virtual scene, Batman!
Congratz to y'all, guys!
You're almost Unreal :)
How much do those LED screens cost?
Does the virtual camera match the depth of field of the physical camera? Because it would look weird if the whole background was evenly blurred by the physical camera.
I guess one could implement a realistic DOF gradient depending on focal length and position. Should be easy enough, looking at what they have accoplished already .
I wonder whether the issue of aliasing can be overcome by using the data from the engine rendering and then compositing it with filmed footage.
This is blowing my mind!
What do you do if the director wants a low shot pointing up? Can the sky section be extend to match the perspective of the back section?
In the Laika Studios would they use the same technology?
Mind blowing!
Now UE4 calling me to use in VFX Virtual Production. I can not wait anymore. Lets Dive in .........
Oh I thought in the beginning he was in the actual location lol oof
Goes to show how well this technology already works ^^
So directors have to have video editing and color correcting skills too?
Simply amazing!