I'm showcasing the technology starting at around 2:50. Also check out some of the projects I created in this playlist: ruclips.net/p/PLnW6Tpl5Y1w6x8WhX4L9XoDv6_nrHXpqQ
As others have already stated here in the comments, the thing you promote here so much is not really useful for most productions. Filming an actor in the greenscreen and placing him as a 2D plane in the 3D scene is very limiting as the actor cannot move around. And it completely falls apart as soon as you need more than one actor interacting. If you have any kind of perspective you cannot show full body with believeable floor contact and shadows as persons further back would appear as if they are floating. Moving the virutal camera around is also very limited since you cannot really change the perspective other than moving in or out from the subject and have slight parallax movements. Anything more than that would give away the illusion. Also this method is nothing new, it has been done for years, long before Unreal Engine was a thing for filmmakers. Having the lighting of the scene hitting the plane is nice, yes, but also not realistic as you have no light interaction so you still have to fake the scene lighting on the greenscreen and yet again no movement is possible. So basically you are back to the live tracking method so you can recreate the scene lighting on set and the actor can move around freely and then you are back to the same problems the LED walls have with accuracy of the tracking etc. And it might be true that the keying is getting better through software improvements, but none of your examples from your own work show results that would be good enough for an actual professional production. Lots of blurry fringes or even transparent areas on the actors etc. Also comparing LED with conventional greenscreen stuff you do in your living room is kind of absurd. Of course no one can afford an LED setup for home use. It was never advertised as such. So the price point is no argument here. Even a somewhat accurate and professional live tracking solution on a greenscreen for unreal would be too expensive for most hobbiest. Even though things are definitely improvig and costs go down, at the moment I'd say the conventional ways are still the most relieable for hobbiest and small productions with low budgets, in the sense of shooting on a greenscreen with tracking markers and use 3D tracking in post. I also had some good results with CamtrackAR on the iphone for 3D tracking but this limits you to phone footage and 30 fps. Unless you shoot parallel with a cinema camera and use some trickery in unreal to combine the phone tracking with the footage of the cinema camera. But it is still an exciting concept I hope will be improved upon over time. Sorry if this comes across as largely negative. I still think it's super fun to experiment with all these new techniques and tools we have today and I'm really exciting what more will come in the future. It's just your claim in the title that really is misleading and not held up very well by what you present here.
Hey Hethfilms, thanks so much for your comment. I think it’s a good critical perspective on this workflow. And a valuable addition here in the comment section. The one thing I want to add is you can achieve quit a lot with this workflow when you plan your shots carefully ahead. Of course not everything is possible. But there are still a lot of shots that you can achieve and make look believable. Anyways… again thanks so much & talk soon :)
Also it just came into my mind. You should be able to normalize the scaling of the actor when he moves in Aximmetry in real-time. I will figure this out
When looking at grants recently, I did explore the cost of an LED Wall myself, and it was around £150K for a 4m x 3m setup... so yeah, exceedingly unlikely for indies. However, there are still better alternatives than green screen. An ultra short-throw projector (such as the Optoma UHZ65UST) and a projector screen at least 100" diagonal width, when lit properly, provides similar image quality as an LED wall, for a fraction of the cost.
We've tried decent ultra short-throw projectors too (we used a Christie), and unfortunately they're not a patch on a proper LED wall. There are two issues - light output and black levels. The second issue is actually the more significant one. Each micro LED in an LED wall is surrounded by a black, light-absorbent material. This means that spill light from spotlights / panel lights etc. on the wall is not the end of the world - you still get good blacks (and hence contast). With a projection screen, any spill / bounce light that hits the screen absolutely kills the black levels. This is a significant issue, as in practice it's very restrictive to light the front of an actor without any light hitting the screen. The light output of an LED wall also means you can film in something other than pitch darkness, whilst still getting good results. LED walls are an order of magnitude more expensive (at least), but are much better to work with. They also support features like Genlock - important when syncing the Unreal PC to the camera - that projectors don't.
@@axi0matic I am not saying that, for an established Indie team, an LED wall isn't a great idea. However, if you are either a solo operation or just launching into VP, it is highly impractical for someone to afford such a setup unless they have been working in the arena for a while. Even if one were to rent time at a local facility, this is currently £2,000/day minimum. If one could secure a MegaGrant, maybe - failing this, a projector is the only "affordable" step up from green screen.
@@rushboardtechuk Fair, but you said a projector “provides similar image quality as an LED wall”, which unfortunately isn’t the case. I’ve seen the odd video where they’ve been used quite effectively though. There’s also a recent video by a team that did the VFX for a BBC3 comedy series, which used green screen with great results. Well worth a look.
@@axi0matic yeah I apologise about it being "similar". A more accurate interpretation would be "subjective" based on an entry-level budget (i.e., no more than $10K).
Just a thought on how to fix the projected lighting onto the subject which the video wall is supposed to fix... why not use coloured RGB lights in close proximity to the subject, then connect those to a responsive lighting system like Dreamscreen or Philips Hue Sync Box - then connect that into your PC running Unreal Engine so that as the camera moves through a scene, the scene sends lighting signals to the LED lights - the net effect being the subject is lit as the virtual camera moves.
That's a great thought & approach!. You should give it a try!! I'm using some professional LED lights now that I control via DMX using the Unreal sequencer which gives me fantastic results :) Will share this soon
@@Moons_mooniverse YAS - I also thought about DMX but didn’t know you could connect in to UE or Unity with it - that could add some interesting effects into the room / environment too I guess?
Modular lightfield panels might be the high end future. Its a 360 volume but everything is in 3D. No glasses. No eyestrain. Multiusers at the same time. Its going be a huge thing at concerts, theme parks and alternative to a traditional night at the movies. A "holodeck" or location based VR but without a headset nor avatars. Real people in the light of impossible worlds.
this is just a clickbait, he has no idea on what he is talking. Look at that chroma, like, wtf this is awfull and yet he claims that it is the future. InCameraVFX is not about "metaverse" and virtual reality, this is just sad.
Have to disagree with you. I’ve never seen a single example of live keying that is suitable for a feature film work flows. A live keyed version is always recorded for reference but the majority of the time compositors still work with the original green screen layer and background layer to create the final image.
Have you ever seen Ghostframe? I believe that is the best example as to what a hybridized system could look like for a post production work flow. Also 3D props really just aren’t the same as tactile props…
No it wont replace it because a LED wall lights the production as well. There are drawbacks with super bright directional lights but this is the way forward in most cases
Led walls might have issues too, if you change your mind later on. So, even with this tech, green screen will always be there in my opinion. On another note, I really appreciate your alternative mindset for filmmaking. I'm searching for more ways to make my guerrilla way into it. Thanks, man!
With LED walls you can do either. Or the more interesting idea is a hybrid. The virtual landscape is everywhere except immediate behind your actor which is show as green on the LED wall. This technique allows for all of the natural lighting of a LED wall plus gives the option to change things in post due to the tracked green screen. Also there is little chance for green screen spill with this technique.
You really compare blockbuster cinema with livestream keyes/effects??? :D 1) In cinema, every hair and semitransparent shadow/reflection counts. You never ever will pull a single key (like you do for Live-Production) for a final vfx matte. You pull many, different keys and use every possible separations' technique to bring every detail back to have no lose of the original quality/grain/color of the footage.' 2) If you have LED Wall you have also perfect green If you want ;-) ...even in little parts of the virtual set (If you want) 3) VFX worked now 20 years in front of green and also Virtual Production started 8 years ago with greenscreen and mocap. But it was always a nightmare to act and direct in front of this green hell, movies suffered noticeable in quality and the look of 100% CG gave everything a computer game touch. So more and more physical set-buildings came back on Set and now also the light and environment are back. That's a very good thing for Actors, Directors and DOPs. They work now again on the same ideas and have not to believe in something that could happen in the postproduction (maybe). 4) AR and AI will have a massive impact o VFX and will change a lot again. But until we have no AR contact lenses in our eyes or 3D holograms on Set, Hollywood will stick with that virtual LED environments :-)
Yes, you can make the cutouts but in all examples, you showed you cant say the result is that good. massive halo flickering and so on. With the LED wall you missed the point of light. in front of a greenscreen, you don't have the light from the environment or reflections. If you going ta make a case it is a 100% replacement of the human in the game engine.
no , u r wrong about green screen, yes there are advancements in tech , but ur green screen keying is also not perfect , there are lines we can see around ur neck which should be removed , and there is a big problem with hair keying ...softwares usually smooth down the messy hair , but in production we need that detail so thats why keying takes time and should be done properly
I will improve this in future videos. I know that I have to get to the point faster. Maybe for this video set the speed to 2x. Because I’m also showing what software is able to do now which is really impressive!!
What they meant when they said "you're actually there" is that the actors can see the background and that's immersive, making it easier for them to act. What does that have to do with VR headsets? You say that keying is much easier today and then show your examples, are you really sure those examples are movie-ready? If you don't think that your shots would require a whole lot of retouching and rotoscoping in post you're delusional. Your edges are being eaten away by the green spill. I also don't understand the connection between virtual production and the metaverse you're trying to make. The metaverse is internet stuff, how does that relate to content production?
it seems to me you haven't understood what the LED volume is for... mate... LIVE ACTION FOOTAGE. mixed with real interacting lighting. you are talking about something completely different. if your movie will be only static shots.... ok hahahaha wtf man... so the future is bad composited green screen ?? a B-movie? come on man...
No, what I’m actually talking about is live (real-time) footage. Both action or no action. With real-time green screen keying and camera tracking (not only static shots). What I haven’t mentioned in the video is controlling real lights with virtual lights in UE via DMX. Will upload a video how I do this soon!
I'm showcasing the technology starting at around 2:50. Also check out some of the projects I created in this playlist: ruclips.net/p/PLnW6Tpl5Y1w6x8WhX4L9XoDv6_nrHXpqQ
I think your examples perhaps somewhat undermine your arguments. They're certainly not bad little hobby projects but the results are all very amature.
The LED wall however adds an element of lighting and reflections that are nearly impossible to replicate.
No question. That’s why I also want to have an LED wall :)
As others have already stated here in the comments, the thing you promote here so much is not really useful for most productions. Filming an actor in the greenscreen and placing him as a 2D plane in the 3D scene is very limiting as the actor cannot move around. And it completely falls apart as soon as you need more than one actor interacting. If you have any kind of perspective you cannot show full body with believeable floor contact and shadows as persons further back would appear as if they are floating. Moving the virutal camera around is also very limited since you cannot really change the perspective other than moving in or out from the subject and have slight parallax movements. Anything more than that would give away the illusion. Also this method is nothing new, it has been done for years, long before Unreal Engine was a thing for filmmakers. Having the lighting of the scene hitting the plane is nice, yes, but also not realistic as you have no light interaction so you still have to fake the scene lighting on the greenscreen and yet again no movement is possible. So basically you are back to the live tracking method so you can recreate the scene lighting on set and the actor can move around freely and then you are back to the same problems the LED walls have with accuracy of the tracking etc. And it might be true that the keying is getting better through software improvements, but none of your examples from your own work show results that would be good enough for an actual professional production. Lots of blurry fringes or even transparent areas on the actors etc. Also comparing LED with conventional greenscreen stuff you do in your living room is kind of absurd. Of course no one can afford an LED setup for home use. It was never advertised as such. So the price point is no argument here. Even a somewhat accurate and professional live tracking solution on a greenscreen for unreal would be too expensive for most hobbiest. Even though things are definitely improvig and costs go down, at the moment I'd say the conventional ways are still the most relieable for hobbiest and small productions with low budgets, in the sense of shooting on a greenscreen with tracking markers and use 3D tracking in post. I also had some good results with CamtrackAR on the iphone for 3D tracking but this limits you to phone footage and 30 fps. Unless you shoot parallel with a cinema camera and use some trickery in unreal to combine the phone tracking with the footage of the cinema camera. But it is still an exciting concept I hope will be improved upon over time. Sorry if this comes across as largely negative. I still think it's super fun to experiment with all these new techniques and tools we have today and I'm really exciting what more will come in the future. It's just your claim in the title that really is misleading and not held up very well by what you present here.
Hey Hethfilms, thanks so much for your comment. I think it’s a good critical perspective on this workflow. And a valuable addition here in the comment section. The one thing I want to add is you can achieve quit a lot with this workflow when you plan your shots carefully ahead. Of course not everything is possible. But there are still a lot of shots that you can achieve and make look believable. Anyways… again thanks so much & talk soon :)
Also it just came into my mind. You should be able to normalize the scaling of the actor when he moves in Aximmetry in real-time. I will figure this out
u
When looking at grants recently, I did explore the cost of an LED Wall myself, and it was around £150K for a 4m x 3m setup... so yeah, exceedingly unlikely for indies. However, there are still better alternatives than green screen.
An ultra short-throw projector (such as the Optoma UHZ65UST) and a projector screen at least 100" diagonal width, when lit properly, provides similar image quality as an LED wall, for a fraction of the cost.
We've tried decent ultra short-throw projectors too (we used a Christie), and unfortunately they're not a patch on a proper LED wall. There are two issues - light output and black levels. The second issue is actually the more significant one. Each micro LED in an LED wall is surrounded by a black, light-absorbent material. This means that spill light from spotlights / panel lights etc. on the wall is not the end of the world - you still get good blacks (and hence contast). With a projection screen, any spill / bounce light that hits the screen absolutely kills the black levels. This is a significant issue, as in practice it's very restrictive to light the front of an actor without any light hitting the screen.
The light output of an LED wall also means you can film in something other than pitch darkness, whilst still getting good results. LED walls are an order of magnitude more expensive (at least), but are much better to work with. They also support features like Genlock - important when syncing the Unreal PC to the camera - that projectors don't.
@@axi0matic I am not saying that, for an established Indie team, an LED wall isn't a great idea. However, if you are either a solo operation or just launching into VP, it is highly impractical for someone to afford such a setup unless they have been working in the arena for a while. Even if one were to rent time at a local facility, this is currently £2,000/day minimum.
If one could secure a MegaGrant, maybe - failing this, a projector is the only "affordable" step up from green screen.
@@rushboardtechuk Fair, but you said a projector “provides similar image quality as an LED wall”, which unfortunately isn’t the case. I’ve seen the odd video where they’ve been used quite effectively though. There’s also a recent video by a team that did the VFX for a BBC3 comedy series, which used green screen with great results. Well worth a look.
@@axi0matic yeah I apologise about it being "similar". A more accurate interpretation would be "subjective" based on an entry-level budget (i.e., no more than $10K).
Just a thought on how to fix the projected lighting onto the subject which the video wall is supposed to fix... why not use coloured RGB lights in close proximity to the subject, then connect those to a responsive lighting system like Dreamscreen or Philips Hue Sync Box - then connect that into your PC running Unreal Engine so that as the camera moves through a scene, the scene sends lighting signals to the LED lights - the net effect being the subject is lit as the virtual camera moves.
That's a great thought & approach!. You should give it a try!! I'm using some professional LED lights now that I control via DMX using the Unreal sequencer which gives me fantastic results :) Will share this soon
@@Moons_mooniverse YAS - I also thought about DMX but didn’t know you could connect in to UE or Unity with it - that could add some interesting effects into the room / environment too I guess?
Modular lightfield panels might be the high end future.
Its a 360 volume but everything is in 3D. No glasses. No eyestrain. Multiusers at the same time. Its going be a huge thing at concerts, theme parks and alternative to a traditional night at the movies. A "holodeck" or location based VR but without a headset nor avatars. Real people in the light of impossible worlds.
Oh, that’s a good point!!
Great big picture way of think. I was thinking about led walls, but it's true, the ROI is terrible. Marvellous channel!😃
Thanks so much Lautaro!! :)
Do you have a course that teaches how to use Aximmetry? they way you are using it right now?
Okay I turned of at 09:47. You abviously have no idea about filmmaking and the gap between your little indie setup and stuff for big cinema.
this is just a clickbait, he has no idea on what he is talking. Look at that chroma, like, wtf this is awfull and yet he claims that it is the future. InCameraVFX is not about "metaverse" and virtual reality, this is just sad.
I'm pretty sure Hollywood knows about your keying features
Wow this is amazing
Can you please gives us a ready to go Virtual Production file? please help! thanks
Have to disagree with you. I’ve never seen a single example of live keying that is suitable for a feature film work flows. A live keyed version is always recorded for reference but the majority of the time compositors still work with the original green screen layer and background layer to create the final image.
Have you ever seen Ghostframe? I believe that is the best example as to what a hybridized system could look like for a post production work flow.
Also 3D props really just aren’t the same as tactile props…
No it wont replace it because a LED wall lights the production as well. There are drawbacks with super bright directional lights but this is the way forward in most cases
Led walls might have issues too, if you change your mind later on. So, even with this tech, green screen will always be there in my opinion. On another note, I really appreciate your alternative mindset for filmmaking. I'm searching for more ways to make my guerrilla way into it. Thanks, man!
Thank you Estrangeiro! I like that you call it the guerrilla way :)
With LED walls you can do either. Or the more interesting idea is a hybrid. The virtual landscape is everywhere except immediate behind your actor which is show as green on the LED wall. This technique allows for all of the natural lighting of a LED wall plus gives the option to change things in post due to the tracked green screen. Also there is little chance for green screen spill with this technique.
surprised that you can also put 3d objects in front of the character. Is that the nature ability of the composition in UE?
Correct. This is the nature ability of the composition in UE. It’s super cool!!
You really compare blockbuster cinema with livestream keyes/effects??? :D
1) In cinema, every hair and semitransparent shadow/reflection counts. You never ever will pull a single key (like you do for Live-Production) for a final vfx matte. You pull many, different keys and use every possible separations' technique to bring every detail back to have no lose of the original quality/grain/color of the footage.'
2) If you have LED Wall you have also perfect green If you want ;-) ...even in little parts of the virtual set (If you want)
3) VFX worked now 20 years in front of green and also Virtual Production started 8 years ago with greenscreen and mocap. But it was always a nightmare to act and direct in front of this green hell, movies suffered noticeable in quality and the look of 100% CG gave everything a computer game touch. So more and more physical set-buildings came back on Set and now also the light and environment are back. That's a very good thing for Actors, Directors and DOPs. They work now again on the same ideas and have not to believe in something that could happen in the postproduction (maybe).
4) AR and AI will have a massive impact o VFX and will change a lot again. But until we have no AR contact lenses in our eyes or 3D holograms on Set, Hollywood will stick with that virtual LED environments :-)
Why would you want a separate green screen? It's totally redundant.
Does your dad own a green screen factory?
think big production. You are thinking about youtube videos. this is not for you bro.
Yes, you can make the cutouts but in all examples, you showed you cant say the result is that good. massive halo flickering and so on. With the LED wall you missed the point of light. in front of a greenscreen, you don't have the light from the environment or reflections. If you going ta make a case it is a 100% replacement of the human in the game engine.
no , u r wrong about green screen, yes there are advancements in tech , but ur green screen keying is also not perfect , there are lines we can see around ur neck which should be removed , and there is a big problem with hair keying ...softwares usually smooth down the messy hair , but in production we need that detail so thats why keying takes time and should be done properly
Get to the point…
I will improve this in future videos. I know that I have to get to the point faster. Maybe for this video set the speed to 2x. Because I’m also showing what software is able to do now which is really impressive!!
@@Moons_mooniverse or perhaps just give us chapters so that we can choose if we want the build up or not. Cheers!
This doesn't work for professional productions like a high budget movie or series.
it's so clear you have no idea what you're talking about
What they meant when they said "you're actually there" is that the actors can see the background and that's immersive, making it easier for them to act. What does that have to do with VR headsets?
You say that keying is much easier today and then show your examples, are you really sure those examples are movie-ready? If you don't think that your shots would require a whole lot of retouching and rotoscoping in post you're delusional. Your edges are being eaten away by the green spill.
I also don't understand the connection between virtual production and the metaverse you're trying to make. The metaverse is internet stuff, how does that relate to content production?
clickbait
it seems to me you haven't understood what the LED volume is for... mate... LIVE ACTION FOOTAGE. mixed with real interacting lighting. you are talking about something completely different. if your movie will be only static shots.... ok hahahaha wtf man... so the future is bad composited green screen ?? a B-movie? come on man...
No, what I’m actually talking about is live (real-time) footage. Both action or no action. With real-time green screen keying and camera tracking (not only static shots). What I haven’t mentioned in the video is controlling real lights with virtual lights in UE via DMX. Will upload a video how I do this soon!
"I don't think this is the future". GTFO.