Wow, I was not expecting this. I was expecting the video to replay exactly without lighting information from the scene, but it is actually lighting the video from the scene, which is amazingly cool.
Terrific! I am just starting out with Unreal and film making and I would love to see a tutorial on the Octane plugin. Your results have been so good that I can't see not using it if I can figure out how. Help me, Josh. You're my only hope.😉
Awesome great idea to use Rotoscoping because the one thing I’m realizing is I’m getting heavy spill from the green screen, the interactive light and matching the light with the scene is really doing a lot immersing the live action into the unreal scene, so I totally understand how not having that heavy green screen spill and light reflection, because of the proper light you need to have the screen well lit for a comp. :)
Yes absolutely. Plus in small spaces theres so much more spill as the green bounces off walls. Plus I'm not a big fan of the despill plugins either, they can do a lot of damage to colour accuracy if used too heavily.
Apples and oranges. Octane gives physically accurate path tracing. Lumen might be more useful when fully launched but it wont compete with Octane for cinema.
Hi! I cannot make out of focus with the camera my green screen footage... Is there any setting for that? How is it possible UE doesn't allow to focus/blur the subject like in a real situation?? It makes the footage much more realistic when you get slightly out of focus ... Thanks a lot in advance and congrats for your great tutorials!
Hi I have responded to your comment on another video. If you still need help or advice I have a discord server you could join that might help. discord.gg/rSFaHqZa
have you tried using "Masked" mode instead of "Translucent" so that you can have the 2d plane cast shadows in the environment? There are some drawbacks with that method so I'm trying to figure out the best way to cast shadows natively in UE.
I have tried masked mode but it really chews up a good matte. The opacity is 'dithered' so doesnt give cinematic results in my opinion. However translucent planes can both casts and recieves shadows via octane for certain. I did work on some real time alternatives but was never 100 percent satisfied. I might wait until the full ue5 release before pursuing that further.
Hi, thank you, very interresting. Few questions : what are the nodes I can add to the material to manage contrast and brightness of my footage ? and how can I have the shadow of my footage on the ground (the shape) ? do you know if octane works on a Mac ?
for contrast, you can use a multiply or power node. I have videos on shadows. I have a quick search. Im sure they'll pop up. For octane, take a look on otoy.com
Hi. I love your tutorials. I did step by step but my plane have light all the time even when i tern off all lighting. What i do wrong and why world light not effent the plane?
Thanks Joshua. Do yu have a tutorial that goes over the whole process of rotoscoping in AE, exporting in format suitable for UE (EXR format?), saving 3D camera in AE, loading that camera in UE and aligning to your short footage? What I have done so far is export an EXR sequence from AE with background removed, then created camera via Cine4K. Now just trying to figure out how to pullo it all together! thanks again@@JoshuaMKerr
well I just want a programmatic way of drawing my media file pics and videos into a timeline that I can walk over so people can see like a better death video @@JoshuaMKerr
You can make adjustments to the image plane using the material editor. You can also do a more traditional grade with the colour settings in the post process volume.
First time Unreal Footage looks not like a game or animations and more realistic. Great job. Will try out rendering with Octane. Now I am doing live capture with Off world. Very cool way to do Virtual Production
@@JoshuaMKerr it is a plug in - OFFWorld plug in. I use it to stream out of the Unreal. Instead of using sequencer and rendering shot by shot it streams & help capture video real-time . But it comes with less flexibility that you mentioned in this video. I guess I will do a video on this plug in and tag you. It is a really cool alternative to sequencer, but not 100% replacement
Good question. You can add audio to sequencer for certain and I have definitely seen the option to export the audio through the legacy movie capture option. Also although I haven't tried it movie render queue has a .wav audio option too.
Thanks for the tutorial! One question. The media in my scene always plays back and renders really roughly around the actors key (the file is correctly keyed) any ideas on how to improve in engine quality or final render quality?
@@JoshuaMKerr Just curious, have you had any issues with fog and inserting pre-keyed media? Mine always renders in front of the fog no matter where it's placed. Not sure if it's a limitation with the engine.
Theres a few settings to try deoending on the issue. Look in the material for 'compute fog per pixel' then theres also 'allow fogging' (I think). Output velocity might also help.
I love UE but to import prekeyed footage is overly complicated when it doesn’t have to be. They should have something similar to Blenders import images as planes. One click and it’s all set up for you. They could easily do this in UE…..rather than having to jump through all these hoops and blueprints. It could all be automatically done and ready to go.
@@JoshuaMKerr ok you prob know but in 5.1 it’s basically just drag and drop now for prekeyed footage and works out if the box 😊, only thing is it looks like specular is too high and can’t find how to change it in 5.1. Have you given it a bash yet please?
@@JoshuaMKerr yeah it’s def much easier now. Just got to figure out what video formats are supported as some didn’t work, and before I could click on the material and change specular etc (at least I think that’s why the video looks washed out) but can’t figure out how to do this in 5.1 yet but so happy they have streamlined this process.
Very helpful, thanks, I have one problem, as soon as I apply the keying the plane brightness is not changing with the directional light anymore, it will remain always the same bright, before applying the keying playing only the media player on the plane everything is good, changing the directional light affecting the plane nicely. If I place a point light that affects the plane but the directional light not. Any tip, why's that?
That's a tough one to answer without seeing your material setup. I have a new discord server that ive started to help people with their virtual production problems. If you join and post your problem in the forum it might be easier to figure out. Here's a link: discord.gg/C4RtzVUU
@@JoshuaMKerr hey, I have a total shot, and the subject stands on a mirror floor or rough tiles and there is no reflection on the floor, shadow yes But no reflection
@@JoshuaMKerr i Use a plane witch is, transculate and there is a movie showing, a keyed out render, i tryed both, Lumen and reytrace But if i change to reytrace, the shadow cast the whole Plate and not the subject only, i guess i need to make fake reflections by douplicating the Video and hide it in game
There is... but it's not very good so it's better to colour correct in another package first. In your material you simply multiply your footage by a vector3 and select what colour you want to tint your footage with.
Did you ever get into relighting live actors/video in Unreal - but with 3D depth of their face? Currently exploring this. Tried depth maps via volumetric video with a dual stereoscoptic lens but the depth maps didnt have enough depth for the face to appear fully 3D in Unreal. Thinking about trying ARkit livelink face app and combining the mesh with the video feed using a HMC. Or other method you might recommend? Thoughts?
Real-time would be a major plus, but I was more asking which route/method in general did you take this week to give your video footage of your actors face a 3D dimensionality so that it could be relit by Unreal in a very realistic way? (I mentioned two ideas above)
i have a question, i want to use prekeyed footage with a translucent material, i can cast shadows but i want to cast lightshafts if the actor is moving? how to do that? maybe u can help ? best regards
@@JoshuaMKerr hey joshua, i always (sometimes not but often raytraced ) use Lumen, if its possible in the scene, BUT - the shafts are not showing, only if i change the blending mode to masked, but the problem with masked is , the edges are very wired. and have some pixel even if i export it wit no antialiasing and 16 samples for example. and the other problem with masked is - my footage looks choppy - it looks like the fps is not matching so ,for ex. 25 fps is used in my opinion.
@@JoshuaMKerr I think this will work, BUT i dont want to do this on every shot 😀 this will take to long, is it possible to duplicate the prekeyed plane > turn it to masekd > hide it from scene except the Shadows and Shaftst? if yes how to do this?
This is an awesome video! I’ve just come across your channel and it’s very helpful! I know this is a year old but I have a question. If anybody could help it would be very much appreciated. Basically, everything works perfectly in sequencer but when I render it out anything I have through a media player/playing on a plain just comes out showing the alpha channel and not the actual image sequence I see in engine?? Any ideas what I’m doing wrong? Cheers
@@JoshuaMKerr hello mate, thank you for the reply and so quick too! I posted just before heading out for work so I’ll jump on the discord tomorrow now. Thank you again and keep up the great work!!
Hi josh! Great video! I followed it step by step and got the same results. However, when I render the movie out, I get the black plane and not the transparent. What tid you do to maintain your alpha when rendering the clip? Thanx again for a great video! 🙂
This was great! I was finally able to get some prekeyed footage into Unreal on a Mac...just to see how absolutely terrible it plays back on this machine haha. Curious what are the specs on the machine you are on right now? Seems like it's playing back well. I've been trying to hold out for GPU prices to drop, but may just end up giving up on that and going the prebuilt route with the prices right now.
Until recently I was working with a 2070 which certainly did the job. You can allocate more ram to image sequences in unreal settings. might cover that in the next video.
Hi how do you render this out, i did the same thing with pre key and import to UE as a media player like you did, but when i render with movie render queue, it doesnt render the media player, its just a black box for the plane with media texture
@@nk0599 If you're completely new to unreal engine I suggest reading the unreal docs. Here's a link to the info on level sequences. docs.unrealengine.com/4.26/en-US/AnimatingObjects/Sequencer/Overview/
When i import my green screen keyed footage i always end up with this black outline on the subject. But in after effects it looks clean did you ever have that problem
@@JoshuaMKerr yes I e been doing exr in after effects my key is great but when I bring it into unreal there is this slight black outline so frustrating. That’s why I usually stick to Element3d but I love unreal
I remember something like this. Just out of curiosity, could you try two png sequences? One rgb and the other an alpha matte. So you end up with two media tracks in sequencer. I feel like it's an exr issue.
I do all of my keying in after effects before bringing in to Unreal so theres loads of tools for edge feathering etc. Unreal doesnt have tools like that.
@@JoshuaMKerr I did my keying in AE too and rendered in PNG Seq but when import into UE, the edge are sharp and seems like cutout when composited. The feathering seems not working in UE.
Thanks for the tutorials, the following is happening to me, I do everything as you explain in the previous video, I save the EXR images, then I load them in Unreal from Img_Media_Source and they load fine, but when I hit Media_Player to load the sequence the program closes . I also tried with .PNG images and the same thing happens to me, what could it be? I am working on UE5, I appreciate if you can give me some light. Greetings!
Hi Raul, Im not working much within UE5 at the moment so will let you know if I experience this. As UE5 is not a full release yet it might be best to work in an earlier build.
Hello! I've tried to do the same thing but with UE5 and the Actor inside the material I create is always on focus, no mater what. I tried to change the material in not transucent ( only for test ) and it worked. But as you know you need the material be transucent. I couldn't find a solution other than fake a little bit the defocus in post or place the actor directly on davinci fusion and do the post-production there. Any suggestions?
@@JoshuaMKerr Great! It worked! Notice that it works also with the masked blanded mode( playing with opacity mask ) but it has different reactions to lights. Can be usefull too. Thank U man! Now I'm trying to study the perfect workflow between this one and the other with post production directly in Davinci Fusion with the render in UE only of the background.
Hey Joshua i have another question, do u know how to face the plane from prekeyed footage always to the camera? with a specified distance? for example if i move the camera the plane is always showing to the camera
Example i have a greenscreen footage with movements inside the footage SO - i dont want 3d track the footage - i want to put it on a plane - and lock or link it to the camera inside unreal 5 - so if i move the camera - the footage should always stay facing to the camera , i hope u understand :)
There are a few methods but I think this one works best. Go to the material editor where your media texture is and add a 'screen position' node then hook it to your media textures UV input. This fits the video to the camera view. Then add a 'sprite' node and hook it directly into the world position offset. This makes the plane camera facing. So back in the world you just need to position the plane and scale it so it's always bigger then the camera view. You can make the plane huge if you want, the video will always be scaled to fit the view. Might make a video tutorial
Hi, yes I rendered using Octane through the sequencer. With the octane plugin active in Unreal there should be a dropdown in the sequencer export settings Octane Render(image). This will export an image sequence in a format of your choosing.
Hey johsuar i have i Problem wich is causing me headache since days, i have a sequencer scene with 5 Media tracks wich have 5 Different img sources, the preview is very very laggy in th sequencer, my fps Drops from 60fps to 0 and the sequence is almost freezing so its almost impossible to Set key frames to camera moves, what can i Do? The img sequence is, 720p png sequences with alpha masks, (fire Video) so i have beeb searxhing on Google why the fps is dropping maybe u can help?
Tricky one to solve. Sequencer can be a bit laggy in my experience. I would suggest you try going into project settings > Img media and experiment with raising the cache memory amount. Let me know if this helped you.
Does anyone know how to color correct a green screen transparent media file? I tried using the color region, but for some reason it doesn’t affect my green screen plain. It works perfectly on anything else in the scene.
@@JoshuaMKerr Do you have a video on how to make corrections to the media material? I understand that I’m going to have to really dive in and learn blueprints.
@@devillefilms8244 I have steered fairly clear of blueprints so far but the material editor is very useful. I don't have a video on this yet though I'm afraid.
Seems like I have trouble with the transparancy with my source media. I see all green around the runwayml rotoscoped pngs inside the joshintrotex window. Not the chesslike little black and white pattern. I exported the mp4 from runway out of premiere pro. as a png sequence with alpha channel. Any suggestions?
I've never exported my media from Premiere Pro, only after effects. I'd suggest joining my Disord and posting a video of the issue in the support forum.
@@JoshuaMKerr Thanks, Joshua. I had to use a UltraKey Filter and tell PP to set green as transparant. Export png frames with Alpha Channel. Keep up the brilliant content.
It is possible but the shot would need to be matchmoved either in after effects of Blender then imported to Unreal. I am experrimenting with this for a future video. So far I have just been using tripoded shots and adding camera shake in Unreal.
I have a problem when it comes to shadows. How do I get the two sided material to cast shadows, which actually move according to the movement of the mask. When I tried to render drawn shadows, it took the mask from the first image and took it as shadow, the shadow does not change when the image sequence moves on +If I play with focus and out of focus area with my camera the plane with the media on it is always in focus and never blurred. How can I make the media be blurred as well when everything on the same distance else is blurred with the focus function of the sequencer camera
You might want to take a look at my video about shadows on the channel. Look for Crazy Shadow Trick. The depth of field issue is quite common and to be honest UE still has a way to go to get it all working nicely. Look in the material for 'output velocity'. If taht doesnt work there is also an option in the material called 'translucency pass' ... I set this to 'Before DOF'
@@JoshuaMKerr Thanks for answer. The "before dof" thing works. But it makes niagara systems and emitters that are positioned behind the keyed img sequence seem as if they were in front of it
@@JoshuaMKerr I found an own version for the shadow. I made a copy of the same material and that copy I changed from translucent to masked (because on masked the shadow works). Then I copy pasted my media plane and put the masked_material on it. Only problem is: I have to place the masked object 0.001 m behind the translucent object so that you see the translucent one affected by light, but get the shadow of the one behind. Is there a way to take the masked object or take any object and hide the mesh but let the shadow stay?
I found the solution! I take a shadowpassswitch: On the upper knot I connect a texture which is 100% transparent (so I have no colored mesh) and on the bottom knot I put the media texture, so I have the shadow
I really hope that you still replying to the comments to this video. So during rendering it's crashing and it comes with " GPU crashed or D3D device removed. There was only one successful attempt when it rendered fine. It crashes only when i render video with green screen. Thanks.
@@JoshuaMKerr exr , resolution 1280*720. My RAM is only 8.00 GB, maybe it's not enough for unreal engine and projects like that? I was told that i need at least 16 . Thanks for respond btw
Hello How do you do to have your audio file included in your rending and export please ? i DID EVERYTHING I HOPE RIGHT BUT WHEN I RENDER no matter which format i don't have audio vith ( audio of my original video + tHANK YOU
My video was rendered as a png image sequence so it was synced later in premiere pro. It is possible to render a video with audio I think. Have you added the audio to the sequencer?
@@JoshuaMKerr Yes i know that but i understand well now .it is better to synch your original audio sound in a video software and for audio that you add in UE export it from UE. Thank you
hi my mind is currently boggled im trying to set a character into the distance like you have here me walking along a cliff, i have keyed it in after affects (great tutorial btw) and now i have the footage in here on the unreal plane and its playing back perfectly fine. Now i know there are 2 things, take recorder and the sequencer. Basically i now want to be able to record the virtual camera looking at the plane so i can move the camera while the image plane is playing in the distance (so its all in sync when i play it back), it looks like i can do that through the sequencer, i have tried hitting record on sequencer and its just confusing me. Could you help me understand how i can record the live virtual camera movement in real-time (im using live link which works fine ). and then when i play it back I want it to have the virtual camera movement i did (through live link iPhone) and the image plane recorded in sync. so if im walking along a cliff the green screend image of me is married up to the camera movements i made.
Hi, you're on the right track. Add your camera to take recorder and record the camera movement. That will capture the animation keyframes. You can then go into your recorded take and copy the keyframes. I would then go to your sequence where your media plane a d camera are and paste the recorded keyframes in there. Let me know if this makes sense. I have a discord where people can ask questions if that's of interest.
i've been looking for a way to bring in pre keyed footage for months thank u so much. ur channel is so underrated man
Thanks so much! Glad its helpful.
Wow, I was not expecting this. I was expecting the video to replay exactly without lighting information from the scene, but it is actually lighting the video from the scene, which is amazingly cool.
You need to watch the video about switchlight
@@JoshuaMKerr (Title of video?)
Wow! The end result looks so good!
Thanks
Terrific! I am just starting out with Unreal and film making and I would love to see a tutorial on the Octane plugin. Your results have been so good that I can't see not using it if I can figure out how. Help me, Josh. You're my only hope.😉
That video is on the docket next for sure. I expect it to be up in the next two weeks.
Thanks for these tutorials. Just about to start my first virtual production!
Thats exciting! What genre?
@@JoshuaMKerr A music video jumping on the hype for Top Gun Maverick
Awesome great idea to use Rotoscoping because the one thing I’m realizing is I’m getting heavy spill from the green screen, the interactive light and matching the light with the scene is really doing a lot immersing the live action into the unreal scene, so I totally understand how not having that heavy green screen spill and light reflection, because of the proper light you need to have the screen well lit for a comp. :)
Yes absolutely. Plus in small spaces theres so much more spill as the green bounces off walls. Plus I'm not a big fan of the despill plugins either, they can do a lot of damage to colour accuracy if used too heavily.
Have you figured out a way to create shadows and reflections from the images you imported in? Would love to see a tutorial on that.
Octane renders these with shadows and reflections.
@@JoshuaMKerr Good to know. Does Lumen not do the same?
Apples and oranges. Octane gives physically accurate path tracing. Lumen might be more useful when fully launched but it wont compete with Octane for cinema.
@@JoshuaMKerr Thanks for the info!! I thought with all the hype around lumen that it was equal to all the other path/ray tracers out there.
@@JoshuaMKerr Thanks for the thorough answer! Looking forward to seeing the content on this channel
Hi! I cannot make out of focus with the camera my green screen footage... Is there any setting for that? How is it possible UE doesn't allow to focus/blur the subject like in a real situation?? It makes the footage much more realistic when you get slightly out of focus ... Thanks a lot in advance and congrats for your great tutorials!
Hi I have responded to your comment on another video. If you still need help or advice I have a discord server you could join that might help. discord.gg/rSFaHqZa
How to render green screen video Sir please 😢😢😢😢😢
have you tried using "Masked" mode instead of "Translucent" so that you can have the 2d plane cast shadows in the environment? There are some drawbacks with that method so I'm trying to figure out the best way to cast shadows natively in UE.
I have tried masked mode but it really chews up a good matte. The opacity is 'dithered' so doesnt give cinematic results in my opinion. However translucent planes can both casts and recieves shadows via octane for certain.
I did work on some real time alternatives but was never 100 percent satisfied. I might wait until the full ue5 release before pursuing that further.
There's an option in the material called "Cast Dynamic Shadow as masked", that makes it possible for a translucent plane to cast masked Shadow
@@martinlehmann9138 never knew this! I will need to check this out!
Hi, thank you, very interresting. Few questions : what are the nodes I can add to the material to manage contrast and brightness of my footage ? and how can I have the shadow of my footage on the ground (the shape) ? do you know if octane works on a Mac ?
for contrast, you can use a multiply or power node.
I have videos on shadows. I have a quick search. Im sure they'll pop up.
For octane, take a look on otoy.com
Hi. I love your tutorials. I did step by step but my plane have light all the time even when i tern off all lighting. What i do wrong and why world light not effent the plane?
is the texture plugged into base colour or emissive?
@@JoshuaMKerryes. plug to base color. Like you did
Interesting. Its hard to say without seeing your full setup. Perhaps try re creating the material in a new project to see if the issue still happens.
@@JoshuaMKerr ok. Thanks
Great video! Does the newer Media Plate feature supercede this now?
Not always would be my best answer. I've had some issues with Media plate for certain situations and often go the traditional route. Still a good tool
Thanks Joshua. Do yu have a tutorial that goes over the whole process of rotoscoping in AE, exporting in format suitable for UE (EXR format?), saving 3D camera in AE, loading that camera in UE and aligning to your short footage? What I have done so far is export an EXR sequence from AE with background removed, then created camera via Cine4K. Now just trying to figure out how to pullo it all together! thanks again@@JoshuaMKerr
Really interesting channel man! Much appreciated!
Much appreciated! Really glad you enjoy the content.
Had no idea this was possible. Thank you so much!
No problem!
Thanks for the vid. how do I do this on scale for multiple files accessing by date and placing a timeline in 3d space?
I'm really not sure. Sounds like you have a big project in mind.
well I just want a programmatic way of drawing my media file pics and videos into a timeline that I can walk over so people can see like a better death video @@JoshuaMKerr
Hello Joshua , thanks for the tutorial ....is there anyway to color grade the image in UE5 ?
You can make adjustments to the image plane using the material editor. You can also do a more traditional grade with the colour settings in the post process volume.
@@JoshuaMKerr Thanks for the fast response . are you on Patreon ?
I will be soon. Ill take that as a compliment though ;)
First time Unreal Footage looks not like a game or animations and more realistic. Great job. Will try out rendering with Octane. Now I am doing live capture with Off world. Very cool way to do Virtual Production
Ooh. What is Off World? sounds interesting
@@JoshuaMKerr it is a plug in - OFFWorld plug in. I use it to stream out of the Unreal. Instead of using sequencer and rendering shot by shot it streams & help capture video real-time . But it comes with less flexibility that you mentioned in this video. I guess I will do a video on this plug in and tag you. It is a really cool alternative to sequencer, but not 100% replacement
Ah I see. Yes definitely would be interested to take a look. Although I tend not to render in realtime for the sake of quality.
@@JoshuaMKerr Was my concern as well till people suggested recording as lossless file, which leave me with huge files 😂 but quality is good
@@KonstantynVL Thats fair enough for colour depth but it wont give you the benefit that multi sample renders get.
Thank you! Can you export in synch sound like this?
Good question. You can add audio to sequencer for certain and I have definitely seen the option to export the audio through the legacy movie capture option. Also although I haven't tried it movie render queue has a .wav audio option too.
Thanks for the tutorial! One question. The media in my scene always plays back and renders really roughly around the actors key (the file is correctly keyed) any ideas on how to improve in engine quality or final render quality?
You migth have the material set to masked rather than alpha premultiplied (or translucent).
This was awesome, super helpful. You deserve way more subs!
Thanks man. Knowing it's helped you is enough. Im sure the channel will grow too :)
@@JoshuaMKerr Just curious, have you had any issues with fog and inserting pre-keyed media? Mine always renders in front of the fog no matter where it's placed. Not sure if it's a limitation with the engine.
Theres a few settings to try deoending on the issue. Look in the material for 'compute fog per pixel' then theres also 'allow fogging' (I think). Output velocity might also help.
Thanks for the tutorial
very welcome
I love UE but to import prekeyed footage is overly complicated when it doesn’t have to be. They should have something similar to Blenders import images as planes. One click and it’s all set up for you. They could easily do this in UE…..rather than having to jump through all these hoops and blueprints. It could all be automatically done and ready to go.
I hear you. considering Epic want to be taken seriously amongst filmmakers you'd expect this would be a feature.
@@JoshuaMKerr Exactly! Hopefully they will implement this sooner than later! Thanks for replying.
@@JoshuaMKerr ok you prob know but in 5.1 it’s basically just drag and drop now for prekeyed footage and works out if the box 😊, only thing is it looks like specular is too high and can’t find how to change it in 5.1. Have you given it a bash yet please?
Yes I read about it but am yet to give it a proper test. No doubt theres a video to be made about it.
@@JoshuaMKerr yeah it’s def much easier now. Just got to figure out what video formats are supported as some didn’t work, and before I could click on the material and change specular etc (at least I think that’s why the video looks washed out) but can’t figure out how to do this in 5.1 yet but so happy they have streamlined this process.
You are a life saver Joshua
Glad it was helpful
Very helpful, thanks, I have one problem, as soon as I apply the keying the plane brightness is not changing with the directional light anymore, it will remain always the same bright, before applying the keying playing only the media player on the plane everything is good, changing the directional light affecting the plane nicely. If I place a point light that affects the plane but the directional light not. Any tip, why's that?
That's a tough one to answer without seeing your material setup. I have a new discord server that ive started to help people with their virtual production problems. If you join and post your problem in the forum it might be easier to figure out. Here's a link:
discord.gg/C4RtzVUU
hmm how to cast reflections in something like a mirror?
Good question. Describe the shot and angle, what do you want to see?
@@JoshuaMKerr hey, I have a total shot, and the subject stands on a mirror floor or rough tiles and there is no reflection on the floor, shadow yes But no reflection
What reflection method are you using? lumen, ratracing?
@@JoshuaMKerr i Use a plane witch is, transculate and there is a movie showing, a keyed out render, i tryed both, Lumen and reytrace But if i change to reytrace, the shadow cast the whole Plate and not the subject only, i guess i need to make fake reflections by douplicating the Video and hide it in game
Interesting. Ill have a look into this further.
Is there any way to adjust the the color (brightness, contrast, hue, sharpness) of the EXR media on the plane inside UE5?
There is... but it's not very good so it's better to colour correct in another package first.
In your material you simply multiply your footage by a vector3 and select what colour you want to tint your footage with.
Did you ever get into relighting live actors/video in Unreal - but with 3D depth of their face? Currently exploring this. Tried depth maps via volumetric video with a dual stereoscoptic lens but the depth maps didnt have enough depth for the face to appear fully 3D in Unreal. Thinking about trying ARkit livelink face app and combining the mesh with the video feed using a HMC. Or other method you might recommend? Thoughts?
Oh my. Stay tuned! Releasing a video on this next week.
@JoshuaMKerr o
Oh jesus! LOL. You have to at least give me a hint of the direction you ended up going so I'm not wasting time this week. :)
@@brettcameratraveler although it is unfortunately all pre keyed footage for the moment. There's no quality real-time solutions yet
Real-time would be a major plus, but I was more asking which route/method in general did you take this week to give your video footage of your actors face a 3D dimensionality so that it could be relit by Unreal in a very realistic way? (I mentioned two ideas above)
Yes of course. Look at Switchlight. That's all I'll say for now
i have a question, i want to use prekeyed footage with a translucent material, i can cast shadows but i want to cast lightshafts if the actor is moving? how to do that? maybe u can help ?
best regards
What method are you using to cast the shadows with the plane. I think you can cast lightshafts when not using raytracing.
@@JoshuaMKerr hey joshua, i always (sometimes not but often raytraced ) use Lumen, if its possible in the scene, BUT - the shafts are not showing, only if i change the blending mode to masked, but the problem with masked is , the edges are very wired. and have some pixel even if i export it wit no antialiasing and 16 samples for example. and the other problem with masked is - my footage looks choppy - it looks like the fps is not matching so ,for ex. 25 fps is used in my opinion.
@@JoshuaMKerr and i forgot, I turned on Volumetric fog - so the shaft is showing on all the Masked objects
Go take a look at my video 'Crazy Shadow Trick'. I think the first part will help. Let me know
@@JoshuaMKerr I think this will work, BUT i dont want to do this on every shot 😀 this will take to long, is it possible to duplicate the prekeyed plane > turn it to masekd > hide it from scene except the Shadows and Shaftst? if yes how to do this?
This is an awesome video!
I’ve just come across your channel and it’s very helpful!
I know this is a year old but I have a question. If anybody could help it would be very much appreciated.
Basically, everything works perfectly in sequencer but when I render it out anything I have through a media player/playing on a plain just comes out showing the alpha channel and not the actual image sequence I see in engine??
Any ideas what I’m doing wrong? Cheers
Hi there. Interesting problem. I think it would help to see your material and sequence setup. Have you joined the discord?
@@JoshuaMKerr hello mate, thank you for the reply and so quick too! I posted just before heading out for work so I’ll jump on the discord tomorrow now. Thank you again and keep up the great work!!
it's great !
Hi josh! Great video! I followed it step by step and got the same results. However, when I render the movie out, I get the black plane and not the transparent. What tid you do to maintain your alpha when rendering the clip? Thanx again for a great video! 🙂
Never mind. I got it. It was the last bit that I messed up 😀
Glad you sorted it.
I wish it was a simple drag&drop a video file...
yes that would be good. but image sequences are best for unreal Im afraid. this may change in future
This was great! I was finally able to get some prekeyed footage into Unreal on a Mac...just to see how absolutely terrible it plays back on this machine haha. Curious what are the specs on the machine you are on right now? Seems like it's playing back well. I've been trying to hold out for GPU prices to drop, but may just end up giving up on that and going the prebuilt route with the prices right now.
Until recently I was working with a 2070 which certainly did the job. You can allocate more ram to image sequences in unreal settings. might cover that in the next video.
@@JoshuaMKerr Awesome, thanks!
Hi how do you render this out, i did the same thing with pre key and import to UE as a media player like you did, but when i render with movie render queue, it doesnt render the media player, its just a black box for the plane with media texture
It should work through MRQ but it sounds like your sequence isnt playing. Is it set to auto play in the sequence details panel?
@@JoshuaMKerr How do you do that? please help me out i just start out with unreal engine, thank you
@@nk0599 If you're completely new to unreal engine I suggest reading the unreal docs. Here's a link to the info on level sequences.
docs.unrealengine.com/4.26/en-US/AnimatingObjects/Sequencer/Overview/
Go josh!
Thanks man.
When i import my green screen keyed footage i always end up with this black outline on the subject. But in after effects it looks clean did you ever have that problem
Have you tried with exr or perhaps png? Are you in 5.1 also?
@@JoshuaMKerr yes I e been doing exr in after effects my key is great but when I bring it into unreal there is this slight black outline so frustrating. That’s why I usually stick to Element3d but I love unreal
I remember something like this. Just out of curiosity, could you try two png sequences? One rgb and the other an alpha matte. So you end up with two media tracks in sequencer. I feel like it's an exr issue.
Hi May I know if there is any way to soften the keyed edges, something like edge blur so that the edge will not look so sharp?
I do all of my keying in after effects before bringing in to Unreal so theres loads of tools for edge feathering etc.
Unreal doesnt have tools like that.
@@JoshuaMKerr I did my keying in AE too and rendered in PNG Seq but when import into UE, the edge are sharp and seems like cutout when composited. The feathering seems not working in UE.
Are you using a masked material or translucent?
I'm using masked material. Seems like masked material will be able to cast shadow.
I thought so. Masked can cast the correct shadow but cant handle semi transparency. You need a translucent material for any kind of feathered edge.
Thanks for the tutorials, the following is happening to me, I do everything as you explain in the previous video, I save the EXR images, then I load them in Unreal from Img_Media_Source and they load fine, but when I hit Media_Player to load the sequence the program closes .
I also tried with .PNG images and the same thing happens to me, what could it be? I am working on UE5, I appreciate if you can give me some light.
Greetings!
Hi Raul, Im not working much within UE5 at the moment so will let you know if I experience this. As UE5 is not a full release yet it might be best to work in an earlier build.
"Make sure you hit save so if it crashes you have it!"
me: "My ue5 never crashes. I'm too cool to save it.
5 minutes later...
crashes.
DAMMIT!
I feel your pain.
Hello! I've tried to do the same thing but with UE5 and the Actor inside the material I create is always on focus, no mater what. I tried to change the material in not transucent ( only for test ) and it worked. But as you know you need the material be transucent. I couldn't find a solution other than fake a little bit the defocus in post or place the actor directly on davinci fusion and do the post-production there. Any suggestions?
Check the material settings. There should be a render after depth of field dropdown option in there. Also experiment with output velocity.
@@JoshuaMKerr couldn't find any depth option and the output velocity was there but did nothing :(( thank u aniway. Are u using this workflow with UE5?
Looks like the option is now called 'translucency pass'. I do use this in UE5
@@JoshuaMKerr Great! It worked! Notice that it works also with the masked blanded mode( playing with opacity mask ) but it has different reactions to lights. Can be usefull too. Thank U man! Now I'm trying to study the perfect workflow between this one and the other with post production directly in Davinci Fusion with the render in UE only of the background.
@@TonysMind Welcome down the rabbit hole of media planes. I have found Octane to help a lot if you're not worried about realtime.
Hey Joshua i have another question, do u know how to face the plane from prekeyed footage always to the camera? with a specified distance? for example if i move the camera the plane is always showing to the camera
Example i have a greenscreen footage with movements inside the footage SO - i dont want 3d track the footage - i want to put it on a plane - and lock or link it to the camera inside unreal 5 - so if i move the camera - the footage should always stay facing to the camera , i hope u understand :)
Do you need the footage to always fit the camera view?
@@JoshuaMKerr yes
There are a few methods but I think this one works best. Go to the material editor where your media texture is and add a 'screen position' node then hook it to your media textures UV input. This fits the video to the camera view. Then add a 'sprite' node and hook it directly into the world position offset. This makes the plane camera facing.
So back in the world you just need to position the plane and scale it so it's always bigger then the camera view. You can make the plane huge if you want, the video will always be scaled to fit the view.
Might make a video tutorial
@@JoshuaMKerr Thank u very much i will try
Thanks a lot. That’s so good tutorial
Glad you enjoyed it. let me know if you have questions.
This is so cool!
Thanks!
I LOVE YOU 😆😂 THANKS
Aww cheers
how did u render the sequencer in octane? i tired but not seeing that option in render setting. can you pls help here
Hi, yes I rendered using Octane through the sequencer. With the octane plugin active in Unreal there should be a dropdown in the sequencer export settings Octane Render(image). This will export an image sequence in a format of your choosing.
Hey johsuar i have i Problem wich is causing me headache since days, i have a sequencer scene with 5 Media tracks wich have 5 Different img sources, the preview is very very laggy in th sequencer, my fps Drops from 60fps to 0 and the sequence is almost freezing so its almost impossible to Set key frames to camera moves, what can i Do? The img sequence is, 720p png sequences with alpha masks, (fire Video) so i have beeb searxhing on Google why the fps is dropping maybe u can help?
Tricky one to solve. Sequencer can be a bit laggy in my experience. I would suggest you try going into project settings > Img media and experiment with raising the cache memory amount. Let me know if this helped you.
@@JoshuaMKerr thank you very much 🤟🤘🤘
Did it help?
Hey johsuar, yes it did
@@JoshuaMKerr on 27th November u will See the results of my first ue5 music video, Stay tuned
Does anyone know how to color correct a green screen transparent media file? I tried using the color region, but for some reason it doesn’t affect my green screen plain. It works perfectly on anything else in the scene.
I have tended to make any corrections before bringing in the img sequence. Otherwise I do make tweaks to contrast etc in the material itself.
@@JoshuaMKerr Do you have a video on how to make corrections to the media material? I understand that I’m going to have to really dive in and learn blueprints.
@@devillefilms8244 I have steered fairly clear of blueprints so far but the material editor is very useful. I don't have a video on this yet though I'm afraid.
Seems like I have trouble with the transparancy with my source media. I see all green around the runwayml rotoscoped pngs inside the joshintrotex window. Not the chesslike little black and white pattern. I exported the mp4 from runway out of premiere pro. as a png sequence with alpha channel. Any suggestions?
I've never exported my media from Premiere Pro, only after effects. I'd suggest joining my Disord and posting a video of the issue in the support forum.
@@JoshuaMKerr Thanks, Joshua. I had to use a UltraKey Filter and tell PP to set green as transparant. Export png frames with Alpha Channel.
Keep up the brilliant content.
What if the shot is not on tripod?
It is possible but the shot would need to be matchmoved either in after effects of Blender then imported to Unreal. I am experrimenting with this for a future video. So far I have just been using tripoded shots and adding camera shake in Unreal.
@@JoshuaMKerr Would love to see how to do it once you crack it😄🙏🏽
me too
how do you fix no shadow
Is it a translucent plane that doesn't have a shadow? Are you using raytracing
@@JoshuaMKerr i am using raytracing and its a masked plane
@@aerick1 Is the c as t shadow checkbox selected? Also check the light source shadow settings
@@JoshuaMKerr i already have and the only thing that seems tow work is when I check dynamic shadow. but when I do this, it casts a really janky shadow
Thanks!! Suscrito! 🙌🏽
No worries
I have a problem when it comes to shadows. How do I get the two sided material to cast shadows, which actually move according to the movement of the mask. When I tried to render drawn shadows, it took the mask from the first image and took it as shadow, the shadow does not change when the image sequence moves on
+If I play with focus and out of focus area with my camera the plane with the media on it is always in focus and never blurred. How can I make the media be blurred as well when everything on the same distance else is blurred with the focus function of the sequencer camera
You might want to take a look at my video about shadows on the channel. Look for Crazy Shadow Trick.
The depth of field issue is quite common and to be honest UE still has a way to go to get it all working nicely. Look in the material for 'output velocity'. If taht doesnt work there is also an option in the material called 'translucency pass' ... I set this to 'Before DOF'
@@JoshuaMKerr Thanks for answer. The "before dof" thing works. But it makes niagara systems and emitters that are positioned behind the keyed img sequence seem as if they were in front of it
have a look at the translucency sort priority setting in the objects details panel
@@JoshuaMKerr I found an own version for the shadow. I made a copy of the same material and that copy I changed from translucent to masked (because on masked the shadow works). Then I copy pasted my media plane and put the masked_material on it.
Only problem is: I have to place the masked object 0.001 m behind the translucent object so that you see the translucent one affected by light, but get the shadow of the one behind.
Is there a way to take the masked object or take any object and hide the mesh but let the shadow stay?
I found the solution! I take a shadowpassswitch: On the upper knot I connect a texture which is 100% transparent (so I have no colored mesh) and on the bottom knot I put the media texture, so I have the shadow
I really hope that you still replying to the comments to this video. So during rendering it's crashing and it comes with " GPU crashed or D3D device removed. There was only one successful attempt when it rendered fine. It crashes only when i render video with green screen. Thanks.
What is the resolution of your footage and what format is it?
@@JoshuaMKerr exr , resolution 1280*720. My RAM is only 8.00 GB, maybe it's not enough for unreal engine and projects like that? I was told that i need at least 16 . Thanks for respond btw
8 should be totally fine. I don't even have 16
Hello
How do you do to have your audio file included in your rending and export please ? i DID EVERYTHING I HOPE RIGHT BUT WHEN I RENDER no matter which format i don't have audio vith ( audio of my original video +
tHANK YOU
My video was rendered as a png image sequence so it was synced later in premiere pro. It is possible to render a video with audio I think. Have you added the audio to the sequencer?
@@JoshuaMKerr
Yes i know that but i understand well now .it is better to synch your original audio sound in a video software and for audio that you add in UE export it from UE.
Thank you
Great thanks for sharing
no problem. any questions let me know.
hi my mind is currently boggled im trying to set a character into the distance like you have here me walking along a cliff, i have keyed it in after affects (great tutorial btw) and now i have the footage in here on the unreal plane and its playing back perfectly fine. Now i know there are 2 things, take recorder and the sequencer. Basically i now want to be able to record the virtual camera looking at the plane so i can move the camera while the image plane is playing in the distance (so its all in sync when i play it back), it looks like i can do that through the sequencer, i have tried hitting record on sequencer and its just confusing me. Could you help me understand how i can record the live virtual camera movement in real-time (im using live link which works fine ). and then when i play it back I want it to have the virtual camera movement i did (through live link iPhone) and the image plane recorded in sync. so if im walking along a cliff the green screend image of me is married up to the camera movements i made.
Hi, you're on the right track. Add your camera to take recorder and record the camera movement. That will capture the animation keyframes. You can then go into your recorded take and copy the keyframes. I would then go to your sequence where your media plane a d camera are and paste the recorded keyframes in there. Let me know if this makes sense.
I have a discord where people can ask questions if that's of interest.
Does it also work in Unreal Engine 5?
It certainly should.
Thank you!
Let me know if you run into any problems
Just use aximmetry
I am looking into Aximmetry at the moment. However I do like the results from rotobrush and keylight for a deferred render pipeline.
mine just plays a black screen
Maybe join my discord and post your issue in the support forum.
My plane becomes black while rendering. Please help
Do you have your level sequence set to auto play
Solved! i had dropped material texture on different place@@JoshuaMKerr thanks for responding bro. learned a lot from you
What's you pc setup atm? Thanks
Im using a 3080 GPU right now. Maybe 64gb ram.
@@JoshuaMKerr nice! I'm using 3060 12gb, 32gb ram. It runs okay lol