TRACK THIS! - Camera Tracking From Blender To Unreal Engine
HTML-код
- Опубликовано: 1 авг 2024
- Camera Tracking From Blender To Unreal Engine
Intro 00:00
Strat Tracking In Blender 01:12
Cleaning the Tracking 09:01
Solving The Camera 10:51
Send Tracking To Unreal 15:31
Scaling The Camera 19:26
Aligning the Footage 22:40
Bringing Scenes To Life 25:19
Casting Shadows 26:30
Side By Side 28:48
Blender For Unreal: github.com/xavier150/Blender-...
In this video I use some 'tricky' footage to camera track and matchmove using Blender. Then I send that tracking data to Unreal Engine to build a 3D scene.
The footage is tricky because it was not captured with vfx in mind. The shutter was only set at 1/50 meaning the footage has some motion blur, the movement is a bit shaky and also there are no useful tracking points in the footage either,. All of this makes it difficult for the tracking software in Blender to find good track points.
I originally tried tracking the shot in After Effects which gave a pixel error of 1.50. This is generally too high for any matchmove with the better results being less tha 1 and the best under 0.5. This is my attempt at getting a better result as Blender gives more control.
Once the tracking is done I bring the data to Unreal Engine and show how to set up the camera and footage so it sits nicely in the 3D space.
🔥BECOME A PATRON - for exclusive content and credit on new videos
🔗bit.ly/3pcBuBn
---------------------------------------------------------------
😁 Subscribe to Joshua M Kerr
🔗bit.ly/3l4U9wo
---------------------------------------------------------------
🎦 JOIN THE NEXT GEN FILMMAKERS DISCORD
🔗bit.ly/3kUzqvj
---------------------------------------------------------------
👀WHAT TO WATCH NEXT..
📺 An Unreal Engine Short Film In Three Days
bit.ly/3DwvPtI
📺 The Smallest Virtual Production Studio
bit.ly/3WW3KCY
📺 Virtual Production Playlist
bit.ly/3HnUeDe
---------------------------------------------------------------
I can feel your spirit to figure out things your own way, being creative, thinking outside the box. Thanks for giving me some ideas I've never seen anywhere else and probably never would've come up with myself!
Good luck with your creative work. I would love to see what you make from your new ideas.
The amount of information you needed to absorb to get this all working... amazing.
This was great to find! I've been pushing learning tracking through blender for too long and finally learned through this video. Great work on this!
Wish you a very prosperous year mate.
Looked all over for this workflow using free blender and free unreal as apposer to cinema 4d, adobe after effects etc and couldn’t find anything until I found this video.
Your willingness to relinquish detail and showing clearly the steps involved including the buttons to press enlarged on screen is unprecedented!!
Your channel is way underrated, and my wishes for you at that you x10 your subscribers this year!!
Thank you so very much for this. I have subbed and will be checking out all your other videos in the coming days and weeks.
All the very best and again, much much appreciated! 👍👍👍👍
Thanks for this comment. It's super cool to hear your thoughts.
Im glad you enjoy my videos that and the tutorials have been useful. Im sure there will be more tutorials to come and lots more crazy short films.
Thank you
your tutorial is BADASS!
Thanks!
Great work 👏
Thanks cous!!
You are brilliant sir!
Thanks! Hope it was helpful.
Thank you so much for this!
No bother. Hope it works for you
great job!
Thanks!
thank you so mach ..
great tutorial
Thanks
I just tried it. it worked! Go to scene, units, and set to 0.01
Brilliant!
Super helpful, Joshua! Thanks alot. I'm using in the moment Syntheyes as a tracker and go directly to Unreal, but great to see your workflow. I'm also using Blender. Love it.
Glad you enjoyed the walkthrough. I haven't used syntheyes, how have you found it?
@@JoshuaMKerr hi, I was working in a matchmove department for almost 2 years back in 2008-2010 and they've just it heavily. It's a cheap genius matchmove program. 🔥💯
@@JoshuaMKerr but I gotta say also, that I'm using more and more Blenders Tracking Features. They are awesome, too.
Thats a top tip. I might have a peek into that.
nice dude!
Glad its useful
Nice man, that was exactly I was looking for, hope I can replicate your workflow and somehow use it in my project ;)
Any time mate.
Great 🔥
Amazing!!!... Thanks
Glad you liked it!
Handsome! Thanks for the lesson!!! Useful chips that others don't talk about!
Glad it was helpful!
Excellent video! Thank you for sharing this knowledge. Except for the Octane stuff at the end (it's not longer free and only available as subscription, right?), this is very useful and quite inspiring.
Watching it after the CGMatter tutorial series on Blender 2.8 Motion Tracking, so it makes a lot of sense now! Parenting the camera in UE5 to an Empty, and scaling that was the missing piece of this puzzle.
Hoping you'll do an updated version of this video for UE 5.3 and Blender 4.0+ without Octane, and using the send2ue plugin provided by Epic Games. Also, _please_ without distracting background music.
Thanks for the feedback, I might do an update of this video. Also, Octane remains free.
Nice one! My cat is all grey with fold ears, not sure if trackers would even stick to her at all haha
Yeah cats are tricky subjects haha
Great tutorial, many thanks for uploading! Do you think this technique would work for tracking camera movement while filming a subject that is moving, for example, a person who is talking, and moving their arms and head about? If not, would you have any tips on how we could do that? Cheers!
It can absolutely work, because you can delete any points that try to track the subject so the camera solve is more accurate.
This is something unreal needs to improve on. Tried C4d, blender, AE, Maya, but there is too much guess work, especially with bigger movements, flaws shows up pretty quickly. I guess its why people prefer tracking apps which are accurate enough to then adjust afterwards. I did read that someone used 3Dvisualizer to cinema4d to datasmith UE and got what I consider perfect results. But since those are expensive softwares, I never got around to try them.
dude honestly, how did you even figure this out. ur so cool
Really glad you enjoyed the work I'm doing. It takes a bit of figuring out, I just want to make cool things and help others do the same.
i wanted to make an impressive virtual product then someone recommended me this and im so grateful reeeeeee
Спасибо большое очень спас
You're welcome
this is a great tutorial! have you used Byplay at all? would be good to ee an actual tutorial that takes from Byplay into unreal (as I can't find any!)
I haven't tried but it looks interesting! It really comes down to whether the tracking data is good enough and translates well from the phone to cine camera footage. Otherwise the process of going from blender to unreal would likely be the same.
how can i export camera + obeject at the same time while using this addon
I think if you select the object there will be an option in the plugin to change it from 'auto' to 'set recursive'.
is this method still a good way in 24? or are there better ways to do it?
Should still work for traditional matchmove tracking workflows.
I cannot select the tracking curves in blender in order to clean them up. how to enable curve selection?
Sorry, Im not sure offhand, I dont remember having to do anything special. Perhaps someone will jump in and help with this.
hey man i keep getting this error message when I import the camera track to the sequencer: "Failed to find any matching camera for CineCameraActor. Importing onto first camera from fbx Camera3D1" what is this??
I haven't come across this error, sorry. It's been a while since I've used this workflow.
Hi Josh, love your content! A life saver for transferring motion tracking data to UE. A question tho! Whenever I save the UE project file or begin exporting cinematic sequence from UE, my cine camera sequence unlinks itself from the Empty “Camera Mover” Actor and resets. How do I prevent it from repeating?
Hi. Ive had this happen before but im not sure what solved it. Have you tried deleting and re-creating the mover and camera.
Have either of you had any luck figuring this out? I'm trying to use a process similar to this , (importing an FXB from blender for camera animation,) and have been trying to use an actor to properly scale and manipulate the move in my scene. But every single time I go to render, the camera resets its position and comes unlinked just like you said. I'm looking all over for a solution.
Solution: Create the actor, camera and plane outside of the sequence itself and link it afterwards and then import the tracking data to the camera you created before.
hope it helps!
cheers
small addition: theres a small yellow lightningbolt indicator in the outliner, whenever you've created a 'spawnable' actor inside a sequence. With these it does not work for me, or i haven't figured it out properly
@@doc4678 Thanks for the response! I've been importing the FBX tracking data to the camera inside sequencer, to clarify, would you add the camera to sequencer after it's linked to the actor and then import the fbx? Or would you be doing it another way entirely?
Hello. After connecting the camera to the actor and scaling it, I have a problem with RenderQ. The camera comes out of the actor sub-category and it is the same size as the first one. can you help
I would try creating a new camera and copying the keyframes from your imported tracking animation to a new level sequence.
@@JoshuaMKerr Thank you . Please make a camera tracking tutorial in Blender and a camera set in Unreal. Of course, with rendering in renderQ
is this why background sets for chroma keyeing has "+" in them?? do they use that instead to track their uhmm virtual cameras
That's exactky right. And this is prety much the process of matching the camera motion with your CGI.
Great video! Have one question, after saving scene or starting to render, my Camera from blender detach from Camera mover, and scale is not correct, maybe you know how can i fix this attach? Thanks
I think you need o set the unit scale in blender before bringing it into unreal. About the camera detatching you could try linking the objects in sequencer instead of the details panel.
@@JoshuaMKerr Thanks, will try this!
@@JoshuaMKerr wow it works, attached in sequencer, thank You Joshua you made my day!🤌🏻
No worries. Let me know if you solve the scale issue
@@JoshuaMKerr solved
Update. Found it!
HI Joshua. Thank you for your videos. You do the 'wading through and getting out the other side' really nicely. I'm super hoping you can help me with a question. I'll ask and hope you can answer! I create my camera using ae2blender but when i try to export it to Unreal I a) can't see the camera so i can't 'export recursive'. Do you have any idea what i've done wrong? It's a super simple camera move. Thank you, Joshua. Dave
Haha no worries, Thanks for the kind words. Just in case anybody is searching through comments how did you solve this?
@@JoshuaMKerr I'll hadn't clicked on 'object'.🙂. Incidenlty, unreal5 ... i tried pasting the camera data and nothing seemed to happen. The 'scale to 0.1 feels arbitary. I'm gonna try and find a more precise way to bring in a camera or have you already done so?
how to attach a video with a moving person to the camera tracker in Unreal Engine 5.3
Assuming I understand correctly you should just drag it onto the camera actor to parent it.
Why your refine 11:27 can select length K1 K2 ? My menu is only checkbox can be select.
I'm not sure to be honest. What version of blender are you using?
@@JoshuaMKerr I'm using the 3.3 .
@@pomeloyou Looks like it may have been removed.
hi, i have problem when i creat actor to control camera seq(location, rotation, scale), it still ok then i CTRL+S camera unlink form Actor and return all transform when i import. How to fix that
What version of the engine are you using?
me too here
Hey! This is great..as a Maya user how you imported the footage into unreal?
Hi Salman.The footage was exported from after effects as a png sequence (with alpha) and imported as image media.
@@JoshuaMKerr I'm having problem, its rendering but only the skybox not the footage. I've downloaded Octane Renderer for Unreal but its taking hours..is there way to render from Octane for limited frames? Cheers!
Yes, you can set the frame range in the sequencer export options.
@@JoshuaMKerr That's the thing, when I use the Sequencer it only exports the Skybox but when I use the Octane Render it renders the footage,,
@@JoshuaMKerr Hey..are you on DIscord?
when I parent my cinecam to the empty actor, and move the empty actor to the position I want. The CineCam immediately jumps back to the original position after playing back in sequencer, unaltered by the empty actor. The transform controls in the cinecam are all already set to relative instead of world. Any ideas?
Maybe try creating a new level sequence and new cine camera actor. Then paste the camera transform keyframes across from the imported sequence to the new one.
@@JoshuaMKerrI managed to get it to work, I was thinking of the coordinates all wrong 😅
Aha! good work
@@wjm123 I've got the same issue... ho did you fix it? Thanks!
@@simonevrech4870 hey! you might be doing the same as what I was trying to do by placing an empty actor, copying the location of the cine camera to the empty actor, so that now the empty actor and camera are literally on each other, then moving the empty actor to 0,0,0. And by doing this you realise that the cine camera just jumps back to the original location.
All this is happening because the sequencer already has keyframes for the cine camera at specific co-ordinates (for example. 100,100,0), and if you notice, when you put the empty actor exactly on the cine camera and you parent the cine cam to the actor, the cine camera's co-ordinates will now become 0,0,0 because its 0,0,0 relative to the empty actor. Hence, when the sequencer starts play back, its gonna jump to 100,100,0 RELATIVE to the empty actor.
Once you understand this, you'd know where you went wrong. TLDR: you could just fix this by
1) going to the first keyframe in the sequencer so that the cine cam moves into the first position,
2) then place an empty actor to the location where you'd want the camera to be
3) Parent the cine cam to the empty actor
4) copy and paste the relative location of the cine cam to the empty actor but flip the sign (ie if the cine cam was 100,100,0 put -100,-100,0 for the location of the empty actor)
now you'll have the cine cam moved to the location you want. That how I'd do it, not sure if there's a better way. hope that helps :)
Hey josh. Have you seen the error messsage saying UniScale for blender must be 0.01. Maybe this is why your camera is coming in so big?
Been a while since I tried this but good to know you have the solution.
will it works on unreal engine 5 ?
I haven't tried but don't see why not.
after installing the plugin, i still don't see the unreal option on the right after selecting the camera, Ami doing something terribly wrong here wrong?
The usual suspects are the version of the plugin and version of blender. maybe check those first. Then double check the addon is active.
@@JoshuaMKerr I have a feeling he was talking about not having the unreal engine tab on the side of the screen after installing...Only reason I'm assuming this is because I was stuck on that part too lol. But the solution is to hit "N" to bring up the tabs, and Unreal engine should be on there. I'm a noob, so it's an understandable thing to get stuck on.
Very cool...but such an inconvenient workflow 😬, all the steps and not having the scale match at the end is a killer. I've been liking camtrackAr which can send the app's live camtracking fbx directly to UE4 without any fiddling... It seemed like bledartrack was an even tighter live tracking system, but one is still stuck with the inconvenient fbx export to UE4...
Its just one of many tools. They all have drawbacks and plus points but this does offer extra flexibility especially when live tracking data messes up...which does happen. Good to have a fallback option.
@@JoshuaMKerr Yeah, I still can't get camtrackar to work clean enough, blendertrack app seems to work well, but has the import problems.
I feel you. Its really always down to what level of accuracy and quality you can live with. I dont mind spending the extra time to push it as far as I can.
@@JoshuaMKerr I was noticing at 17:09 that your camera's scale in Blender was at 1.812. I believe this might be where you had scaling issues when transferring to Unreal. Did you ever apply scale to the camera in Blender to reset scale to 1 before sending To Unreal??
Is it still working with Unreal Engine 5? - And the latest version of Blender.
(I know a lot of people looking for this answer.) Regardless, thank you for the tutorial.
It shoud work with UE5. The plugin's latest release was for Blender 3.5
Hi there.I'm wondering how to change the camera's worldposition in UE. I want to focus the object in the other position but not the (0,0,0),how can i do this in blender or UE?Thanks
Can you give a bit more detail. I dont quite understand what youre trying to acheive. If its just to change the origin then you could parent it to an actor somewhere else in the scene. Sorry if thats not what youre after.
@@JoshuaMKerr Thanks for reply.I had tried the attach the CineCameraActor to an actor,and move the actor where i want. But when i start previewing LevelSequence,it doesn't work as i expected.(For example in UE,on the beginning,the CameraActor Position was(0,0,0),Rotation is(0,0,0),I move ed the CameraActor to the Position(10,10,10) Rotation(30,30,30).And clicked preview. The CameraActor went back to (0,0,0).
@@JoshuaMKerr Also,before export the camera in blender,i try to modify the "Additional location",but it seams not working.🥲
Are you setting the camera transform in the level sequence? Sometimes deleting and re creating the camera cuts layer can help.
@@JoshuaMKerr I delete the imported camera cuts,OMG,it works!Thank you so mush🥰
whenever i pate the cmd it accepts it and is green, but doesnt trigger the save sequence as pop up. what am i doing wrong?
Do you have all of the correct plugins installed into unreal as shown. I also came across this problem when using an earlier version of blender for unreal engine. Make sure that blender addon is also up to date.
@@JoshuaMKerr hmm, I'm using blender 2.81 (sub 16)..is that too old?
@@JoshuaMKerr and the add on I'm using is the v0.3.0
This should be fine but you could try with the latest blender too. Version 3.0 of the addon worked for me.
I was using unreal 4.26 and also 4.27. Check that the plugins you need are enabled.
@@JoshuaMKerr thanks I'll give that a try
Does this work in UE5?
I dont see why not. I haven't tried but cant see a problem.
@@JoshuaMKerr I just didn't know if the plug ins worked the same way.
@@treythesmall6723 Let me know if you get it working.
19:25 is a deal breaker
There may be a way to get the exact scale but I'm not sure how it's done. I prefer live tracking to get that level of precision
Last one, Josh! ruclips.net/video/E06q2jjp4YY/видео.html Getting closer. Super useful for filmmaking if it works
Hi, sorry I've not been quick to reply. Lots going on on my end. So you have a scene imported from blender with a tracked camera plus the original footage. And you want to have the pillar in front of the video footage yes?
How is your video material set up. Are you setting the uv yo screen position or similar?
@@JoshuaMKerr Hi Josh. Thanks for replying. So I tracked the camera in AE. wouldn't take in Blender. Used your tutorial to get the camera data out but made UniLevel to .01 which abs. worked. Then ... made an image plate, locked it to the camera, made a material / image sequence etc. And so close. Now I'm trying to see 'through' the image sequence to accurately place actors. I'm trying to embed actors in a pre shot scene, so super relevant to film making. just in case you've come across this 🙂
@@JoshuaMKerr Not 'in front' but both visible, so a person can 'add' elements to a pre shot scene.
ur cat looks the same as one of my cats what
Is yours psychotic as well?
@@JoshuaMKerr no
Lucky
There's gotta be a better way to align your footage with the 3D track/scene. It's really sketchy to be eyeball matching the footage with the 3D camera. Is there no way to just make them exact without fiddling around manually trying to match it? This can't be the best way
Well you could create and export a plane or cube along with your tracked camera. That should help you align with a floor and also get scale correct.
I still don't get it why blender tracking is complicated
It's just a bit of a clunky interface. It's super powerful at tracking and complicated because of the level of control you get.
Never works in unreal
Been a while since I looked at this workflow. Might need to revisit this
Will this work in UE5.3 or not if not how should I do camera track in ur 5.3)?
That was too complicated in unreal...
I may have to take another look and make sure there aren't some easier methods.
@@JoshuaMKerr i tried your process and it didnt load my cam in unreal engine. it didnt read the script. any idea why? im running unreal 5.
This didn't work, it gave me a LogPython Error =(
LogPython: Error: Traceback (most recent call last):
LogPython: Error: File "P:/Video2023/Christine O'Malley/Blender/ExportedFbx/ImportSequencerScript.py", line 319, in
LogPython: Error: print(CreateSequencer())
LogPython: Error: File "P:/Video2023/Christine O'Malley/Blender/ExportedFbx/ImportSequencerScript.py", line 285, in CreateSequencer
LogPython: Error: camera_binding_id = seq.get_binding_id(camera[1])
LogPython: Error: AttributeError: 'LevelSequence' object has no attribute 'get_binding_id'
Its been a while since I mse this tutorial. Make sure you have a version of the blender for unreal plugin that is compatible with your unreal version.
@@JoshuaMKerr Yeah I did =/ I will try the older version maybe, the newest one was a preview version for 4.0 and I am still using 3.6