In my personal opinion, the more control and freedom to pick and choose after the fact filmmakers get, the more soulless films tend to become. This isn't necessarily a 1:1 relationship, because a filmmaker with a strong sense of vision will still focus on what he or she wants from the get go, but it does allow to start filming earlier in the production schedule and make what used to be important decisions after the fact and it also allows for a greater degree of studio interference (and if there is something that tends to suck the soul out of movies it's studio interference). I'm a believer in having to develop a fully realized vision for your film before you shoot a single frame. On the other hand, advances like these fascinate me and I love thinking about the posibilities they open up.
also one of the "small" things that make a big difference why crowd scenes in LoTR still look better is also their insane mo-cap library which was used for movement of pretty much any kind, any fight, any death, any interaction.. and all animation was done through those quality motion captures. while many modern crowd scenes, yeah they use mo-caps too (sometimes it is also a simulation), those mo-caps are not of the same quality. to understand the difference, there is a great video comparing Valve's Left 4 Dead zombie ragdolls with Back 4 Blood (it's been a while, but I think it was the one from Crowbcat). Valve made it of much higher quality and thus the older game holds much better in animation regard than most modern game. Or like when you see Uncharted animation vs random mixamo captures. And same goes for LoTR crowd scenes. Much higher quality of captured movement for animation and simulation. Same difference as with Avatar performance capture vs any modern movie. Avatar is all just captured (yeah, you have always some post-enhancements or clean-ups) and tries to maintain that reality. While other movies either rotomate or mo-cap and then re-aimate it or add animation that is unreal and make it look all spongy while mixed with something more real. That's also why even after 14 years, Avatar 1 seems more real. Because movements sells and grounds it much better than spongey squishy unrealistic movement you see elsewhere. With CG that can look better when paused, but shows intself as a trick when played. But Avatar shows that the movement just seals the deal completely and wipes out the unreality altogether.
Interesting perspective - I mostly agree with this although there are a couple things I disagree with too. You're 100% right in that a lot of the time what makes VFX stick out like a saw thumb is sub-par animation, especially in regards to humanoid rigs (because obviously we are human and spend most of our time observing other people so its easier to notice when it looks awkward/off) so for realistic actions accurate and clean motion capture data is optimal and can save a lot of time. You mentioned Left 4 Dead so I'm going to comment on that example first, for L4D1 Valve used and combined hundreds of motion capture takes for the many actions of the zombies, locomotion, hit reacts, death animations etc however the compromise of this is that a lot of it isn't cleaned up to the highest quality this is especially noticeable when you focus on things such as the fingers for example which stay in the same static idle pose and as far as I can tell aren't even animated - so yes Valve achieved much more believable animations but I don't think its exclusively because they had "higher quality" motion capture to use but instead they had a much larger variety of data to go off and use, also things can get a bit finicky when using video games as an example because there is a whole other technical process involved in actually implementing these animations, anim graphs, state machines, locomotion frameworks such as advanced locomotion system (which a lot of AAA games use now), etc. Moving on to your Avatar example one thing that actually sticks out in the first Avatar movie is that in some of the scenes when the Na'vi are moving at high speed it can actually look a bit awkward especially when compared to the most recent installation in the franchise - this is because the Na'vi obviously have much longer limbs than the actors in motion capture suits, so when for example an actor sways their arms really fast it can look oddly quick because the length from the origin of the pivot (the shoulder/upper arm) all the way down the arm to the hand is much longer - there is a particular scene when Jake (in his avatar form) jumps from the top of a water fall into the water down below which stands out to me the most. Also avatar isn't all motion captured - things such as creatures especially were hand keyed by animators at ILM and sometimes you don't get a good result from the mocap data for whatever reason so hand keyed animation is still involved. And that kind of brings me to my main point, I feel like you downplayed the importance of hand keyed animation in the VFX industry but productions such as Chappie, District 9, Jurassic world, The Mandalorian (the season 1 speeder bike chase comes to mind specifically) and of course the Transformers franchise all feature a large amount of hand key animation and visual effects studios such as WETA digital and ILM often require you to have experience with both motion capture cleanup and hand keyed animation because sometimes motion capture just isn't always an option such as if you're animating a physically impossible action or something such as a dragon for example. So motion capture isn't absolutely necessary - what is necessary is good animation and a team of talented artists can often achieve that.
@@Megasus-ju1mg thanks for the response regarding L4D, I didnt expect they would have an animation for fingers, i was focused on general movement, and imo, that huge library helps *a lot*, same as with LoTR. They tried to implement so many variations that it just looks really alive. And you are right, you need to do plenty of cleanup and fixes. But I think this is where the slippery slope lies? You need to do a cleanup, but it is possible to make it too clean to the point that movement are nice, but slightly odd or too smooth, or how to explain it. Or you clean some keys and then the interpolation will make it less real. Maybe you want clean it so there is a bit less data and jitter, but you can also loose some of that life in it. But I think that might not be the main thing. If you look at modern crowds, most of the time, when they walk, they have this rigid NPC style walk that you see in games, but not in reality and it looks more like they overlayed general animation to whatever character. Generic walk cycle captured by man is applied to women, walk cycle with arms wide apart so the animation fits to any kind of person, thin, thick, plenty of clothes, lack of clothes. I didnt mean to downplay hand animation. I meant to talk only about Na'Vi/Avatar mo-caps/performance-caps. Maybe I should have pointed that out. Because yeah, rest of stuff is hand animated because there probably is not a way to motion capture various kinds of fish (i guess one could, but.. why would anyone bother, if you can do a good hand animation quicker and believable) or birds or creatures, not to mention six legs or limbs or dozens of tentacles.. or infamous Smaug performance session that ditched the data and just hand animated the dragon. And pretty much any monster in the movie is hand animated. And I'd guess that most background apes in Ape trilogy is done just by hand.. and especially fight scenes are surely pure hand animation (also even if you have some reference, you still need to hand animate it). It is a neccessity and can be done so good that you dont even question it. (and there is also rotomation, which is pretty much matching actor frame by frame with 3D character). With mo-cap you are limited to reality, with hand animation, sky's the limit. Which brings me back to Avatar/LoTR vs many other movies and focus on human(oid) characters, beause that's when we notice it the most (and also who knows how many shots did have hand animated character in Avatar, cause something either didnt work or needed a big fix). I guess the main difference is, in the end, between tailor making your mo-cap to what you are actually needing, vs having general library of general movement for general crowd, so you can pull from that library and apply it to any crowd simulation you need. Elves? Put there the general mo-caps of stoic people. Orcs? Put there the general mo-cap of brutes. Walk cycle? Put there the general mo-caps of people walking. Instead of tailor making every mo-cap to the precise nature of your need. I mean, it makes sense, it saves time, money, hours, and it is a thing that most people dont notice or care about. But I think that is that one last percent difference that grounds it even more.
@@Patrix8558 Yee that makes sense, and yep you're definitely correct in a lot of scenarios motion capture is more efficient because as you mentioned its quicker and also you are "constrained" to reality since you know you have accurate data from someone physically doing whatever actions are needed in person. and your point about cleanup also has a lot of merit, at Ubisoft for example they don't actually delete any of the underlining keys from mocap date, they just mould tweaks around it by tweaking curves and using animation layers in a none-destructive fashion and I'm assuming thats how most people do it too - someone inexperienced with mocap can definitely have the tendency to "over clean" it when for realistic motions you need those imperfect arcs and slight micro movements. Just felt the need to underline the importance of hand-keyed animation in the pipeline as, in general, I feel like a lot of people (especially those outside the industry) often make the assumption that minimal work is required when it comes to realistic animation in VFX because "the computer does all the work" - not saying that's your interpretation but I just wanted to highlight that.
Honestly, I highly doubt the motion capture tech of over 20 years ago is that good by modern standards. You can clearly see it in the video when they show the animations they used. They are a bit floaty and not everything is cleaned up properly (it's just that it does not necessarily matter for tiny orcs in the back of the shot). The systems themselfs decent but nothing to write home about in this day and age. I think it is more the variety and the quality of pieces themselves that made those mocaps age well. Vicon launched a 1 megapixel camera in 2000, a 4 MP camera in 2004 and a 16 MP camera in 2008. I literally was in a budget mocap studio that still used old 0.3 megapixel cameras as late as in 2016. I am sure Weta could afford those premium cameras but today 1 megapixel is about the resolution of cheapest infrared motion capture cameras. But in the year 2000 it was the best of the best. In reality you should know the strong points of your system. Those 0.3 MP mocaps we recorded are totally usable today, especially for wide, violent motions-not so much for subtle acting when the jitter becomes quite apparent. The actors and stuntmen who can SELL the action are just as important. The suit does not magically make you into a great performer-if you make an amateurish attempt at moving in a specific way, it is often quite visible you are pretending. Stuntmen and actors experienced at staged fights will move way better than an average joe and will look much more intesting. And they can actually come up with interesting ways of beating each other up. The quality of applying the motion to the 3d character can also vary. In the previews some orcs walk in a funny way. Again, it probably does not draw too much attention in a crowd but definitely would require fixing for a character viewer will be able to see clearly.
@@Megasus-ju1mg yeah, it is easy to forget how time consuming and how heavily skill based many things in VFX are. we even forgot to mention stop motion animation.. that's wizardry.. spend so much time on frame-by-frame basis and you can't just tweak something in the end, just by reshuffling some keys and rerender the scene. and it's also interesting that stopmotion animation (e.g. something like Missing Link) can feel so grounded that you feel the weight of the characters and their movements, many times it feels more weighty than usual CG character animation. hm, the Ubisoft info is intriguing. They always had good parkour, but the connection/interpolation between animations feels somehow more janky than it used to. so they never really do any cleanup, just play with curves all the time? although thinking about it, when you mention animation layers, I guess it makes sense, because you instead just overlay your animation through that and keep some real-life jankiness, but you'll make it do what you want or for what you need. Yeah, now to think of it, this seems like a pretty standard stuff, but obviously not always. this is a bit random, but while we are at it, modern animated movies seem to me that they started to have this same-ish kind of animation. Like.. high quality and really well done animation, but it all moves in such a same manner that even characters are losing their individuality. Like, imagine if every real life actor moved in the very same manner.
I love how you actually comment on these techniques as storytelling tools, which shouldn't be surprising given that you're an amazing storyteller. Please keep it up, I really enjoy everything you put out and watching your channel grow! Let's go 50k!
"There's no way of doing this with real actors"... well, probably not with 100.000 extras. But there are definitely quite many movies which had a ludicrous amount of extras in their scenes. Waterloo is a well known example with roughly 15.000 "infantry" extras + 2000 cavalry (which means there where at least 2000 trained horses on set, too) shown in an extremely complex and dangerous battle scene. Not a Battle scene, but still pretty darn impressive and actually far beyond the 100.000: The mass scenes in Gandhi, where about 300.000 extras were seen on screen at the same time. I don't want to question your really good video, but it's good to keep in mind that such things were never impossible since the dawn of cinema. In arguably (one of?) the earliest "blockbuster big budget" movies ever, Metropolis, almost 40.000 extras were involved, with many thousand seen at the same time in some scenes. Crazy, isn't it?
Ironically, when Todd Howard said "it just works" (about weapon customization in Fallout 4), it just worked. So yes, Massive devs would have hoped for their stuff to "Just Work Todd Howard Style".
Most excellent, I seem to remember a story (possibly apocryphal) that when they first used the army sim in LoTR the two armies ran away from each other rather than fight. I worry more that the CGI overuse by studios and/film makers that don’t understand the pure amount of work and time that has to go into them to look as good as some so.
Thank you!! And man, I find them in a bunnnnch of places - old articles on the internet, news reports, behind the scenes on DVDs, the DVD commentaries, interviews + my own experiences in the VFX industry to tie it all together If you're interested I've got a bunch of the online publicly available sources linked in the description - worth checking out if you wanna know more about this stuff! Thanks again for watching! I really appreciate it! 😁
I think Forest Gump was one the first movies to heavily rely on the crowd cloning technique, for the stadium shots and the Washington DC scenes. I’m not saying it’s the first ever, but I remember it being the first big movie where it was heavily publicized in the making off
There shouldn't be any problem to have a crowd simulator (like massive or Alice) to use photogrammetry (we call it videogrammetry when it moves). You can use the videogrammetry data as if it was motion capture. Crowd sim softwares don't create movement on the fly, they trigger pre-capture motion capture (or hand keyframed) data clips. For exemple, run to stop, stop to walk, walk to jump, etc. You can do the same by capturing videogrammetry data of various actions and have the crowd sim tool merging from one to the other. The main challenge you're going to stumble upon is clothing as it will be impossible to merge between clothes in a way that wrinkles match. But then you could use blend shapes to help with that if you can generate a constant meshes.
I once worked with a young chap called Tim, who came over from New Zealand to work in the Uk. He told me about how Peter Jackson had visited his school (when he was younger) with the idea of using children as Orks for his new film. Tim and a few of his friends acted so stupid and caused so much trouble that Peter Jackson walked out saying it was a bad idea to use children.
As an example of how much better that Antman crowd shot could look, I highly recommend watching the "Mountain" PS2 ad. It's 20 years old now and still looks incredible!
detroit become human and other games already have the missing technology you mention at the end. Fully capturing every animaiton needed for freee movement. They scanned real actors, but the player controls and dictates their movement. The scans, for a game anway, are super realisitc.
It's all fine and dandy, but q-tips are good enough for me lol I mean there's no way crowds will ever be real except on the news when something bad is happening like a riot, so I already knew it's CG no matter how real they made it, it's still CG in my head...lol
Creating the MASSIVE battles. I see you, CGY. What a wonderful bit of inside baseball you've brought us this week. Like, I suspect, many of us, the bonus featurettes on The Lord of the Rings were my first foray into the endlessly intriguing boring shit that goes into the making of movies. Thanks for this exploration.
Must these simulations not have taken ages to compute back then? I mean rendering sure, takes long, but even today most computers struggle with games that do similar things in real time
I imagine they optimize it by having simpler version for sh0ts that are further away. Likevthey would do the frontlines and the point of focus in high detail and then ramp down the movement complexity. This could be done dynamically as the camera moves aswell
@@nebuli55 For movies, there are most likely no optimization as huge studios have huge render farms so they wanna render everything at highest res possible so they wanna be pixel fked by directors but they do use camera culling for shots with lots of geo.
They probably used 3d models of different LOD (level of detail) depending on how close to the camera they are. They would also render them chunk by chunk and combine them later in compositing.
I know it has nothing to do with the video, but the Gandalf shot at 8:37... man, the bluescreen is rough. Crazy how I never noticed it before Also if you pause at 12:11, you can tell where the CG crowd starts quite easily. Still incredibly impressive.
SPEAKING OF CROWDS, Queen of the Damned was filmed with 500 (unpaid) extras. A concert for a bunch of goths might just be be only time this would really work, lol.
16:40 You can actually make more sets of animatiopns like make a preson run, fight but then come from run to fight etc so the computer has both animations and the animations in between to compile them seamlessly it is easier to do than you may expect also games already use procedural animations where computer fills some gaps on itself or for example stops animation of punch when it detects an object like a wall
There is also software that can replace characters based on their distance form the camera. This way the CG character can be replaced with a lower res character once they are far from the camera. The resolution of a CG character is based on the amount of polygons that the character is made up of. CGI character close to the virtual camera in the program need to be made of thousands of polygons to create the detail the character is made of. As the camera gets further and further away, the characters can become a stand-in made of eight or nine polygons. The shape of the character looks like something from a video game from the nineties, but will look fine when it is so far from the camera that even the film would only resolve them in ten pixels. Otherwise, to have a scene with thousands of fully fleshed out men in armor, any system, even today, would bog down and crash under the weight of so many polygons. Therefore, all distant characters need to be replace by low res stand-ins. The remarkable thing about this is that as the camera flies over the scene, the software knows to replace those low res characters with their high res replacement once they are in range of the camera to see detail. Large crowd scenes would be impossible or at least impractical without this feature. Even if you have a computer that can handle that many high res figures, why would you want to if the final render will only see those characters as a few pixels? Render times are long enough as it is and every extra polygon costs you render time, even if they’re not being seen. I have many low res characters I created just for that purpose. They’re made of ten or less polygons and their legs and arms move by use of morph targets rather than bone rigs. Having that many characters bone rigged would be a massive job when a large crowd in the distance looks just as good as long as they’re moving. It even amazes me how realistic those low res characters, without bones, can look at the right distance.
The real question is how the heck those early 2000s computer’s were able to render those simulations. Also awesome video, I always learn so much from your videos.
@@entwistlefromthewho yeah I know, I was just wondering because even today simulations of great detail and complexity require powerful computers that were not around in the early 2000s
@@RenderBenderProductions But that obviously can't be true, because you know that late 1990s computers were performing these simulations. So either your expectation of late 90s/early 00s computers is misinformed or your understanding of how much computing power is needed to perform the task is.
Great discussion of Massive, but the clips didnt line up well. Most of the clips were Hero shots with actors. Would like to see a section where you break down a single crowd battle and highlight a few of the animations within that.
Bohemian Rhapsody’s crowds looked incredible, but at the same time on second or third watches every time they show a sweeping shot of the crowd, it looks really weird and fake, to my eyes.
i always loved and watched vfx with an awe, but isn't this lord of the rings cgi is like a big machinima in a vieogame engine? does it relate to eachother? cause it sounds very similar in principle
the only difference being video game have to render the visuals in at least 30fps in real time, but in film it has to render the image only one time to complete the scene? it's basically like making a machinima in unreal engine but for a cutscene, so you can use a lot of resources but you have to render it for a long time? right?
I think you are wrong about your assumption that these 3D scanned actors won't work with a traditional crowd simulation system. The datasets are compatible or at least can be converted so it doesn't really make a difference to the crowd sim. The method with which an agent is animated is completely separated from the logic of the crowd sim - as you said its all about the appearance of "behavior" via intelligent choice of actions.
Mmm yea maybe, I just thought about the visual quality of blending between the different animations and captured meshes yanno? Because a large part of the volumetrically captured humans looking so good is that it's an exact 1 to 1 capture of the actor doing the movement. And once that gets tied into the crowd sim you're gonna start loosing that quality because it's gonna have to interpolate between the different captured animated states and such loosing that super "life like" appearance and start looking more CG again.
@@CGWHY I don't think the blending will be an issue, I think the end result depends much more on the quality of the performances (and their nuances) and the amount of variations.
I also hope that film makers remember that they are telling a story, which has a plot, characters the audience can appreciate. When CGI dominates, the story gets lost. No human ever said, “Tell me a CGI. “ It is certainly a useful tool, but it is not the story and several recent films seem to have forgotten that.
if you only capture the actors as mesh then rig them with a skeleton you can retarget onto them any kind of animation. but actually, just using meta humans is easier, they already come rigged.
It was fine with Lord of the Rings, but this has really gotten abused as a technique in war movies. I think its because the AI is unconcerned about dying, its movements are overly aggressive. Real warriors are tired, sweaty, careful, short burst explosive, and have weight to their movement. CGI warriors are killer bees that are in perfect unison with zero concerns if they die. It's non-stop floppy swinging everywhere chaos, so what if I get killed or exhausted, I'm CGI! Where's the dust, grime, blood, and broken ground from a large company of mounted warriors? It's not there yet and it feels flat when I see it...we know they are CGI, you have to fool us better.
As someone endlessly passionate about all forms of special effects in film, I cannot overstate how happy I am to have found your channel!
yep the guy is super enthusiastic, love it!
Same. I always watch VFX Artists React from Corridor Crew and I love that so I’m so glad I found this channel. It’s literally movie magic to me.
same here! just subscribed
The Creator of Massive actually won a special Academy Award for his work at the Scientific and Technical Achievement Awards
Didn't know about the last method. Really fascinating. Great video dude!
In my personal opinion, the more control and freedom to pick and choose after the fact filmmakers get, the more soulless films tend to become. This isn't necessarily a 1:1 relationship, because a filmmaker with a strong sense of vision will still focus on what he or she wants from the get go, but it does allow to start filming earlier in the production schedule and make what used to be important decisions after the fact and it also allows for a greater degree of studio interference (and if there is something that tends to suck the soul out of movies it's studio interference). I'm a believer in having to develop a fully realized vision for your film before you shoot a single frame.
On the other hand, advances like these fascinate me and I love thinking about the posibilities they open up.
also one of the "small" things that make a big difference why crowd scenes in LoTR still look better is also their insane mo-cap library which was used for movement of pretty much any kind, any fight, any death, any interaction.. and all animation was done through those quality motion captures. while many modern crowd scenes, yeah they use mo-caps too (sometimes it is also a simulation), those mo-caps are not of the same quality.
to understand the difference, there is a great video comparing Valve's Left 4 Dead zombie ragdolls with Back 4 Blood (it's been a while, but I think it was the one from Crowbcat). Valve made it of much higher quality and thus the older game holds much better in animation regard than most modern game.
Or like when you see Uncharted animation vs random mixamo captures.
And same goes for LoTR crowd scenes. Much higher quality of captured movement for animation and simulation.
Same difference as with Avatar performance capture vs any modern movie. Avatar is all just captured (yeah, you have always some post-enhancements or clean-ups) and tries to maintain that reality. While other movies either rotomate or mo-cap and then re-aimate it or add animation that is unreal and make it look all spongy while mixed with something more real.
That's also why even after 14 years, Avatar 1 seems more real. Because movements sells and grounds it much better than spongey squishy unrealistic movement you see elsewhere. With CG that can look better when paused, but shows intself as a trick when played. But Avatar shows that the movement just seals the deal completely and wipes out the unreality altogether.
Interesting perspective - I mostly agree with this although there are a couple things I disagree with too. You're 100% right in that a lot of the time what makes VFX stick out like a saw thumb is sub-par animation, especially in regards to humanoid rigs (because obviously we are human and spend most of our time observing other people so its easier to notice when it looks awkward/off) so for realistic actions accurate and clean motion capture data is optimal and can save a lot of time.
You mentioned Left 4 Dead so I'm going to comment on that example first, for L4D1 Valve used and combined hundreds of motion capture takes for the many actions of the zombies, locomotion, hit reacts, death animations etc however the compromise of this is that a lot of it isn't cleaned up to the highest quality this is especially noticeable when you focus on things such as the fingers for example which stay in the same static idle pose and as far as I can tell aren't even animated - so yes Valve achieved much more believable animations but I don't think its exclusively because they had "higher quality" motion capture to use but instead they had a much larger variety of data to go off and use, also things can get a bit finicky when using video games as an example because there is a whole other technical process involved in actually implementing these animations, anim graphs, state machines, locomotion frameworks such as advanced locomotion system (which a lot of AAA games use now), etc.
Moving on to your Avatar example one thing that actually sticks out in the first Avatar movie is that in some of the scenes when the Na'vi are moving at high speed it can actually look a bit awkward especially when compared to the most recent installation in the franchise - this is because the Na'vi obviously have much longer limbs than the actors in motion capture suits, so when for example an actor sways their arms really fast it can look oddly quick because the length from the origin of the pivot (the shoulder/upper arm) all the way down the arm to the hand is much longer - there is a particular scene when Jake (in his avatar form) jumps from the top of a water fall into the water down below which stands out to me the most. Also avatar isn't all motion captured - things such as creatures especially were hand keyed by animators at ILM and sometimes you don't get a good result from the mocap data for whatever reason so hand keyed animation is still involved.
And that kind of brings me to my main point, I feel like you downplayed the importance of hand keyed animation in the VFX industry but productions such as Chappie, District 9, Jurassic world, The Mandalorian (the season 1 speeder bike chase comes to mind specifically) and of course the Transformers franchise all feature a large amount of hand key animation and visual effects studios such as WETA digital and ILM often require you to have experience with both motion capture cleanup and hand keyed animation because sometimes motion capture just isn't always an option such as if you're animating a physically impossible action or something such as a dragon for example. So motion capture isn't absolutely necessary - what is necessary is good animation and a team of talented artists can often achieve that.
@@Megasus-ju1mg thanks for the response
regarding L4D, I didnt expect they would have an animation for fingers, i was focused on general movement, and imo, that huge library helps *a lot*, same as with LoTR. They tried to implement so many variations that it just looks really alive.
And you are right, you need to do plenty of cleanup and fixes. But I think this is where the slippery slope lies? You need to do a cleanup, but it is possible to make it too clean to the point that movement are nice, but slightly odd or too smooth, or how to explain it. Or you clean some keys and then the interpolation will make it less real. Maybe you want clean it so there is a bit less data and jitter, but you can also loose some of that life in it. But I think that might not be the main thing. If you look at modern crowds, most of the time, when they walk, they have this rigid NPC style walk that you see in games, but not in reality and it looks more like they overlayed general animation to whatever character. Generic walk cycle captured by man is applied to women, walk cycle with arms wide apart so the animation fits to any kind of person, thin, thick, plenty of clothes, lack of clothes.
I didnt mean to downplay hand animation. I meant to talk only about Na'Vi/Avatar mo-caps/performance-caps. Maybe I should have pointed that out. Because yeah, rest of stuff is hand animated because there probably is not a way to motion capture various kinds of fish (i guess one could, but.. why would anyone bother, if you can do a good hand animation quicker and believable) or birds or creatures, not to mention six legs or limbs or dozens of tentacles.. or infamous Smaug performance session that ditched the data and just hand animated the dragon. And pretty much any monster in the movie is hand animated. And I'd guess that most background apes in Ape trilogy is done just by hand.. and especially fight scenes are surely pure hand animation (also even if you have some reference, you still need to hand animate it).
It is a neccessity and can be done so good that you dont even question it. (and there is also rotomation, which is pretty much matching actor frame by frame with 3D character). With mo-cap you are limited to reality, with hand animation, sky's the limit.
Which brings me back to Avatar/LoTR vs many other movies and focus on human(oid) characters, beause that's when we notice it the most (and also who knows how many shots did have hand animated character in Avatar, cause something either didnt work or needed a big fix).
I guess the main difference is, in the end, between tailor making your mo-cap to what you are actually needing, vs having general library of general movement for general crowd, so you can pull from that library and apply it to any crowd simulation you need. Elves? Put there the general mo-caps of stoic people. Orcs? Put there the general mo-cap of brutes. Walk cycle? Put there the general mo-caps of people walking. Instead of tailor making every mo-cap to the precise nature of your need.
I mean, it makes sense, it saves time, money, hours, and it is a thing that most people dont notice or care about. But I think that is that one last percent difference that grounds it even more.
@@Patrix8558 Yee that makes sense, and yep you're definitely correct in a lot of scenarios motion capture is more efficient because as you mentioned its quicker and also you are "constrained" to reality since you know you have accurate data from someone physically doing whatever actions are needed in person.
and your point about cleanup also has a lot of merit, at Ubisoft for example they don't actually delete any of the underlining keys from mocap date, they just mould tweaks around it by tweaking curves and using animation layers in a none-destructive fashion and I'm assuming thats how most people do it too - someone inexperienced with mocap can definitely have the tendency to "over clean" it when for realistic motions you need those imperfect arcs and slight micro movements.
Just felt the need to underline the importance of hand-keyed animation in the pipeline as, in general, I feel like a lot of people (especially those outside the industry) often make the assumption that minimal work is required when it comes to realistic animation in VFX because "the computer does all the work" - not saying that's your interpretation but I just wanted to highlight that.
Honestly, I highly doubt the motion capture tech of over 20 years ago is that good by modern standards. You can clearly see it in the video when they show the animations they used. They are a bit floaty and not everything is cleaned up properly (it's just that it does not necessarily matter for tiny orcs in the back of the shot). The systems themselfs decent but nothing to write home about in this day and age. I think it is more the variety and the quality of pieces themselves that made those mocaps age well.
Vicon launched a 1 megapixel camera in 2000, a 4 MP camera in 2004 and a 16 MP camera in 2008. I literally was in a budget mocap studio that still used old 0.3 megapixel cameras as late as in 2016. I am sure Weta could afford those premium cameras but today 1 megapixel is about the resolution of cheapest infrared motion capture cameras. But in the year 2000 it was the best of the best.
In reality you should know the strong points of your system. Those 0.3 MP mocaps we recorded are totally usable today, especially for wide, violent motions-not so much for subtle acting when the jitter becomes quite apparent. The actors and stuntmen who can SELL the action are just as important. The suit does not magically make you into a great performer-if you make an amateurish attempt at moving in a specific way, it is often quite visible you are pretending. Stuntmen and actors experienced at staged fights will move way better than an average joe and will look much more intesting. And they can actually come up with interesting ways of beating each other up.
The quality of applying the motion to the 3d character can also vary. In the previews some orcs walk in a funny way. Again, it probably does not draw too much attention in a crowd but definitely would require fixing for a character viewer will be able to see clearly.
@@Megasus-ju1mg yeah, it is easy to forget how time consuming and how heavily skill based many things in VFX are.
we even forgot to mention stop motion animation.. that's wizardry.. spend so much time on frame-by-frame basis and you can't just tweak something in the end, just by reshuffling some keys and rerender the scene.
and it's also interesting that stopmotion animation (e.g. something like Missing Link) can feel so grounded that you feel the weight of the characters and their movements, many times it feels more weighty than usual CG character animation.
hm, the Ubisoft info is intriguing. They always had good parkour, but the connection/interpolation between animations feels somehow more janky than it used to.
so they never really do any cleanup, just play with curves all the time? although thinking about it, when you mention animation layers, I guess it makes sense, because you instead just overlay your animation through that and keep some real-life jankiness, but you'll make it do what you want or for what you need. Yeah, now to think of it, this seems like a pretty standard stuff, but obviously not always.
this is a bit random, but while we are at it, modern animated movies seem to me that they started to have this same-ish kind of animation. Like.. high quality and really well done animation, but it all moves in such a same manner that even characters are losing their individuality. Like, imagine if every real life actor moved in the very same manner.
I love how you actually comment on these techniques as storytelling tools, which shouldn't be surprising given that you're an amazing storyteller. Please keep it up, I really enjoy everything you put out and watching your channel grow! Let's go 50k!
Incredible video! Please keep this up, so glad I found your channel! Fascinating topics explained in great depth with clear explanations! :D
Thank you so much! I'm really glad you're enjoying it 😅
"There's no way of doing this with real actors"... well, probably not with 100.000 extras. But there are definitely quite many movies which had a ludicrous amount of extras in their scenes. Waterloo is a well known example with roughly 15.000 "infantry" extras + 2000 cavalry (which means there where at least 2000 trained horses on set, too) shown in an extremely complex and dangerous battle scene. Not a Battle scene, but still pretty darn impressive and actually far beyond the 100.000: The mass scenes in Gandhi, where about 300.000 extras were seen on screen at the same time. I don't want to question your really good video, but it's good to keep in mind that such things were never impossible since the dawn of cinema. In arguably (one of?) the earliest "blockbuster big budget" movies ever, Metropolis, almost 40.000 extras were involved, with many thousand seen at the same time in some scenes. Crazy, isn't it?
Keep up the good work man!
12:58 ah yes, the shot with 2 legolas’ in it 😆
This type of vfx has always interested me so great video!
Ironically, when Todd Howard said "it just works" (about weapon customization in Fallout 4), it just worked. So yes, Massive devs would have hoped for their stuff to "Just Work Todd Howard Style".
Most excellent, I seem to remember a story (possibly apocryphal) that when they first used the army sim in LoTR the two armies ran away from each other rather than fight.
I worry more that the CGI overuse by studios and/film makers that don’t understand the pure amount of work and time that has to go into them to look as good as some so.
That is actually true. Peter Jackson talks about it on the Appendices
Fun fact about Gandhi (1982): Over 300,000 extras were used in the funeral scene, the most for any film, according to Guinness World Records.
Awesome content as always!
Where do you actually find the information for these videos? This all seems really in-depth!
Thank you!!
And man, I find them in a bunnnnch of places - old articles on the internet, news reports, behind the scenes on DVDs, the DVD commentaries, interviews + my own experiences in the VFX industry to tie it all together
If you're interested I've got a bunch of the online publicly available sources linked in the description - worth checking out if you wanna know more about this stuff!
Thanks again for watching! I really appreciate it! 😁
I think Forest Gump was one the first movies to heavily rely on the crowd cloning technique, for the stadium shots and the Washington DC scenes. I’m not saying it’s the first ever, but I remember it being the first big movie where it was heavily publicized in the making off
crowd copy-pasting was first used in Gladiator, if i am not mistaken, and then in Contact.
There shouldn't be any problem to have a crowd simulator (like massive or Alice) to use photogrammetry (we call it videogrammetry when it moves). You can use the videogrammetry data as if it was motion capture. Crowd sim softwares don't create movement on the fly, they trigger pre-capture motion capture (or hand keyframed) data clips. For exemple, run to stop, stop to walk, walk to jump, etc. You can do the same by capturing videogrammetry data of various actions and have the crowd sim tool merging from one to the other. The main challenge you're going to stumble upon is clothing as it will be impossible to merge between clothes in a way that wrinkles match. But then you could use blend shapes to help with that if you can generate a constant meshes.
Love this channel, been watching all videos for the last two days! Subscribed and sharing with friends!
I once worked with a young chap called Tim, who came over from New Zealand to work in the Uk. He told me about how Peter Jackson had visited his school (when he was younger) with the idea of using children as Orks for his new film.
Tim and a few of his friends acted so stupid and caused so much trouble that Peter Jackson walked out saying it was a bad idea to use children.
Great video and Im glad you didn’t put Rings of Garbage in the same video as LotR. It doesnt even deserve the name 😂
As an example of how much better that Antman crowd shot could look, I highly recommend watching the "Mountain" PS2 ad.
It's 20 years old now and still looks incredible!
First half of the video: Well yeah, this is mostly just stuff I already knew... Second half of the video: Floor shatters from my jaw dropping on it
I always thought Braveheart was the first movie to do huge battle scenes, at least it was the first one I remember.
Starship Troopers used crowd simulation software in the 90s.
detroit become human and other games already have the missing technology you mention at the end. Fully capturing every animaiton needed for freee movement. They scanned real actors, but the player controls and dictates their movement. The scans, for a game anway, are super realisitc.
This is so fascinating!! I'm always so intrigued by practical and CGI effects. Just amazing.
One of the earliest examples of digital crowd compositing was for the coronation scene in Elizabeth (1998)
Great as always, thanks!
Imagine being the forerunner of something, and *still* be the best of them all.
It's all fine and dandy, but q-tips are good enough for me lol I mean there's no way crowds will ever be real except on the news when something bad is happening like a riot, so I already knew it's CG no matter how real they made it, it's still CG in my head...lol
Creating the MASSIVE battles. I see you, CGY. What a wonderful bit of inside baseball you've brought us this week. Like, I suspect, many of us, the bonus featurettes on The Lord of the Rings were my first foray into the endlessly intriguing boring shit that goes into the making of movies. Thanks for this exploration.
Must these simulations not have taken ages to compute back then? I mean rendering sure, takes long, but even today most computers struggle with games that do similar things in real time
I imagine they optimize it by having simpler version for sh0ts that are further away. Likevthey would do the frontlines and the point of focus in high detail and then ramp down the movement complexity. This could be done dynamically as the camera moves aswell
@@nebuli55 For movies, there are most likely no optimization as huge studios have huge render farms so they wanna render everything at highest res possible so they wanna be pixel fked by directors but they do use camera culling for shots with lots of geo.
@@xanzuls even the best computers in 1998 were an order of magnitude slower than they are now. Plenty of optimisation was needed
They probably used 3d models of different LOD (level of detail) depending on how close to the camera they are. They would also render them chunk by chunk and combine them later in compositing.
I know it has nothing to do with the video, but the Gandalf shot at 8:37... man, the bluescreen is rough. Crazy how I never noticed it before
Also if you pause at 12:11, you can tell where the CG crowd starts quite easily. Still incredibly impressive.
SPEAKING OF CROWDS, Queen of the Damned was filmed with 500 (unpaid) extras. A concert for a bunch of goths might just be be only time this would really work, lol.
i still prefer the orwell crowd method. take a piece of cart-board, poke some holes in it, move a small light under it, film that.done.
Of course, there is the alternative of using the Red Army for battle scenes.
16:40
You can actually make more sets of animatiopns like make a preson run, fight but then come from run to fight etc so the computer has both animations and the animations in between to compile them seamlessly
it is easier to do than you may expect
also games already use procedural animations where computer fills some gaps on itself or for example stops animation of punch when it detects an object like a wall
You should look at Bahubali 2015 (from the filmmaker of RRR ) final battle scene and that entire movie was made on small budget of 22m usd
There is also software that can replace characters based on their distance form the camera. This way the CG character can be replaced with a lower res character once they are far from the camera. The resolution of a CG character is based on the amount of polygons that the character is made up of.
CGI character close to the virtual camera in the program need to be made of thousands of polygons to create the detail the character is made of. As the camera gets further and further away, the characters can become a stand-in made of eight or nine polygons. The shape of the character looks like something from a video game from the nineties, but will look fine when it is so far from the camera that even the film would only resolve them in ten pixels.
Otherwise, to have a scene with thousands of fully fleshed out men in armor, any system, even today, would bog down and crash under the weight of so many polygons. Therefore, all distant characters need to be replace by low res stand-ins. The remarkable thing about this is that as the camera flies over the scene, the software knows to replace those low res characters with their high res replacement once they are in range of the camera to see detail.
Large crowd scenes would be impossible or at least impractical without this feature. Even if you have a computer that can handle that many high res figures, why would you want to if the final render will only see those characters as a few pixels? Render times are long enough as it is and every extra polygon costs you render time, even if they’re not being seen.
I have many low res characters I created just for that purpose. They’re made of ten or less polygons and their legs and arms move by use of morph targets rather than bone rigs. Having that many characters bone rigged would be a massive job when a large crowd in the distance looks just as good as long as they’re moving. It even amazes me how realistic those low res characters, without bones, can look at the right distance.
The real question is how the heck those early 2000s computer’s were able to render those simulations. Also awesome video, I always learn so much from your videos.
The early 2000s wasn't the Dark Ages....
@@entwistlefromthewho yeah I know, I was just wondering because even today simulations of great detail and complexity require powerful computers that were not around in the early 2000s
@@RenderBenderProductions But that obviously can't be true, because you know that late 1990s computers were performing these simulations. So either your expectation of late 90s/early 00s computers is misinformed or your understanding of how much computing power is needed to perform the task is.
render 40,000 characters at 2k resolution in less than 512Megabytes of RAM... doesn't sound too hard ;)
Great discussion of Massive, but the clips didnt line up well. Most of the clips were Hero shots with actors. Would like to see a section where you break down a single crowd battle and highlight a few of the animations within that.
Bohemian Rhapsody’s crowds looked incredible, but at the same time on second or third watches every time they show a sweeping shot of the crowd, it looks really weird and fake, to my eyes.
The fat guy in the 1:30 minute mark is a Spanish RUclipsr. It feels weird to see Spanish memes transcend the language barrier.
I am a fan of the old epics that just did it legit and cost a fortune compared to other films of their day.
This reminds me of terracotta warriors in China
the Dr Who crowd scene could have made this better by simply blurring the BG a little, didn't help the whole shot was in perfect focus...
i always loved and watched vfx with an awe, but isn't this lord of the rings cgi is like a big machinima in a vieogame engine? does it relate to eachother? cause it sounds very similar in principle
the only difference being video game have to render the visuals in at least 30fps in real time, but in film it has to render the image only one time to complete the scene? it's basically like making a machinima in unreal engine but for a cutscene, so you can use a lot of resources but you have to render it for a long time? right?
I think you are wrong about your assumption that these 3D scanned actors won't work with a traditional crowd simulation system.
The datasets are compatible or at least can be converted so it doesn't really make a difference to the crowd sim.
The method with which an agent is animated is completely separated from the logic of the crowd sim - as you said its all about the appearance of "behavior" via intelligent choice of actions.
Mmm yea maybe, I just thought about the visual quality of blending between the different animations and captured meshes yanno?
Because a large part of the volumetrically captured humans looking so good is that it's an exact 1 to 1 capture of the actor doing the movement. And once that gets tied into the crowd sim you're gonna start loosing that quality because it's gonna have to interpolate between the different captured animated states and such loosing that super "life like" appearance and start looking more CG again.
@@CGWHY I don't think the blending will be an issue, I think the end result depends much more on the quality of the performances (and their nuances) and the amount of variations.
sec30 whats the movie?
Cecil B DeMille seemed to manage with real people. No copy and paste?
Very interesting!
I also hope that film makers remember that they are telling a story, which has a plot, characters the audience can appreciate. When CGI dominates, the story gets lost. No human ever said, “Tell me a CGI. “ It is certainly a useful tool, but it is not the story and several recent films seem to have forgotten that.
Lord of the Rings was not filmed in/with/by hollywood, just sayin.
if you only capture the actors as mesh then rig them with a skeleton you can retarget onto them any kind of animation.
but actually, just using meta humans is easier, they already come rigged.
It was fine with Lord of the Rings, but this has really gotten abused as a technique in war movies. I think its because the AI is unconcerned about dying, its movements are overly aggressive. Real warriors are tired, sweaty, careful, short burst explosive, and have weight to their movement. CGI warriors are killer bees that are in perfect unison with zero concerns if they die. It's non-stop floppy swinging everywhere chaos, so what if I get killed or exhausted, I'm CGI! Where's the dust, grime, blood, and broken ground from a large company of mounted warriors? It's not there yet and it feels flat when I see it...we know they are CGI, you have to fool us better.
Quality post!
Ok but why didn't they make a game out of this?
Missed potential
WHY THE FUCK DID THEY NOT ADD THIS "MASSIVE" SOFTWARE INTO GAMES
Buddy, it I don't see them, then why can I see them?
Dune don’t let me down
Spoiler ⚠️
If we don’t see Paul and the fedaykin absolutely maul the emperor n his sardaukar in arrakeen in part 2 I’ll actually cry
Great video but please stop saying insane to describe everything.
Whoa! Hehe