Phase-Functioned Neural Networks for Character Control
HTML-код
- Опубликовано: 29 апр 2017
- We present a real-time character control mechanism using a novel neural network architecture called a Phase-Functioned Neural Network. In this network structure, the weights are computed via a cyclic function which uses the phase as an input. Along with the phase, our system takes as input user controls, the previous state of the character, the geometry of the scene, and automatically produces high quality motions that achieve the desired user control. The entire network is trained in an end-to-end fashion on a large dataset composed of locomotion such as walking, running, jumping, and climbing movements fitted into virtual environments. Our system can therefore automatically produce motions where the character adapts to different geometric environments such as walking and running over rough terrain, climbing over large rocks, jumping over obstacles, and crouching under low ceilings. Our network architecture produces higher quality results than time-series autoregressive models such as LSTMs as it deals explicitly with the latent variable of motion relating to the phase. Once trained, our system is also extremely fast and compact, requiring only milliseconds of execution time and a few megabytes of memory, even when trained on gigabytes of motion data. Our work is most appropriate for controlling characters in interactive scenes such as computer games and virtual reality systems.
Кино
This could be the future for consistent good animation while traversing the world.
And yet Ubisoft would some how manage to fuck it up lol
+Rebumai No worries. Ubisoft are already dead. We're just waiting for them to be buried at this point: www.vg247.com/2017/04/26/vivendi-takeover-of-ubisoft-will-happen-this-year-report/
but ubisoft always had good animations :( AC, Watch Dogs, you can even take For Honor as an example
Let's hope Ubisoft will fight Vivendi. Vivendi are just so much worse than Ubisoft... Just google it.
I actually like Ubisoft's recent conduct. They're trying to be good, it seems. But publishing business is just such a sea of sharks overall, the least scrupolous tend to win. There are still rumours of them not treating their people too well, but oh well, can't be any worse than Konami now or EA a while back.
I could watch this model walking for hours. It's so realistic and dynamic...
Almost like watching a real person, but less creepy.
You can see similar character mobements in GTA V and Assassin's Creed
In games cheap poor animations are making me not want to play games. But when animations are realistic then it is different story.
Zippy Parakeet I feel like as clunky a franchise as it is, Assassins Creed has some of the best walking and moving animations
@@snoopyontheshed yep, movements are quite natural
I can't even get my character's feet to align properly to the ground in UE4, lol.
because ue4 is shit
@@elgunlee dam look over here guys, what a fukin badass
Use Unity lol
what if unity is also shit?
Make your own engine.
I like his hat.
my HAT... It has. Three corners.
you are cringe that is a baby song
me too XD
another settlement needs our help
I think the model used is haytham from assassin's creed
Amazing. A way to improve realism and general appearance without falling back to improving graphics and other methods that often imply a substantial increase in resource consumption. Well, I'm assuming the latter, that this will only require a small increase in physics calculations. The industry has, for some reason, focused almost exclusively in realism looking right, and much less so in it also "feeling" right. Hopefully this will be only the first step toward remedying that.
I especially liked the bit where the character walked backwards or sideways depending on the camera angle. pretty sure all of the character animation I've seen so far will always walk forwards, even though you're looking behind you.
*so far have walked forwards
Backwards walking isn't limited by state machine complexity in modern titles, but by the practical issue that moving backwards as fast as forward looks physically implausible. There was lots of backwards running in old Unreal Tournament and other classic titles when you watch other characters.
I don't think it's exactly a lightweight technique, would need a ton of planning ray casts, and in complex open world engines you only have budget for a few dozens of those per frame. Plus not even sure how large the NN is, it's not fast. Sure on main character it'll be fine, but in a competitive game where you have to run this on a dozen characters, or in crowd simulation, well, it seems like it would be an issue.
ah you're right.. I didn't think about that. but if they really wanted, there are plenty of games that walk until you use the run command, so you could have them walk backwards until that command is given, then move forward in that direction. in the end, I guess it's really unnecessary to add that animation.
Imagine not needing parametric blend trees or nav/height meshes anymore! The ramifications of this are huge, especially in AR/MR, when the landscape's topology isn't known beforehand. Awesome work!
mmmm I would assume there is still a significant amount work needed to topologize the walkable terrain; or rather, new approaches would be required. As you can see in this video, the character is walking over a single plane with the obstacles blended out of the plane, like a sheet laid over a series of objects. The more topology that the control scripts have to crunch, the heavier the demand. They have optimized the example by using obstacles that are simple ramps. Perhaps you could tell the control module to treat elevation changes in the immediate vicinity that are beyond a certain rate of change in a specific manner. This brings up a whole host of interesting variables that could be fun to play with... for example the look-ahead radius. You could amplify the move speed and rate of topology analysis to create realistic-moving sci-fi android enemies that move in a creepy, superhuman manner. The list goes on
It just works.
I FOUND YA
Yup
Only 3 likes. Not this time, Justin Y.
It's just you.
nice input
The quaker oats guy is getting real good.
underrated comment.
are you fucking kidding me? this is incredible! I'm watching 2:30 onwards right now and I'm shocked at how good this is. it reminds me of how impressed I was seeing those tech demos of the euphoria animation blending physics engine that rockstar picked up and used in GTA IV. I hope I get to play around with this in future games.
hey dad
wait, why have I found you here? lol
this fucking AI walks better than me...
skynet
So you're a character from Goofball xD
@@junpeiiori4720ha
These guys should spend some time with the ones at Boston Dynamics
boston dynamics doesn't uses AI/ML. They use Control theory.
yeah i was thinking the same, same maths right
Control theory and machine learning are pretty related as far as mathematics and computer science goes. Both are basically a series of optimization algorithms to achieve a desired result, and in fact can be used in conjunction to great affect. Not sure if BD does this, but I wouldn't be surprised if they do.
@@IshanJain Boston dynamics doesn't use control theory. They use Robotics.
See how funny is that?
The future is so bright for gaming, machine learning is literally going to "change the game"
That's really cool, now apply it to a scene with 200+ characters with this technique.
Would be nice to see their interaction with each other.
Rockstar games, you're next.
"lol no. U idiots will buy our shark cards anyways" -Rockstar
Nothing worse than just slightly nudging the analog stick and having your character move 8 feet
William Kjellgren nah from what we've seen gta V downgraded the euphoria physics from gta iv
Hitting people with cars in gta IV was the most satisfying thing :P . Not anymore in gta V
Don't forget about drunk movement in GTA IV, those were the best falling simulator.
This is so cool. Neural networks are capable of optimizing so many different tasks, and we're just barely scratching the surface of their potential.
This is amazingly realistic and this program makes it looks easy. I was especially impressed at 4:41 when the character awkwardly walks down the stairs, it looks so realistic it's incredible.
This is the opposite of the ARMA (3) animations.
This is more directed towards singleplayer 3rd person RPGs like Assassins Creed, The Witcher series, rather than Military simulator games.
I do not see people enjoying these in a MMORPG or any other kind of online genres (with some exceptions of course, GTA being one of them imo)
@Mighty Memestar "because it probably takes up a lot more of the computer's resources"
It doesn't. He explains that in the video, and also in the description.
Imperial Guardsmen I don’t think you get the joke
*This* a billion times over. And how about good old-fashioned climbing/ mantling:
*COD 2* : Mantling: ruclips.net/video/IpMmuNBhMe8/видео.html
*Rainbow Six: Raven Shield* : Ladders, ruclips.net/video/Hh7JdcpXt8M/видео.html .
And don't get me started on Arma's vehicle mounting. SMH.
This adds so much realism even at its current demo stage. I just wish it didn't stay under covers, forgotten and unused.
Please, somebody, create a Unity and Unreal plugins to use this feature, ideally with some trained presets so indie developers could use it immediately without having to train from their own motion capture.
Except that is extremely unlikely to happen. If someone would make a plugin to run the network in an engine like Unity/Unreal, you'd still need a character with this exact skeleton in order not to get animation issues. They trained the network specifically for this character, so if indie developers would like to use it on their own characters, they need to train it for each unique skeleton (but characters can share skeletons, so you only need to train once for each unique skeleton).
Secondly, anybody who would have the money/time to implement it into Unity/Unreal would never give it away for free, because it will instantly make their animation look better than literally 100% of the rest of the industry, including animation done by the best studios in the world that have spent millions of dollars on perfecting their current technology. So any plugin made for this will likely cost into the thousands -if not millions- of dollars. (and on top of that, such studios would probably instantly hire whoever made that plugin and make them take it offline so nobody else can profit if the plugin was free/cheap).
Thirdly, this is just a demo in a custom engine. The video gives no information about settings in the engine and what is done by the engine. So it's likely that anyone who tries to implement this into another engine will face some problems (especially since there is a lot of math involved due to the nature of neural networks).
Not only is it impressive, it's a milestone, there's no way to compete with a neural network with an architecture strictly designed to "create" real-time animations of execution. I wonder how many companies are interested in this state-of--art software, what a beautiful job.
I feel like this is part of a master or PhD thesis and whoever made his just hates their life now, but has done so much for the industry thank you. This is very cool
WAIT, ISNT THIS ASSASINS CREED???
that's the least interesting part of this
is it in the final game?
The motion looks like Death Stranding to me.
So this is why the new Assasins Creed Games are so unresponsive :/
Looks like Assassin's Creed 3 to me
Guys, this is fantastic! I want to buy shares of your company, because your technology is the future of video games!
Can’t even imagine how smart you gotta be to do things like this... impressive
it seems like so recently that just having your character able to put their foot up on a step up instead of just one foot hanging off was amazing. now we have this.
Why am I watching this at 2am?
HaSTaxHaX 04:43 am lol
because it's good stuff
It legit JUST turned 2 AM when I read this comment. Creepy.
It's 1:41 here
1am here, I love game dev videos but still this is much more theoretical and advanced than the usual content that I consume. It seems like youtube puts me in the weirdest corners of the site later in the evening. I wonder if their algorithm adjust for local time.
CPU usage:
Game: 20%
Character Movement: *80%*
fuck yeah finally a way to utilize those idle cores you have in your gaming rig
Guys a trained model barely needs any computation. Its takes way less than rendering a single frame, its just a batch of multiplications and additions. The training which happens on the development side is what takes time.
its unlikely to really need a lot of computational power. The training of the network does, but all you are doing is basically finding a set of variables for the function that produces the desired result. Once you have found the variables you can just hardcode them into your game engine. Actually running this function with the variables you determined earlier sholdnt take a lot of CPU/GPU time. Certainly less than the game physics and graphics.
FIFA?
But we can use it for filmmaking like basic animation
This is gonna honestly set the new standard for player character movement animation.
Five minutes of watching a yellow pirate chase a black snake around monochrome terrain has never been this cool before.
We're nearly there god
Rockstar is probably very interested in this. I can already see it happening. Red Dead Redemption 2 delayed indefinitely.
Tommy Sasaki yet it was just announced at e3 this year 😂
It was announced a year ago
RDR already had Euphoria anyway.
just seen gameplay.
just played it
His enthusiasm is unmatched
my god thats beautiful, now just make him swim and run and i will watch the video on repeat forever and never be bored, keep it up
It will be interesting to see what effect this approach has upon the pacing of game character movement. When a character moves with convincing "energy cost", where a character's movement slows down when obstacles interrupt its momentum, our brains will become less likely to believe that game characters can perform impossible feats, such as running up a mountain side without becoming tired. Thus, games will need to incorporate "energy cost management" as a game mechanism that deliberately affects the pacing of game character movement.
There's a small unknown company called Bioware, which needs this tech right f..cken now!
Max Shepard they sure do
There’s another small indie dev team that you’ve probably never heard of called rockstar games that could definitely stand to use this as well
Max Shepard
Yeah and they also need better face animations
A very humble, small and unknown indie company called Bethesda totally needs this right now too !!!
Fuck BioWare. Bethesda needs this tech like 10 years ago.
Absolutely brilliant! I can't wait to get this as a feature in Unity some day and have all my characters "learn" how to move like that.
That's really REALLY impressive. This has to be implemented in the next generation of games. I'm impressed on how well it works.
How many neurons do you use for each layer? I haven't found that mentioned in the paper,
Wow this looks smooth. I can't wait until this gets integrated into game engines. Are you guys going to write an API or have you just made the specification?
It would be sick if this was added to Unity's new Machine Learning add-on
this technology is worth millions... they do explain how they did it but not the specifics. Every big studio, or Unreal/Unity would pay big bucks to include this type of tech. If I was them I would not release this as open source. Let the bidding war ensue! Priority pricing for CDProjekt Red and Bethesda :)
This is incredible and I would love so much to learn every detail so that I could make this myself too!
I really love how animation systems like this look and feel
amazing this will open up so many possibilities when it comes to environmental design. This will literally open up a lot more games if this gets implemented. I can remember dark souls 2 trying such a thing with different states while being in different kind of states ( less or more hollowed / damaged ) but failed sadly to accomplish this in a responsive matter. I really like this, this is at least a step forward in game design as a whole.
What you are thinking of with Dark Souls 2 is not the same as this. Character movement changing based on effects like Hollowed, Damaged, is just a Finite State Machine (FSM) that blends between animations and Inverse Kinematics (IK), just like every other game, including more advanced and realistic movement like what can be seen in The Last of Us 2 gameplay from this year's E3 conference. This is an entirely new approach to solving the above problem using a neural network that is likely to revolutionize the gaming industry if the processor load is not excessive.
Only problem that should've been tackled first is the actual 'foot' of a character HAS to 'stick' to the ground(on most of the surfaces). Just look how the foot is sliding across the ground. I see this issue in many games if not i all of them.
one step at time..
Skemooo Wow.
It isn't hard to implement feet inverse kinematics, but yeah most of the game makers don't bother...
Fantastic work, this already looks better than anything I've seen in any game.
The potential for this is incredible
Can this be applied to fantasy-model games where mobility may have more than a normalistic interpretation? i.e. Flying/floating/unreal speed... etc.
Yes, all it takes is to give animation weights and alter smoothing/interpolation values.
I think you would need to train it with motion data of people "flying" and mid-air double-kicking, of course the actors will be suspended from cranes, but they will be able to do the most natural-looking motion which would then be fed to the training data of that neural network.
All these idiots commenting about demos and snake oil probably have no programming experience or understanding of game dev. This looks pretty damn awesome and can't wait to dissect the paper and possibly implement it myself!
Well, the naysayers are retarded, but not for the reasons that the OP thinks. Yes, obviously something this abstract and complex needs detailed papers and evidence. However, that doesn't really excuse you from looking like a total buffoon when someone says "Hey guys, here's this cool new thing I'm working on" and you reply with "FAAAKE FAAAAAAAAAKE FUCK YOU ASSHGLE RELEASE A PAPER OR IT'S FAAAAAAAAAAAAAAAAAAAAAKE"
The problem/solution with the internet is that it gives everyone a voice. Before this would have only been released to small circles of like-minded academics, and nobody here would have even known it exists yet.
@MrBorderdown
Oh look here's the paper:
theorangeduck.com/media/uploads/other_stuff/phasefunction.pdf
If you just took 5 minutes to google the names at the start of this video you would see that this is coming from a PHD student and a professor from the University of Edinburgh, as well as a graphics researcher from method studios (they worked on guardians of the galaxy, doctor strange, captain america). They presented this at SIGGRAPH 2016 and there is no indication that they are selling it.
Also you clearly don't even know what you're looking at on his channel. Hes posted a couple small games, implemented A* pathfinding, a fluid simulation and some shaders. I'd really like to see your accomplishments. And I like how you call him "third world" just because he looks slightly brown. You're just embarrassing yourself.
I appeared to have triggered this random dude.
Broski come back to me when you actually have something to back up your words. You're just another random internet anon saying shit. At least I have something to back up the fact that I'm a gamedev.
I'll even accept the fact that what I make is low quality stuff. I'm no professional and I just graduated. But I'm much closer to being someone who actually contributes to society instead of spewing random racist injectives at people. Take stock of your life and actually do something with it. Peace.
I guess the only solution is to destroy freedom of speech. :) Or at least build a totalitarian society where internet use is limited only to the elites. Best solution might be to simply kill 99% of the population through mass genocide.
I reported the dude for harassment...
and Sai, you're work is pretty cool :) keep going and you'll make it!
This looks sooo good and natural. Absolutely love it!
This guys are the real immersion scientists.
I'd like to see it performing together with actual game mechanics besides walking - like swinging weapons, being hurt and jumping repeatedly for no reason like an idiot, which we all love doing.
I've seen more than enough "revolutionary" technologies that were absolutely unusable for games. If this one will take a whole lot of crutches to perform character movement with other sorts of simultaneous movements - it is just yet another fun experiment that has nothing to do with real games development.
RaptorAnton an article and its paper stated that currently it doesn't work with complex movements such as hand movements, interacting with objects in the scene or climbing steeper slopes.
So yeah, whether or not future research will allow it to efficiently make a character run up a steep slope while playing with his chode is still unknown :)
You could just animate the hand movement by hand. Combining this N.N. with an arm movement engine and a system for fixating the feet would result in stellar animation.
Looks very similar to Uncharted
Nikita yesss
For the people confused by this video, it's a replacement for state machines and blendtrees for animation systems, not animators.
Simply amazing. I hate, and I mean HATE, playing games without realistic movement- and this looks absolutely perfect!
Isn't Ucnharted 4 very close to this ?
Yeah probably, exept maybe with the "stepping over small obstacles"
Hornyblend I said that. Wanna play with me?
I feel like the Tales series (notably Zestiria and Berseria) must have also utilized a similar neural network. Can someone confirm?
no.
it just has fluid animation with consistent physics and IK, not "very close to this"
animators need not worry just like mo-cap its an additional tool that allows animator to put out better quality not replace the m
well it uses like 50 animations to blend between them procedurally so the animators job is not by far outdated
mmm.... I would say AI will probably be able to do this too in not too long - Honestly everyone's job is in danger in the coming 30 years lmao
Truly outstanding work. Congratulations on the fantastic results and thank you for all of the hard work. I'm always thrilled to see new steps forward (no pun intended) in software development that cuts down on some of the tedious tasks of content creation.
Commentary aside. This is some really awesome tech. I can instantly appreciate this. I hope you'll make some good money off of this because this is truly awesome.
Impressive, but isn't it heavy on resources?
There are numbers in the paper. On an i7-6700, lowest CPU cost mode take 0.8 ms to evaluate but 125 MB of memory; lowest memory cost mode takes 1.8 ms to evaluate but only 10 MB.
theorangeduck.com/page/phase-functioned-neural-networks-character-control
0.8ms but for? 0.8 ms to generate path between 2 points? 0.8ms for each frame? Just trying to imagine if it would be possible to implement that in game, where you need to tick a lot of entities in limited time/resources, or computers are still too weak for that. And if that 125MB will be this same for multiple entities, or some of memory can be shared.
But still, looks pretty great, stairs still seems te be a bit awkward, but probably this can be adjusted with some additional work. Already epic for animations, not sure if ready for real-time stuff.
Yea, if you've got the entire game on the main thread. Other than that, it should be ok : )
If this is 0.8ms for each frame, then not really... if you creating a game, and have like 30 characters that needs to be animated in real time... you can't just create new thread for each of them, most of PC will be not able to handle this + you will need a lot of RAM.
Whether that's affordable depends on your game and target hardware, not every game needs 30 full quality characters animating at 60 Hz simultaneously. You wouldn't necessarily have to run this at the full tick rate either, you might be able to halve or quarter it and interpolate or combine with IK or something. I'm not too knowledgeable about NN approaches for animation so I'm not sure how this compares to others, but just generally the quoted costs wouldn't make me immediately disregard this.
Star Citizen?
But it will not :)
I have no idea how this works, but it's fantastic.
unbelievably smooth! you guys rock!
that dude is drunk
Pirate
Back in my days, gamers need to press the jump button to jump and climb and shit..
dont you get tired of constantly trying to avoid various obstacles, pitfalls, or being mindful of your actions in general? GOOD NEWS! now you just sit back and feel smooth about the movements as your computer finds a way to complete the objective on its own, not to worry! you can still feel proud, you did have to press forward afterall!
I feel like both of you missed the entirety of the point of this video.
It's about getting realistic animations, not auto-jump/climb or invisible walls. Those two are nothing special or interesting.
Or you are just trolls which is also an option as always with such comments.
Jesus these comments are stupid.
Ooooo I like that idea, where you look is where you're character looks and their walking is different, like walking backwards and sideways while facing your target of vision
Game Studio wars aside, Neural networks are continuing to change (and optimize) the way we play games. Real time neural network driven physics reduce game download sizes in animation data, and lessen the load on the CPU (by a fraction per iteration that is far less than the amassing of floating point vector math sent to our GPUs currently). Can't wait to see this in the GPU market, hardware-accelerated neural net driven physics. Exciting.
sell it and you will be rich
And how it works?
He can patent it.
I think you're right, but if that becomes industry standard, every company would pay him a lot of $$$ for solving their problems.
EDIT: That sounds weird... I mean HIRE him to solve other problems they have.
You actually can patent a method of doing something. It has been done several time before in history. But that doesn't mean he should.
And personally i find that kind of behavior anti-productive for society as a whole, this mentality is pretty prevalent in the industry and there is a kind of silent agreement to not engage in that bullshit.This isn't a product, it's a technological advancement and theory, applied to show the results.
He _could_ repackage it and sell it as middle ware, or go the route of consulting to companies to implement it. But fuck all that. _THIS_ video is a step forward in technology and knowledge shared freely with the wider community. It warrants the author respect and appreciation. Be thankful for the existence of institutions and individuals who spend their time in pursuit for knowledge.
That's exactly how it works. It'll be licensed to game engines & studios.
THIS SHOULD BE ON THE MARKETPLACE !
Nice work. Especially the terrain work.
This is ridiculously good. Bravo
this was actually incredible!!!
Thats nice. When I think about this being in all games...its great!
This is extremely impressive!
Insanely cool. Definitely gives me inspiration to get back into the technical/programming side of game animation.
Amaizing. Great job!
Incredible work again!
This looks amazing I'm stunned
This is definitely the future of videogame animations
Video Kojima is definitely using this method for Death Stranding, you can tell from the gameplay. Nice to see something like this at work
Beautiful work!
beautiful work, what a novel implementation!
For Honor does this and it just looks damn good.
Fantastic job, how fluid the movement was wow!
You guys are awesome! Can't wait to play a game with such incredible capabilities!
Well great. You've taught the machines how to mimic human mannerisms and physics.
Step 79 of 284 of Skynet achieved.
Awesome work guys, this is state of the art!
This looks incredible cool!
This is so splendid. Such an impressive neural network. I would be interested to read/learn more about the phasing mechanism - it gives such mind blowing and smooth results!
This is pretty impressive!
Very interesting work, looking forward to more stuff like this. These methods could be implemented in numerous ways.
Holy crap... This is brilliant
This can make a good football/soccer game. I want to be a game developer, now I can only dream on what I could do with this amazing thing you made.
This looks awesome, thanks for your work on this. Nothing breaks immersion like janky character movement.
Very nice results. It would be interesting to see something like this also take props/tools into account. For example an old man walking with a cane, or a worker walking with a shovel, or carrying a heavy box.
This is so good !! Nice work
This is brilliant work guys, it looks very convincing compared to what I'm seeing even in upcoming video game release.
Thank you Yoshiboy2, very cool
Well done, this is quite impressive.
WOOOW that is simply the best looking charachter movement iv ever seen
Awesome work! Hope to see this presented at Siggraph 2017!
Simply impressive!
awesome!!!!! I want it in all games, right now!