"I fear not the AI that has trained in 10 billion simulations once. But I fear the AI that has trained in one simulation 10 billion times." - Bruce Lee probably
Ai trained in 10 billion simulation will kill the other Ai in all the simulations except that unique simulation in which he was trained 10 billion times -Zombie killer 2021
This is a very good comment. In the real world people don't have unlimited energy reserves. That's why most average people can barely fight for 3 minutes on average before gassing out. Since muscles require energy and different actions use different amounts of energy, your idea makes a lot of sense for realism. In the real world, if you had unlimited energy, you would act completely different.
Agreed. In addition to that the reward mechanism needs to be more complex than "I touched the enemy and didn't get touched." I note the bots are just tapping each other for the most part, though there was that one decent knock. But things like momentum should ramp up the reward considerably so that a proper full contact punch is preferenced over light jabs (although this is actually not the case for fencing).
It should also have some kind of way to alternate punches occasionally because something that was obvious is that the computer is always going to go for the most "advantageous" move it can seeing as it swung with the leading hand 100% of the time showing it has no interest or understanding of diversionary tactics and advanced problem solving.
An important thing to remember with these learning algorithms is that they're going from "less skilled than an infant" to "basic boxing" in *a week*. It sounds like a long time because we're used to computers operating in milliseconds, but imagine going from not realizing you have limbs to walking around and throwing punches in 7 days. That's a huge amount of learning, even in this simplified system.
It's a week of computing time, it's not a week inside the simulation. The simulation goes on for a billion steps at 30 Hz, so about 30 million seconds or about a year around the clock, or 4 hours of playtime every day for 6 years, which is interesting because it's in the same ballpark as the time humans require.
@@DontfallasleeZZZZ Time is relative based on perspective and scale of where it's measured. A great example is Geological time, vs the span of human existence. Then you go out to space and have all kinds of weird time effects like time dilation. In this case we have "computer" time.
Furthermore even if we talk about a "big" neural network it's nothing compared to a human brain. Also the sensors i.e. the input variables are hardly comparable to what we can use and train on which of course goes hand in hands with the many sensory values to process. In the early days someone working with AI claimed we try to make a creature with insect-like neuron knots behave intelligently. Considering that it's amazing what can be done.
The funny thing is that due to their bodies having the same measures, they've learned that cross countering was the best strategy. It would be interesting to see the same experiment, but with different measured characters.
Yes, exactly. Also, they're point fighting. Not trying to win by disabling the opponent. Id like to see this again with better AI boxers, and each have a health bar. Max damage for certain headshots, certain bodyshots (liver, solar plexus maybe). And then see what they come up with. Will they take little jabs to the face to land a huge cross to the chin? That would be amazing to see the techniques after thousands of simulations
@@jimmythe-gent Rope-A-Dope?! BTW using an AI-engine they came up with a chess engine (AlphaZero) that beat all other more-algorithmic engines and 'solved' the game of GO, and poker, that previously were thought to be immune. Progress for the machines, bring 'em on!
Once the AI get so advanced you should save copies of individual behavioural patterns, name them, and start an arena. Maybe live stream fights? Would this not be awesome?
I think they're going to load up simulations of prime Ali vs Tyson. Then go to other sports like prime Michael Jordan vs Kobe Bryant or Lebron to see who is the goat.
Having taught martial arts for a few years, it was surreal seeing how the ai was moving at different stages of their learning process. It looked remarkably like someone actually learning to fight.
I think the reason the blue AI kept losing when the red AI fell over is due to over training for one possible outcome. The blue AI expected an attack and only knew how to win when it was being attacked but didn't know how to proceed when not attacked. It overcompensates for the expected attack and then falls over.
@@comradpingu5745 I'm guessing you're joking, but the serious answer is no. The A.I. just does what is going to win it the match in the best (probably fastest) possible way. Though its reactions all depend on what values it gets for inputs.
I fear it may turn into pay to win because it's expensive to train NNs. Those with the money could train far faster and therefore get much smarter AIs.
It's interesting as the style the simulation parameters create is quite accurate for a certain type of fighting. I assume the instructed goal is to touch against the opposition's head and not get hit by the way they are moving is very remniscent of points based boxing jabs but more accurately the probing stages of bare knucklefights where the consequences of getting caught by grazing hits is much worse so lunging and swinging back is a solid strategy when you can win with stiff jabs.
As a fencer I'm really looking forward to what will it really evolve into. Just like when AI started becoming better in chess, we learnt a lot from them, and I believe same can be applied to more physical sports. What a time to be alive!
the thing is that you can actually learn loads from AI, you can literally makes one live infinite lifetimes just to do one thing, the only problem is most of them create some glitch that is impossible for humans to do lol.
@@unknownr3802 then improve the physics of the simulation and the limitations of the bot so it approaches reality. But you do have a good point. Turn based strategy games are somehow fundamentally different than sports for learning from AI
@@yevgeniyvovk9788 that is true, i saw a paper one time where it was sort of physics based, it had joints that get locked after a certain angle etc but the machine found a way to glitch the joints into spasming the joint then flying the machine into a certain direction, then the machine learnt how to control where it got flown to, so sometimes even adding more physics and limitations actually help you robot glitch it, but you could definitely improve it to some degree.
We can learn from AI by their decision making capability not their executions. They will be perfect in term of executions or find glitches in the simulation That is why we can learn a lot from turn-based game. Talking about decision making, sports have been using data science for quite sometimes now.
Kind of crazy to hear that they only use 90 Seconds of motion capture data. Too bad you don't have 90 seconds of Bruce Lee motion capture data. Be interesting if old movie footage could be used as motion capture.
in fact being able to use general video footage would be fantastic. one could teach avatars to dance on their own, walk down the street on their own - interacting with each other to create background street action. imagine the same for a huge crowd or a restaurant scene with waiters, diners, baristas, etc. creating all this background action is very expensive yet improves the believability of a story greatly. fine if you have the budget, not so good if you don't - until we avatars like these. the possibilities may soon be limitless.
You can re-create it as motion capture data, or maybe use AI to turn it into motion capture data but handrafting would be better with today's technology.
@@zsomborszepessy4351I know of a few games that have that adaptive difficulty, but it’s usually a mechanic and not ai actually learning to kill you. I know in Tlou for example, enemies will flank you while others will aim at where they think you are until you make a move. So many times I’ve panicked when that would happen so I thought I could be quicker and headshot an enemy, but they have the advantage already aiming at me so I get hit 90 percent of the time. Enemies will also learn to try and stealth attack you especially when they’re the last ones standing. I know games like stalker and the old fear games have some of the best ai, but idk how well they adapt to what you’re doing. The ai in tlou 2 is really tough on the hardest difficulties, but I think that’s mostly because they basically have aimbot. If I’m in their sights, they will almost always headshot me no matter how much I’m moving. You have to basically catch them by surprise since most head on fights will be game over.
The rear hand/power hand should offer an increased reward (just like a real cross offers increased power and damage if it lands) over the jab hand to stop it from becoming a stiff jab stalemate every exchange, and having fighters with slightly different dimensions as many others have said would also be a good change
@Siss Derella because violence is human nature. it is in our blood, there is nothing to "overcome". millions of years of evolution has made us, and there's been a lot of violence in that time. in fact, those who were weakest and incapable of defending themselves were weeded out.
@@Anon-nv7bp "because violence is human nature." No, survival is human nature just like any other animal. Humans do not wage war because they enjoy murdering and getting killed. Its to acquire resources for survival or just greed.
I want to see this in games with dynamic outcomes for side characters. It would be awesome to also somehow be able to help improve their potential by helping a character, giving it access to a few more million steps of training data or something like that :3
so like pokemon, but instead of food giving it preset level powerups, the in game food lets your agent train for 250M more steps. BRB opening up unreal engine
would be super cool yeah. One bit that is a bit.. a bit of a pity is that both seem to have the same model, would be very interested to see how they would perform toward opponents with different tactics etc. then also for your game idea, you can choose not only your favorite to train, but also who they train against!
1:13 In fighting games, we call this "conditioning" the opponent. We train the opponent to react to an action in a certain way, and then we punish them for reacting in that predictable way.
As an ex fencing coach myself, I was watching the boxing component going "Those lunges look more akin to epee fencing". Sure enough, 10 seconds later it's applied to fencing, albeit foil hitbox. Very interesting stuff. Maybe my combination of fencing knowledge, gamdev and machine learning could have some niche crossover here. Hmm. Edit: One major difference I could see is that the swords in this simulation didn't flex the way regular ones do, and particularly using that foil hitbox, that leads to extremely linear plane along the direction of the piste. Giving the AIs the ability to flick the blade with the appropriate movement and inertia would lead to far more dynamic fighting, since there would be an incentive for that lateral motion. Better still would be this for simulating epee, since it makes sense to disregard the foil's priority rules. In which case they would need the bell guard protecting the hand, which would be very easy to add to the simulation. Giving the AIs both the ability flick the weapon over the bell guard the way humans do, and the reward of doing so, would lead to more realism - and also clear the way to more high stakes combat options, like rapier to first blood.
@@twicevictorious8828 Sorry, I just saw this. The primary issue with taking it to VR is that there's no proper haptic feedback; the collision of the blades for parrying, beat attacks, glissade, etc is pretty core to the sport. Additionally, even if you could manage that, you'd want it on a system that can fully track your whole body, otherwise you couldn't specifically move parts to dodge. :)
It would be interesting to see how these AIs develop different strategies for different physiques. Different height and reach would be pretty simple to implement.
I would love to see a video where you teach the ai different fighting styles and then had them fight other styles to see how they would adapt over time
That would be a great training tool for MMA professionals preparing for a fight. Not sure how hard it would be to incorporate grappling and submissions though.
This is so exciting for someone like me who loves VR boxing games. They’re great and all but the enemies just aren’t dynamic enough to keep it interesting for long. Can’t wait till this is realtime!!!
@@unliving_ball_of_gas You don't need to store 1B steps of training data, you only need to store and run the final AI. The training will have been done pre-release of the game. After that you don't need the data anymore.
@@alansmithee419 Huh, that's how it works? Because I have this game "Evolution" basically you build a body and a neural network and it'll train itself to walk, run climb up stairs, jump, etc. and the more you train it, the laggier it becomes. You could download the game on mobile/pc if you like. Edit: The name is "Evolution" by Keiwan Donyagard
@@unliving_ball_of_gas Optimizing a trained model is a whole other thing. Yes, if it's not optimized it will become so laggy to run in ordinary computers. I don't think that Evolution game has good (or even any) optimization process.
Looks like the Toribash characters. Wonder if they could put the two together, either with players fighting against these AIs or providing training data for the AIs.
@@CameronKujo I haven't played it in ages, I just remember the very distinctive models. I didn't think they were the same, but they look similar, and I was more wondering if they would be compatible enough to quickly retrain the AI for toribash.
I cant wait until we can watch AI Boxers box , or create our own AI and challenge other players AI, with different fighting styles. it doesn't even have to be limited to boxing. man that would be so cool
@@ShawnJonesHellion real AI will exist, mainly because governments don't understand the dangers. in the words of elon musk: i tried to get people to slow down AI, to regulate AI, this was futile. i tried for years. main reason he's working on neurolink btw. if you can't beat them, join them
Lately I've been wondering about how long it's going to take before characters in video games are able to move realistically like this, rather than using premade animation cycles and other "shortcuts". This video talks about the millions of steps in the learning process, but once the learning has gotten to a sufficient place, can that movement "model" be applied to characters in a game? Is it versatile enough for that? Or is the issue more about processing power?
you could probably train an AI to create realistic movement animations, but integrating an ai just for character movement is out of the question. It would be way too resource intensive and highly inefficient.
There’s something hilarious about that blue man trying to run past the red one. It’s like he’s spawned in and immediately starts frantically sprinting like a cartoon character 😂
It's freaky how close to real life movements the AI can get. You could of honestly said that footage was mocapped and I wouldn't have argued. Imagine if we instructed it to fight using weapons, or even group fighting! The possibilities are really endless..
Stage 1 of boxing looks like someone snuck into the lab & cheekily programmed the AI to think that it was a Jedi Master that could vanquish it's opponent by simply staring them down & using the Force to knock them over.
I LOVE the AI fencing. I've fought Historical European Martial Arts and SCA heavy rapier/cut and thrust for 20 years, so I think it's really cool to see the AI learn how to sword fight
The AI get into an interesting jab war but I wonder if it's because both characters have exactly the same dimensions or are they marginally different? In real boxing, opponents can be different shapes and sizes with variable arm length, height, reach, punch range, speed, stamina etc... If you had two such AI of different dimensions, perhaps the fight would turn out differently as the AI which is getting out-jabbed might change it's tactics completely?
Could also be due to the physics simulation. It seemed like if the models swung too hard they would take themselves off their feet so maybe the model found that the best solution was to just go for light jabs
Hope there are follow-ups on this. Very interested to see how the AI moves with a bit more parameters, like requirements for the acceleration of the strike, or achieving a peak transfer of inertia. I wonder if the simulated weight of the limbs was especially accurate, it seemed that way. It would be amazing to see how the AI moves to deliver and avoid perfectly applied striking. Cool to see that in a lawless sandbox, the real world method of movement is still validated.
So amazing! Thank you for sharing Karoly, just watching them essentially 'learn to walk' is inspiring enough. So amazing watching this stuff! Thank you again for this video!
Seeing previously unknown strategies would be exciting, just as we saw AlphaGo come up with novel strategies. Masters of those games have even stated it's often more interesting to watch the AI matches, than the more familiar/predictable human matches.
I live watching AI “learn” what to do. I also like that you include previous research so that we can come along on the journey. Excellent work, as always!
Actually, I would on the contrary be surprised if the researchers stopped at 130M steps. The reason was mentioned: the agents didn't even had a clue that they could get higher rewards by touching the opponent. So it was mainly a matter of random to start doing so, but until that it would be quite unreasonable to quit. But as always, it was still amazing to watch how did the imitate boxing :)
2:58 the Eureka moment :) amazing result! @Dr - i love the way you've told the story of the process here. In your next vid, could expand on what form the 'reward' takes? I get in principle how this strengthens the neutral path, but what is the general math to it? Thanks for your great work covering these paper dropping advancements ;)
there are no truly optimum tactics in football. the game is fluid and players adapt strategies in real time based on the situation theyre faced with in that moment.
Squaring off in the ring, staring their one another down,waiting until someone collapses from exhaustion alone? Now that's some gentleman's boxing as Andre Filipe originally intended.
And here I am, waiting for AI enemies in games to stop having limited motion rules and no learning. I can't wait for this to be implemented in single player shooters with enemies that adapt to you, extra immersion.
Not gonna happen. At least not in most shooters. AI like that would be a PAIN to go up against, unless they had very well-signaled "barks" for what they're doing. You don't necessarily want the smartest AI in a game, you want the most fun.
@@nerd_nato564 you could hamper it in various ways reduce the time it's allowed to react in, add randomness to its motion so it doesn't have pinpoint precision. Also picking an earlier generation that hasn't become super human yet. I think it would be great. Every game would be different and would be much more like fighting a real opponent
@@rickymort135 That's not the issue though. You don't necessarily want human-like AI in your game. You want AI that is smart enough to not fuck up basic things, but also predictable enough to not feel unfair. Some games would indeed benefit from learning AI like this, but the issue isn't a superhuman AI, it's the unpredictable and creative AI.
I don't know if that would make it more or less fun. The hardest difficulties are already cheating, they know exactly where you are on the map regardless of anything and have a lock on your head from across the map.
@@themonsterunderyourbed9408 That's exactly the point, this sort of enemy AI will not need to cheat in order to feel engaging and difficult. I'm very much into multiplayer games when it comes to the competition, because it's way more engaging to fight real people, but I hate the communities of most multiplayer games. If someone can bring real human playing abilities to AI in single player games, I'd be thrilled.
Would be interesting to see several ai bots put in a incredibly hard puzzle game where they need to cooperate to win.IT also will have this physics engine ofc and would be hilarious.
Would've been cool if they 'saved' the AI after each X steps, and then have them fight each other, just to show how drastically the AI improved and how hard it can demolish its previous iterations.
I think you would get more interesting results and different tactics on your agents if they had different length of their limbs, because now it looks like they came up with symbiotic situation where they have pretty much the same odds to touch the opponent and get same amount of points each while mirroring each other's moves and reactions. It's a continuous stalemate.
I want to see these trained for 100T steps. Would they invent a new style of boxing? Would they be able to teach us new tactics? Can't wait to see what kind of martial arts AI could come up with in 5 years.
Stage 3 Boxing AI looks less like learning to punch and more like dancing, but at least that got them to move fist to opponent's face, even if they did it by moving everything closer..
OMG I just predicted the future 😂, later in the video, 2minutepapers (sorry if that's not your name, youtuber, I can't check fsr) said that its like "two drunkards trying to duke it out" 😂😂😂
they are currently working off of >circle opponent >jab >evade jab hopefully they can handle additional instructions >circle opponent >chose between a jab, feint, or hook >decide which hand to throw this attack >decide to aim low or high >decide to block or evade >decide to change circle direction this way it has a range of attack options and possible responses to an opponent but I fear this would take more than a week to train an AI how to do. but then we can make them grapple. grapple over each other or grapple over $Object (be it a piggy bank, brick, gun, sword, game objective ball, toy etc)
"I fear not the AI that has trained in 10 billion simulations once. But I fear the AI that has trained in one simulation 10 billion times." - Bruce Lee probably
No bro, this was Sun Tzu. Page 205 guyz believe me
Ai trained in 10 billion simulation will kill the other Ai in all the simulations except that unique simulation in which he was trained 10 billion times -Zombie killer
2021
@@warrenarnold why denigrate yourself and a good joke with a terrible joke like that bro
VERY GOOD
Nah, it wasn’t Bruce Lee who said that, it was Luce Bree.
we should try to make a simulation where movement costs them energy to see if they would avoid too many small and fast movements
This is a very good comment. In the real world people don't have unlimited energy reserves. That's why most average people can barely fight for 3 minutes on average before gassing out. Since muscles require energy and different actions use different amounts of energy, your idea makes a lot of sense for realism. In the real world, if you had unlimited energy, you would act completely different.
Very true
probably it will make computing 1 year
Agreed. In addition to that the reward mechanism needs to be more complex than "I touched the enemy and didn't get touched." I note the bots are just tapping each other for the most part, though there was that one decent knock. But things like momentum should ramp up the reward considerably so that a proper full contact punch is preferenced over light jabs (although this is actually not the case for fencing).
It should also have some kind of way to alternate punches occasionally because something that was obvious is that the computer is always going to go for the most "advantageous" move it can seeing as it swung with the leading hand 100% of the time showing it has no interest or understanding of diversionary tactics and advanced problem solving.
"Everyone has an algorithm 'till they get punched in the mouth." - AI Tyson
underrated comment
here at 7 likes remember me when you get a thousand! :)
AI Tyson, i knew him since the gpt lab
AHAHAHAHAHAHHA
This comment made me cry... lool so damn funny good shit
An important thing to remember with these learning algorithms is that they're going from "less skilled than an infant" to "basic boxing" in *a week*. It sounds like a long time because we're used to computers operating in milliseconds, but imagine going from not realizing you have limbs to walking around and throwing punches in 7 days. That's a huge amount of learning, even in this simplified system.
It's a week of computing time, it's not a week inside the simulation. The simulation goes on for a billion steps at 30 Hz, so about 30 million seconds or about a year around the clock, or 4 hours of playtime every day for 6 years, which is interesting because it's in the same ballpark as the time humans require.
@@DontfallasleeZZZZ great points, I was wondering about that!!
@@DontfallasleeZZZZ Time is relative based on perspective and scale of where it's measured. A great example is Geological time, vs the span of human existence. Then you go out to space and have all kinds of weird time effects like time dilation. In this case we have "computer" time.
Furthermore even if we talk about a "big" neural network it's nothing compared to a human brain. Also the sensors i.e. the input variables are hardly comparable to what we can use and train on which of course goes hand in hands with the many sensory values to process.
In the early days someone working with AI claimed we try to make a creature with insect-like neuron knots behave intelligently. Considering that it's amazing what can be done.
But why do they always start from scratch. Couldn't they utilized pretrained networks or classic algorithms for a starter and build on that?
"After 130 million steps of training, it can not even hold it together"
My life
Lmao
😔
rip
:'(
"look what it does at *420* million" aww it tries to take a hit
The funny thing is that due to their bodies having the same measures, they've learned that cross countering was the best strategy. It would be interesting to see the same experiment, but with different measured characters.
Yes I guess small differences in mass would create a much more divers game.
where are those cross countering, i see none
Yes, exactly. Also, they're point fighting. Not trying to win by disabling the opponent. Id like to see this again with better AI boxers, and each have a health bar. Max damage for certain headshots, certain bodyshots (liver, solar plexus maybe). And then see what they come up with. Will they take little jabs to the face to land a huge cross to the chin?
That would be amazing to see the techniques after thousands of simulations
@@jimmythe-gent Rope-A-Dope?! BTW using an AI-engine they came up with a chess engine (AlphaZero) that beat all other more-algorithmic engines and 'solved' the game of GO, and poker, that previously were thought to be immune. Progress for the machines, bring 'em on!
@@raylopez99 yeah the ai machine beat that other chess ai- i think it was called "fish ...xxx..something"
0:50
I mean, I would also stop playing if my friend suddenly had a seizure.
Blue: And then he turned himself into a ball. The funniest shit I've ever seen.
@@BVW16 Kek
This comment made my day. Thank you sir.
lmao, made my day XD
I think he "juked" the other player
I’d love to see a boxing simulation where one character has a shorter build or shorter wingspan and see how it adapts to its disadvantage
that would be cool
HEAD MOVEMENT BABY BOB AND WEAVE
@@boyarvalishin9565 LET EM HAVE IT MAC BABY
I'd love to see boxing with wings
Yeah that'd be really cool
Once the AI get so advanced you should save copies of individual behavioural patterns, name them, and start an arena. Maybe live stream fights? Would this not be awesome?
Yeah men that's incredible idea
I think they're going to load up simulations of prime Ali vs Tyson. Then go to other sports like prime Michael Jordan vs Kobe Bryant or Lebron to see who is the goat.
@@jamescarbon3853 Everyone gangsta till you see one of them doing the ali shuffle
Noted
@@builderdude9488 AI could probably learn to showboat too if you give them enough time lmao
Having taught martial arts for a few years, it was surreal seeing how the ai was moving at different stages of their learning process. It looked remarkably like someone actually learning to fight.
"this ai showcases agents that can learn boxing"
red guy falls for no reason whatsoever
"wait a minute -- that's the soccer ai, sorry"
lol so true
@@yaboi1288 neymar AI
*football
@@GigaChadL337 European lies
@@GigaChadL337 It can be either holy fuck, depends on where you're from
I think the reason the blue AI kept losing when the red AI fell over is due to over training for one possible outcome. The blue AI expected an attack and only knew how to win when it was being attacked but didn't know how to proceed when not attacked. It overcompensates for the expected attack and then falls over.
same thought
So by training AI to do things for rewards, it created AI capable of being ass-holes? Interesting...
Just like real life humans (most of the time)
a true counterpunching merchant
Basically it hasn’t been studying the game itself, but another AI’s analysis.
@@comradpingu5745
I'm guessing you're joking, but the serious answer is no. The A.I. just does what is going to win it the match in the best (probably fastest) possible way. Though its reactions all depend on what values it gets for inputs.
Now I want to see an AI version of Robot Wars. Well described combatant rules, unlimited training. Last Bot standing wins.
I fear it may turn into pay to win because it's expensive to train NNs. Those with the money could train far faster and therefore get much smarter AIs.
@@techpriest4787 good point. 200 hours on a cloud standard set up then. I'm hyped to see a dude find an iron skillet and ko the opponent.
bring back Bamzooki!
@@techpriest4787 tbh with you I don't mind I want to see the best
Like Battletech style? That'd be cool.
This was strangely motivating we all start off stumbling but over time we learn and grow I’m glad these two stickmen can now box
It's interesting as the style the simulation parameters create is quite accurate for a certain type of fighting. I assume the instructed goal is to touch against the opposition's head and not get hit by the way they are moving is very remniscent of points based boxing jabs but more accurately the probing stages of bare knucklefights where the consequences of getting caught by grazing hits is much worse so lunging and swinging back is a solid strategy when you can win with stiff jabs.
As a fencer I'm really looking forward to what will it really evolve into. Just like when AI started becoming better in chess, we learnt a lot from them, and I believe same can be applied to more physical sports. What a time to be alive!
the thing is that you can actually learn loads from AI, you can literally makes one live infinite lifetimes just to do one thing, the only problem is most of them create some glitch that is impossible for humans to do lol.
@@unknownr3802 then improve the physics of the simulation and the limitations of the bot so it approaches reality. But you do have a good point. Turn based strategy games are somehow fundamentally different than sports for learning from AI
@@yevgeniyvovk9788 that is true, i saw a paper one time where it was sort of physics based, it had joints that get locked after a certain angle etc but the machine found a way to glitch the joints into spasming the joint then flying the machine into a certain direction, then the machine learnt how to control where it got flown to, so sometimes even adding more physics and limitations actually help you robot glitch it, but you could definitely improve it to some degree.
We can learn from AI by their decision making capability not their executions. They will be perfect in term of executions or find glitches in the simulation
That is why we can learn a lot from turn-based game.
Talking about decision making, sports have been using data science for quite sometimes now.
I'm so sorry to hear that ur a fence
Kind of crazy to hear that they only use 90 Seconds of motion capture data.
Too bad you don't have 90 seconds of Bruce Lee motion capture data.
Be interesting if old movie footage could be used as motion capture.
in fact being able to use general video footage would be fantastic. one could teach avatars to dance on their own, walk down the street on their own - interacting with each other to create background street action. imagine the same for a huge crowd or a restaurant scene with waiters, diners, baristas, etc. creating all this background action is very expensive yet improves the believability of a story greatly. fine if you have the budget, not so good if you don't - until we avatars like these. the possibilities may soon be limitless.
You can re-create it as motion capture data, or maybe use AI to turn it into motion capture data but handrafting would be better with today's technology.
It can. As long as the footage is full body and high enough resolution and we can ai upres it then derive motion capture data from simple video
Someone should choreograph his moves in slow motion and speed it up infinity to match Bruce Lee's speed
It may be possible to skip the motion capture data and just train longer
Should’ve taught Tyron Woodley some of this
@Mal Theri tron is trained to 200mil times only, de does dancing part Allright
lmao! 🤣
Tyron's agent didn't learn to throw the knockout punch
What a time to be alive
@@ayoubmerzak1733 that's the only punch he learned actually...
Would love to see a future where AI in video games can dynamically adapt to what your doing, hopefully without them becoming impossible to defeat.
I see a rise in controller buying
thats already happening, been happening for a while now actually
That would be interesting... a video game that gets harder, the more you play against it. I think this would really help people to learn strategy.
@@zsomborszepessy4351I know of a few games that have that adaptive difficulty, but it’s usually a mechanic and not ai actually learning to kill you. I know in Tlou for example, enemies will flank you while others will aim at where they think you are until you make a move. So many times I’ve panicked when that would happen so I thought I could be quicker and headshot an enemy, but they have the advantage already aiming at me so I get hit 90 percent of the time. Enemies will also learn to try and stealth attack you especially when they’re the last ones standing. I know games like stalker and the old fear games have some of the best ai, but idk how well they adapt to what you’re doing. The ai in tlou 2 is really tough on the hardest difficulties, but I think that’s mostly because they basically have aimbot. If I’m in their sights, they will almost always headshot me no matter how much I’m moving. You have to basically catch them by surprise since most head on fights will be game over.
The rear hand/power hand should offer an increased reward (just like a real cross offers increased power and damage if it lands) over the jab hand to stop it from becoming a stiff jab stalemate every exchange, and having fighters with slightly different dimensions as many others have said would also be a good change
Now we can recreate a virtual Colosseum the Romans could only dream about with animals, humans and even aliens fighting against one another.
@Siss Derella nah I want the Virtuasseum
@Siss Derella “AcT LiKE aN InTeLlIgEnT sPecIeS”
@Siss Derella because violence is human nature. it is in our blood, there is nothing to "overcome". millions of years of evolution has made us, and there's been a lot of violence in that time. in fact, those who were weakest and incapable of defending themselves were weeded out.
@Siss Derella laame
@@Anon-nv7bp "because violence is human nature." No, survival is human nature just like any other animal. Humans do not wage war because they enjoy murdering and getting killed. Its to acquire resources for survival or just greed.
I want to see this in games with dynamic outcomes for side characters. It would be awesome to also somehow be able to help improve their potential by helping a character, giving it access to a few more million steps of training data or something like that :3
So instead of the side character walking into walls, it just falls down immediately.
so like pokemon, but instead of food giving it preset level powerups, the in game food lets your agent train for 250M more steps. BRB opening up unreal engine
would be super cool yeah. One bit that is a bit.. a bit of a pity is that both seem to have the same model, would be very interested to see how they would perform toward opponents with different tactics etc. then also for your game idea, you can choose not only your favorite to train, but also who they train against!
@@yevgeniyvovk9788 Oohhh that sounds really cool
@@LunaticCharade like boxing archetypes: southpaw, counterpuncher, ect... great idea
"and, you know what's coming?"
Boxing AI vs. Fencing AI?
AI MMA.
Fencing AI will probably win because of it's reach, historically weapons seem to be an advantage.
... vs. Gorilla AI. (Mortal Kombat's fonts) FIGHT!
Alien AI vs. Predator AI
@@nonamenoname1942 Finish Him!!!
1:13 In fighting games, we call this "conditioning" the opponent. We train the opponent to react to an action in a certain way, and then we punish them for reacting in that predictable way.
2:32 my boxing teacher when I not ready for practice in 1 minutes fr
As an ex fencing coach myself, I was watching the boxing component going "Those lunges look more akin to epee fencing". Sure enough, 10 seconds later it's applied to fencing, albeit foil hitbox. Very interesting stuff.
Maybe my combination of fencing knowledge, gamdev and machine learning could have some niche crossover here. Hmm.
Edit: One major difference I could see is that the swords in this simulation didn't flex the way regular ones do, and particularly using that foil hitbox, that leads to extremely linear plane along the direction of the piste. Giving the AIs the ability to flick the blade with the appropriate movement and inertia would lead to far more dynamic fighting, since there would be an incentive for that lateral motion.
Better still would be this for simulating epee, since it makes sense to disregard the foil's priority rules. In which case they would need the bell guard protecting the hand, which would be very easy to add to the simulation. Giving the AIs both the ability flick the weapon over the bell guard the way humans do, and the reward of doing so, would lead to more realism - and also clear the way to more high stakes combat options, like rapier to first blood.
Dude if you could design a game or simulation with that fencing experience and knowledge I'd play the hell out of it even after quarantine
@@twicevictorious8828 you should try Hellish Quart in the meantime.
Well, you can still try to make a decent bot for your potential VR-game.
@@twicevictorious8828 Sorry, I just saw this.
The primary issue with taking it to VR is that there's no proper haptic feedback; the collision of the blades for parrying, beat attacks, glissade, etc is pretty core to the sport. Additionally, even if you could manage that, you'd want it on a system that can fully track your whole body, otherwise you couldn't specifically move parts to dodge. :)
Thank you
It would be interesting to see how these AIs develop different strategies for different physiques. Different height and reach would be pretty simple to implement.
After 10b steps the AIs learn to just abandon the ring and go to bar and called it a day
I would love to see a video where you teach the ai different fighting styles and then had them fight other styles to see how they would adapt over time
That would be a great training tool for MMA professionals preparing for a fight. Not sure how hard it would be to incorporate grappling and submissions though.
0:15 i don’t know why but this section made me laugh so hard
Bro this so meme
This is so exciting for someone like me who loves VR boxing games. They’re great and all but the enemies just aren’t dynamic enough to keep it interesting for long.
Can’t wait till this is realtime!!!
The fighting is realtime. It's the training that took ages. Once it's trained it can be used for whatever you want almost instantly.
@@alansmithee419 That's the problem. I don't think 1B steps of training-data could fit in a tiny phone-size storage.
@@unliving_ball_of_gas You don't need to store 1B steps of training data, you only need to store and run the final AI.
The training will have been done pre-release of the game. After that you don't need the data anymore.
@@alansmithee419 Huh, that's how it works? Because I have this game "Evolution" basically you build a body and a neural network and it'll train itself to walk, run climb up stairs, jump, etc. and the more you train it, the laggier it becomes. You could download the game on mobile/pc if you like.
Edit: The name is "Evolution" by Keiwan Donyagard
@@unliving_ball_of_gas Optimizing a trained model is a whole other thing. Yes, if it's not optimized it will become so laggy to run in ordinary computers. I don't think that Evolution game has good (or even any) optimization process.
Looks like the Toribash characters. Wonder if they could put the two together, either with players fighting against these AIs or providing training data for the AIs.
Hey, do you know what is that? I thought it was Toribash, but it isn't, how can i test the AI in there?
@@alfredogonzalez2576 You'd have to check their paper, but I don't think it's Toribash.
@@PeterBarnes2 it’s not toribash. And if it is, it’s an old model. I can’t believe people still play the game. Props
@@CameronKujo I haven't played it in ages, I just remember the very distinctive models.
I didn't think they were the same, but they look similar, and I was more wondering if they would be compatible enough to quickly retrain the AI for toribash.
Goddamn, I haven't heard the word 'Toribash' in years.
0:45
RED used 'Curl into a ball'!
It was super effective!
The first example is more of an illustration of the limitations of the model rather than of it uncovering some special "hypnotising" technique.
1:30 "AI agents that can learn boxing" ...*proceeds to fall over*.
To be fair, if I'm playing that game and the opponent breaks all their bones at once I'll also forget about trying to cross the line
I cant wait until we can watch AI Boxers box , or create our own AI and challenge other players AI, with different fighting styles. it doesn't even have to be limited to boxing. man that would be so cool
That'd be really cool
as if real ai is allowed to exist
There's something similar in starcraft broodwar's SSCAIT. Give it a look!
Epic major scale fighting simulations
@@ShawnJonesHellion real AI will exist, mainly because governments don't understand the dangers.
in the words of elon musk: i tried to get people to slow down AI, to regulate AI, this was futile. i tried for years.
main reason he's working on neurolink btw. if you can't beat them, join them
around 3:28, they really start to exhibit true point scoring strikes, as well as counter punching. Really impressive.
Lately I've been wondering about how long it's going to take before characters in video games are able to move realistically like this, rather than using premade animation cycles and other "shortcuts". This video talks about the millions of steps in the learning process, but once the learning has gotten to a sufficient place, can that movement "model" be applied to characters in a game? Is it versatile enough for that? Or is the issue more about processing power?
probably like 5 or 10 years i would say.
you could probably train an AI to create realistic movement animations, but integrating an ai just for character movement is out of the question. It would be way too resource intensive and highly inefficient.
This man has just taught an AI how to beat the daylights out of something, so beautiful
3:15 a man of culture I see
1:19 This is what we call a feint. Pretty awesome that the AI figured out this concept, albeit a kind of ludicrous version of it.
it looks like high level (or bullshit who knows) Systema
2:47 live simulation of the Logan Paul match
1:07 Homie celebrated by hitting the griddy
"Heres your reward."
"What is it?"
"A single point."
"Okay."
1:43 Me before I even start studying
I recognize the glove-to-glove dancing in a circle as resembling a real heavyweight boxing match
2:01 "I am afraid this is just passing out without any particular benefits."
Huh. I have a few friends who I can say that about...
There’s something hilarious about that blue man trying to run past the red one. It’s like he’s spawned in and immediately starts frantically sprinting like a cartoon character 😂
3:33 Those bots fights like my drunk neighbors. btw which language and engine used to make that?.
They learned to dance faster than to do boxing.
Should have kept learning the dancing moves, then it might have been more fun !
It's freaky how close to real life movements the AI can get. You could of honestly said that footage was mocapped and I wouldn't have argued. Imagine if we instructed it to fight using weapons, or even group fighting! The possibilities are really endless..
That's at least partly because motion capture was used as training data.
1:00
When AI is so good it figures out how to use exploits.
That's a pretty common thing for ai trained to play video games
@@SuLokify
Yeah. I'm thinking devs could use this to help find bugs.
Stage 1 of boxing looks like someone snuck into the lab & cheekily programmed the AI to think that it was a Jedi Master that could vanquish it's opponent by simply staring them down & using the Force to knock them over.
I LOVE the AI fencing. I've fought Historical European Martial Arts and SCA heavy rapier/cut and thrust for 20 years, so I think it's really cool to see the AI learn how to sword fight
The AI get into an interesting jab war but I wonder if it's because both characters have exactly the same dimensions or are they marginally different? In real boxing, opponents can be different shapes and sizes with variable arm length, height, reach, punch range, speed, stamina etc... If you had two such AI of different dimensions, perhaps the fight would turn out differently as the AI which is getting out-jabbed might change it's tactics completely?
Could also be due to the physics simulation. It seemed like if the models swung too hard they would take themselves off their feet so maybe the model found that the best solution was to just go for light jabs
Hope there are follow-ups on this. Very interested to see how the AI moves with a bit more parameters, like requirements for the acceleration of the strike, or achieving a peak transfer of inertia. I wonder if the simulated weight of the limbs was especially accurate, it seemed that way. It would be amazing to see how the AI moves to deliver and avoid perfectly applied striking. Cool to see that in a lawless sandbox, the real world method of movement is still validated.
Another impressive step forward. Keep up the good content
Me every time I fall down from now on: "I'm using hypnotic adversarial collapsing on my opponent."
80 year old boxer be like 0:10
I'm tooo oooold fooorr thiiis ( "back cracked"), aaaaaahhh
We’re really setting the stage with this. Now Terminator won’t need to steal a weapon upon arrival!
Really Excited for the future of the robots! This can be used for security? Or maybe for a box training? What a time to be alive!
So amazing! Thank you for sharing Karoly, just watching them essentially 'learn to walk' is inspiring enough. So amazing watching this stuff! Thank you again for this video!
The explanation about the hypnotic adversarial collapsing sounds like something that would happen in Baki
"He's just standing there, menacingly!"
Imagine in the future we have MMA fights contested by AI:) i would love to see that!
Me too
No, too boring. There is no pain, and so no overcoming the pain, and so no excitement.
@@vladimirdyuzhev spotted the casual
Seeing previously unknown strategies would be exciting, just as we saw AlphaGo come up with novel strategies. Masters of those games have even stated it's often more interesting to watch the AI matches, than the more familiar/predictable human matches.
@@vladimirdyuzhev you could easily program pain, health, stamina etc.
"Learn boxing and even mimic gorillas" Punch-Out Reboot spoiled the secret boss again.
Is it really spoiled? We all knew it was gonna be DK again.
I live watching AI “learn” what to do. I also like that you include previous research so that we can come along on the journey. Excellent work, as always!
Actually, I would on the contrary be surprised if the researchers stopped at 130M steps. The reason was mentioned: the agents didn't even had a clue that they could get higher rewards by touching the opponent. So it was mainly a matter of random to start doing so, but until that it would be quite unreasonable to quit. But as always, it was still amazing to watch how did the imitate boxing :)
What a time to be alive!!! 4:10
2:58 the Eureka moment :) amazing result!
@Dr - i love the way you've told the story of the process here. In your next vid, could expand on what form the 'reward' takes? I get in principle how this strengthens the neutral path, but what is the general math to it? Thanks for your great work covering these paper dropping advancements ;)
What a time to be alive
Humans hate being told the optimal path. So telling them the optimal path isn't the optimal path.
there are no truly optimum tactics in football. the game is fluid and players adapt strategies in real time based on the situation theyre faced with in that moment.
It'd be amazing to see them have more than one strategy evolve
"I'm gonna use a psychological attack to win"
*crumples into the floor*
Squaring off in the ring, staring their one another down,waiting until someone collapses from exhaustion alone? Now that's some gentleman's boxing as Andre Filipe originally intended.
4:45 This is obviously the future of combat sports. We just don't understand how overpowered this technique is yet, but we will....
it would great to see a paper on the latest in motion capture using ordinary video footage...
You're thinking what I'm thinking ?
Training an AI on Muhammad Ali or Bruce Lee data ?
“You can box anyone you want, all you have to do is practice.” - Pablo Picasso
AI should not make art, but it absolutely should fight for my entertainment
I'd really love to see the fencing one, being a fencer myself it got me hype af
We need to have some sort of virtual humans Olympics
Yes, nerds, its our time to shine🌚😅
This reminds me of the game Toribash, but it’s AI controlled and real time!
I like this AI learning video the most because it involves punching
Imagine showing up to a boxing match just for every fight to end with one fighter passing out before even throwing a punch.
if they want it to mimic human boxing more they would have to account for things like stamina, pain, and concussions/knockouts work.
And here I am, waiting for AI enemies in games to stop having limited motion rules and no learning. I can't wait for this to be implemented in single player shooters with enemies that adapt to you, extra immersion.
Not gonna happen. At least not in most shooters. AI like that would be a PAIN to go up against, unless they had very well-signaled "barks" for what they're doing. You don't necessarily want the smartest AI in a game, you want the most fun.
@@nerd_nato564 you could hamper it in various ways reduce the time it's allowed to react in, add randomness to its motion so it doesn't have pinpoint precision. Also picking an earlier generation that hasn't become super human yet. I think it would be great. Every game would be different and would be much more like fighting a real opponent
@@rickymort135 That's not the issue though. You don't necessarily want human-like AI in your game. You want AI that is smart enough to not fuck up basic things, but also predictable enough to not feel unfair. Some games would indeed benefit from learning AI like this, but the issue isn't a superhuman AI, it's the unpredictable and creative AI.
I don't know if that would make it more or less fun. The hardest difficulties are already cheating, they know exactly where you are on the map regardless of anything and have a lock on your head from across the map.
@@themonsterunderyourbed9408 That's exactly the point, this sort of enemy AI will not need to cheat in order to feel engaging and difficult. I'm very much into multiplayer games when it comes to the competition, because it's way more engaging to fight real people, but I hate the communities of most multiplayer games. If someone can bring real human playing abilities to AI in single player games, I'd be thrilled.
True respect to the developers for not interrupting the learning for a week.
This is the second video I have seen from Two Minute Papers. Excellent cutting edge content. Well done.
This is so damn cool, seeing ai tryna knock each other out
'Off distribution activations' essentially means red horrified it's opponent into submission via extreme and rapid contortion.
Would be cool to implement these agents to a VR boxing app and let them learn fighting against humans 😂
With motion capture on the human. I'm sure it would train up quick, but then box like a human and not innovate which would be better to see.
Would be interesting to see several ai bots put in a incredibly hard puzzle game where they need to cooperate to win.IT also will have this physics engine ofc and would be hilarious.
The one with the intentional ragdolling to trick the opponent was hilarious and clever.
@MeOwO 😂
Would've been cool if they 'saved' the AI after each X steps, and then have them fight each other, just to show how drastically the AI improved and how hard it can demolish its previous iterations.
I think you would get more interesting results and different tactics on your agents if they had different length of their limbs, because now it looks like they came up with symbiotic situation where they have pretty much the same odds to touch the opponent and get same amount of points each while mirroring each other's moves and reactions. It's a continuous stalemate.
I want to see these trained for 100T steps. Would they invent a new style of boxing? Would they be able to teach us new tactics?
Can't wait to see what kind of martial arts AI could come up with in 5 years.
Imagine Boston Dynamics programming this into atlas
The fencing one would be interesting, especially if you have 100 Atlas's training several hours each day.
Stage 3 Boxing AI looks less like learning to punch and more like dancing, but at least that got them to move fist to opponent's face, even if they did it by moving everything closer..
when you and your friend are drunk and try to duke it out: 1:31
OMG I just predicted the future 😂, later in the video, 2minutepapers (sorry if that's not your name, youtuber, I can't check fsr) said that its like "two drunkards trying to duke it out" 😂😂😂
they are currently working off of
>circle opponent
>jab
>evade jab
hopefully they can handle additional instructions
>circle opponent
>chose between a jab, feint, or hook
>decide which hand to throw this attack
>decide to aim low or high
>decide to block or evade
>decide to change circle direction
this way it has a range of attack options and possible responses to an opponent
but I fear this would take more than a week to train an AI how to do.
but then we can make them grapple.
grapple over each other
or grapple over $Object (be it a piggy bank, brick, gun, sword, game objective ball, toy etc)