DeepMind’s AI Trained For 5 Years... But Why?
HTML-код
- Опубликовано: 27 сен 2024
- ❤️ Check out Lambda here and sign up for their GPU Cloud: lambdalabs.com...
📝 The paper "From Motor Control to Team Play in Simulated Humanoid Football" is available here:
www.science.or...
arxiv.org/abs/...
My latest paper on simulations that look almost like reality is available for free here:
rdcu.be/cWPfD
Or this is the orig. Nature Physics link with clickable citations:
www.nature.com...
🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Benji Rabhan, Bryan Learn, B Shang, Christian Ahlin, Edward Unthank, Eric Martel, Geronimo Moralez, Gordon Child, Jace O'Brien, Jack Lukic, John Le, Jonas, Jonathan, Kenneth Davis, Klaus Busse, Kyle Davis, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Matthew Valle, Michael Albrecht, Michael Tedder, Nevin Spoljaric, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Rajarshi Nigam, Ramsey Elbasheer, Richard Sundvall, Steef, Taras Bobrovytsky, Ted Johnson, Thomas Krcmar, Timothy Sum Hon Mun, Torsten Reil, Tybie Fitzhugh, Ueli Gallizzi.
If you wish to appear here or pick up other perks, click here: / twominutepapers
Thumbnail background design: Felícia Zsolnai-Fehér - felicia.hu
Károly Zsolnai-Fehér's links:
Twitter: / twominutepapers
Web: cg.tuwien.ac.a... 
Наука
A few years ago you helped ignite my passion of machine learning. Since then I’ve learned to code, have educated myself vastly in machine learning. I’m currently building my own small NN. Thanks for opening my eyes to something amazing
That is absolutely amazing, so happy to hear your story! Hope the neural network will go on to do great things!
@@TwoMinutePapers You are big my fellow scholar
The first time I learned about them was SethBling'd Mario NN
@@TwoMinutePapers this must be a great feeling to know you are an inspiration for others
@@ThatGuy-kz3fx neural nutwork
It would be cool if there were simulated sport competitions where teams used their own AI model against each other
They do this already with chess, computer chess competitions, the latest winner is AlphaZero-based Leela, just beat the reigning champ StockFish.
@@raylopez99 Didn't Stockfish win?
Loving the idea.
I would like to see this also in FIFA.
There are competitions like that. Not sure if there are any for sports, but I've definitely heard of some other competitions:
- Minecraft village/ city generation AIs
- Minecraft resource gathering AIs (get diamonds etc.)
- Ticket to ride AIs (board game)
- bad piggies/ angry birds AIs (not sure which one)
- chess AIs
...this is what I remember without actively looking for them. If you look for competitions like that, I'm sure you'll find some.
To get a feeling of how difficult this is: imagine QWOP with 56 keys instead of 4, and you don't just have to run but to play football, in 3D instead of 2.
@eetm but your 🧠 is just a processor playing 3d qwop in the dark.
To get a feeling of how difficult this is, try to consciously control all of the muscles that are required to walk.
Or just appreciate the fact that your brain can do that for you while you think about more important things.
@@dzambi I didn't expect this level of existential crisis on a chill Sunday....
Also the buttons are pressure sensitive. ;)
All things should be related to QWOP
they should do this with real rules (throw ins and cornershots etc), and with 11v11, run it for weeks on different computers, see if they come up with some kind of great strategy, see what formation and stuff they pick.
And add stamina so they have to learn to economize their energy.
And goalkeepers
And a referee.. who always seems to be one sided too for the real effect.
@@letMeSayThatInIrish yeah... genetic, height, weight, heart illness, traumas :D
@@alihms lmao
I notice that they still seem to move really unnaturally -- their upper bodies seem very flail-y. I wonder if that would get ironed out if they were given some cost to excess movement (just like humans get tired).
I was just typing something similar before I saw your post. I even used the word "flaily". Adding fatigue to the simulation feels like it might have significant impact.
Along similar lines, if there was some cost to getting hit, like being slow to move for some time, I wonder if there would be an emergent consensus to avoid causing damage to your opponent, like an emergent moral code. Would they even develop a tit-for-tat rule enforcement?
I was also about to say this. It seems to be a commonly overlooked issue when training AIs in movement.
They may be moving more efficiently than real football players. People don't always take the most efficient paths when moving, we have to consider extraneous social variables of how our movement looks. It's possible that it takes more energy to restrict the movement of limbs than to incorporate their inertia into the body's trajectory.
@@maelstrom2313 if that were the case, real soccer players would move like this. No one cares about looking stupid if they win (cf. the Fosbury flop, which looks stupid but is standard, because it lets you jump higher). There's also no reason to think that these should be moving efficiently, because there's no incentive for them to be doing so (they don't get tired).
It's crazy how the AI just iterates and comes up with through passes, lobs and cruyff turns.
i am a little bit sceptical, with this, there are probably thousands of hours of footage and they show the best most human like stuff, these are things that just comes from the noise. If the ai would be any good they would shoot the ball directly to the empty net every time, then it would learn its better to put one player to the goal
@@DailyCorvid You dont need a ref with robots. That's the point.
Imagine training an AI to do a task for 100 years in 10 minutes, then exporting it and importing it into a robot to achieve the task perfectly in the real world
What a time to be alive!
Think i saw something about them doing that with a ping pong machine, right?
I wonder if they inject noise into the simulation to simulate the imperfections of reality.
Your point holds but i dont think that doing things in simulated world corresponds 1 to 1 with the real world. It would still need lots of training in real world (because the simulated world wouldn't hold all the variables that the real world holds and those small inconsistencies add up)
I have always ponder that if our universe is simulated, maybe it is in a supercomputer that does million of iterations, where every iteration takes milliseconds but for us is eternity
@@josesandv More like every millisecond does millions of iterations
Even after all these years, you still amaze me with how understandable you make these papers for people like me. Thanks for really spending time in digesting this information to someone who's not in the field or can't allocate enough time to dive deep.
You are too kind - thank you so much! 🙏
I see some serious meme potential in this
We can make a religion out of this
What is the game the AI is playing? It looks like non Americans trying to invent their own version of football 🏈
@@MarcillaSmith Why do you call it football when you bring the ball using hands? Shouldn't it called Handball?
@@LinggarMaretvaCendani No it should be called soccer cuz it socs.
@@LinggarMaretvaCendani handball is when you can't afford a racquetball racquet.
This actually could be a really cool esports team. Like imagine if madden had AI agents to play the team. It was actually pretty entertaining to watch.
I see great potential in deadly sport types played by AI.
Worth checking out altered state machine and their upcoming games (FIFA AI League and AIFA)
Nah
@@Ulexcool Hater
@@michaelatorn8380 any examples of what sports specifically, can't think of any other than extreme diving
I think the pre-training behaviour pretty much captures the behaviour of actual football players
Exactly :-)
I was gonna say 😂
Ah, the bugs kick in sometimes.
From the expert opinion of someone who's never watched football before
@Vixan knowing football, but not acknowledging flopping from a foul? Who doesn't watch football?: you
If this simulation gets more accurate, it could influence how football is played in real life. Imagine when the richest clubs can afford to create such simulations and optimise their players.
they wouldn't need to be rich probably
8:22 **gets lightly tripped over, falls down and has pain seizures**
The most realisting thing in the entire video.
I want more Ai Football!!! I wanna watch a full match! Looks so fun. Imagine what they might do years from now??
Boston Dynamics vs Real Madrid when?
Also I want to see 100 vs 100 players and if this can be transferred onto soldiers like in Totally Accurate Battle Simulator.
0:53 me when mom says I can't have another pack of fruit gushers until I eat some broccoli
I would love to find out with a model like this what the ideal play style would be according to game theory and physical limits of the players, but still having to follow all the rules. For example: Would it actually be better even for the goal keeper to move out and play in the field, vs. staying in the goal? I guess humans didn't figure out optimal gameplay yet for soccer and this could lead to new and crazy strategies.
It was my initial thought as well. Using AI we could try so many new strategies. And it wouldn't be hard to give the different AI players different properties to reflect the properties of the real players of a professional team. On eis a faster runner, another have better stamina, a third one have great control of the ball etc.
How epic is it gonna be when a fellow scholar makes a soccer game like FIFA but using these AI's
FIFA AI League game is about to be released on the App Store
Just imagine the labour curve going down suddenly after a couple of weeks… nobody knowing why… and AI looks like it would start to discuss and talk to each other on the field rather than playing. THAT would give me goosebumps. ;D
What a time to be here.
I, for one, welcome our new AI football overlords.
1:13 This is proof that not knowing how to play football can cause extreme cases of seizures.
This is the kind of niche thing that I personally really enjoy reading in my own time, but it's so much better when you have a PhD holder narrating for you and showing you all of the nuance that you might have missed.
They're using their body to shield the ball and then turning around to beat the pressure from opponent... Wow.. That's something one of the best midfielders of all time Xavi did often too... Incredible
I think one of the major improvements in the last few years we've seen in AI training (besides training size/time) is how we train intermediate goals to speed up training.
If we simply took the untrained AI with the joint body, and asked them to learn soccer, it would take forever with minimal improvement.
But as we've seen in this video, they first train the AI to enable it to walk, dribble, and kick the soccer ball.
Once the AI knows how to perform general ball handling tasks, it can use that understanding to more quickly learn to score goals.
This might just be my perception though.
the next logical step would be (and it's probably already done to some extent, in one way or another) to split the AI into a "teacher" (supervisory) and a "pupil" (learning) AI, whereby the former one would be in charge of figuring out and setting the incremental goals for the latter to accomplish
Remarkable. Ex footy player here, wondering how long till these analyses show how Messi's additional value was in the distribution of labour - a 'team player'.
You voice are a IA too? o.O
The fact that it learned to embellish an injury, only to pop right back up is great
The talking sound so robot like. Its like you have all the words with your voice already recorded and then you just type the words which automatically creates the voice
I can't wait to see the Olympic AI Opening ceremony!
I hope you have seen "Lawyer Explains Stable Diffusion Lawsuit (Major Implications!)" by corridor crew uploaded a few hours ago. He even uses your catchphrase at the end! What a time to be alive!
Two Minute Papers, thank you so much for providing us such incredible content.
I wish there was like a place I could go and watch these AI do their thing in real-time, I could watch that stuff for hours.
The pure PANIC in their running HAHAHAHAHA
Don't forget the guy having a mental breakdown at 4:22 and then immediately snapping back to the game
@@enderyu It is all very relatable.
Just imagine if this ai was put into a robot and you had to play soccer against it. It'd be horrifying with how they move - imagine if they even looked like people 💀
Still absolutely insane progress with this kind of physics-based ai agents! It's crazy!
you do remember the part where he said that the ai trained without a refree? Gosh you need an ambulance
RoboCup Humanoid League
@@asrar4907 Ahha yeah you'd definitely need at least an ambulance after playing against these
I think they did that already
2:20 We used to do the same trick when we were kids, passing over the wall when we played soccer.
7:32 AI really pulled a Blue Lock there💀
My heart desires a whole tournament with these little AI's. Crazy ragdoll physics and wild flailing bodyparts included. They are so fun to watch! Can you upload more footage?
One day there's gonna be a better way to 'start' these training projects. It just seems wrong that they should start with such little knowledge. I'm sure something will arise as we work towards the future.
By the way this was damn impressive! Just the mere fact that it could side-step and through-ball shows how much of an understanding it really has about the game, a surprisingly deep one!
@@DailyCorvid loving this
"5 years pass by in 3 days. Are you thinking what I'm thinking?"
Well I'm thinking about that Black Mirror episode with a clenched stomach...
In the first seconds: "... this Increadible AI..." meanwhile the AI spazzing on the floor xD
these little foot ball player are so fun to watch i could watch that for the whole day
It would be awesome if they can also simulate some form of fatigue, some movements look too energetic or wasteful.
4:22 that's pretty much Messi's signature move.
it would be so sick to see full teams progress and having a goalie learning how to goalie and adding in rules like offsides, out, fouls, penalties. would like to see what formations they would come up with or if they would stick with the all forward all back game plan
So first they trained to dive. Pretty realistic.
Neymar's fault
even more fascinating than usual! this has some extraordinary implications both for AI-generated video and possibly for humans, if 'division of labour' could be translated back to humans in the form of more collaboration. for AI video it surely means we are quite a bit closer to being able to create a background street or restaurant scene and not have to animate every single crowd artiste (a nightmare). this would be hugely helpful in itself, but if/when combined with the ability to direct avatars at a high level (eg "please go and pour some wine into that flute"), it will bring us closer to being able to make virtual movies, especially for difficult or dangerous scenes, which could then be intercut with real actors.
I was thinking teams could learn new approaches or possible plays.
"please" 😂
One of my favorite 'agent' training examples. So wild.
Ok, now put the code into 22 boston dynamics robots and let's go
The red teammate at the top looks like something out of a nightmare
5:55
I love the pretrained red guy in the middle of the pitch. Give him to glow sticks and he looks like he's at a rave.
on multi-agent ai, do we have to relearn a strategy for 11 v 11 instead of the current 2 v 2? i assume so because in theory its a different game?
Physics agents are so interesting, I wonder if there is any demo of this that I can find so I can run it myself. Would love to train goalkeepers and play 6v6 or even branch out to something like fencing!
Those pre training moves are what I look like when I play sports
If 11 a side and the full rules of the game were added then this could eventually be used to discover new tactics. Similar to how the chess ai developed its own strategies that humans could never think of.
*DeepMind - Magical Skills, Goals & Assists - 2023 | HD*
I imagine a.i agents will optimize sports tactics and train real players in the (near) future
I wonder if it's possible to train certain skills first, like getting up, sprinting and kicking separately, then continue with the game training.
Imagine if they could do AI training for robot control (in real life)...
I like your storytelling, sir. What a time to be alive!
1:05 That poor guy in the middle of the field trying so hard to get back up!
Reinforcement learning 2022 trend: get a lot of data and through it into GPUs training with visualization -> obviously get something look eye-catching
I have just been convinced that flopping is a mandatory practice in soccer by AI
The tantrum after being knocked over is really interesting. It doesn't seem to be the optimal playing strategy, and I doubt the AI had its feelings hurt. Why doesn't it get up and get back to the game?
Downtime was the most optimal play maybe lmao
probably just like bugs (the insect), it doesn't know how to. it didn't ever learn how to properly get up, because only goals are rewarded. that's the main focus
"AaAnd. NoOow. BuUut."
How long before we watch
world cup AI
I studied multiagents theory at university and I can't wait to see this expermient go with more players !
BTW if anyone has sources about theory of agents interactions in tthis context
This guy was an opera singer in his previous life
So so they make all 4 players (2 on each team) learn from scratch?
What happens if they drop an Ai in with a team mate that already knows how to play?? Do they learn faster? What happens if they can learn from a team that is better or worse than them???
Sooo interesting. I’d love to learn how to do more of this stuff. I have a few raspberry pi’s that I’m tinkering with but I bet you need a whole bunch more processing juice to do this type of stuff.
I hope one day you can make a video that breaks down how to get into a project from scratch or starting from a GitHub library.
What a time to be alive indeed.
Just ask SkyNet 😉
"What happens if they drop an Ai in with a team mate that already knows how to play?? Do they learn faster? "
in principle they definitely should, because there will be quicker and stronger negative feedback = penalty for incorrect behavior
this sort of learning isn't really done by copying or mimicking, but by adjusting the neural network attempting to avoid the penalties
"the corner of the internet where we look at blue curves and make happy noises" is the best quote ever
They looked exactly like professional soccer players at the very beginning of training.
5:25, I'm afraid that they created Neymar from the ground up :O
0:53 ok thats scarily accurate to how a seizure looks like
3000 years later: AI can toss a ball telepathically.
At 4:20 you can see they have learned to fake injury, truly mastering the professional level of football.
They're training them on rotoscoped humans playing, so was the feints with the footwork introducing "noise" that was creating the initial shaking/quivering? When they're running they take lots of tiny steps, which makes me think that's a latent trait of the human data set foot-faking distorting the training?
That red AI trying to feint injury crack me up
There's something really funny about seeing the ai fall over and have a seizure when it hasn't had any training
this gives "soccer fever " a whole new meaning
The through ball at 4:29 is amazing 🔥🔥
As far as I I understood what I saw, they weren't really passing.
It was more like a combination of
0) Run towards the ball
1) Shoot as close to the goal as possible while avoiding the enemy (including bouncing off of the wall)
2) If teammate has the ball then position towards the middle and the goal (essentially where teammate will shoot), which is indeed impressive
It may not sound like there is a difference, but there is, because passing to a teammate so he can shoot at the empty net is not the same as shooting the ball next to the enemy, missing the goal, and hope teammate gets there (who gets there, because he expects the missed shot)
To make it more realistic the ai should no only know which movements are possible but also in which position how much force the body can create and also how fast the whole body or certain parts of it get tired and take all this into account. Also they should have a field of vision with moving eyes that can only focus on a certain point directly and just aprroximate the rest and not know what the others thinks but predicting it from their action. After that it might look exactly like real soccer. Maybe it's possible to use it in a video game. The player just selects who they want to control and points in the direction and shoots and everything else is done by the AI.
5:15 It seems that it was trained with Neymar footage haha
As a sports fan, it would be great to see 11 on 11 at a larger scale, add the rules of the game, and factor in variables such as height, weight, strength, speed, endurance, etc.
Bruh! Did that AI get pissed and then go for revenge after standing up😅? That’s actually amazing 😅
Say your prayers Pep Guardiola, AI is coming for your job
Before this AI gets to play a full games, its players will need to learn the throw-in, invisible walls just don't cut it.
0:54 Okay, both red guys are legit having a seizure, and no one is there to help them. That's honestly so accurate to society bro.
6:40 its interesting how that curve looks similar to the Dunning Kruger Effect
0:53 It's the Italian technique seeking a foul or, when possible, a penalty.
0:54 Perfect simulation of Italian football teams?
Pure gold, they play like half of my classmates on PE :D
The researchers failed to teach them the most important skill necessary to play soccer: curling into a ball and crying when coming into contact with an opposing player.
The children I "freed" from the basement when I am not satisfied with their sports skills (I might've set the intensity of their shock collar a bit too high): 0:07
The fact that these NNs are learning to collaborate is truly scary...
5:14 Neymar: "Finally a worthy rival"
Watching all this physics simulations and last Boston Dynamics video about Atlas and all the struggles they get, i'm interested when BD will use machine learning instead of direct behavioral coding?
I'm pretty sure they already do. They have a simulation they use to train stuff, then make adjustments if the output doesn't work in real life
That joke about the tantrum player was so damn funny haha
Very interesting
If they could also simulat the muscle exhaustion or energy usage for the movement, they could have a more efficient way of running. Because for now they run very crazy, it seems they have these kinds of mouvements from the infinite energy they have to move in the simulation.
Maybe also some injury system.
And of course, like mentioned in the video, a referee.
this channel is so underrated. Also its quite scary
Next step: add fatigue and more players for complex teamplay (and hopefully not kill the game by only defensive play :) )!