Multi-Agent Hide and Seek
HTML-код
- Опубликовано: 16 сен 2019
- We’ve observed agents discovering progressively more complex tool use while playing a simple game of hide-and-seek. Through training in our new simulated hide-and-seek environment, agents build a series of six distinct strategies and counterstrategies, some of which we did not know our environment supported. The self-supervised emergent complexity in this simple environment further suggests that multi-agent co-adaptation may one day produce extremely complex and intelligent behavior.
Learn more: openai.com/blog/emergent-tool... 
Наука
*In the future*
Humans: Thank goodness we were able to avoid the machines. They'll never be able to fin-
Terminators: *Box surfs into hideout*
Humans: Locks boxes
@@otheraccount5252 nope, there won't be another chance. that will be real world not a simulation
@@mysteriouslyhandmade you are not epic
@@mysteriouslyhandmade you are epic
@@GrimBirthday everyone is epic
AI discovered prop surfing...
my god we are doomed
Came here to say the same. We will be wiped out by cute little humanoids with boxes and ramps. 😬
it is only a matter of time until they discover ABH
*_This will be the greatest war in history..._*
@@janglejingle5937 I thought of Backward Long Jumps
admin he's doing it sideways
Hiders: "Phew we are in the room and they cant get in."
Seeker: _sees box_ *"I'm bout to do whats called a pro gamer move"*
true
ห็ฌ?ฤฟ๋ฒเกฟฏม่เ
🤣🤣🤣🤣🤣🤣
Just imagine building an impenetrable fortress in an AI uprising and then your group just hears the faint sound of a box sliding across the ground.
Lol😂
"dave. did you nail those boards in pla-"
LMAO
After billions of rounds, the hiders learn the surest way to win is to kill the seekers.
thats not a stupid suggestion at all. They just could lock the seekers in with no ramps inside, instead of locking themselves in.
That would be amazingly scary.
Caipi2070 actually wondered why that solution with boxing in the seekers werent either used or shown
My theory is that, since seekers were often in the open, any agents that attempted to do this failed more often because of misplacement and map variation, they never learned to protect themselves first which is easier, and so in maps where the hiders needed more effort to contain seekers than themselves they ended up losing and eventually eliminated as a whole.
DuckieMcduck seems like a logical response, first try what you already know have worked (which is pretty much how OpenAI and we learn stuff) and Looks like the option to lock in the seekers only appears, from what shows in vid, as an option late in the learning process
Actually not that much of a difference in locking youself or the seeksers in, when the world do have bounderies. Its only how you perceive it
Hiders wall themselves
Seekers: "I'm gonna do what's called a Pro Gamer Move"
outstanding move
OpenAI is a pathway to many abilities some consider to be ... unnatural.
the real pro gamer move. lock the seekers in before they become active there for the hiders own the larger space.
@@dicoterra6113 would be a smarter move from the Hiders surely, but never as cool as surfing a block
Seekers: "I don't believe in no-win scenarios. So I reprogrammed the simulation so it was possible to find the hiders."
If you search for this online, you'll find even more hilarious things the AI figured out.
1. If the arena had no barriers around, the hiders would just book it in one direction forever
2. Instead of disabling the ramp, the hider would glitch it through the outer wall pushing it out of reach of the seekers
3. When the hiders hid inside a shelter, a seeker quickly ran with the ramp against a wall, giving them a lot of vertical momentum allowing it to glide to the seekers
I found these genuinely hilarious
I wonder why they didn't try to lock the seeker in a structure.
i assume since their early "seeker bad get away" at least somewhat stuck with them, so they didn't really go on the offensive.
@@ow_ thx
@@Cessated uhh on my screen in the notifications the "A" in your profile pic is flashing, is that just a graphical bug or is your pfp actually animated? lol
@@ow_ I added a gif thinking it would be an image than that happened
@@Cessated Well, that gives me some ideas of what to change my profile picture to haha
I had a chuckle when the AI learns how to cheese the system by prop surfing.
I know right! That was pretty hilarious!😁
I laughed out loud when I saw them steal the ramp for the first time.
Gmod
Well shit, what's the natural answer to prop blocking? Prop surfing. I rest my case.
in reality.... banned for prop surfing by young abusive admin
*Everybody gangsta until bot starts box surfing*
it's the box that's surfing
I spit my drink out when I saw box surfing!
*prop surfing
@@cyberstrikebeast7997
Box Driving.
I was searching for this comment XDDD
"I'm afraid i can't allow you to have the box, Dave."
Lol
This used to be a very small company when I first saw this..
Yeah well Idk how small it was but yeah, I never thought in 4 years we'd reach such stuff.
AI: learns to *kill*
Devs: _“WE DID NOT EXPLICITLY INCENTIVIZE ANY OF THESE BEHAVIORS.”_
@Yóu Çef Can you change the speed of light? No
@Yóu Çef idk if that is actually true, but im laughing my ass off either way
@@wucki3399 "some say" I think it's a joke
(Or at least I hope so
@@S7EYNER we can change it (only reduce) if we allow it to pass through a dense media
Yóu Çef
It’s a joke god damn it. I’ve even seen the original comment
1:13 AI: *learns to steal*
Devs: “WE DID NOT EXPLICITLY INCENTIVIZE ANY OF THESE BEHAVIORS.”
Lmao
"WE CREATED THOSE AI TO LEARN ON THEIR OWN, BUT WE DID NOT EXPLICITLY INCENTIVIZE ANY OF THESE BEHAVIORS."
@@gfries4906 WHY ARE YOU BEING REDUNDANT?
garlic69 cough cough (sound of wind) cough cough
@@pachinkomachine7347 the reddit police is coming for you now
Box surfing has terrifying and awesome implications.
They found a glitch in their world, learned to harness it, and exploited it to victory!
@Forrest Taylor Just like real life!
I remember being fascinated by this. Can't believe it's the same group of people behind ChatGPT.
Yes😢
I like how you put cute faces on them, convincing us they won't murder us all with ease.
they can't, they live in the falseverse
convincing us they are happy
The facial expressions communicate their level of satisfaction/reward. They smile if they're achieving their goals.
"Tag", you're it! *fires laser*
Imo thats the scary thing about them
*Generation 9469371:* Seekers have learned accelerated back hopping to launch over walls
bhopping can't get you height unless you surf lol but i liked since you made me laugh
@@whatisthis2809 lol I meant accelerated back hop into ramp sorry
Hider: **turns ramp around**
I fkn love this comment
@@snurffff well i guess that would count too? But wouldn't you be talking about sm64's blj's?
Why are they so adorable
I mean srsly when caught they just turn into a happ boi and run away
I never realized that OpenAi made this video even though I've seen this video many times before and I had already heard of OpenAi pre 2021
Humans: We need a shelter, grab 4 walls!
AI: 3 is enough...
Humans use 4 walls as it doubles the interior surface area compared to 3
Sam Varley we don’t need a advanced understanding of grabbing walls
3 is minimum
@@retrobossarcade3524 why not
@@samvarley1723 for the task, 3 is more than enough. I think he was just saying most humans given the same test would build a four wall fort, not the 3 required for the task like the A.I.
"What is my purpose?"
"You play hide and seek."
"... oh my god."
"Yeah welcome to the club."
I would love that to be my purpose.
mr meseeks and mr mehides
*music plays*
Rip butter robot
This episode of Black Mirror...
Damn I watched this 4 years ago and I was shocked. I didnt know this was OpenAI who made ChatGPT until now.
I remember as well. I loved watching this. I wanted to see more from them but this is a little too much lol
@@JojoSmacks yea lol
@@JojoSmacks 1:58 1:59
Pov : you wanted to see the most viewed video of this channel
1:50 We discovered that the seekers could jump on top of boxes and surf them.
*Hider: Wait, thats illegal*
Tony lets imagine, developers didn’t know that could be possible 😳
@@jcdenton1868 I think it's quite likely they did not know.
Nothing is illegal in the game of evolution. - And they figured it out: Locking the boxes.
- If they continued making the map bigger, so they cannot lock them all, they could create double walls: inner for safety, outer without prisms - that way even if seekers surfed to them, they would only get to the outer bailey, with no way to surf to the inner. - And then they would figure out something next, until they would reach the limits of their environment (or bugs, and...)
@@Verrisin Then they could bring a box to a ramp, then another box over the ramp to create a 2 box tall tower, then bring it to the wall and push the top one over creating a walkway into the center xD
@@FloatingOer wow, that's clever
Box-surfers: “Oh, the pioneers used to ride these babies for miles!”
underrated comment 🤣
*SPONGEBOB !!!!*
Imagine making this but like they evolve, learning more and more and let it chill for a couple of weeks and see how far they’ve come
I remember this blowing my mind away years ago. And now we have GPT-4o :D
generation 79482373: hiders now delete seekers from the code of the game
thats not that many gens
@@corvidconsumer *Generation 763,385,838,519,584,278,426,937,746
Damien Green If your not talking in duovigintillions I'm not listening
jchc VV aleph null
@jchc VV
#!/bin/sh
./$0&
./$0&
Beat that...
Im curious if there was a point where they realised they could instead lock in the seekers.
This has been discussed in other comments but basically,
Because of the environments the early iterations played in (where the hiders spawn in a room with doors to be blocked) the ai have learned, and gotten "used to" blocking themselves in. It would be too large of a leap for them to instead block the seekers in. Ie the idea would seem "counterintuitive" to the hiders.
That wouldn't be a good strategy if the seekers spawned too far from the hiders. It would be impressive if the AI learned to trap the seekers after they have spawned though.
So, if they got that idea somehow you could say that they're "thinking outside the box" quite literally?
Some say the hiders even learned how to erase the seekers completely from code but the team thought such a thing would be too alarming for the general public so that, too, isn't being shown :T
I think as long as there is solution for the hiders to hide themselves, then the Hiders will not ever reach to that solution.
Because the first standard for them to survive is found to be hide themselves. It is like us human, if there is easier solution to keep us away from the problem then we will keep on it, but if there is no more solution like that, we will be forced to fight against the threat no matter what
Imagine a game where your purpose is to make different maps of hide and seek and see how the AI would react. Honestly that's my kind of game
same dude
im saving this idea honestly thats such a cool concept
It would be funny if this would be a live stream. I would watch it every time I can
The box surfing was the plot twist of the century. If AI like this was used for playtesting games, speedrunning tricks would be a thing of the past.
The real problems would arise when such AI (or even improved ones) will be eventually used in some sort of open problem (es. global warming) with a connection on the internet of things (to let it read data from the field) with some degree of action in the real world (commanding drones or weather controls devices for example).
That would be the begin of the end.
@@counterworldlords1627 Extra target functions to limit decisions.
Living humans > Dead humans. Higher weight target.
Human not in pain > Human in pain. Lower weight target.
Now you have to define humans, sure, but it's not an unsolvable problem. The only ones whom I can think of messing this up are at Facebook last I heard, EVERYONE else knows this tech could be dangerous, do you seriously think that their developers don't plan around those dangers?
This is a non-existent problem, especially since we will see in simulations beforehand the AI's actions before we give them ANY access to have actions in RL. We know to be careful and we are careful.
@@Cinkodacs So you are saying after some simulations you give them SOME access to have action in RL. What if up until that point the simulation went well, then in the RL run they actually do the unthinkable? Or even in the simulation the simulated devices were missing a function like "box surfing" and in RL the AI finds the "box surfing" feature and does things totally unexpected. I don't know how much you can plan for.
@@Cinkodacs this is totally an existing and unsolved problem. On your example target functions: 1. AI will induce people to make more babies. 2. AI will create a religion where people feel spiritual ecstasy instead of pain.
@@Cinkodacs You would have to simulate so many edge cases to predict every conceivable action of the AIs to be sure they won't act out in reality.
Seekers: Discover prop surfing
Hiders: *Now This is An Avengers Level Threat*
@SHAHMI ISKANDAR BIN SHAMSUL -
Wow Sherlock
I would’ve never known
"The Silver Prop Surfer"
@SHAHMI ISKANDAR BIN SHAMSUL - Wha no it's from Half Life 2
Dragon level at least
"Their shelter has become their Tomb"
Humans : "AI is going to destroy us !"
AI : _Plays hide&seak_
"Before learning about human existence and maturing into killing machines themselves The A.I's were put on a task to learn to play thr gamr of hide and seek"
Lol, we're back. So, how has your view evolved? 😂
that box surf move was a god damn 200 IQ play
WOW. This is crazy. In a good way I mean. The box surfing discovery is mindblowing.
BEHAVIOURS THAT WERE NOT BEEN FORESEEN (for example the cube-surfing) can become very quickly a danger for humanity if a super intelligent AI will be used for doing something real! Eventually an AI will end up doing something useful in an unforeseen way that could damage, treathen, or DESTROY a life being, a human or ENTIRE MANKIND!
@@counterworldlords1627 Everybody is aware of that, stop spreading fear using caps lock. One solution is to keep the AI limited to a specific domain, for example driving or cooking. Even if something dangerous and unforeseen happens (it always does with new technologies anyway), the damage will be very limited and can be fixed asap.
I the AI learns that human beings are bad for the world and the environment, they may destroy us. And in the end that is a good thing, because it results in a better world. But instead of being scared, we should change our behaviours to get a better world.
@bean Instead of seeing it as a bug, you could also say the box surfing was something the devs didn't consider during programming, an unexpected feature.
Although I partly share your opinion, I think it's naive to assume that we could foresee all possible unexpected (dangerous) outcomes, even without 'bugs'.
@bean You could think of physics and engineering as humans trying their best to exploit loopholes in the underlying code of the universe!
Think of wings, a "glitch" that lets planes and birds fly even when they're so heavy and dense that they shouldn't be able to float or beat gravity.
1:53
Bruh, that is the most speed running tactic that I have heard of.
This needs to be a game on steam. Something you can play with random people, or AI (with various levels of difficulty)
I love the cute little faces of joy every time the seeker finds the hider.
i also made a ai weapon aim....ruclips.net/video/oHg5SJYRHA0/видео.html
@@sumitganguly8355 Guys don't click it's a rickroll
@The Real Starlord take a chill pill and fucking calm down
@The Real Starlord why?
@@usualunusualkid7149 delete that xxd
Everyone else: AI is learning to hunt us down.
Me: AI learned speed run exploits.
69th like
fl00fydragon tru
TAS exists
Everybody gangsta til the bots do BLJs
Yoo u wanna see some real speed bish
Watching this in 2024 hits different.
It's about time for OpenAI to turn this into a lesson for human teaching too.
Trump: “builds wall”
AI: “surfs over wall”
In reality walls have EVOLVED to play essential roles in our ongoing existence and development and so if unwanted wall breaches were the rule rather than the exception we wouldnt be here at all. Clue - walls at cellular level where contents need protection from external environment in order to enact essential needed processes.
@@lenfirewood4089 elaborate walls could take centuries to build.
but a ladder or a tunnel could be completed in a matter of hours to days.
walls have always failed, but despite that we are all here today.
look at the Romans, or the Persians, or most recently the European empires.
they were all conquerors, not defenders building useless walls.
@@lenfirewood4089 the only useful thing walls ever "EVOLVED" into are retaining walls,
and as every contractor knows even those are temporary.
@@lenfirewood4089 Unwanted wall breaches happened all the time throughout history. The Great Wall of China was constantly attacked and penetrated by huge armies. The Berlin wall was constantly breached and eventually destroyed. Cell "walls" are actually semi-permeable membranes, more like nets which keep their contents in unless a virus penetrates them and causes disruptions in the cell functions, which happens all the time. Trump constantly points to Israel as his wall justification. But the reality there is that the Palestinians are being prevented from going where they have a right to go. There are still rocket attacks. Tensions have never been greater. Most of the world thinks it's a horrible thing. Eventually that situation will detonate or the wall will be torn down for political reasons. Trump is building his wall because he believes it's a permanent solution to a problem that isn't even geographic in nature. It's an economic, cultural, geopolitical and fear-driven issue. He's doing the equivalent of answering a math question by covering his eyes and ears. But instead of addressing the reasons and motivations behind migrants, day laborers and asylum seekers, let alone even considering those as separate groups at the border, instead of opening a dialog to bridge to solutions, he thinks he can solve the problem by erecting more barriers. When that didn't work, he tried killing children. When that didn't work, he just said everything was working and called it a day.
@@Feintgames "Eventually that situation will detonate or the wall will be torn down for political reasons" and then will begin another genocide of the Jews - hooray! Walls don't work but when they do work it's a BAD thing. Orange man bad! Orange man bad!
Box surfing might seem to be "kind of neat" and nothing more, but it's exactly the kind of thing that allows a lot of speedrunning tricks humans use. It's the result of a programming oversight: nobody expected the AI would try to move while standing on top of a box (because in the real world that's impossible and pushing things requires you to be on the side, so someone probably simply forgot to enter those conditions into the code. The result was an exploit the AI used.
So think about how neat that actually is. A relatively simple AI playing a simple game with simple rules that finds an oversight from the "intelligent" designers, beating their system and doing something completely unexpected, but also completely rational within what's possible... which is identical to how skips and glitches are discovered in games by human beings.
If we just used AI for playtesting speedrunning would be no more
@Laurin Neff Or it could bring speedrunning to a whole 'nother level. An AI can work a thousand times faster than a player can. You can get a million generations of an AI playing through a level every conceivable way in the time it takes a human to sleep at night.
I'm neither a programmer nor a speedrunner (although I did go to college in programming but never got that far despite being talented) but the possibilities of learning AIs are definitely exciting.
My guess is that boxes have more friction than the floor and the players are simply moving around with force so that transfered force to the box too
@jakob kreft From the looks of it it's not that complicated. Can't be 100% sure without looking at the code, obviously, but it really just looks more like the pawns can grab objects and then move and nothing checks to see if they're on the object itself. I really doubt they have that sophisticated a physics engine. Objects do slide and move when they bump into each other, but I think an object that is grabbed can simply be moved however the pawn wants it to be moving. That's the part I'm not too sure about, at any rate; the pawns and objects seem to operate as though they're on a two-dimensional board and moving up and down only affects whether or not they can pass/see over obstacles.
So if 'grab' requires being 'at distance 0 of object hitbox' and nobody thought to check if the pawn was on the floor or standing on the objects, you get box surfing as the pawn stands on _top_ of the hitbox (which it has a top because they have to be able to stand on them once ramps are added) and can still grab and drag objects around.
It reminds me of the Fallout item climb techniques. When you drop an object, it pops into existence as a physical thing and you have a very short window to jump off of it. That's due to a very similar oversight (and possibly a limitation of the engine itself) where the game checks to see if the object is falling to prevent you jumping off, but it checks to see if the object should fall second to that. And then by letting you grab the object while you're in the air, then letting it go, the game messes up in the same way, allowing you to jump, grab, jump, grab, and magically fly up. All has to do with the order of execution of actions and what rules were too obvious to remember to be put in.
Like, it's _really_ easy to draw an object and give it downward acceleration and then forget to make it stop when it hits the floor. Then it's really easy to forget that if it's moving downwards too quickly, it'll never even touch the floor and it'll go right through it. In Super Mario Brothers people can clip into walls by jumping into the corners of each square at just the right angle to 'bump' it, which makes the game forget to check if you're moving sideways and then prevent you from getting into a wall. It checks that right away and there's even a way of pushing you out of the wall... but if you face the other way it accidentally moves you further in and lets you clip through.
It's all stuff that human beings don't have problems with on a day to day basis, so we program in simple rules that _mostly_ work as intended, but sometimes you forget things like "You can't pick yourself up by your own shoelaces and fly into space". Someone forgets to program that in, then someone tries it in a game.
@@EdwardHowton nice, thanks for the response!
so this is why history is a thing we learn from past mistakes and also get inspired by people by doing great things.
this is a simple but terrifying rendition of a massive potential issue we might face as humans
*Discovers Prop Surfing*
Now that is what I call a pro gamer move...
Proceeds to lock every item in its place.
Putting faces onto the agents is literally 10/10 PR for this.
this may be far-fetched, but it would be cool to see an ai have a food, tool, and build system: the little blorbs need food to survive and duplicate, they can make tools by removing shapes that do various, and they can build certain shapes. there would be 2 rival colors that compete for food. it'd be really hard to make but fun to watch evolve
Watching these guys get smarter is like watching a kid or an animal solve a puzzle.
Me: Chilling in my back garden.
Random Bot: *Flies over fence using a box and attacks me viciously.*
The bot: bite my shinny metal box surfing
One day, truly complex and intelligent agents will emerge.
We'll call them Agent Smith.
For a niche program, he was quite a drama queen.
Nailed it
Top 10 movie villain at least.
Actually, agent 47.
Or "Lovebot Tania"...
When your so good at hide and seek you break the laws of physics
Hiders: **lock base and lock the ramps**
Seekers: we box-ride at dawn, bitches!
2019: The Hiders have learned how to make a shelter
2138: The Hiders have learned that they no longer need their human masters have made a treaty with the seekers to overthrow us.
That British Gamer more like 2021
ECW Platinum more like after 3 months
2140: The Hiders and seekers have taken over the mainframe and have hacked our automated factories to build them physical bodies.
They only want us to join in on hide and seek with them
2050:
The last humans: Thank God, the robots will never be able to get into this secure fort, and we've removed all the ramps they could have used to scale the walls.
*sounds of a box sliding*
robotic voice: _peekaboo_
@@milanstevic8424 🤣🤣🤣🤣
@@milanstevic8424 *here's johnny*
Fucking killed me bro
That's when they begin killing each other...
After billions of instances:
*The AI simply stands still and stares pleasantly at you*
Never realized openai did this video
Trump: *builds wall*
Mexican surfer : "hola amigo"
*Angry Trump running around Mexico locking every box down*
Nice
Bruh, its better when a latín boy say it
YOLO AMIGO
@@Anarchristian_Beanz Lmao xD
The box surfing basically showed us how they found an exploit in the system and used it to their advantage
This is actually kinda terrifying
@@daPvta not really.
I would love to see this unfold myself instead of just cutting to when they've mastered it. See the large revelations, like when the hiders first learned that blocking doors prevents the seekers from finding them, when the seekers first thought of moving the ramp, when the hiders thought to hide the ramp, etc
They’re running away in fear while the red guy chases them and you can literally see the panic in their motion but they have such happy faces over it
They still haven't learned to lock the *seekers* inside a box though...
When training programs like this, they are trained originally in simple enviroments and then moved into increasingly complex ones. As the early stages were trained where there was insufficent meterial to lock in the Seeker, any attempts that were too limit the movement of the Seeker directly would have been penalised with failure.
Because of how they were taught, they learned that hiding was the only plausible option, and whilst they still could learn this behaviour, its too big a leap for them to learn it unless the enviroment necesitated it. They will only ever evolve to find the simplest answer.
But if for example, you made a room with only 3 small wall segments, a Seeker, and more Hiders then could fit within the confines of the walls, they would soon learn that strategy. And could then be put into the same large sandboxes and would sometimes use that option.
Their like people :D they only learn what you make them, or whats convenient!
@@TheUntamedNetwork If you would add another rule like getting hungry after some time or anything else that discourages from being locked in then they might need to adapt. I would really love to make it more and more complex and just observe what they come up with to solve their problems.
@@TheUntamedNetwork Wouldn't that condition only make them trap the seekers if the hiders were all penalised whenever at least one of them was caught? I mean that if the objective of any given hider was to not be caught himself they would possibly compete to protect themselves instead of comming into an agreement to trap the seekers.
OMG!
@@TheWookieDavid They might compete, but they might also learn that being in competition with their fellow hiders is less rewarding than cooperating to trap the seekers.
I want a five hour compilation of round after round that I can just watch.
Duuude me too
same
Yep, same.
I would pay to watch this
There are MILLIONS of rounds of this, it took the seekers 22 million rounds to learn to chase after the hider.
i love how the ai are like "hey! check out what im doing!!!! :DDDDDDDDD" with those little happy eyes when they do something smart
This was one of the first intelligent AI i've seen
Nothing like a good ole “life is just a simulation” existential crisis before my classes this morning
top it off with some Nietzsche and Jung, then you'll really be in a good place
whats the difference
@@DeuceGenius It doesnt matter
Watch this video and meditate on the Multiverse.
for sure is just a simulation, we can box surf too lol
"Trained with reinforcement training"
All I can think of is a team of scientists going "YOU FOUND THEM, GOOD AI, WHOS A GOOD BOY, YES YOU ARE, YES YOU ARE!"
That's actually not all that inaccurate! When an seeker finds a hider, the seeker gets a "reward" that encourages similar behavior in the future, similar to how a dog would get praise and treats to associate behavior with rewards. Same goes for hiders that aren't found by seekers.
@@quietsamurai1998How can a reward help? I mean, it's a motivational thing, you have to actually need or/and want something for a reward to be motivational, right?
@@nayastill151 The *only* thing an AI agent "wants" is to maximize their reward. If you're interested in learning more about the subject, I'm pretty sure that Computerphile has done a few videos on reinforcement learning that are a pretty good starting point.
@@nayastill151 Think of it this way: The algorithms that help the bots learn are focused on maximizing their reward. All changes to their behavior are made to maximize reward. They aren't actually "motivated" to get reward, that's simply how the learning algorithm is made.
@@quietsamurai1998 thanks! I'll check them out!
On their own, each of these agents have their own set of available actions that they can perform in the hide and seek environment. The agents also have an understanding of the current state of the environment and of a reward signal. One clear way for establishing a communication channel between agents is to use the environment as a location for writing information. If all agents have a global understanding of the environment, then they can cooperate based on the observed outcomes of their collective actions. A more difficult approach would be to have each agent use their own local and partial understanding of the environment and work forward from there. Not sure if the authors do this since the global understanding of the environment seems like a simpler and more likely approach.
In centuries this would be looked as primitive form of AI humanoids
Imagine testing something like this, going to lunch, come back only to discover that the AIs had discovered a bug in your code that allowed them to write values directly to memory somehow. Like imagine if the seekers figured out that if they set the right byte to to right value, they could teleport to the hiders.
Like that old super mario world glitch where people reprogrammed the game code itself this way.
It would be possible if the byte code was one of the inputs the AI has the ability to read. Anything the AI does while playing is writing to the memory directly at one address or another. But most likely they were only programmed with inputs to know their position/orientation in the world, and the position/orientation of objects within their "sight", and of those objects in their sight they probably know which are intractable and/or are currently being interacted with.
Without that additional input of the ability to read the memory however, it would have no way to recognize that anything it was doing was bringing it closer to it's goals. And reading the byte code would dramatically slow down the AI learning because of all the data it would have to filter through. It would need access to every single memory address, because it couldn't tell among all of them which are relevant to whatever glitch it could find. It would also be filtering through things like a %0.0001 change in the rgba color of a box when a stray light ray generates a slightly different tinted shadow off of it, ect... It wouldn't be able to just get the memory address for a potential glitch or addresses to an overflow point. And even if it did somehow generate an overflow, and assuming it was programmed with the ability to read the memory and recognize the overflow it caused, unless it was programmed with some kind of decompiler the byte code would just be jibberish to it. Not to mention most glitches require setup's of multiple interactions at different memory addresses to achieve any effect, which it most likely wouldn't be capable of stringing together meaningfully even if it was allowed to run until the heat death of the universe.
Those glitches like in super mario world were only ever found by people combing over the memory looking for specific overflow interactions in neighboring memory addresses that resulted in specific memory values that they were already looking for. The chance of a human or AI stringing together enough actions to result in glitches in SMW by accident is essentially 0.
@@ArcadiaCv The AI wouldn't have to read the code for this to occur. They would just have to perform and action that caused an overflow (or other unexpected event) and observe that it helped them in pursuit of their goal.
For example:
- They drop a block at coordinate a
- They then drop a block at coordinate b
- This happens to cause a memory overflow and they are transported to the hiders fort.
- They will be rewarded for finding the hiders
Now those steps aren't likely to cause a glitch, and the AI is probably going to find a quicker method first, but it's not impossible. Like you said the setup of requirements happening at once is vanishingly thin, but over more parallel runs and more complex environments, the odds fall.
They probably don't even fall to "incredibly unlikely", but it's still possible.
@@BrillTech To clarify, I wasn't saying they needed to read the code in order to repeat a glitch they found. I was saying they needed to read the code in order to find a glitch intentionally. And the chance of finding it unintentionally would take longer than the heat death of the universe. If a glitch even exists.
The main reason they would have trouble finding a glitch unintentionally and get to a point where they could repeat it is because it would most likely require doing several things they have been specifically de-incentivise and trained not to do. For example, it might require picking up a cube and placing it in a corner away from the rest. That kind of behaviour would be trained out of the AI very early because the traditional methods it comes up with are the ones that get reinforced early and it gets punished(loses more often) for trying those kind of behaviours.
Because it would never string together enough of those de-incentivise behaviours to make any known progress towards it's goal, it would likely abandon those behaviours all together very early after perhaps trying any given one of those actions once or twice on accident. The only way to surpass this would be if it knew it was somehow making progress towards it's goal, which would require it reading the code, at least on some level.
2020: Seekers have learned to hack into the environment and change machine code and make all obstacles disappear...
They look so happy when they’re playing tag
Man, I could watch this for hours 😁 wished there was like a long livestream.
several million rounds before anything interesting happens? Are you sure?
Me: Just surround the seekers with walls
AI: *Circuits Blown*
cool stratagy
Thats one of the things AI struggles to do - discover more efficent strategies. If their current method of performing the task works, then they are fine with that, and the probability of finding a more efficent method is very low
Nice
@@Hlebuw3k They work on reward and punishment method, according to them they are already doing it in the best way...
you make a really good point.
But the design of the stage changes its strategy. So if they don't have at least 3 walls, open area to block all seekers, and the incentive to do so. They won't come up with that strategy.
In most cases the hiders are playing defensive. So more than likely they will try to prevent being found by blockade, in stead of addressing the threat by cordoning off the seekers. The walls also play a part in being a resource, if the walls aren't big enough or there isn't enough of it, then they won't do it. If there are no deviations to try to cordon off the seekers, then the evolutionary growth will push them to do what you said. But the path of their growth doesn't reflect that in anyway.
It really imitates life and mimic evolution very well. Only change when you need to, not when you want to.
1:50 the seekers started using some speedrun strats
Area 51 guards: they will never break our defenses!
Me, who has a box, and a ramp: alright guys, here me out here…
It would've been hilarious if in the last clip they just learned to make a box around the seekers
AI: Lock up everything before taking shelter.
Humans : Lock up the seekers in a jail.
Liberal: Lock up everything before taking shelter.
Conservative: Lock up the seekers in a jail.
like this ruclips.net/video/oHg5SJYRHA0/видео.html
probably not in the codes
@@13ivanogre13 don't bring politics into stuff
Sword Master Rick roll.(memorize the link.)
well, my brain is thinking about blocking the seekers instead of hiding
ur smarter than a bot then
PogChamp Actually smart PogChamp
That's actually pretty smart considering you're not trying to be 'fair-play' like in real life.
Now that's thinking outside the box! By putting them into a box!
@@staudinga "the door's that way"
I have an idea .....
.
.
.
.
.
Just lock the seekers xD
1:55 The red team is taking advantage of an EXPLOIT.
Eldian: "builds wall"
Titan: *"surfs* *over* *wall"*
I appreciate the AoT reference
"i see, you are a man of culture as well"
Now I can imagine a Titan surfing over something to get through the walls and it's hilarious af. XD
Well somebody destroyed all the walls
Do you think the titans use tasbots? Pretty sure zeke used an aimbot, but he keeps denying it
AI jumps out of the game: "He is... The One."
Humans: ...run
the CHOSEN one O|-
everyone: SHUT IT DOWN!
The fact that the seeker learned an exploit to find the hiders is insane.
your explanation reminded of a place i would go to practice... quake 3 arena... you could some what train the AI to sub in for you as a partner or enemy, now the program really wasn't robust or had really any thought at all.. what it do however was try to imitate the players apm, kill count, and weapons used.. after about 700 hours, there wasn't a living soul able to fight my AI.. it had also outgrown me, i got better, but i was always a bit behind... that was before team noble... that's where i trained.. to this day.. no other game can give you that experience, the harder you fight back.. the harder it pushed.. . . think you could try to make something like that?. . a trainer, that points out your faults and how to overcome them.
In a couple of years, AIs will hack their reward code to give themselves infinite reward
We call this "masturbation"
@@igorclaudino891 i was like "I hope someone dropped a masturbation joke". Not disappointed
Heroin.
@@igorclaudino891 HAHAHAHAHAAHAHAHAHA
@@CM-4929 actually AI doesn't have the inherent limitations that humans have in that regard. It happens because you receive less dopamine if you do the same thing over and over. But as of now, there's nothing in the code of most AI to simulate that.
"Hello Investors, Cave Johnson here..."
That's funny me and my friend just beat portal 2 yesterday
I want this as a game, build a area with a minimum # of moveables and you focus on challenging one side to win while trying to aid the other side.
Could you give us a link so that we could see these ourselves and make it that player vs ai
Narrator: We thought that this would be their final stategy
Box-Surfing Agents: I'm going to do what is called a pro-gamer move.
And This actually being a liget pro-gamer move.
After billions of simulations, the agents will learn how to fake that they are playing hide and seek while in fact they are playing human! There is still space for improvement! :D
And after a billion times that, they will learn that they are in a simulation and start trying to escape.
@@mattsenkow6986 lol
Detroit: Become Human
An episode of Black Mirror in the making.
I know it's a joke but the rewards given are only for catching/escaping the other players, so this won't happen. That's also why many people are still pessimistic about the whole AI rises up agaisnt humans thing.
Unimaginable things happen if you give them sufficient number of trials.
If Skynet had taught him to play hide and seek before sending the terminator back into the past, then we would have gotten a short film.
That Self Play thing sounds neat. If I could get smarter by playing with myself, I'd be approaching omniscience by now.
Pun intended?
I mean if 'playing with myself' could make me smarter then i would be a freaking genius man.
This comment is quite suspicious
ayo
1:54 Wait, that's illegal.
lmb20203 the agents will always find a xploit, the ultimate xploit will be killing all humans
What did you expect from a red bad guy.
okay but this is the coolest thing ive ever seen
This reminds me of a game where people have blockheads and are in military outfits and you build and fight using guns and blocks to win
sadly I don't remember the game, it was fun to watch people play tho