The Iterated Prisoner's Dilemma and The Evolution of Cooperation
HTML-код
- Опубликовано: 17 окт 2024
- The iterated prisoner's dilemma is just like the regular game except you play it multiple times with an opponent and add up the scores. But it can change the strategy and has more real world applications as it resembles a relationship.
THE PRISONER'S DILEMMA • The Prisoner's Dilemma
FOOT NOTES
Additional requirements for an iterated prisoner’s dilemma game
For a one off prisoner’s dilemma, they payoffs can just be like 5 greater than 3 greater than 1 greater than 0 (like in the video). But for an iterated prisoner’s dilemma game to be an iterated prisoner’s dilemma game, the total payout for both cooperating has to be bigger than the total payout for one person cooperating and one person defecting. Basically 2x3 greater than 5+0.
If it was like 8 points for defecting while the other cooperates, it would still follow the on off prisoners dilemma format, 8 greater than 3 greater than 1 greater than 0. But with an iterated prisoner’s dilemma game the best strategy would be to go back and forth between cooperating and defecting (giving them on average 4 each, rather than 3 for both cooperating). Which can happen for sure, but it’s a different sort of game.
With a one-off prisoner’s dilemma this doesn’t matter so much. Trade might change the relationship, but it should still appear as a prisoner’s dilemma. The incentives that make it the prisoner’s dilemma would be the same. Their best strategies would still be to always defect.
Tournament 1 vs tournament 2 (A set number of rounds vs not knowing when it ends)
There is a bit of a logical quirk when the players know how many rounds there are. A big part of game theory is getting into your opponents head, predicting what they will do. Also thinking about the whole game and reasoning backwards.
So when they know how many rounds there are, the last round against any given opponent, has no consequences. If you defect in the first round the opponent can reciprocate and you probably won’t be better off or it. But an opponent can’t reciprocate against any defecting in that last round. And since no matter what the other person is going to do, defecting gives a better payout. A player should defect in the last round... so should everyone really. They will only get more for doing that. We should expect them to do it.
But then if everyone is defecting in the last round, not in response to anything that happened before, then really the second to last round also has no consequences. Nobody is going to defect because you defected in the second to last round. What they’re doing in that next round is already set, if we defect in the second to last round we’re not giving up future gains we could have gotten while cooperating. So everyone should defect in the second to last round because there are no consequences and the payoff is higher. But then if everyone is defecting in these rounds, then the third to last round also has no consequences…. Blah blah, defect the whole time. With this reasoning the correct strategy is to always defect.
But we already know that ALWAYS DEFECT isn’t a great strategy here. Because defecting has consequences.
But reasoning backwards, in this context, only works if everyone is doing it. If everyone has thought that way. If everyone reasoned this way, then everyone is always defecting. Then always defecting is the best strategy against that.
But the people who submitted for this tournament clearly didn’t reason this way.
Maybe because most of the time in the real world, we don’t know how many rounds there are, so we always think our actions will have consequences. Also we rarely interact with one person in isolation. We can build a reputation for being un-cooperative. If we are dicks to people who we are about to never see again those who will may not want to cooperate with us as much. So those who submitted thought like this.
Maybe because they didn’t know about the idea.
But even still, in the context of that tournament it would still better to defect in the last round. I suspect TIT FOR TAT or FORGIVING TIT FOR TAT modified to defect in the last round would have won... in the tournaments at least. If you're done with a relationship and you kill them and take your stuff, the people watching you won'y want to cooperate with you.
For that tournament they actually changed the definition of what a “nice” strategy was to allow defecting in those last few rounds against an opponent. The weird space where the two ideas meet.
Cooperating with whom?
Cooperation and defection refer to between the two players. Not necessarily with outside forces like with the police.
Patreon patreon.com/us...
Well done! Repeated games are one of the more difficult topics to explain, and this presentation should help many more people understand tit-for-tat and cooperation in the repeated prisoner's dilemma.
Ayy Presh! :D
Heyo
I hate you
Mind your glasses
Prestolwolker
silent for half a year and then you reward us with 2 videos in a month?
This is madness... THIS IS ... This place.
Since it is July right now, I suspect school is the reason.
there are quite some channels out there, that seem to be managed by a student. they post videos december/early in the year and around june/july/august. simply the time between semesters at an university. thats why many of these channels cover a specific field in detail, but not much more.
i think the person behind "this place" studies biology, exspecially in the field of evolution.
I'm debating going with a sparta or Patrick joke..
Whenever anyone says "nice guys finish last", I just refer them to this dilemma.
Unconditionally nice guys finish last. No one should be nice to people who are trying to take an advantage of you. Also, most of these "nice guys" are not really being nice, but are (very poorly) trying to manipulate someone. They are just using niceness to get laid, so it is this very artificial kind of niceness, where they say nice things all the time, but never really self-sacrifice, and when they just come across as creepy, they cry how women only date jerks. Then there are the genuinely nice guys, but they don't advertise themselves as "the nice guy", but think that being nice is just part of basic human decency, so they don't make a big deal out of it. I would be extremely wary of people who all the time advertise some quality of theirs. Be it their niceness, intelligence, morality, whatever. It usually means they are trying to take the attention away from some other shady shit they are up to.
Dude, "nice guys finish last" doesn't even refer to romantic/sexual endeavors. Though it can be applied to them, that's not the point of the saying. I wasn't thinking about that at all when I wrote this.
If you google that phrase, +90% of results will be about romance.
Actually, it's a pretty even split. As far as image results go, it's almost entirely about romance, but otherwise, that's far from accurate.
When people refer to "nice guys" they don't refer to tit for that, they refer for the dude who, in this scenario, NEVER retaliates.
You know that they're not milking the video for revenue when the run time is 9:58
That's exactly what I mean, I wasn't trying to be sarcastic, sorry if it came off that way.
Wish it was 2 seconds longer, these people need money if they wanna survive on this site.
In late July monetization will be at 8 minutes
This didn’t age well
@Matthew_ you can monetize videos that are 8 minutes now
I had a contest like this in my computer science class. Each strategy played each other one 200 times. My program started by cooperating, then acted like tit-for-tat unless one of two things was true: either the opponent had defected more often than cooperated or the opponent had defected 5 or more times. I also submitted an alternate program that was the same, except it always defected on the last round.
That strategy would be the only viable non-nice strategy
him: everyone likes points
me:wait... golf.
Golf just has antipoints. You lose points when playing.
I'm a golfer and I still hate all the points that I'm left with at the end of a round and I would like them to pack up, leave and take the kids with them.
You know, it's always nice to have your worldview -- that cooperation is better than competition -- validated statistically.
Right. But it cant be pathologically compassionate, which is why communism always gets the boot. And it cant be pathologically narssistic, which is why cronyism degenerates.
But there is still competition...a competition of competing strategies. There isnt a Jesus Christ strategy where one it has been established it always and everywhere shall be flr all time.
I wish everyone had that mentality now a days. Everyone would all be winners no matter what.
You might enjoy reading Mutual Aid: A factor of evolution, by Peter Kropotkin.
@@m.kastro591 if everyone is the winner, then nobody is, because it would be 8 billion-way draw.
@@SN00888 Life isn't a zero sum game, you can gain without forcing someone else to lose.
Where can I get the letter G? There is no link in the discription
Believe, and the letter G will reveal the link to you.
+louisng114 GG
GG?
i wanted G when it was unique and exclusive, but now it looks like it's multiplying, i don't want it anymore.
roidroid Such impudence! The letter G is not multiplying. It is just the duality of G. One is two, and two are one. It is the same being in two different forms.
Silly you. It's like someone who is wearin lasses askin where their lasses are.
You are, by far, one of the best channel I know on RUclips. I'd put You on the same level as Vsauce or Kurzegesagt, if not even higher.
Mix in Kurzegesagt's facts with Vsauce's philosophy. All three of them great channels. Oh and +1 to This Place for the satire at the end ;)
Why good beats evil in the end, proven with math
No. It proves that the optimal strategy in a prisoner's dilemma scenario is to retaliate immediately, and always be the second to forgive.
That's not what I would call good. It's not necessarily evil either. It's just the best strategy.
The prisoner's dilemma has a reward system designed to encourage cooperation anyway. The fact that a "good" strategy beats out an "evil" one is due to the way the reward system is designed; not an inherent law of nature. Not all reward systems work this way, so we can't call this a proof for good vs evil.
Except in the last round
"Evil will always triumph because good is dumb" - Dark Helmet.
This seems like a stupid thing to ask but could you give an example of a real world reward system that does not reward cooperation?
@@MrFishtoot It's not a stupid thing to ask, because actually that's a good and important question. His comment saying that "this is not a law of nature" is nonsense and false, because this strategy and rewarding system is exactly how nature by itself works. We can see this proof everywhere we look in the evolution of animals and also in today's economy. We humans managed as species to get this far and developed because we learned how to cooperate and as a result we evolved much better comparing to solitary animals or any uncooperating species.
Retaliating when necessary and cooperating when is fair is a very good strategy in life.
What a great video! Most people don't realize it, but game theory is build in to our very genes, and indeed the genes of every living organism. You really illustrate this point in an easy to understand way. I remember reading about these tournaments in The Selfish Gene, a book that quite literally changed my view of what life is in a radical way.
I really like that book. I learned about evolution in high school and university but didn't really get an understanding for it till that book. I can't believe how old it is though. We should have been learning evolution through that lens the whole time.
This Place
The gene-centered view of evolution is very useful model for thinking about how life works, but it's also a very challenging and counter-intuitive way of thinking about how life works. We humans love to feel important, and it's hard to feel important if one sees oneself as merely a vehicle constructed by their genes for the "purpose" of gene replication.
The more you learn about biology, though, the more this view rings true. Consider the sea squirt, a creature that develops a brain during its larval stage, but as soon as it reaches the next part of its life cycle it finds a nice comfy spot to anchor and live out the rest of its existence without the need to move or react to its environment in any way. Then the first thing it does is digest its own brain and nervous system since they're no longer useful to the genes of that organism.
We only have a brain, and by extension a consciousness, because it's useful to our genes for their vehicle to be able to move around and react in real time to happenings in their environment. Then you can start to think about all the many and varied ways that our genes manipulate our consciousness to their own ends, and you start to realize how deep that rabbit hole goes.
This is precisely why I think metacognition is one of the most important things we can do, unless we consider our own thoughts and why they occur to us, we're slaves to the unthinking will of our genes; but as soon as we see the chains for what they are, they melt away. And make no mistake, what is best for us is not the same as what is best for our genes. Our genes will tell us that sacrificing ourselves for two siblings or eight first cousins is a roughly even trade.
Our genes know nothing of human flourishing, they simply run on the unthinking calculus of how to best replicate; and most of us are running their operating system on the hardware of our minds without even thinking about it, we're walking around on complete auto-pilot. That's why The Selfish Gene is more than just an interesting book, it's an important book; and I'm glad that you're bringing many of the concepts contained therein to a new generation.
It feels like after your comment got This Place's attention you saw it as an opportunity to share thoughts so you immediately tried to say everything you could that was relevant lol
bwoy12345
Sure. I have quite a bit to say about the gene-centered view of evolution and I don't know very many people who understand it well enough to talk about it with.
waiting for the day that people discover this channel and it blows up
ill be watching like a proud father
New strategy : TIT FOR 1 1/2 TATS basically TIT FOR TAT, but it defects every other enemy defection, doesn't matter if they are back-to-back or 100 rounds away. Plus, if the opponent defects at the same time, it becomes TIT FOR TAT as long as the opponent keeps defecting, then when the opponent cooperates again, it switches back to itself, as in defect on the second defection. It may not do as well as regular TIT FOR TAT, but it might just be better than FORGIVING TIT FOR TAT, since it would still retaliate on strategies like TESTER, while still preventing defection echos.
"Tit For Tat but I'll Remember This."
Yeah, I was wondering what a "less forgiving" tit for tat would be like. For instance, it "forgives" the first time, but then if they do it again, the programming swaps to regular tit for tat mode unless they "play fair" for some number of rounds, after which it goes back to being forgiving.
You could call "only forgives once" version "fool me once" or something like that. Then you could make several variants that have different requirements for rounds of fair play before switching back to "forgiving" mode, and see which works out the best. You could also try variants like having it so each time they "betray" it sticks to regular tit for tat mode even longer before going back to being forgiving, and/or after a certain number of betrayals it *never* leaves tit for tat mode.
Kinda like we forgive, but not forget?
Smart
"Video's over now."
Three videos in and I already love you. That was excellent.
I love how the Prisoner's Dilemma has so many applications to the real world. Evolution, company rivalry, and even how people interact with each other.
I have watched 3 to 4 video about Prisoner's Dilemma but this is the first one with such in depth look into the strategies, really liked it. Thanks making this!
almost 100000 subs ?! this channel grows fast (it is what it deserves for this great quality)
They aren't growing fast enough. They deserve a lot more.
+Aveyago I agree
This channel beats almost every other channel I'm subscribed to. The visuals and the small touches are priceless.
I really wasn't expecting such a complete video on this subject
I love this video so much! I see life as an (endless) iterated prisoner's dilemma, I was just never able to put it in words why I think we are all better off cooperating (with an occasional punishment). Thank you for making this! :)
This is such a well-animated and well-explained video! I wonder if it makes sense to defect on the very last move because you can’t be retaliated against?
EDIT oooh I just read the description and I see you discussed it there
Hey Cary!
Oh so I see you've been planning this for a while... 👀
cool beans
What is the Harrington strategy? I tried searching, but I cannot find about it anywhere.
Could you explain the HARRINGTON strategy? I couldn't find any articles.
The book didn't give much description of it. Harrington isn't the name of the strategy (nor is JOSS), these are the last names of the people who entered them into those tournaments. That may be why you can't find info on them. Most of the strategies in the book were referred to that way.
But here is where the passage talking about it. Or if that is a temporary link search for "harrington evolution of cooperation" and it should bring it up.
books.google.ca/books?id=GxRo5hZtxkEC&pg=PT33&lpg=PT33&dq=harrington+evolution+of+cooperation&source=bl&ots=j4qSHfDXTE&sig=71hE72G-aiTDFzYqvttJnczbNn8&hl=en&sa=X&ved=0ahUKEwiu6dnRwN3NAhUj9IMKHa7iDHcQ6AEIIjAA#v=onepage&q=harrington%20evolution%20of%20cooperation&f=false
If you'll forgive me asking, what is "the book"? It sounds interesting.
click "read more", there's a link
I think he provided one, I read about this in the Selfish Gene.
Exploitative strategies don't go extinct in the real world… As they say, "There's a sucker born every minute." An example would be con-men.
Read Axelrod's "Evolving New Strategies", watched this, reread Axelrod... much greater clarity now. Great video. Bravo
Where exactly can I order my G's ? I almost ran out recently and need to stock up a_ain. Damnit, I ran out of them, _reat...
Heres some g's gggggggggggggggggggggggggggggggggggggggggggggggggggggggggg
gg
Try Avoid7. Don’t use a seventh-symbol.
This is by far my favourite video on youtube right now. actually - aside from music or other amusement related videos i would say it is the most easy to follow and interesting videos ive seen sofar. i really am happy to find this masterpiece
I remember hearing of this, in a "Social Darwinism" context. People always asume that the *most complex* one are the best adapted.
But this is an actuall test where one of the least complex tactics (just copy the other sides last move) won. It was way less complex then any other strategy entered (except maybe "always play nice") but still won out.
I've rewatched this video about 6 times. I always find myself back to it.
When you consider that life consists of a bunch of squishy computers, it's hard not to appreciate our mental algorithms, which are so complex regarding strategy, that we make and seek out videos such as this.
I feel like I can listen to you talk for hours. Great job again. Keep it up!
This would make an amazing evolution AI
Dude my professor showed this too us in class and then i found the rest of your stuff on this channel, def. earned my sub.
This dumps a good part of religious and moral philosophy. Thanks for making it as simple as possible but not simpler.
You're back! I'm so happy. Love your art style, explanations, and videos in general. Sharing the crap out of them.
here's an idea, T4T but it always defects on the last turn
This is literally the most interesting video I've EVER seen, and overall one of the best videos I've seen.
this is that game! The evolution of trust!
Amazing new video Jesse! Why don't you have a billion subscribers yet?? Can't wait to translate this one and the older prisoners dilemma video alongside the others... once the caption translations are turned on that is.
Giggling like a child because of the plaque that reads T4T
Intelligent thinking of linking this dilemma to evolution of cooperation. Well done and thank you!
Where can I buy the letter G?
I really love your videos! from your voice to the visuals and amazing soundtrack!
Now I'm just wondering who got a worse score than a random strategy
Kabitu1 maybe an always defector?
Matheus Tran
Nah, HOTZ.
Extremely mind blowing and comprehensive video on this topic! Big thanks!!
This is supposed to be like life isn't it?
i think so
It's supposed to demonstrate why certain strategies (like tit-for-tat) evolve in environments where the reward system is similar to that of the prisoner's dilemma. These sorts of environments are often, though not necessarily always, present in real life scenarios. In that sense, yes; it can be used to demonstrate the benefits of strategic patterns in certain (but not all) real life scenarios.
The important factor here is the reward system. As reward rules change, strategies will change to fit them.
his avatar is earth and his name is this place. so yes
No.
It demonstrates how morality can arise from natural selection.
The fascinating thing about this is that it even explains interspecies cooperation. Kropotkin's Mutual Aid was a hugely popular work when it was first pulished, and I think we've spend much of the last century in denial of the evidence for cooperation, because no one could think of a reason it might exist. Now we know! Thanks!
Idea: Tit for traitor:
tit for tat
but it defects last round
so no reaction can be had
Yeah, but since both defecting is worse for the group, once tit for traitors kill off a portion of the tit for tats, they'll get less than the tit for tats get with each other.
In the 2nd tournament, the host did not disclose how many rounds there would be specifically to avoid this, as it goes against the spirit of the experiment
Another idea: reverse tester:
Betrays twice, then cooperate, then acts as tit for tat
I love your videos! The animation and lecture styles used really appeal to me.
What can I do to support this channel?
Virtues Last Reward anyone?
Awesome game, have you played/ are playing Zero Time Dilemma?
ok, this is getting a bit creepy now
I randomly decided to try 999 a week ago, around 5-6 years after a friend of mine suggested it. I did google a few things up (ice-9, glycerin, "can I find Snake?") then I find out there's a new game (ZTD) and bump into a 999 live gameplay 00:00 today and I find this comment here. Morphogenetic fields playing tricks on my unconscious
I hadn't seen any ZTD or VLR news before playing and finding stuff
I was hoping to find a comment referencing VLR :P
^^
You make some of the most informative and insightful videos on youtube
Are you an accordion player?
no but I really like it
I did a presentation on this in grad school. We held a little iterated prisoners dilemma with starburst candies as the reward :). Then we broke down the math of it a little bit. You explained it very well, and I appreciate your animations. As a funny aside: I met my wife in that class :).
can this can explain why morals are they way they are? (I think so)
Excellent observation!
Also governments.
Yes it is often used in state of nature discussions and explains why the state was established.
Doesn't explain the state in the modern sense if you read anything by Kropotkin.
Only so long as organisms gain benefits based on a system of rules similar to the prisoner's dilemma.
Depending on the distribution of rewards, you will see different variations in strategy evolve.
My brother recommend this channel I'm so happy he did
I made the portuguese translation :D
Brigaldo
Ty and gj :D
Your videos are so original and engaging, I'm surprised you're not much, much more poplar!
Can't wait to get my letter G in the mail!
Excellent video. Summed up and explained the last few chapters in the Selfish Gene really well 👍
God I love your voice. Have my baby?
i hate his voice ugh
He'll kill it
Really good way of putting this. Well done
What will you be doing for 100k subs?
nothing. but I give you permission to eat a cupcake in my honour.
+This Place , Can I have milk too?
milk is for winners
Milk for all!
The party game Jack Box has a prisoner's dillema like question pop up in one of the minigames. Every year when we play it at the LAN party, the question comes up "take the money, and everyone who didn't loses the game. But if everyone takes the money, everyone loses the game. If no one takes the money, nothing happens" and the answer is to always take the money. As that scenario requires that everyone else in the group also takes the money for you to lose the game.
"Hi welcome to trivia night!"
Actually tit for tat still wouldnt be optimal. The optimal one would be tit for tat where you always pick "defect" on the last round, since that would be the most beneficial move that would have no consequences.
"And they didn't do a set 200 rounds; that way nobody would know when the interaction would end."
Pro tip: When the people who clearly know a lot about a topic (like Jesse, or the people running the tournament) tell you something, they're *probably right*, especially when your counter-point feels oh so clever. Before you try correcting them maybe try re-examining the evidence with your Thinking Cap on.
He just didn't listen to the whole thing.
Uh, i was obviously talking about the first scenario.
+luboisfat in which case forgiving tit for tat is more optimal, as are many many others methods.
TheAdmiralBacon But it clearly says in the graph that Tit for Tat is the best option.
Great content as usual! Thank you :) Hope the channel keeps growing.
You explain this very complicated.
I like the different viewpoint to this topic, very interesting and informative.
You can exploit tit-for-tatter by deflecting on the last round if you know what is the last round.
That's the JOSS strategy stated in the video.
+remilia scarlet no, JOSS deflects randomly, nat only on last iteration.
failing@commenting Oh,ok.
That strategy you said could work for the last round of the first one since the first tournament have a fixed 200 rounds.
It wouldn't be as good against itself.
Right like building trust with someone then stabbing them in the back. as you leave.
20 minutes of content in 10 minutes of video. great stuff
I like the way you ended the video. "Video's over now." Perfect.
I like how this video kinda explains why many animals have some kind of moral (many experiments have shown that humans aren't the only animals with a sense of morality)
Fantastic explanation and I like that you used the tournament results.
There is a very good interactive game called The Evolution of Trust which shows basically all of this stuff but in a game. It's really cool!
the best person to present ideas : This Place
7:47 that's interesting. It's like the Tit4Tat here is committing a suicide attack, taking out both it's opponent AND ITSELF, for the protection of the rest of the Tit4Tat family.
Did Veritasium copy this video? his story seems pretty similar with some (imo unconvincing) application of the ideas plus some new stuff at the end. I guess maybe the story for the prisoner's dilemma can only be told in so many ways? but tbh i'm a little suspicious.
Veritasium directly has this video on their references so they definitely took inspiration. But the Veritasium video has more than enough original content (notably, interviews with Strogatz and Axelrod, the actual sources of both videos) to justify its own existence
Title:
Game Theory and its Applications in the Social and Biological Sciences
Very well presented and captivating animations.
Pretty interesting! And manages to explain a lot of stat/ideological concepts in a pretty absorbable way! Nice!
the evolution of trust feels like a game version of this video.
Huzzah! A fellow person of culture.
Huzzah! A fellow person of culture.
You should definitly make more videos. I really appreciate them :)
Wonderful analysis. I highly recommend that everyone actually read Axelrod's book: it's actually incredibly accessible to everyone and it's incredibly informative and thought-provoking.
Bravo! Delightful video on an enlightening topic. Thanks!
very interesting, got some new information on the prisoner's dillema. good job
This made me rethink Game Theory, thanks!
This video actually explained evolution with the help of math. Great job.
Great video, what did you use to animate it?
Wow the videos just keep coming... it's like christmas :D
Thank you for everything you make.
"That's why I'm telling you about it only at the end" 10/10 would watch again
this might be your best video
"That's a healthy piece of real estate!" - Everyone's favorite blue midget Homestar, Homsar on the letter G
As usual, amazing video. Good work!
Just as good if not better than the veritasium version. Fantastic video.
Waiting for the day people discover this channel and it booms.
Great video. For future videos my request is more complicated subjects. All your previous videos explained stuff so well i understood them twice. Why not make it twice as complicated, so I and others only understand it once?
dude love your video keep the good stuff coming
It's taking me a bit to wrap my head around this but it's really cool
well done, this video summarized todays endless 3h lecture very nicely. Now I got the gist of what my professor wanted and can include the complicated equations he used to describe those models...meh
Wow, TMM brought me here and that was amazing, thank you!
YES! A new VID! These are great.
I wonder how much longer this channel has before it becomes really big.
I only visit this channel once a year. And it keeps getting better every time I return.