I think a major problem with this approach is that you're giving it the same world every time. The AI is not learning to look for resources, it's learning to how to get to where the resources are in that particular map.
I disagree and the counterpoint is examples like racing games They're trained to run the same course every time, yet when they're set loose on tracks they've never seen before, they still perform well. They aren't just training to memorize inputs, it knows what it wants and how to get it.
Dustin James True, but minecraft has a lot more variables than a racing game, making it significantly harder to generalize. There is also the fact that the AI tends towards a specific path rather than looking around for resources, suggesting it is memorizing the location of the ores instead of looking around for them.
The achievements would be too far apart to guide the AI properly and the only way the Ai would open the portals would be by doing the right actions completely randomly. You'll need a lot more goals than Achievement and mining to make it complete the game.
One major issue is that it seems that instead of being smarter, they just get better at beelining to the nearest chunk of high value ores by exploiting generations of discovery. If you repeat this, try varying the map every few epochs to make them more intelligent. You could also add things like the penalty for losing health or hunger.
Yeah, Basically The bot will just try to look at as much blocks at possible for new generations, and with 1000 repeats or so, the bot will know everything about the world, so it'll just travel to highly valued blocks then down a value and so on. The only thing it *actually* learns is to sense x and z position. Its like challenging 50 players to mine on diamond blocks spread out in a 25x25 stone cube as fast as possible with unlimited blocks and an op pick while giving the whole team a minute to talk and plan. It gets to a point like 25 or 10 try's where all the players go to a specific position and dig down in the most efficient, boring, way.
Ideas: Higher reward per second for higher health (i.e, +20 points per second with full health, +1 point per second on half a heart, 0 points total for death). Maybe even just leave it at death = 0 points when you die because that's just a flat out failure. Higher reward for more hunger, same as health. Higher reward for wearing armor (correlated with the actual protection value of the armor) SUPER high reward for correctly crafting items, giving it the incentive to learn High reward for combat engagement, but lose all reward on death. I think that should create a really interesting combat dynamic as it tries to avoid low health and death while still trying to kill mobs Basically... Reward/penalize every single aspect of the game according to how a real player feels about each aspect (Player doesn't like low health, armor, or food, so the AI doesn't like low health, armor, or food)
if it keeps getting points for just having high health it might encourage it to not do anything so it should stop giving point for high health if it doesnt do anything
@@tonitruivena8554 I was thinking it would be discouraged because it would have literally hundreds of times the amount of points for doing things, but if this is the case then maybe penalize it just ever so slightly for lower and lower health
Small reward for every experience point earned. This should encourage engagement with allot of mechanics. ... Turn on advancements, and maybe they will go after them for the X.P. reward. ... Large punishment for dying, which is mostly undone if they can recover their items before they despawn. (The differnece is equal to the value of any lost E.X.P./items.) ... If you could encourage them to build, that would be epic! Maybe some human-given build-judging points could encourage that?
It would have been more interesting, if each set would have been run on a different seed. With that, you could have had natural negative feedback for digging straight down to caves or lava. Also, to get to diamonds you have to dig down to below 16. It would have been interesting to see if the AI could have developed an efficient strategy to getting to diamond level fast without risking dying. Such as real players use: staircases, spiral staircases and digging two spaces at the time and teetering between them.
@@longnoseboi thousands of simulations is a really small number of iterations for a neural network. People doing research in the field often end up with hundreds of millions of iterations. Not saying anyone should have to go that far, but only then you find the interesting behaviours if the whole thing is well planned.
@@rushi7312 Their point is that you can't DO hundreds of millions of iterations in minecraft because they'd have to be run in minecraft, which works on real time. They had to make a simulation which could be run very quickly so that they could do lots of iterations of it quickly.
@@chloechristley8335 I'd have done it if it were easy... But I know it isn't. Tho I don't believe it's impossible to teach the AI how to do what it is supposed to do but without it being "really" in a "live" Minecraft world. I'm pretty sure there's a way to give it data to crunch and train on without actually being in-game. The problems come with world/loot gen tho. Once it's generated it's data. But finding a way to make the whole engine work without having to do any rendering etc would be a challenge.
@@DinoNuggies4665 "Don't Dig Straight Down" has been a community rule anyone who's ever played MineCraft oughta know. Because there's a fair chance of finding a lava pond or ravine which could kill you. If you HAVE to dig straight down a long way, dig a staggered 2x1 or staircase, so the block you're digging isn't the one you know is safe.
One thing you could add is torch placing as it's important in mining. Reward efficient torch placement and high enough light levels so that mobs cannot spawn. Efficient torch placement would be only placing torches when needed (not placing them too close) and also maximizing their coverage (not placing them too far).
Some general stuff about making this video: the command to let you see the bots more clearly is /effect give @s night_vision 10000 and you should go into gamemode spectator so you don't have to dig tunnels while the bots are doing their thing. Good idea on the x-ray pack tho
this isn't even the first of such attempts. There are many and more polished experiments on NN in Minecraft. Microsoft itself is doing research on the same, but with different goals that help in real life.
@@Kathrynerius But players still have to set goals for Baritone, it doesn't have a goal by itself. That's what this video is about.Not Bots but actual AI.
@@nuduw So far I almsot find Baritone's pathfinding more impressive than this tho still. I know he just wanted the bot to mine (which is kinda a flaw almost I feel) but besides, having the bot only on one seed in one location, with a diamond pickaxe already makes this only be the most inefficient and overcomplicated X-Ray ever. I imagine he did it because he had certain restrictions with C++ and Minecraft being in Java, but imagine if he had each generation on another seed, with no tools at all and told it some common things a player would know, like what tools you need for certain blocks, crafting recipes, at which heights ores are most likely to generate, a little more about the movement system (like what the fastest way of moving are), how to use sword axe and shield (with the cooldown) and generally encourage it too look around more. I'm sure it would be a lot of work and not everything on here would need to be done to have it probably be more interesting, but just imagine how good the bot would get (like I actually think it could become like an actual person playing and better). The visualization he did is awesome tho.
As someone who deals with RL a lot I see a couple problems. 1. It will overfit to finding diamonds simply because it has the se world during the course of training. 2. The bots interact with each other which is a problem because behaviourbof one agent influences the behaviour of others. Basically when they mine they compete for blocks or even help each other out by exposing more blocks. This is all fine as long as it is intentional. 3. I think your evolutionary alg is quite slow and other learning algorithms such as PPO would be a better fit. With that said I am impressed by the amount of work you put into this project and the results are not to be ashamed of!!!
Thanks, glad you enjoyed! Most of the standard material like programming, AI techniques, and required math I learned while getting my degree. Things relating to visualization (like the minecraft mod and network graphics) were mostly self studied through online forums and existing repositories.
Awesome! Just started learning the basics of AI, and that was the most helpful explanation of a neural network I've seen yet. Also, really well made, great production. Thanks for making the video, I enjoyed it thouroughly.
I'd suggest rewarding the bot for "Smart mining" like have it go after ore in the normal progression of the game and stop rewarding it when it gets to much of say coal or stone, to the point where it might even avoid less valuable ores like coal in favor of finding something like diamonds
this is the first video that *actually* tells me how a neural network works. all the others just throw up the nodes grapic and wave away the "this number goes to this node which goes to this one"
Really cool and good example. I wouldn’t mind seeing a little more progress on this. Random of 100 worlds, and instead of negative for digging down, large % of worlds should start with lava a few blocks under or a big fall to discourage but not limit digging down. Have them collect 100 of each ore and wood, teach chopping trees, and switching tools. That would be interesting to see a short series with this as a goal. Either way, awesome work!
Thank you for including weights in your explanation of neural networks! I've seen a few videos on this subject, and you're the first person I've seen to actually include them in the explaination
I think programing so that they feel hunger and a fear responce to certain mobs were it can choose to attack or run based on certian specified parameters would help pramote more comples seaking behaviors
You don't have to program a fear response, assign a negative score to getting hurt and it will *learn* to fear mobs. That's the whole point. You could make a very robust AI if you weren't limiting yourself to neural nets.
The f3 information page also provides a "targeted block" readout, if you allow them to access that Information as an input similar to how you gave them coordinates it could help promote behaviors of looking around and intentionally mining ore when they see it instead of blind tunnelling, which I think would be the logical next step for the ai to do better mining
There's so much more you can do with this. You can add inputs like hunger and rewards from eating food. You can add rewards for getting the most number of points with the least amount of work etc. Thanks for your vid.
The biggest issue I can see is that since you've just used the same map every time it looks like it's memorizing features instead of dynamically responding My biggest suggestions would be: 1. Use different chunks or shuffle through a hundred 2. Use a recurrent network like a lstm so it can use past information to it's advantage
@@mouradqqch1767 Stuff like where it had already dug so it doesn't overlap dig patterns, caves, and if you wanted to expand the game to include crafting the tools previous ore deposits could be useful. It's a method to try out and while it may be helpful it could also not help at all
Just found this channel and finishing watching this episode. Wish there was more since ik interested in this stuff. Hope you'll upload again soon. Awesome vid, and enjoyed it
I know I'm super late to this, but if you want it to explore more organically, I think you should reward it for seeing an ore block and for moving towards said ore block. Making sure that this action can only occur once so they don't just end up spinning in circles for points.
i remember watching a video about someone who made an ai that could play Minecraft by only giving it the screen (thought it was compressed) as input, and basically giving score for surviving and removing score when it died. could anyone point me to that video? this video reminded me of it
How can such an amazing amount of intelligence and work get put into this project and yet fundamentally mess up. The bots should not mutate from the single best bot. They should mutate from the best 50% or 10% or what ever seems like its working the most. Also, this simulation makes it such that the buts learn what best to do in that specific map. You need to make the bots in Minecraft instead of a simulated program. Even if that's not possible cause it's too slow, that literally how it needs to be done in order for random seed generation and accurate sim world to game world translation since both are the same. I really like this video and the concepts in it. Genuinely good job. You just need to make those changes and you will have an absolute banger of a video.
Have you ever worked on a large, long term project from scratch? That's exactly how 😂 All that work and intelligence makes it easy to lose things through the cracks of you don't have inhuman levels of planning structure.
@@SophiaAstatine I am the definition of not having "inhuman levels of planning structure". The insult was a joke at me. I was being ironic cause I spend weeks one projects just to have a completely fundamental issue pop up that screws it all up. Then I went on to explain some of the things I would have done differently cause I did find that to be a genuine problem. But yeah most of what I said was a joke.
I challenge you to create an AI model that teaches them to survive. Maybe they'll learn to build, fight and farm by themselves if they can access their own inventories and interact with objects? Sounds like a much larger project but simply imagine what an AI companion mob could accomplish alongside players
It would be quite interesting to see how the bot learns if you give it milestones. Eg, collecting wood gives it a large amount of points but only the first time. The same for making a crafting bench, etc. I think a bot that can start in any chunk simulation, make a pickaxe, upgrade to iron, and then make a bucket. Seems like the bot invented strip mining pretty quickly, so the hardest part would be the progression ladder.
This is really interesting as a first step, I would love to see how far this can go. For instance, what if you punished it when it died? What if you told it some basic information, like created an association between mountainous biomes and emerald ore, so it knew (without spending millenia training) that it could find emerald ore in mountains. What about making it guess how far it can jump so it actually tried to parkour? This could go in some really interesting directions depending on where you want to take it. Plus, its literally the first actual description of what the fuck neural networks are I have seen so far. Most just say "it like neurons, next question" (another thing that would be cool is if you composited the weights of all of the networks in a generation, with the total score they earned deciding how much they impact the baseline for the next generation. This would more closely mirror genetics since a single mutation never becomes homogenous throughout an entire species over one generation and by creating composites instead of deciding one victor you allow for it to learn from all of the "failed" attempts slightly.) On a side note, is there any specific reason you choose the network layout that you did? Like how many nodes in each collumn?
A neuron in a neural network is quite simple. It is just multiplying the elements in two lists (the input and the weights) to get a number, then adding an extra number (bias) to get the linear output, and finally applying a nonlinear function to the linear output such as tanh or sigmoid or relu to get the neuron output. I wouldn't really call it neural, it's mostly multiplying numbers together.
This is awesome, are you going to make a tutorial? I've been trying to achieve a neural network in Minecraft, but I haven't had as much success as you lol
This is absolutely a delight to see. I'm surprised at all the criticism and "unsolicited advice" people are offering. Like, let them make their own ANN's. Roko's Basilisk would be proud, sir! **secret handshake** (haha)
Best approach would be trim the time down to train the ai by coding on already learnt ethics in real life by us and then training forward. Like In real world, jumping up and down or standing in one place for long time without any reason would be considered unhealthy, waste of time in AI world. So just as you coded not to dig not more than 2 blocks down, similarly code few things not to do more than set amount of time, but also allows the AI to bypass such limitations if priority meets the needs.
I would love for you to continue this project a bit to be honest, or maybe if someone can wrote the code in python so i can make do some work with the project. It would be cool to bot only watch the AI mine but also to see it build and even craft, as well as for it to learn to obtain needs for survival and other complex game play features.
Good job, I think the maps should change every few epochs (or every epoch) so that they don't just memorise the locations of where the ores are, and therefore become smarter.
Now turn this into a mod, and villagers will finally be useful, they will build their civilization from stone age to modern age, with them working of course, not just giving them materials This is my type of mod too, so I don't feel lonely playing Minecraft lol
Perhaps in a decade we’ll be able to run a mod like that. That neural network trainer is simulating a world where the AI is the only decision maker, i.e. no mobs, no players, no other AIs. If you want neural network villagers that aren’t crap you’ll have to train them significantly in the actual game with actual players or else they’re going to act stupid anytime players get close to them. Running neural networks isn’t cheap either, and given the complexity of Minecraft and human behavior when interacting with Minecraft, these neural networks may be far too small to produce interesting behavior noticeably different from current villagers. So you may end up needing the latest gen tech just to load even a small village, and given that neural networks run the best on GPUs, your framerate is likely to struggle as it tries to render and think for a dozen AIs simultaneously. Honestly that’s probably a bit optimistic even. Give it a decade though and I wouldn’t be surprised if it becomes a reasonable thing to add.
@@Treviisolion an alpha go style neura network for this would be interesting. It watched many many professional games and learnt how to play intuitively by watching how professionals make their decisions
@@olliejobson6371 Possibly, though go is a much simpler game than Minecraft. There's only one goal, only one other player, and a limited number of actions, and it still took a ridiculous amount of processing power to match and finally beat humans consistently at it. My money would be on whatever the team that made an AI to escape spawn in 2b2t did, where you build an AI to do what you want combined with a bit neural network learning rather than brute force learning everything.
I am going to school for software engineering and I hope to do AI classes because this stuff is just phenomenal. The visualizations you had in the video were very well made too. Seeing you use the same IDE I use to code made me really hyped because I could see myself doing projects like this myself sometime in the future. Thanks for the inspiring video Poppro :)
Actually, the seconde set of reinforcement learning with your rule of never digging straight down was not the right thing to do. Your ai learned that digging straight down was the most efficient way to win according to the map you gave it. Implementing an artificial rule to stop it from using this strategy is counter productive, because now you are partially forcing it to comply to how you think the game should be played. Some, in the comment section, have pointed out how you could have had better result without this new rule (which created some new strange and counter productive behavior in your ai). First, players do not dig straight down due to fall damage, lava and cave full of mobs. So you could have implemented a simple death mechanism, when it falls and die 0 points, when it goes into lava 0 points and when it dies from mobs 0 points. Second, depending on where you position your player on the same map, digging straight down to diamond level could be the optimal strategy, sometime (like in the 1st run) there is no lava or caves in between the two Y position. therefore you could have placed randomly your player on the same map and let it learn the different strategies or you could have changed the map every other epochs. Still, a great video showcasing C++ ai programming and java minecraft mod programming!
Or even, subtract say, 2 points for taking damage, falling into lava, even dying. With that detriment to its scoring, it should figure out to avoid those, or learn to minimize it after a while. Definitely don't punish it for digging straight down
One day the ai will speed run and play minecraft all by itself from setting a random seed and making a name for the world, getting diamonds and netherite, starting a server for other ai models and showing off builds it has made and even become a RUclipsr making minecraft videos.
As others have pointed out, there are a few problems with the way you went about training. for the world, perhaps having a selection of 8 or more different maps, one of which being chosen at random for each epoch. this teaches the neural net not the specific world they're in, but rather how the blocks that surround them should influence the decisions they make. Another problem I noticed was that many times the network would decide to just stand still, or just look around. perhaps you could put in place a punishment for standing around and doing nothing for too many decision cycles (be lenient so they have time to 'see' the world), with a sort of forgiveness towards that punishment for turning to look around, but capping how many times that forgiveness works before actually making a movement/action. making the punishment occur quicker after each movement that is not a successful mining action.
![Noob To Max With DRAGON REWORK In Blox Fruits [FULL MOVIE]](http://i.ytimg.com/vi/LnBMOinoOvA/mqdefault.jpg)
![I.N "HALLUCINATION" | [Stray Kids : SKZ-PLAYER]](http://i.ytimg.com/vi/n5B5q1Hwt_U/mqdefault.jpg)
I think a major problem with this approach is that you're giving it the same world every time. The AI is not learning to look for resources, it's learning to how to get to where the resources are in that particular map.
yes
I agree
It’s basically learning a set seed speedrun lol
I disagree and the counterpoint is examples like racing games
They're trained to run the same course every time, yet when they're set loose on tracks they've never seen before, they still perform well.
They aren't just training to memorize inputs, it knows what it wants and how to get it.
Dustin James
True, but minecraft has a lot more variables than a racing game, making it significantly harder to generalize.
There is also the fact that the AI tends towards a specific path rather than looking around for resources, suggesting it is memorizing the location of the ores instead of looking around for them.
I think that the AI should also be rewarded by completing advancements, so it will actually progress to beat the game and not just mine all the time.
Or the in game achievements as a guide to completing the game
@@evancalgary7684 In the latest versions the achievements have been renamed to advancements.
already done? ruclips.net/video/baAa6s8tahA/видео.html
The achievements would be too far apart to guide the AI properly and the only way the Ai would open the portals would be by doing the right actions completely randomly. You'll need a lot more goals than Achievement and mining to make it complete the game.
@@Enju-Aihara that isn't a neural network though
One major issue is that it seems that instead of being smarter, they just get better at beelining to the nearest chunk of high value ores by exploiting generations of discovery.
If you repeat this, try varying the map every few epochs to make them more intelligent.
You could also add things like the penalty for losing health or hunger.
Yeah, Basically The bot will just try to look at as much blocks at possible for new generations, and with 1000 repeats or so, the bot will know everything about the world, so it'll just travel to highly valued blocks then down a value and so on. The only thing it *actually* learns is to sense x and z position. Its like challenging 50 players to mine on diamond blocks spread out in a 25x25 stone cube as fast as possible with unlimited blocks and an op pick while giving the whole team a minute to talk and plan. It gets to a point like 25 or 10 try's where all the players go to a specific position and dig down in the most efficient, boring, way.
@@Vaiave yep, that’s all it does.
They don’t get more intelligent, they just get X-ray.
@@cecilrhodes2153 To put it simply, yes, you're right.
They can't lose health or hunger 🐶🐶🐶
@@brysmann4291 sure they can, if they would ‘fall’ they’d lose health, though I suppose the hunger thing isn’t really possible here.
Ideas:
Higher reward per second for higher health (i.e, +20 points per second with full health, +1 point per second on half a heart, 0 points total for death). Maybe even just leave it at death = 0 points when you die because that's just a flat out failure.
Higher reward for more hunger, same as health.
Higher reward for wearing armor (correlated with the actual protection value of the armor)
SUPER high reward for correctly crafting items, giving it the incentive to learn
High reward for combat engagement, but lose all reward on death. I think that should create a really interesting combat dynamic as it tries to avoid low health and death while still trying to kill mobs
Basically...
Reward/penalize every single aspect of the game according to how a real player feels about each aspect (Player doesn't like low health, armor, or food, so the AI doesn't like low health, armor, or food)
if it keeps getting points for just having high health it might encourage it to not do anything so it should stop giving point for high health if it doesnt do anything
@@tonitruivena8554 I was thinking it would be discouraged because it would have literally hundreds of times the amount of points for doing things, but if this is the case then maybe penalize it just ever so slightly for lower and lower health
Small reward for every experience point earned. This should encourage engagement with allot of mechanics.
...
Turn on advancements, and maybe they will go after them for the X.P. reward.
...
Large punishment for dying, which is mostly undone if they can recover their items before they despawn. (The differnece is equal to the value of any lost E.X.P./items.)
...
If you could encourage them to build, that would be epic! Maybe some human-given build-judging points could encourage that?
@@agsilverradio2225 probably very difficult to get to work in an AI
@@agsilverradio2225 I'd guess it would probably be easier to code a new minecraft-like game with AI in mind than to get something like that to work.
It would have been more interesting, if each set would have been run on a different seed. With that, you could have had natural negative feedback for digging straight down to caves or lava. Also, to get to diamonds you have to dig down to below 16. It would have been interesting to see if the AI could have developed an efficient strategy to getting to diamond level fast without risking dying. Such as real players use: staircases, spiral staircases and digging two spaces at the time and teetering between them.
this ai couldn’t do that as for whatever reason it’s running in a simulation of the game which would lock it to while number
@@longnoseboi thousands of simulations is a really small number of iterations for a neural network. People doing research in the field often end up with hundreds of millions of iterations.
Not saying anyone should have to go that far, but only then you find the interesting behaviours if the whole thing is well planned.
@@rushi7312 Their point is that you can't DO hundreds of millions of iterations in minecraft because they'd have to be run in minecraft, which works on real time. They had to make a simulation which could be run very quickly so that they could do lots of iterations of it quickly.
@@chloechristley8335 I'd have done it if it were easy... But I know it isn't.
Tho I don't believe it's impossible to teach the AI how to do what it is supposed to do but without it being "really" in a "live" Minecraft world.
I'm pretty sure there's a way to give it data to crunch and train on without actually being in-game.
The problems come with world/loot gen tho. Once it's generated it's data. But finding a way to make the whole engine work without having to do any rendering etc would be a challenge.
The fact that the bots survived digging down without drowning in lava is just, cursed.
I suppose you don't dig down a lot?
well, the bots that picked spots with lava beneath them didnt pass on their genes and the ones that knew where the diamonds were did
@@matthewe3813 It was one map, not a million random maps, so there was no selection process.
@@robertsandlin366 Well, it was a selection process but not on how to avoid lava, just on where it's most efficient to dig down.
@@DinoNuggies4665 "Don't Dig Straight Down" has been a community rule anyone who's ever played MineCraft oughta know. Because there's a fair chance of finding a lava pond or ravine which could kill you. If you HAVE to dig straight down a long way, dig a staggered 2x1 or staircase, so the block you're digging isn't the one you know is safe.
One thing you could add is torch placing as it's important in mining. Reward efficient torch placement and high enough light levels so that mobs cannot spawn.
Efficient torch placement would be only placing torches when needed (not placing them too close) and also maximizing their coverage (not placing them too far).
I personally never light up my mines because of mob spawning mechanics mean they never spawn close to me. I just turn up my brightness
@@sheepy0125 normal for alot of players tbh..
Some general stuff about making this video: the command to let you see the bots more clearly is /effect give @s night_vision 10000 and you should go into gamemode spectator so you don't have to dig tunnels while the bots are doing their thing. Good idea on the x-ray pack tho
add "true" to the end to that command and you won't have those annoying particles
Wow, really cool stuff! Keep up the good content, I'm looking forward to your next video. :)
Thanks!
Never actually taught someone would make a Neural network in Minecraft, this is amazing to be higher
this isn't even the first of such attempts. There are many and more polished experiments on NN in Minecraft. Microsoft itself is doing research on the same, but with different goals that help in real life.
taught
@@nuduw 2b2t baratone playing the whole game for players already, lol.
@@Kathrynerius But players still have to set goals for Baritone, it doesn't have a goal by itself. That's what this video is about.Not Bots but actual AI.
@@nuduw So far I almsot find Baritone's pathfinding more impressive than this tho still. I know he just wanted the bot to mine (which is kinda a flaw almost I feel) but besides, having the bot only on one seed in one location, with a diamond pickaxe already makes this only be the most inefficient and overcomplicated X-Ray ever. I imagine he did it because he had certain restrictions with C++ and Minecraft being in Java, but imagine if he had each generation on another seed, with no tools at all and told it some common things a player would know, like what tools you need for certain blocks, crafting recipes, at which heights ores are most likely to generate, a little more about the movement system (like what the fastest way of moving are), how to use sword axe and shield (with the cooldown) and generally encourage it too look around more. I'm sure it would be a lot of work and not everything on here would need to be done to have it probably be more interesting, but just imagine how good the bot would get (like I actually think it could become like an actual person playing and better). The visualization he did is awesome tho.
This is crazy cool, I could legit watch someone... Commit their entire life to this endeavor. I would be their biggest fan
As someone who deals with RL a lot I see a couple problems.
1. It will overfit to finding diamonds simply because it has the se world during the course of training.
2. The bots interact with each other which is a problem because behaviourbof one agent influences the behaviour of others. Basically when they mine they compete for blocks or even help each other out by exposing more blocks. This is all fine as long as it is intentional.
3. I think your evolutionary alg is quite slow and other learning algorithms such as PPO would be a better fit.
With that said I am impressed by the amount of work you put into this project and the results are not to be ashamed of!!!
I'm not even 2 minutes into the video and it's professional as hell. You deserve more subs than what you all ready got mate. Keep up the great work!
Friggin genius...
Could you tell me where and how you learned the material to make this project happen?
Thanks, glad you enjoyed! Most of the standard material like programming, AI techniques, and required math I learned while getting my degree. Things relating to visualization (like the minecraft mod and network graphics) were mostly self studied through online forums and existing repositories.
@@thepoppro you should make it so they get points for advancements so they dont just dig straight down
This is so amazing, i love neural networks!
Awesome! Just started learning the basics of AI, and that was the most helpful explanation of a neural network I've seen yet. Also, really well made, great production.
Thanks for making the video, I enjoyed it thouroughly.
I'd suggest rewarding the bot for "Smart mining" like have it go after ore in the normal progression of the game and stop rewarding it when it gets to much of say coal or stone, to the point where it might even avoid less valuable ores like coal in favor of finding something like diamonds
this is the first video that *actually* tells me how a neural network works. all the others just throw up the nodes grapic and wave away the "this number goes to this node which goes to this one"
Really cool and good example. I wouldn’t mind seeing a little more progress on this. Random of 100 worlds, and instead of negative for digging down, large % of worlds should start with lava a few blocks under or a big fall to discourage but not limit digging down. Have them collect 100 of each ore and wood, teach chopping trees, and switching tools. That would be interesting to see a short series with this as a goal. Either way, awesome work!
For your first video this is has really high quality. Loved the vid
"See if you can find the bug while you watch"
Alex: **STARTS FUCKING FLOATING AND GOING THROUGH WALLS**
Thank you for including weights in your explanation of neural networks! I've seen a few videos on this subject, and you're the first person I've seen to actually include them in the explaination
I think programing so that they feel hunger and a fear responce to certain mobs were it can choose to attack or run based on certian specified parameters would help pramote more comples seaking behaviors
That's a good idea.
Fight or flight.
*creeper*
F L I G H T !
That's interesting idea.
I think it'd probably cjose to live underground and then come up to the surface to hunt for food
Bro that last sentence better not have been an attempt to say "...promote more complex seeking behavior"
You don't have to program a fear response, assign a negative score to getting hurt and it will *learn* to fear mobs. That's the whole point.
You could make a very robust AI if you weren't limiting yourself to neural nets.
ai: walks past tons of iron and coal and proceeds to mine in straight lines
Poppro: I'd say this is a great success
This video taught me how neural networks ACTUALLY work. Thanks man!
The f3 information page also provides a "targeted block" readout, if you allow them to access that Information as an input similar to how you gave them coordinates it could help promote behaviors of looking around and intentionally mining ore when they see it instead of blind tunnelling, which I think would be the logical next step for the ai to do better mining
There's so much more you can do with this. You can add inputs like hunger and rewards from eating food. You can add rewards for getting the most number of points with the least amount of work etc. Thanks for your vid.
i know basically nothing about programming and neural networks, but this was still entertaining
This video is hella underrated. Very neat idea for a project!
I hope you continue with this long term... it will be amazing when an AI is advanced enough to build a computer inside Minecraft.
The biggest issue I can see is that since you've just used the same map every time it looks like it's memorizing features instead of dynamically responding
My biggest suggestions would be:
1. Use different chunks or shuffle through a hundred
2. Use a recurrent network like a lstm so it can use past information to it's advantage
What kind of past information would be useful?
@@mouradqqch1767 Stuff like where it had already dug so it doesn't overlap dig patterns, caves, and if you wanted to expand the game to include crafting the tools previous ore deposits could be useful. It's a method to try out and while it may be helpful it could also not help at all
I think this serie it's a really cool concept and you had all my support
Looks like it's on it's way to making a strip mine.
Hopefully it learns to place torches before the mobs start spawning.
This fact that you can do this only based on your knowledge and resourcefulness is beyond me. Congratulations on this badass video man.
Every Human Ever: NEVER DIG STRAIGHT DOWN!!!!!!!!!!
Chad AI: Hold My Coolant.
Just found this channel and finishing watching this episode. Wish there was more since ik interested in this stuff. Hope you'll upload again soon. Awesome vid, and enjoyed it
I know I'm super late to this, but if you want it to explore more organically, I think you should reward it for seeing an ore block and for moving towards said ore block. Making sure that this action can only occur once so they don't just end up spinning in circles for points.
"haha stupid bot so funny"
"digging it just like papa, i'm so proud"
11:15
"IT BECAME TOO DANGEROUS! KILL IT NOW!!!"
That's the content I'd like to see more on RUclips :]
I love how you documented the process and shared it, Please make more like this!!!
i remember watching a video about someone who made an ai that could play Minecraft by only giving it the screen (thought it was compressed) as input, and basically giving score for surviving and removing score when it died. could anyone point me to that video? this video reminded me of it
Imagine you just start your survival game with it and you able to find some iron… when you see
Bot has achieved *[Eye Spy]* advancement
In the future, I get to have a friend who plays with me, and never leaves me with a unfinished castle.
wow, this is really awesome! what a great first video
How can such an amazing amount of intelligence and work get put into this project and yet fundamentally mess up. The bots should not mutate from the single best bot. They should mutate from the best 50% or 10% or what ever seems like its working the most. Also, this simulation makes it such that the buts learn what best to do in that specific map. You need to make the bots in Minecraft instead of a simulated program. Even if that's not possible cause it's too slow, that literally how it needs to be done in order for random seed generation and accurate sim world to game world translation since both are the same.
I really like this video and the concepts in it. Genuinely good job. You just need to make those changes and you will have an absolute banger of a video.
Have you ever worked on a large, long term project from scratch? That's exactly how 😂 All that work and intelligence makes it easy to lose things through the cracks of you don't have inhuman levels of planning structure.
@@SophiaAstatine I am the definition of not having "inhuman levels of planning structure". The insult was a joke at me. I was being ironic cause I spend weeks one projects just to have a completely fundamental issue pop up that screws it all up.
Then I went on to explain some of the things I would have done differently cause I did find that to be a genuine problem.
But yeah most of what I said was a joke.
@@SpicyMelonYT very detailed written joke.
@@SophiaAstatine no, as I said, the first part was joke, the rest was constructive criticism and the appreciation
@@SpicyMelonYT Hope he reads it and makes more videos. Would be awesome with a follow up video but the guy seems to have vanished.
That was the best neural networks explanation ever, it finally clicked. I have been making basic neural nets without even knowing. Omg thx so much
i love how he just casually writes "how to minecraft" in his search bar
Explains a Neural net in the simplest way I have ever seen and understandable with an example included !! very good!
I challenge you to create an AI model that teaches them to survive. Maybe they'll learn to build, fight and farm by themselves if they can access their own inventories and interact with objects? Sounds like a much larger project but simply imagine what an AI companion mob could accomplish alongside players
When you have no friends to play minecraft with, so you download a mod.
@@golett0331 Modern problems require modern solutions
It would be quite interesting to see how the bot learns if you give it milestones. Eg, collecting wood gives it a large amount of points but only the first time. The same for making a crafting bench, etc.
I think a bot that can start in any chunk simulation, make a pickaxe, upgrade to iron, and then make a bucket. Seems like the bot invented strip mining pretty quickly, so the hardest part would be the progression ladder.
cool, good job! please create more content like this, i was very intrigued :)
This is really interesting as a first step, I would love to see how far this can go. For instance, what if you punished it when it died? What if you told it some basic information, like created an association between mountainous biomes and emerald ore, so it knew (without spending millenia training) that it could find emerald ore in mountains. What about making it guess how far it can jump so it actually tried to parkour? This could go in some really interesting directions depending on where you want to take it. Plus, its literally the first actual description of what the fuck neural networks are I have seen so far. Most just say "it like neurons, next question"
(another thing that would be cool is if you composited the weights of all of the networks in a generation, with the total score they earned deciding how much they impact the baseline for the next generation. This would more closely mirror genetics since a single mutation never becomes homogenous throughout an entire species over one generation and by creating composites instead of deciding one victor you allow for it to learn from all of the "failed" attempts slightly.)
On a side note, is there any specific reason you choose the network layout that you did? Like how many nodes in each collumn?
A neuron in a neural network is quite simple. It is just multiplying the elements in two lists (the input and the weights) to get a number, then adding an extra number (bias) to get the linear output, and finally applying a nonlinear function to the linear output such as tanh or sigmoid or relu to get the neuron output. I wouldn't really call it neural, it's mostly multiplying numbers together.
Great explanation! Would love to see more of what this MC AI is able to do in the future
This is awesome, are you going to make a tutorial?
I've been trying to achieve a neural network in Minecraft, but I haven't had as much success as you lol
He has the code on git
This is absolutely a delight to see. I'm surprised at all the criticism and "unsolicited advice" people are offering. Like, let them make their own ANN's. Roko's Basilisk would be proud, sir! **secret handshake** (haha)
Best approach would be trim the time down to train the ai by coding on already learnt ethics in real life by us and then training forward. Like In real world, jumping up and down or standing in one place for long time without any reason would be considered unhealthy, waste of time in AI world. So just as you coded not to dig not more than 2 blocks down, similarly code few things not to do more than set amount of time, but also allows the AI to bypass such limitations if priority meets the needs.
your explanation of neural networking is probably the first that I actually understand
This is gonna blow up soon
ikr we need more
We all started somewhere, and it is even true that bots also start first by digging straight down.
How does this not have more views?
"My computer taught itself to play minecraft"
*That's how it all started-*
I would love for you to continue this project a bit to be honest, or maybe if someone can wrote the code in python so i can make do some work with the project. It would be cool to bot only watch the AI mine but also to see it build and even craft, as well as for it to learn to obtain needs for survival and other complex game play features.
Good job, I think the maps should change every few epochs (or every epoch) so that they don't just memorise the locations of where the ores are, and therefore become smarter.
You should make the rest of the footage available on patreon or you could have a neural network course describing how to replicate this
crazy stuff dude, great job
I love how the bots first instincts are to dig straight down
why does this guy only have like 10k subs, i think he deserves so much more
It will be cool if we forced Ai's to play together and communicate and forming nations
Wow. This quality of content from a channel with only 2.5k...
Good Job. I'm just amazed
Now turn this into a mod, and villagers will finally be useful, they will build their civilization from stone age to modern age, with them working of course, not just giving them materials
This is my type of mod too, so I don't feel lonely playing Minecraft lol
Perhaps in a decade we’ll be able to run a mod like that. That neural network trainer is simulating a world where the AI is the only decision maker, i.e. no mobs, no players, no other AIs. If you want neural network villagers that aren’t crap you’ll have to train them significantly in the actual game with actual players or else they’re going to act stupid anytime players get close to them. Running neural networks isn’t cheap either, and given the complexity of Minecraft and human behavior when interacting with Minecraft, these neural networks may be far too small to produce interesting behavior noticeably different from current villagers. So you may end up needing the latest gen tech just to load even a small village, and given that neural networks run the best on GPUs, your framerate is likely to struggle as it tries to render and think for a dozen AIs simultaneously. Honestly that’s probably a bit optimistic even. Give it a decade though and I wouldn’t be surprised if it becomes a reasonable thing to add.
@@Treviisolion an alpha go style neura network for this would be interesting. It watched many many professional games and learnt how to play intuitively by watching how professionals make their decisions
@@olliejobson6371 Possibly, though go is a much simpler game than Minecraft. There's only one goal, only one other player, and a limited number of actions, and it still took a ridiculous amount of processing power to match and finally beat humans consistently at it. My money would be on whatever the team that made an AI to escape spawn in 2b2t did, where you build an AI to do what you want combined with a bit neural network learning rather than brute force learning everything.
I'm not sure if Tango released his villager mod
props to you for codding the ai in cpp instead of python
neural network teaches itself to break the first rule of minecraft
then gets changed and becomes depressed when it does
Will definitly look forwards to the continuation of this series
what if you have the AI watch like 1000s of "minecraft first night survival" or speedruns so it can map some common and rewardful actions?
That is way easier said than done.
This video was super intresting and very well made! subbed!
It's called baritone.
Great job there ! Recode an entire tool for NN etc is huge !
Loved the content, if you need servers for testing I can provide, we need this to grow
I like how by the end the network is just solidifying what we already know, strip mining works
Using xray instead of spectator mode is really like slapping mojang
spec doesnt show ores tho
I am going to school for software engineering and I hope to do AI classes because this stuff is just phenomenal. The visualizations you had in the video were very well made too. Seeing you use the same IDE I use to code made me really hyped because I could see myself doing projects like this myself sometime in the future. Thanks for the inspiring video Poppro :)
Actually, the seconde set of reinforcement learning with your rule of never digging straight down was not the right thing to do. Your ai learned that digging straight down was the most efficient way to win according to the map you gave it. Implementing an artificial rule to stop it from using this strategy is counter productive, because now you are partially forcing it to comply to how you think the game should be played.
Some, in the comment section, have pointed out how you could have had better result without this new rule (which created some new strange and counter productive behavior in your ai). First, players do not dig straight down due to fall damage, lava and cave full of mobs. So you could have implemented a simple death mechanism, when it falls and die 0 points, when it goes into lava 0 points and when it dies from mobs 0 points.
Second, depending on where you position your player on the same map, digging straight down to diamond level could be the optimal strategy, sometime (like in the 1st run) there is no lava or caves in between the two Y position. therefore you could have placed randomly your player on the same map and let it learn the different strategies or you could have changed the map every other epochs.
Still, a great video showcasing C++ ai programming and java minecraft mod programming!
Or even, subtract say, 2 points for taking damage, falling into lava, even dying. With that detriment to its scoring, it should figure out to avoid those, or learn to minimize it after a while. Definitely don't punish it for digging straight down
Remembering my Armor Stand KI making out of Skript. Could walk anywhere and jump one block up to upstairs. I loved this one.
Are you going to continue the series?
From a newcomer standpoint, your work is truly impressive!
Just use spectator mode instead of digging with xray
with spectator he cant see the ores they are going for?
@@no_name_idea7386 yes you can if you don’t look inside blocks
This is really cool, I would love to see this continued
wish they taught me this in school but no i gotta learn about f@*king minerals
The ,, why did it cost this much? does it play games itself'' has the whole new meaning
we need 2nd part!
I think you should have rewarded the bot for doing things in order(like breaking wood, crafting it into planks, and so on and so forth)
No man would dig straight down. You just made basically Baritone-lite.
One day the ai will speed run and play minecraft all by itself from setting a random seed and making a name for the world, getting diamonds and netherite, starting a server for other ai models and showing off builds it has made and even become a RUclipsr making minecraft videos.
My 7 years old son will love this XD
thank you
As others have pointed out, there are a few problems with the way you went about training. for the world, perhaps having a selection of 8 or more different maps, one of which being chosen at random for each epoch. this teaches the neural net not the specific world they're in, but rather how the blocks that surround them should influence the decisions they make.
Another problem I noticed was that many times the network would decide to just stand still, or just look around. perhaps you could put in place a punishment for standing around and doing nothing for too many decision cycles (be lenient so they have time to 'see' the world), with a sort of forgiveness towards that punishment for turning to look around, but capping how many times that forgiveness works before actually making a movement/action. making the punishment occur quicker after each movement that is not a successful mining action.
Imagine being in a server and you find a bot and then the bot creeply stare at you and it's head following you in a full 360 degrees
0:43 When diamond was the last one, I was so sure it would not understand how to use a nether portal because they did not mention ancient debris
0:43 Emerald and Redstone: Am I a joke to you?
Really appreciate people making great content even with so few subscribers
This is slowly getting into the recommended pages