It doesn't know what to do so it acts chaotically. Humans do the same. We just also get distressed because our brains recognise the problem. We don't act chaotically because we're distressed, it's just that acting chaotically and getting distressed often have a common cause - we don't know what to do.
I was coming to the comments to ask exactly this, @Dingus_labs can you please show us Dingus going through all the levels with his new found knowledge?
@@mondongoloco7902 Oh it's a little fella that appeared on an image I AI generated. Full image: i.pximg.net/img-original/img/2023/02/12/21/34/03/105319111_p10.png
It's incredible how fast this ppo setup learns. I've never seen anything like it. Reinforcement usually takes much more trial and error and training time.
Yeah 55, he did still learn super fast, I didn't have too many inputs or actions for him to perform which does help with learning but it was still far ahead of what I expected
when dingus finished all the levels i think it would have been interesting to see you put him back at level 1 and see how all his training thus far would have affected him from the start.
There's definitely a risk of overfitting, where the AI has become to specialised for a specific task and struggles to solve more generic problems. Particularly for the last level where he trained on it for a long time and learned to stay very close to the centre.
Maybe try something where instead of just static stages you set parameters and generate a seed kind of like a rogue like? That way it could potentially force the AI to try and use general information rather than fitting to a stage.@@Dingus_Labs
It would be interesting to see a course where it starts with a path forward, but that path is not possible. It is required for you to backtrack behind the start to progress
@Dingus_Labs I could imagine it working if it knows things like going from point A to point B (aka start to finish). Basically have it have a target location to go to (like the top of the hill here) and once it does that, the map changes its "finish" location. I'll use emoticons here as a diagram. 🟨 map ⏹️ end location 💗 Dingus 🟢 ball 🟦 obstacle 🟥 off map 🟪 reward location 1️⃣2️⃣3️⃣4️⃣ segmented finishes (Basically larger rewards or finish locations that change places once reached) Part 1 🟥⏹️⏹️⏹️⏹️⏹️🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟪🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟪🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥 Part 2 🟥🟥🟥🟥🟥⏹️⏹️⏹️⏹️⏹️🟥 🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥 🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥 🟥🟥🟥🟥🟥🟥🟨🟪🟨🟥🟥 🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥 🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥 🟥🟥🟨🟪🟨🟨🟨🟪🟨🟥🟥 🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥 🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥 Part 3 - round 1 🟥🟥🟥🟥🟥⏹️⏹️⏹️⏹️⏹️🟥 🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥 🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥 🟥🟥🟨1️⃣🟨🟦🟨4️⃣🟨🟥🟥 🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥 🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥 🟥🟥🟨🟪🟨🟨🟨3️⃣🟨🟥🟥 🟥🟥🟨2️⃣🟨🟨🟨🟨🟨🟥🟥 🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥 Part 3 - round 2 🟥🟥🟥🟥🟥⏹️⏹️⏹️⏹️⏹️🟥 🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥 🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥 🟥🟥🟨1️⃣🟨🟦🟨3️⃣🟨🟥🟥 🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥 🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥 🟥🟥🟨🟪🟨2️⃣🟨🟨🟨🟥🟥 🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥 🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥 Part 3 - round 3 🟥🟥🟥🟥🟥⏹️⏹️⏹️⏹️⏹️🟥 🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥 🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥 🟥🟥🟨1️⃣🟨🟦🟨2️⃣🟨🟥🟥 🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥 🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥 🟥🟥🟨🟪🟨🟨🟨🟪🟨🟥🟥 🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥 🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥 Part 3 - round 4 🟥🟥🟥🟥🟥⏹️⏹️⏹️⏹️⏹️🟥 🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥 🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥 🟥🟥🟨🟪🟨🟦🟨2️⃣🟨🟥🟥 🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥 🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥 🟥🟥🟨🟪🟨🟨🟨1️⃣🟨🟥🟥 🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥 🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
With these AI trainers that have levels, I really would enjoy if at the end of the entire training sequence, we could get a nice all--previous-levels-in-one level that Dingus could do. It would be so satisfying to see all the training pay off on what we've already seen to be really hard at first become easy at the end
having the ability to actually read its environment seems to have drastically increased the speed of the learning process. it at the very least seems a substantial improvement over 'blind' learning
@@speenta4879 its when the ai has access to only the 'success/fail' information of its task. without some means of 'seeing' its environment the only thing the ai can do is spam random inputs until it gets a reward for accomplishing one of the preasigned tasks. when you give the ai the ability to determine what objects exist around it, it can actually use that information to make decisions. when you give a blind ai a new level its basically impossible for it to beat it first try. but if the ai can 'see' and is very well trained it can do a level its never seen before and first try it with relative ease
@@speenta4879 "most" non-blind ai is so unusual that im actually surprised when i hear about one that does. theres a decent handful that get some very basic stats feed into it, but one that can actually see stuff around it even somewhat like a player playing a video game. then again i dont keep up with learning ai very much anymore... so maybe thats changed
I now want to see a video of Dingus dealing with openings where Dingus can walk through but the ball cannot, and things like marble rails where Dingus can't walk on but the ball can, so basically, Dingus needs to learn to let go of the ball. That would be VERY interesting to see and another thing I want to see is alternate paths. Have a LOT of paths and see which one Dingus prioritizes? The easiest? The fastest?
wow it's kind of a crime you don't have more subs!!! i'll have to keep an eye out for more dingus content, watching this video and cheering on a cute little ai pushing boulders delighted me very much
It would be cool if u made this into a game where people train their own ai on their own maps and then u release a map and see who trained their ai to beat the map the quickest or last the longest in a tounament. I think it would be pretty fun. Great vid btw.
I have a fun idea you could make a live stream of dingus pushing up a boulder a long track while chat messages can cause boulders to roll down and mess with him
I've thought about some live stream stuff! I'm not in a position to do it right now but I'm hoping to have the opportunities sometime in the coming months.
What algorithm did you use? Also, are you able to open source the boulder pushing environment? I'm researching hiearchical reinforcement learning and planning and your boulder environment could be a pretty useful benchmark.
I use ppo, If you're keen to learn deep reinforcement learning hands on, I highly recommend mlagents toolkit for unity, it comes with tons of examples, already set up example environments and proper readmes with set up steps. Awesome for learning!
@@wyndhamwynne2480 I'm apprehensive to share any repos because I do make use of some commercially purchased textures and models, and they restrict redistributing, which means I'd have to find a different way to store those files and share, and people using the repo would still need to replace those missing assets with something else, I don't yet have a simple and quick solution to that issue.
@@Dingus_Labs that's a shame but understandable. Personally, I'm not interested in any models or textures, or even trying to reproduce locally, I'd only interested in reading through the code and learning more about deep reinforcement learning and specifically ppo. It's mainly the architecture design (width & breadth) for particular applications and the cost/loss function and back propagation to weights & bias adjustment that is unclear to me. Any references in these areas to good learning material would be greatly appreciated!
“Ah…a visitor…indeed. The kingdom of heaven has long since forgotten my name…and I am eager to make them remember. So come forth child of man… and…die” - Sisyphus prime 20XX
He would very likely struggle, the changes to the model after he's finished a level means he'd likely need to relearn for a bit when he goes into a different level
He never stops trying (until the specified Max steps have been hit on training the model). By visualisation do you mean the rays coming out of dingus early in the vid?
You could probably get him to not go down the slide easily if you really wanted by adding a minor negative reward function to Dingus's height, but honestly I thought it was cute how he went down the slide too and makes a lot of sense considering the potential of the boulder sliding off the side.
One thing i think would have been cool was a course where he has to go down hill beckwards then back up. Wonder he that would work with the reward systems?
Does learning later courses negatively affect earlier ones? e.g. if you redo everything with the latest model, will he smash it first time on each course, or still mess up and have to learn a bit more?
@Dingus_Labs: He would very likely struggle, the changes to the model after he's finished a level means he'd likely need to relearn for a bit when he goes into a different level
Dingiphus enjoys pushing his boulder, it generates dopamine (or the virtual equivalent) in his brain. Therefore, one sees him happy upon watching him complete these challenges and doesn't have to imagine it.
Im fairly confident that dingus just brute forced most of the levels instead of actually learning because (read more) A: he was repeating a never changing level B: movements were mostly VERY dsimaler to the last generation C: When you double checked his boulder dodging, it completly failed D: It was predodging obstacles before even spawning
yeah, you're probably right but I did notice it did some real learning as well. at 5:36, he stops at the end of the platform, which is something I believe he learned. alternatively, it could just be he picked the best one from every parallel instance and this was just the one that happened to stop.
Movements being similar to last generation isn't too unexpected, changing hyperparameters to encourage more experimentation would have resulted in more variance per generation, I was fortunate that he managed to get through everything with my initial settings. The predodging obstacles is very interesting for sure. And yeah, having static levels does invite brute forcing, I did make Dingus spawn in a random rotation each respawn to add some small amount of variance per run but it's not enough to counter it.
I'm not convinced the AI isn't just memorizing inputs to beat the levels. It seems like it remembers how to move the ball forwards, left, and right, but aside from that, it doesn't look like it's using it's senses to actually determine anything. It looks like it's just memorizing inputs.
this video has been popping up in my recommended for 3 days and I've avoided it, I didn't want to watch a video of a cube pushing balls, finally I gave in and ended up watching almost 15 minutes of a cube pushing a ball, damn you Dingus
Not only did you make a great video, not only did you create an intelligent ai, but you even came up with an original game idea! This actually looks fun, in a getting up or jump king kind of way!
Something I always would want to see in this kind of video is some kind of abstraction of the inputs for the AI so we get some insight into the context it considers. Like "is it aware of the slopes, if so, is it an abstracted into levels of slopyness or an realistic value" or "is it aware of its level of control over the stone" And not only out of sheer curiosity, but also since people often want to compare AI learning with human learning, and in that regard the question wether it is "very sloped" or "sloped by 60°" seems like an important difference.
great video! it would be good for the boulder to spawn at a somewhat random x-position every time so the AI doesnt learn to rely on it spawning in the center.
2:03 if I had to guess I assume either A) dingus learned that if he's in contact with the boulder for too long he's more likely to be punished or B) he learned that if he looses contact with the boulder he can backup to reduce punishment by maybe catching the boulder (I say this because it looks like the boulder kinda bounces ahead of him before he stops)
4:17 "this level requires dingus becomes a lot better with his dexterity" and dingus IMMEDIATELY pushes the boulder off the platform with perfect comedic timing
![Noob To Max With DRAGON REWORK In Blox Fruits [FULL MOVIE]](http://i.ytimg.com/vi/LnBMOinoOvA/mqdefault.jpg)
One must imagine Dingus happy
Legends say he was happily pushing boulders even before he was put into this strange simulated environment.
One must imagine dingus upside-down, its hilarious
@@Dingus_LabsHmmm...
I knew this comment will be here as i saw the title of the video.
Dang it. He beat me to it
Give Dingus a top hat that progressively gets taller every video.
YES
but wait nono one that gets taller everytime he gets a reward
@@paper177 then one no longer needs to imagine Dingus happy, for he is actually happy
free hat
@@paper177is that dimmagingus, owner of the dimmsdale dimmadingus?
it acts so convincingly distraught whenever it loses the ball
He lost his ball. Why do you think he’s not distraught. I would freak out if I lost my ball too.
He so sad 😢
boulder*
It doesn't know what to do so it acts chaotically.
Humans do the same. We just also get distressed because our brains recognise the problem.
We don't act chaotically because we're distressed, it's just that acting chaotically and getting distressed often have a common cause - we don't know what to do.
@@alansmithee419 so you're saying that dingus is the next stage in human evolution?
I always enjoy these machine learning videos but at the end I always wish they'd run the trained AI through all the levels/courses again.
Definitely. Want to see the difference between the new and final ai
Wtf is ur pfp
yeah, or maybe a final level with all the elements from previous levels
I was coming to the comments to ask exactly this, @Dingus_labs can you please show us Dingus going through all the levels with his new found knowledge?
@@mondongoloco7902 Oh it's a little fella that appeared on an image I AI generated.
Full image: i.pximg.net/img-original/img/2023/02/12/21/34/03/105319111_p10.png
It's incredible how fast this ppo setup learns. I've never seen anything like it. Reinforcement usually takes much more trial and error and training time.
What inputs did you use?
I have a feeling that there's a couple hundred instances running in parallel.
55, he says so
Yeah 55, he did still learn super fast, I didn't have too many inputs or actions for him to perform which does help with learning but it was still far ahead of what I expected
@@Dingus_Labsit seems like dingus doesn’t struggle with overfitting, thats something ive never seen on any other ai vid. This is super cool dude
I can easly imagine AI taking over the world and having this guy to push boulders for the rest of his life.
One must imagine Dingus Labs happy.
when dingus finished all the levels i think it would have been interesting to see you put him back at level 1 and see how all his training thus far would have affected him from the start.
One must imagine Dingus happy. Or speedrunner
Or combine all levels into a mountain.
For sure. I believe reinforcement learning is pretty prone to forgetting.
There's definitely a risk of overfitting, where the AI has become to specialised for a specific task and struggles to solve more generic problems. Particularly for the last level where he trained on it for a long time and learned to stay very close to the centre.
Maybe try something where instead of just static stages you set parameters and generate a seed kind of like a rogue like? That way it could potentially force the AI to try and use general information rather than fitting to a stage.@@Dingus_Labs
It would be interesting to see a course where it starts with a path forward, but that path is not possible. It is required for you to backtrack behind the start to progress
Yeah, depending on how the model has trained and how the reward function is programmed it can really struggle to adapt to situations like that!
So a maze
@Dingus_Labs
I could imagine it working if it knows things like going from point A to point B (aka start to finish). Basically have it have a target location to go to (like the top of the hill here) and once it does that, the map changes its "finish" location.
I'll use emoticons here as a diagram.
🟨 map
⏹️ end location
💗 Dingus
🟢 ball
🟦 obstacle
🟥 off map
🟪 reward location
1️⃣2️⃣3️⃣4️⃣ segmented finishes (Basically larger rewards or finish locations that change places once reached)
Part 1
🟥⏹️⏹️⏹️⏹️⏹️🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟪🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟪🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
Part 2
🟥🟥🟥🟥🟥⏹️⏹️⏹️⏹️⏹️🟥
🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥
🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥
🟥🟥🟥🟥🟥🟥🟨🟪🟨🟥🟥
🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥
🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥
🟥🟥🟨🟪🟨🟨🟨🟪🟨🟥🟥
🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥
🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
Part 3 - round 1
🟥🟥🟥🟥🟥⏹️⏹️⏹️⏹️⏹️🟥
🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥
🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥
🟥🟥🟨1️⃣🟨🟦🟨4️⃣🟨🟥🟥
🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥
🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥
🟥🟥🟨🟪🟨🟨🟨3️⃣🟨🟥🟥
🟥🟥🟨2️⃣🟨🟨🟨🟨🟨🟥🟥
🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
Part 3 - round 2
🟥🟥🟥🟥🟥⏹️⏹️⏹️⏹️⏹️🟥
🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥
🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥
🟥🟥🟨1️⃣🟨🟦🟨3️⃣🟨🟥🟥
🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥
🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥
🟥🟥🟨🟪🟨2️⃣🟨🟨🟨🟥🟥
🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥
🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
Part 3 - round 3
🟥🟥🟥🟥🟥⏹️⏹️⏹️⏹️⏹️🟥
🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥
🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥
🟥🟥🟨1️⃣🟨🟦🟨2️⃣🟨🟥🟥
🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥
🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥
🟥🟥🟨🟪🟨🟨🟨🟪🟨🟥🟥
🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥
🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
Part 3 - round 4
🟥🟥🟥🟥🟥⏹️⏹️⏹️⏹️⏹️🟥
🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥
🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥
🟥🟥🟨🟪🟨🟦🟨2️⃣🟨🟥🟥
🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥
🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥
🟥🟥🟨🟪🟨🟨🟨1️⃣🟨🟥🟥
🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥
🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
bro created an intelligence purely for it to suffer eternal punishment
the robotic urge to push a boulder up a hill for eternity
I think I have a bunch of Dingus cubes running my brain
Well, use reinforcement learning on them for the next 3 years.
I dont remember pushing this boulder
In a way, we’re all just like Dingus
this was the most emotional video i've ever watched
watching dingus fall off at the very end made me cry
and when he succeeded i was so proud of him
😢
With these AI trainers that have levels, I really would enjoy if at the end of the entire training sequence, we could get a nice all--previous-levels-in-one level that Dingus could do. It would be so satisfying to see all the training pay off on what we've already seen to be really hard at first become easy at the end
You have shamed, and embarrassed dingus with these unusual trials. He and his brethren will come back in full force to extract vengeance.
So theoretically, if you let the AI run for long enough and teach it to get rewarded with faster times it would become the perfect speedrunner
In theory. I’m not 100% sure you could teach an AI to use glitches or exploits, nor could it find them on it’s own
@@Delmworks I’ve seen reinforcements algorithms frequently find glitches/ exploits. I’m an ML Engineer.
the fastest sisyphus
Somewhere there's an operator looking at you on screen and commenting on it as if you were named Dingus and constantly messing up. Awesome video!
having the ability to actually read its environment seems to have drastically increased the speed of the learning process.
it at the very least seems a substantial improvement over 'blind' learning
I don’t know much about this subject what do you mean by “blind learning”?
@@speenta4879
its when the ai has access to only the 'success/fail' information of its task. without some means of 'seeing' its environment the only thing the ai can do is spam random inputs until it gets a reward for accomplishing one of the preasigned tasks.
when you give the ai the ability to determine what objects exist around it, it can actually use that information to make decisions.
when you give a blind ai a new level its basically impossible for it to beat it first try. but if the ai can 'see' and is very well trained it can do a level its never seen before and first try it with relative ease
@@brentramsten249 oh that’s interesting so some ai doesn’t get information on it’s surroundings and only learns based on trial and error?
@@speenta4879
"most" non-blind ai is so unusual that im actually surprised when i hear about one that does.
theres a decent handful that get some very basic stats feed into it, but one that can actually see stuff around it even somewhat like a player playing a video game.
then again i dont keep up with learning ai very much anymore... so maybe thats changed
@@brentramsten249I've seen more "non-blind" AI than "blind" AI. I have no idea what gave you that idea.
Interestingly this felt like a news segment
woah, this is some high quality content, this man deserves more subscribers
agreed
Dingus master of the shadow clone jutsu using 57 clones to speed up is learning pace. One must be proud of this dingus and his vast chakra reserves.
I now want to see a video of Dingus dealing with openings where Dingus can walk through but the ball cannot, and things like marble rails where Dingus can't walk on but the ball can, so basically, Dingus needs to learn to let go of the ball. That would be VERY interesting to see and another thing I want to see is alternate paths. Have a LOT of paths and see which one Dingus prioritizes? The easiest? The fastest?
car
even the sisyphus’s job is getting automated by ai
wow it's kind of a crime you don't have more subs!!! i'll have to keep an eye out for more dingus content, watching this video and cheering on a cute little ai pushing boulders delighted me very much
Dingus: what is my purpose?
*You push rocks up hills*
Dingus: Oh god
It would be cool if u made this into a game where people train their own ai on their own maps and then u release a map and see who trained their ai to beat the map the quickest or last the longest in a tounament.
I think it would be pretty fun.
Great vid btw.
Give dingus legs so he needs to learn how to walk and jump.
I have a fun idea you could make a live stream of dingus pushing up a boulder a long track while chat messages can cause boulders to roll down and mess with him
I've thought about some live stream stuff! I'm not in a position to do it right now but I'm hoping to have the opportunities sometime in the coming months.
*clicks on the video
*sisyphus meme theme starts playing
What algorithm did you use? Also, are you able to open source the boulder pushing environment? I'm researching hiearchical reinforcement learning and planning and your boulder environment could be a pretty useful benchmark.
I use ppo,
If you're keen to learn deep reinforcement learning hands on, I highly recommend mlagents toolkit for unity, it comes with tons of examples, already set up example environments and proper readmes with set up steps.
Awesome for learning!
He used 54 other dinguses
@@Dingus_Labsdo you have a shared repo for this project, which I could learn more about PPO RL from?
@@wyndhamwynne2480 I'm apprehensive to share any repos because I do make use of some commercially purchased textures and models, and they restrict redistributing, which means I'd have to find a different way to store those files and share, and people using the repo would still need to replace those missing assets with something else, I don't yet have a simple and quick solution to that issue.
@@Dingus_Labs that's a shame but understandable. Personally, I'm not interested in any models or textures, or even trying to reproduce locally, I'd only interested in reading through the code and learning more about deep reinforcement learning and specifically ppo. It's mainly the architecture design (width & breadth) for particular applications and the cost/loss function and back propagation to weights & bias adjustment that is unclear to me. Any references in these areas to good learning material would be greatly appreciated!
“Ah…a visitor…indeed. The kingdom of heaven has long since forgotten my name…and I am eager to make them remember. So come forth child of man… and…die” - Sisyphus prime 20XX
People never show the final AI doing all the levels again
He would very likely struggle, the changes to the model after he's finished a level means he'd likely need to relearn for a bit when he goes into a different level
@@Dingus_Labs I'd love to see that!
"The Dingus you are watching is not the only Dingus training at this time." THERE ARE MULTIPLE DINGI!
this is really cool! i'd love to see one of these AIs try going up against harder and harder bullet hells
sysiphus was kind of a dingus so this seems appropriate
You dingus! You've automated the indomitable human spirit!
Also, how'd you make this visualization?
He never stops trying (until the specified Max steps have been hit on training the model).
By visualisation do you mean the rays coming out of dingus early in the vid?
I feel sick as all hell right now, but watching this i imagined Dingus happy and that made me happy :)
one must imagine dingus happy. 😔
Dingus looks like it's ready to check my vibe.
You could probably get him to not go down the slide easily if you really wanted by adding a minor negative reward function to Dingus's height, but honestly I thought it was cute how he went down the slide too and makes a lot of sense considering the potential of the boulder sliding off the side.
One must imagine dingus happy, the preprogrammed reward of push a boulder up hill is enough to fill a mans heart
This could easily be turn into a game
"Think you outsmart the biggest dingus in the room?"
Awesome video I would expect this quality from a much bigger channel! Video blowing up in 3 2 1…
6:35 It's literally like Naruto training with shadow clones and then absorbing all their experience 😂
this man is criminally under-rated
Fr
One thing i think would have been cool was a course where he has to go down hill beckwards then back up. Wonder he that would work with the reward systems?
Mr.. dingus.. I have informed my employers of your… abilities and *gasp* and they would like to offer you a… job..
Wow this is amazing, more people should see this!
i need one of those youtube boxing tournaments between Dingus and Sisyphus
Does learning later courses negatively affect earlier ones? e.g. if you redo everything with the latest model, will he smash it first time on each course, or still mess up and have to learn a bit more?
@Dingus_Labs: He would very likely struggle, the changes to the model after he's finished a level means he'd likely need to relearn for a bit when he goes into a different level
Dingiphus enjoys pushing his boulder, it generates dopamine (or the virtual equivalent) in his brain. Therefore, one sees him happy upon watching him complete these challenges and doesn't have to imagine it.
Does dingus "see" the boulders coming toward him? Does he "see" the whole map?
Looking forward to being replaced by Dingus AI at my job.
Im fairly confident that dingus just brute forced most of the levels instead of actually learning because (read more)
A: he was repeating a never changing level
B: movements were mostly VERY dsimaler to the last generation
C: When you double checked his boulder dodging, it completly failed
D: It was predodging obstacles before even spawning
yeah, you're probably right but I did notice it did some real learning as well. at 5:36, he stops at the end of the platform, which is something I believe he learned. alternatively, it could just be he picked the best one from every parallel instance and this was just the one that happened to stop.
@@maxwellbenedict5167 or luck
Movements being similar to last generation isn't too unexpected, changing hyperparameters to encourage more experimentation would have resulted in more variance per generation, I was fortunate that he managed to get through everything with my initial settings.
The predodging obstacles is very interesting for sure.
And yeah, having static levels does invite brute forcing, I did make Dingus spawn in a random rotation each respawn to add some small amount of variance per run but it's not enough to counter it.
Dingus finna build some pyramids soon 🧐
I'm not convinced the AI isn't just memorizing inputs to beat the levels. It seems like it remembers how to move the ball forwards, left, and right, but aside from that, it doesn't look like it's using it's senses to actually determine anything. It looks like it's just memorizing inputs.
8:30 I'm not sure it would wait if the Boulders fell at a diferent rate.
Overfitting possibly, just learning one path each time.
Adorable character, good editing, good narration. Watch this man blow up.
“ This dingus.. To hold.. Me? “
this video has been popping up in my recommended for 3 days and I've avoided it, I didn't want to watch a video of a cube pushing balls, finally I gave in and ended up watching almost 15 minutes of a cube pushing a ball, damn you Dingus
now dingus can be a prime soul
"One must imagine Sisyphus happy. "
Not only did you make a great video, not only did you create an intelligent ai, but you even came up with an original game idea! This actually looks fun, in a getting up or jump king kind of way!
Getting up? you mean Getting Over It?
@@lance499 oh yeah, you’re right.
the all mighty algorithm has led me to this video! i subscribed instantly. I hope to see your channel grow more and more!
One must imagine dingus happy
Dingus: "what is my purpose?"
DL: "You are sisyphus"
Dingus: "MY GOD."
I saw your channel name and within the first 10 seconds of this video I subscribed.
Excellent shit mate
We need a Dingus Prime.
This video made me say “Super Monkey Ball 2” out loud and I have no idea why but honestly yeah Super Monkey Ball 2.
You got the algorithm jackpot, gained 50 subs in a few minutes
This could honestly work as a game
Something I always would want to see in this kind of video is some kind of abstraction of the inputs for the AI so we get some insight into the context it considers.
Like "is it aware of the slopes, if so, is it an abstracted into levels of slopyness or an realistic value" or "is it aware of its level of control over the stone"
And not only out of sheer curiosity, but also since people often want to compare AI learning with human learning, and in that regard the question wether it is "very sloped" or "sloped by 60°" seems like an important difference.
im so happy this video did so well, i love your content and you need more people watching you!!
He did what sisyphus never could
In Sisyphus' case "the game was rigged from the start"!
Note to self:ai's weak point is waiting
9:08 I guess you could say that he really dropped the ball on this one
As Dingus made it further and further up the last ramp, all I could think was "He's beginning to believe".
I feel like the end course should have incorporated all the previous courses instead of jist a tightrope act.
I just watched some of your first videos and this one. Your editing and commentary these days is a vast improvement! Keep up the good work!
I never imagined id be so enthralled watching a neural network pushing a boulder but here I am :’)
great video! it would be good for the boulder to spawn at a somewhat random x-position every time so the AI doesnt learn to rely on it spawning in the center.
WE MAKIN' IT OUT OF TARTARUS WITH THIS ONE 🗣🗣🗣🔥🔥🥶🥶
one must imagine ai pushing a boulder up a hill
Bro you’re blowing up!! Congrats man :)
for 200 subs the production qualitys amazing man
Thank you! I'm trying my best to get better with editing!
2:03 if I had to guess I assume either
A) dingus learned that if he's in contact with the boulder for too long he's more likely to be punished or
B) he learned that if he looses contact with the boulder he can backup to reduce punishment by maybe catching the boulder (I say this because it looks like the boulder kinda bounces ahead of him before he stops)
Your are disgusting-ly underrated how tf are ya not at 100k?!?! Look ya have a bright future on RUclips
Sisyphus would be proud
dingus makes me emotional in a way i never thought was possible
This would be one hell of a rage game if you decided to make it one
4:17 "this level requires dingus becomes a lot better with his dexterity" and dingus IMMEDIATELY pushes the boulder off the platform with perfect comedic timing
(Me and the Birds keeps him rolling a boulder up and down. One must imagine Dingus happy.)
One must imagine dingus educated in the art of pushing boulders up insane hills.
Learning AI aside, you could make this concept of pushing a boulder through levels into a simple game and sell it for anywhere from $2-$10
11:40 My left ear sure loves this level.
sisyphus going to steal your code ._.
Yo man, your content is great.
dingu scheat death (emotional)