If you'd like to see more explorations into new technology, subscribe for more! A few things I noticed after editing the video: 1. You can see that as I stand still that everything starts to "melt" and lose a sense of cohesion. 2. There's a sheep that appears at 13:45 which is the only Mob I saw in all of my gameplay sessions. 3. At 04:36 I mention switching inventory slots, and it literally would pick different slots than I requested because it was simulating how normally it scrolls around in the real game. 4. This one threw me for a loop by the way: There's actually no "Cursor" in the game, it's just simulating a cursor based on the mouse movements that it receives. 5. 09:25 ASIC means "Application Specific Integrated Circuit" - Hardware that enables better performance than a GPU in this instance. - Thanks @anto_fire8534 for the clarification. 6. I played later and got bubbles by swimming, but then couldn't get rid of the bubbles, so I was just "swimming" through the sky and caves.
I tried switching to the nether by looking at lava pools in the desert but I was unsuccessful. It morphed into other similar blocks instead found in the overworld, I believe this is because there was no nether content given during the model training
this is a pretty good simulation of having a dream - weird things are happening, everything constantly changes and it stops right before the best moment
I've actually been really interested in this concept because in theory ai is simulated neurons and we see a lot of the same things in ai generated video that happen in dreams where the scene completely changes or objects move, change or disappear all together. Kind of interesting to see.
I will say that even in dreams there is more consistency. You can roughly remember where you were and what you were doing in your dream. But in this AI, everything is constantly changing.
I got a chamce to play this yesterday. To me, the coolest part about this is that its not a game. Theres no game engine, no real inventory, even your hotbar is "fake". This is a video generator, based on your input, and the last frame rendered. Really cool tech
Maybe a mod that tries to generate the world from an image provided by the ai and let's you actually play minecraft but sorta letting the AI model continue to try and figure it out and use that to further generate new blocks.
You look at the time; look again and it is different but you don't know either time. Just if it is forward or back. You push a light swith bit no matter how hard you push it won't move. The person you were with has changed. The light swith is now a pull chord type but still winter pull. You check the time again and it is later but you can't say when. You read your notepad for advice and you k ow what it says bit can't see the words. You loom between the words and see smaller words written in the sides of the words written. The closer you look you notice a pattern. Like afibanarchi sequence bit with interlocking shapes bit with a distinct work but you can't see the word. .you wake up
Would be super complicated if it was made to work any good, constantly replacing/ filling blocks based on real time feedback would probably run absolutely terribly aswell lol, unless you limited it to like 10 blocks under you and only 2 chunks out then maybe
@@a1m.. i think a low render distance would make it even better lmao or even if the chunks reset themselves when you looked away, or your inventory randomizes itself when you close out of it, those would probably be more doable
0:26 I just want to say this is EXACTLY how it works in my lucid dreams. There's a wall, I start picturing a wall, and I can open it and I can suddenly move where I wanted to
That's sort of what it is- all video of Minecraft distilled into a disgustingly huge matrix of numbers which, when fed inputs, produces stuff that's kind of like a simulacrum of Minecraft. What you think of when you think "minecraft" is also sort of a distillation of all the pictures and video you've seen of minecraft.
@@asdf30111 in my playthrough, i remember looking at the sky to teleport somewhere, and then when i looked back at the ground, i couldn't move, and then i jumped because it turned out i was in a block
This is basically exactly what my dreams are like! even down to the point of being kicked out right as you find something cool. The internal inconsistency and small context window is exactly my dreams, even the fuzzy graphics and shifting objects, it is literally just my dreams.
@@Miresgaldir same it’s so weird Even down to the way you can’t really read/write things in dreams because your brain can’t generate the words correctly, it’s so surreal to go lucid and walk up to a sign and notice that it’s just your subconscious trying to nonsensically jumble a word together.
@@Insert_Creativity_Here I love your user name 😂 yes that too! The disorientation. I actually just loaded up the model for myself just then, and the best way of moving around, is just by manipulating the screen to look like a different environment and your instantly teleported - almost like a dream 😊
Does anyone else feel a sense of horror watching this? I think somthing like this could be made into a pretty lit horror vr, trying to run away from *something* but nothing makes sense.
JackBond1234 said they'd love to see a horror game like this being sold as gnerative AI when really it's all hard-coded and gets eerie and fuck-y fast. I get the appeal! And I share your sense of dread aswell: It's so foreboding not knowing where you'll end up just after a few blocks, or the thing in front of you shifting and morhping even in scale and distance right as you are staring at it. That was pretty scary! I thought the end was especially creepy, with a horse(??) morphing into a giant blob morhping into nothing but shade... It reminds me of eldritch horror somehow -- now that has some nice potential!! ^^ :D
Fortunately the endgame for Nvidia is neural rendering and they'll probably have consumer GPUs capable of it in maybe the next 10 years or so. Unfortunately they'll probably cost thousands of dollars.
@@someusername1872always assume super-tech like that is 10 times as far away as they say it is. They’ve been saying “fusion energy in next 10 years” for the fast century
i saw a comment about this elsewhere and it won't leave my head: i think the reason that it's cropped into a square, rather than the usual aspect ratio, is to cut off all the facecams from the training data
I was thinking a lot about this, and idk actually if they trained on public videos. They needed to capture the keyboard and cursor inputs as part of the training data, so I think it's likely the had people/bots playing MC and captured the data that way. It was TONS of data though, so they were probably all virtual machines. Also square is pretty standard for a lot of these test/basic models. They mentioned with their future hardware it could be served in 4k. Just guessing though.
Nah, the reason is because training these models with square inputs that are power of twos in size is much more compute efficient. It is very common to intentionally crop all training data into a square for computer vision models.
@@andybarcia4827 This is most likely to be the reason. Otherwise, if they trained on data like streamer gameplay, there would be things like donations, etc...
his content is interesting, and he is a good host (I just subscribed). I think it just needs a more attention grabbing pacing. like, he only said the name of this tool midway throught but I like the way he presents things. it's so relaxing
@@delta0xAI isn’t a newly invented form of life though, it’s just some really fancy equations that usually spit out something statistically likely to look like Minecraft
Does anyone else remember "deep dream?" That was one of the first generative AI publications Google made where they ran the ML model for image search backwards to debug it. It produced insanely trippy output.
I wouldn't really say fully playable...the items,blocks and stuff in your hotbar just appear and dissapear on a whim,and the inventory and ui really just don't work. Its still kinda playable but not really
@@Damian-cilr2you can play it in real time and without 1 frame per 10 seconds frame rate. This is huge. And resource cost is low, since he is playing it in the browser
@@Cubic_cat Nearly all the work is done on their servers. The resource cost of running AI models is very high. That is why Nvidia turned into a mutli trillion dollar company overnight, cause they were wise enough to invest in CUDA cores over the years.
Playing it made my brain think it was dreaming. Afterwards I kept having this subconscious expectation that websites should be morphing and changing when I'm not paying too much attention, kept feeling low level surprise that nothing was memlting or morphing, and kept having to remind myself that that is normal. Pretty damned trippy.
It feels like a dream after playing MC for hours Edit: bro i just realized that the views aren't past over a million damn Keep it up bro you're so cool 🙏
Minecraft Dementia Simulator edition I think it's kinda lit, ngl I think the solution to this is to inspect the game state and embed it into the model as parameters For example, encode the inventory into a set of "bits" and embed it into the model, in this way there will be a persistent memory of the owned items regardless of the context interval Similarly other information can be embeded too like HP, Hunger bar, XYZ coordinates and world seed (idk if that would make any difference), even if it's embeded as a "bias" that would kinda steer the model one way or another
Right, I've seen pseudo-text RPG games working in SillyTavern. You get the internal logic and variables that are stored in the context and update according to the events.
Maybe in the future, with a long enough context, you could store raw data for the time of day, nearby mobs, dropped items, biomes, blocks, the code that moves the player... oh wait
@@ferociousfeind8538 yeah that's true, but its hard do make dementia simulation in deterministic computing And besides, I think it's a fun idea to "merge" models trained in different games and see what happens For example if you merge Minecraft model with a Vintage Story model Or if you merge a Doom with Halo, what kind of crazy "LSD" simulations would result from this? I don't think this works as a product either, but it's an interesting research project
@@ferociousfeind8538ai bros reinvent stuff that already exists every few weeks but way less efficient, they’ve made trains but bad at least 40 times already
This concept would make for a great horror game. Present it as a generative AI experience, but it's actually manually coded, and have the hallucinations become eerie and seem intentional and dark.
I was just thinking something like this. Like with doors you go thru and then you go back thru but it leads somewhere else. And like intentional datamoshing.
There is no real way you can make something much like this without using AI. Unless every single instance of a hallucination is done manually, but that would be nothing like this given how few there could reasonably be.
@@whwhwhhwhhhwhdldkjdsnsjsks6544 You could do this very easily without AI. Just write a shader to give the fuzzy visual look, and have stuff change when you're not looking at it You can still use code to randomise stuff to billions of permutations
@@caut5618 oh, I remember you, you were put into a coma in 2005 and must've just woke up. You're not gonna believe it, but it's been almost 20 years! there was a stock market crash, and then a recession, and then the US elected its first black president, and then everyone thought the world was gonna end, and then a gorilla got shot or something, and then the US elected a washed up reality star as president, and then there was this big pandemic that a lot of people thought was fake, and then- oh, yeah, and using "gay" as an insult - totally lame now. I know, never would've guessed it!
the thing i found the most interesting is that if you are in a dark place and you right-click the ground with your pickaxe in hand, it will place down a torch. it's like it doesn't know what a torch actually is, but it knows that people usually place torches when it's dark.
This is a really interesting concept but I don't think it could ever be more than a gimmick in an actual video game. I agree with others saying it could work in a horror style game, but the incoherence would have to be balanced with monents of consistency to actually sell the vibe. Maybe if a character is walking through an otherwise fully designed game with good mechanics, etc, and the character has a breakdown or nightmare scene that works like this, sure. But even then, a lot of the compelling elements in something like that are story moments like the void of their dream revealing the character's dead sibling or something. Idk, I just feel like once you're increasing the context window to ridiculous amounts and coding in UI overlays whateberver, at a certain point you might as well just make an actual game, with good gameplay feel, story, and level design. This will mostly just give us a distilled/generic version of games we already have with some trippy haze around it.
no you don’t understand, this is a glimpse into the FUTURE of interactive AI experiences!!! this will totally replace all video games and every game dev will be jobless by 2026!!!🤓🤓🤓
@@dogonit1936 people unironically think that and that's both hilarious and sad - imagine thinking it would be better to offload creativity onto a machine...................................................
@@sc0pe103 you see, if we automate all the creative, interesting, fulfilling and/or fun kinds of work, that will finally give us time to fulfill humanity's true purpose: dying in the lithium mines so the last human can be a Martian quadrillionaire
You articulated my thoughts exactly. This is very cool as a Weird Thing™ you get to check out, but AI generated games have as much artistic value as AI generated images. And for me, it matters a lot if I'm having that "conversation" with a real developer or just with a machine.
Watched a couple of these vids so far. Yours was my favorite because youre genuinley excited about it and go into some of the technical stuff, making it interesting to watch
@@UEatPoo I really appreciate that, I’m more in the tech space today, but used to do let’s plays, so it was just this weird coincidental, crossover event. I’ll definitely have more game related content, but not sure if I’ll get into a habit of gameplay like this again
I was going to say that about LSD. Done it only once but this freaked me out because i had a very bad trip. It’s scary when you’re disoriented like this.
@@TruTrolldier Its astounding how easy it is to forget how disorienting psychedelic experiences are. Playing this pulled up some long forgotten memories.
I like the idea of storing the inventory outside of the model. Maybe they could also code something to keep track of blocks outside your FoV. And maybe also some check to make you fall when you're in the air, kinda like a mini physics engine and... oh, wait...
You’re right though. The simulation part is more interesting for the times when you want to fill in the gaps, rather than the whole game. I’d like to see what a mix would look like.
This opens space for a completely new game genre. The AI generated game, one of which the gameplay is going to be completely new every time it is played. Almost like procedural generated game worlds like No Man's Sky or Minecraft, but maybe with the entire gameplay entirely different every time.
I don't think that this is the future of games due to the [totally unpredictable gameplay], but it's still cool research-wise and it may give people insight into how transformers and large models work, manipulate data, and store information.
16:42 close your inventory and turn around, some mysterious evil dark blob appears behind you, you try to investigate it and then the game crashes this is a real life creepypasta
Damn, I saw something like this in my first "simulation". Something dark was standing 6-10 blocks away from me when I turned around on the last seconds of simulation. But it was probably a morphed darkness in some cave that was pretty far away from me.
@@Аноним-п1ж i tried it myself earlier (at 2 am woo what a good idea) and had my inventory opened for an extended period of time, i finally close it and i'm in a monochromatic hellscape that consists of nothing but lines of two colors, with a single small strip of red on the ground i look down at it and i'm now in ACTUAL hell; the nether
I would describe eldritch horror as being able to know something which you can’t understand but I do still totally see where you’re coming from. Actually half way through writing this I realized seeing something you think you understand then It changing completely could totally be interpreted as that.
what i think of whenever i see projects like this is how much less fun these projects will become the "better" they get. The entire charm of AI generated stuff for me is seeing them devolve into nonsense (like that ai generated beer ad). When it gets better it just becomes a video like any other.
yeah theres a charm to the weirdness i miss with flux that i could get with weird interleaved SD prompts i predict it will be less of an issue with such a directly jnteractive thing though and also when we have weights and code available we can start messing with the inference in fun ways
Bro what are you on this looks like actual garbage. You can’t even play for more than five minutes and everything disappears so you can’t make any progress in the game. Are you actually tweaking rn?
Think of this like the Atari days of video games. square moving on screen, pong and all that. Just like that was “bad” visually but impressive technically, this is similar. This is “bad” visually, but impressive technically. And these demos set the stage for things to come.
@@elijahwallis956 12 year old has to have his input because unfortunately there's no laws regarding children leaving stupid nonsensical comments online. Who let this kid have a computer? This is absolutely mind blowing
It's like a dream. You're going somewhere to get a thing to bring back to someone, but on your journey to get the thing the world changes around you and you have trouble getting what you were looking for and even if you do, getting back to where you were is impossible. A dream is just a long free association with little to no ability to back trace through the "narrative". This AI stuff is really teaching us a lot about ourselves.
No idea if it’s just me but, A.I incoherence just makes me itch, it’s such a disturbing feeling. Things like this, and just A.I generated videos in general make me feel this weird, crawling sensation
I feel that for sure. At the end of the vid there was a big black blob that was disconcerting, I backed up from it at first. It makes me nervous to see what's next sometimes.
9:35 etched is the company they partenered with (I believe). ASIC means application specific integrated circuit. it can allow you better performance than cpus and gpus because its designed specifically to do the specific calculations u want at an hardware level, hence why they could run their model in 4K
You look at the time; look again and it is different but you don't know either time. Just if it is forward or back. You push a light swich but no matter how hard you push, it won't move. The person you were with has changed. The light switch is now a pull chord type but still won't pull. You check the time again, and it is later; but you can't say when. Or if it was before you began. You read your notepad for advice and you know what it says, but you can't see the words. You look between the words and see smaller words written in the sides of the words written. The closer you look you notice a pattern. Like the fibanarchi sequence but with interlocking shapesyet with a distinct real 'word', yet you can't see the word. .you wake up
Wait. Isn't this just how the brain works? Just more connections having a more/less correct version of reality? Like, did y'all know that? The reality you see is a predictive algorithm simulation in the brain, and the brain only uses inputs when it's wrong. It would take the brain an impossible amount of energy to calculate using ALL inputs EVERY SECOND, so the brain evolved to have a more/less simulation of the reality. Like color, for example, their is no such thing as color. It's an evaluationary trait that's only in the brain that we evolved to be able to distinguish stuff around us.
Color being a figment is not the same as the brain being a predictive algorithm like the model in this video. Your brain constantly takes in concrete input and definitely doesn't make up as much baseless nonsense as the AI model
@@ZerosiiniFIN I mean, but like. If the ai had more connections as in the same amount as the brain, would it not be the same? Our brains have like what? Billions of connections or something like that? That's only the simplest explanation of the theory. Being basically a predictive algorithm simulation, or more of a "controlled hallucination". Look up: Why reality is a "Controlled hallucination" Posted by Kyle Hill and then you can probably do more indepth research of such topic
@budderman3rd Humans hallucinating is based on actual, concrete, current input. This predictive AI has been trained on terabytes of data and only uses that and the most recent prediction to form the next one. There is no new information other than the player's inout which is not more than a couple of bytes of data. There are some similarities, but I fail to see how your comparison is accurate.
@@ZerosiiniFIN Our reality is basically only in our brains based on such theory. It's a more complex predictive algorithm, that doesn't even use inputs directly.
Honestly that would be a very interesting game mechanic. Imagine a game that changes in the directions you look. For example you go forward and things are normal, you turn and everything starts to change and barely anything is similar. The twist being walking backwards without looking back keeps the old design of the land loaded. So... aslong as you move straight backwards or forward it stays the same, but once you start turning it all changes.
@jme7046 not necessarily. 4D miner is the same all around, you just simply move through 4d space as the game trys to simulate, nothing is technically changing. You just have way more angles of perspective. But visually... maybe a little bit. I'm talking about completely changing the landscape for the most part, with the only 2 things being the same being things you can see directly and things you cannot see whatsoever. Basically moving through space by turning while potentially trying to figure out a path to a point in said space. For example they could turn and find themselfes with the dot being way below them, so they look for something cave like , look at it closely and turn, causing them to suddenly appear in a cave like environment with the dot suddenly being above them or on the same level. It would be quite the brainteaser, hehe.
For a sandbox game this tech could be super interesting. But I can't imagine it would ever work for any other kind of game that would have a fixed endpoint. Do I have that right?
oh shit for real didn't even think of that lol. But actually probably not possible for a while because there would need to be perfect consistancy between both eye positions. And if there is not it would give that weird effect when you look at two different scenes with both eyes
@@SpicyMelonYTthat is literally the least of the concerns of what it would take to implement this for VR. Please just stop endorsing the creation of AI slop.
@@SpicyMelonYT no, I'm a game developer. And I'm tired of seeing the humanity being stripped away from so many aspects of art, programming, writing, hell even games now. I've worked on VR games and know the inner workings behind them, the technology is simply not there yet.
@@bernatrosello4375 I also make games. And the humanity is not being stripped away because these exist. Its because of the lack of laws keeping human jobs. Humanity is still very much there. We like making art, we like making tech. Humans rock. And whats the point of your last statement? You say it like because its not there yet then whats the point? That same damn argument can be made about EVERY SINGLE CREATION EVER. Hell even a game you work on isn't finished until its finished. But you were already imagining what it would look like finished, what it could do, how it can feel. Its there same thing here dude. And you draw this arbitrary line in the sand because you don't like it. That's bad. You know how many artist hated photoshop way back when? And now those people who use it hate the next this which is AI. Video games are fun, they are creative, they give you experiences, and they make money. None of these thing stop happening with AI nor should they.
It's a little bit like a high dimension game. When your rotating and everything changes it could be seen as bug, because it hallucinates. Or you could see it as feature, where you actually move in the forth dimension and that's why your perspective is completely changing
Imagine if, locked away deep in some generated space, there were a key that would help you progress in the game. Your job would be to hallucinate your way there.
This is really promising for future developments in both AI and video games. A major issue with price to performance in video games and hardware has been that the clock speed of processors (GPUs, CPUs) has stopped increasing. Instead, they’re generally going “wider” nowadays, meaning that not all classical software is able to scale to use it. If you think about many game engines, they can’t necessarily use all the cores in a modern CPU to increase the frame rate, for instance. But AI scales remarkably well. In fact, you don’t even need all the processors to be in a single device. You could imagine pre-computing slow moving components in a remote device, and computing fast latency sensitive elements in the device closest to the display, for instance. Imagine not being locked into, say, a single expensive GPU, and being able to combine two cheaper ones from the used market like we used to be able to do with SLI / Crossfire…Or being able to choose to run on CPU, or choosing to run on an NPU, or whatever else have you. The cool thing is that once you get something like this Minecraft demo, it’s actually not that much more technically demanding to do, say, “realistic” graphics (and I mean actually realistic, not video-gamey realistic), or to do any art style you’d like to do. You could also imagine doing more realistic, larger scale fluid simulations, for instance, or crazy physics simulations that are actually realistic…And for essentially “free” without really much more processing.
Exactly. If we eventually have a model trained on physics, objects, and interactions, it's possible it could become a completely new type of game engine. Similar to how ChatGPT is like the ultimate text AI, this could be the ultimate spacial AI. Not to mention incorporating this with generative video like Sora or the recent Mochi1 by Genmo.
I cant wait for this technology to advance enough that actual playable games can be made that utilize it fully. One concept I can see these hallucinations being perfect for is a sequel to LSD Dream Simulator.
eventually we're going to have a model that can simulate an entire operating system as if you were on it. With websites and working pages and programs lol could be wild. The direction this stuff is going is really wild.
Why would you want that over an actual operating system that takes exponentially less computation to run and is more accurate? Imagine an OS that just hallucinates files lmao
what is the potential use case for this supposed to be? you say you are excited but i dont get whats exciting, i get this is just a prototype and things will improve but towards what? what are we improving towards? how does this become anything?
Good question. I mention in a few other comments, but, things I could see happening from this off the top of my head: - a generate anything you want game (like the ultimate Scribblenauts) - new games that haven't existed before that could only be playable like this (assuming there were more constraints and goals etc) - pause a moment in a movie, and suddenly it's a playable game like GTA or whatever - super performative games by streaming low-res and using a trained model to upres (NVIDIA already does this a bit) - maybe I want to make a mod, I define different settings "low gravity, floating blocks" and it could simulate that/convert the generation into an actual mod. If LLMs are the "text and thought", tools with video/gameplay like this can become the "vision and physics". There are probably some better ones I haven't though of.
@@JarrenRocks hmm these all sound not that great... i spose they were just what you thought of off the top of your head but honestly most of this just looks like giving up control or things we can already do or in the case of the movie idea just horrid. oh well hopefully this will be used for good unlike the llms and other image to text ai stuff that just fills the digital world with slop
@@delta0x Well then that's weird. I know that people with clairvoyance have far more solid and structure dreams than the average. Disordered dreams mean disordered self (not referring to medical term), I think that's what is going on here.
@@luis-sophus-8227 everyone has clairvoyance. sometimes i just dont feel like putting a 100% accurate and realistic world in my brain during a time reserved for being away from physicality. its fantasy hours for the little delta
Wait I’m so confused. It’s obviously generating the visuals, but how are things actually changing in the game as well? Is there a mod that goes with it or something to actually change the world? How?
@ We don’t have that kind of technology yet. Something like that will probably take at least another 5 years. This thing has to be fake. Not faked by you, but like the website being misleading to what’s actually going on here.
@@JoshyRB that’s the fun part, this is a research paper, so it’s highlighting a new technology. It’s not a commercial product - you can expect that in 5 years something will be commercially available.
@@JoshyRB that’s the fun part, this is a research paper, so it’s highlighting a new technology. It’s not a commercial product - you can expect that in 5 years something will be commercially available.
It's basically a research paper, so it's being written towards other people in the research field. I just so happen to read those papers and love playing Minecraft 😅
I think training it on 360 video would help with the environment disappearing by just rotating the camera and during "gameplay" just only display a portion of the frame buffer
I imagine the context window could be way bigger if not only the inventory was hardcoded, but the rendering engine also, because then you wouldn’t have to store visuals inside the model. But it is probably really hard to get training data for such a hybrid set up because while there is a lot of video on RUclips, collecting data for a hybrid model would require recording the game engine instead of the visuals.
It this technology succeds it will take valuable jobs from already struggling developers and artists, if it fails, we're stuck with inconsistent messes of games that nobody will end up playing. Ultimately the tech is set up to fail, but even if it wasn't, this business model is horrible
Even successful it'll never replace commercial products, you can't copyright A.I generated content and it wouldn't be allowed to train on works without permission.
I believe the next step of this technology is kind of, incorporating it into the game engine to give it all the context it needs whenever it needs it. Without using context windows for positions and whatnot, you'd likely need to give things tags to tell the AI what everything in a scene is, and with that information it can make some cool stuff, perhaps like Photorealistic Minecraft or even other games, without all the weird hallucinating. Since the AI wouldn't be using its context windows on figuring out what the scene should look like, more could be used to make the scene more consistent with artist vision and previous shots. Interesting stuff, this AI video generation
@@Dr_Pickle I’m pretty sure it would still be more efficient for the servers because generative AI is a lot easier to run then a hyper realistic game where it actually has to be rendered
If you'd like to see more explorations into new technology, subscribe for more!
A few things I noticed after editing the video:
1. You can see that as I stand still that everything starts to "melt" and lose a sense of cohesion.
2. There's a sheep that appears at 13:45 which is the only Mob I saw in all of my gameplay sessions.
3. At 04:36 I mention switching inventory slots, and it literally would pick different slots than I requested because it was simulating how normally it scrolls around in the real game.
4. This one threw me for a loop by the way: There's actually no "Cursor" in the game, it's just simulating a cursor based on the mouse movements that it receives.
5. 09:25 ASIC means "Application Specific Integrated Circuit" - Hardware that enables better performance than a GPU in this instance. - Thanks @anto_fire8534 for the clarification.
6. I played later and got bubbles by swimming, but then couldn't get rid of the bubbles, so I was just "swimming" through the sky and caves.
forgot to pin??
@@value1lol good catch - pinned
@@JarrenRocks it is possible to punch mobs, after punching a sheep it often turns into a pig.
bro like few years will pass and we wont need new games anymore :(
I tried switching to the nether by looking at lava pools in the desert but I was unsuccessful. It morphed into other similar blocks instead found in the overworld, I believe this is because there was no nether content given during the model training
this is a pretty good simulation of having a dream - weird things are happening, everything constantly changes and it stops right before the best moment
I've actually been really interested in this concept because in theory ai is simulated neurons and we see a lot of the same things in ai generated video that happen in dreams where the scene completely changes or objects move, change or disappear all together. Kind of interesting to see.
I agree, brooo these AI generated content are too similar to dreams....
I will say that even in dreams there is more consistency. You can roughly remember where you were and what you were doing in your dream. But in this AI, everything is constantly changing.
Glad to know that I wasn't only one thinking this!
@@bowser3017we only need a bigger context window for that
I see a new hardcore speedrun ahead, beat minecraft in this, just figuring out what to lead it to hallucinate into the endscreen.
Yes I actually morphed to the nether in one session
probably getting 9 coal, crafting it into a block and pressing against it so your whole screen is black, or maybe obsidian
Black blocks filling your vision while there's any particle falling
So this is what dementia feels like
@@neferkar112 I tried it, got stuck in a cave with no torches lmao
I got a chamce to play this yesterday. To me, the coolest part about this is that its not a game. Theres no game engine, no real inventory, even your hotbar is "fake". This is a video generator, based on your input, and the last frame rendered. Really cool tech
@@brandonz404 spot on.
Assuning unlimited compute power, this literally extrapolates to the matrix which is crazy
Kinda feel that thats less cool tbh
@@t33mtech59 This literally doesn't
Not exactly your last frame. The context window is 3 seconds. If it was only based on the last frame it would be an incoherent mess
The best part is the demo ending right before the world either completely breaks or something interesting or scary happens. It just like a real dream
ayo
The clone wars
It’s just a time limit
Minecraft: schizophrenia edition
The lack of self, the lack of memory is emblematic of the enviroment it was birth for, mindless slavery to corperate godcreators.
That's perfect!
More like playing in your sleep, which this video reminds me of the most.
@@RhizometricRealitywhat
@@RhizometricRealitybrother schizoposting on the schizocraft video
Non-Euclidean gameplay
I think you're being rather hyperbolic
Both great comments :)
@@JarrenRocksI believe its pretty spot on
@@Verymusician193 it's a pun!
anyone seen the non euclidean doom?
I would love a datapack that simulates the whole "turn around and you're in a different place" thing, would make for a really interesting playthrough.
Maybe a mod that tries to generate the world from an image provided by the ai and let's you actually play minecraft but sorta letting the AI model continue to try and figure it out and use that to further generate new blocks.
@@gwentarinokripperinolkjdsf683, Gwent... I miss Gwent...
You look at the time; look again and it is different but you don't know either time. Just if it is forward or back. You push a light swith bit no matter how hard you push it won't move. The person you were with has changed. The light swith is now a pull chord type but still winter pull. You check the time again and it is later but you can't say when. You read your notepad for advice and you k ow what it says bit can't see the words.
You loom between the words and see smaller words written in the sides of the words written. The closer you look you notice a pattern. Like afibanarchi sequence bit with interlocking shapes bit with a distinct work but you can't see the word.
.you wake up
Would be super complicated if it was made to work any good, constantly replacing/ filling blocks based on real time feedback would probably run absolutely terribly aswell lol, unless you limited it to like 10 blocks under you and only 2 chunks out then maybe
@@a1m.. i think a low render distance would make it even better lmao
or even if the chunks reset themselves when you looked away, or your inventory randomizes itself when you close out of it, those would probably be more doable
0:26 I just want to say this is EXACTLY how it works in my lucid dreams. There's a wall, I start picturing a wall, and I can open it and I can suddenly move where I wanted to
I like the cartoon physics.
He looker down, and then he fell
Apparently they used Wile E Coyote to train this AI.
Lobotomy Craft v2: Libet’s Delay
Do we have blue search on yt now huh
We do now, its weird @@reclipsel
@@m_r-ock6508 It disappeared right in front of my eyes and idrk if I'm schizo or not
mincarp never b. dame after mr lobtomy 🤑🤑
@@reclipselsame thing happened to me
This feels like the picture that comes to your brain for a nanoseccond when you think aboutthe word minecraft.
That's sort of what it is- all video of Minecraft distilled into a disgustingly huge matrix of numbers which, when fed inputs, produces stuff that's kind of like a simulacrum of Minecraft.
What you think of when you think "minecraft" is also sort of a distillation of all the pictures and video you've seen of minecraft.
And it does model at a certain level of abstraction resembles how biological neurons work
That is FASCINATING
What a hell of an observation yo
Imagine this in vr. You'd probably go insane after a few minutes.
Let’s be honest, it will be quicker
That would be so freaky, I'm already reluctant to play just the regular version lol
and get motion sickness
it's not 3D, so kinda impossible to make this work in VR
@@MrDoboz I mean just plaster what the game is in vr, you might not be able to have hands with it or something but it will kinda have the same effect
So they basically just remade Minecraft Bedrock
Still has less phasing through blocks then bedrock.
shots fired
@@asdf30111 in my playthrough, i remember looking at the sky to teleport somewhere, and then when i looked back at the ground, i couldn't move, and then i jumped because it turned out i was in a block
11 hour already and no returning fire?
@@mccraftingtablelmao6204 Because its true.
This is basically exactly what my dreams are like! even down to the point of being kicked out right as you find something cool. The internal inconsistency and small context window is exactly my dreams, even the fuzzy graphics and shifting objects, it is literally just my dreams.
I'd imagine it's because a sleeping dreaming brain acts in a similar manner to LLMs.
Gotta pull up on the gateway tapes
I always explain AI to people with the analogy of dreams. It's so uncanny
@@Miresgaldir same it’s so weird
Even down to the way you can’t really read/write things in dreams because your brain can’t generate the words correctly, it’s so surreal to go lucid and walk up to a sign and notice that it’s just your subconscious trying to nonsensically jumble a word together.
@@Insert_Creativity_Here I love your user name 😂 yes that too! The disorientation. I actually just loaded up the model for myself just then, and the best way of moving around, is just by manipulating the screen to look like a different environment and your instantly teleported - almost like a dream 😊
Does anyone else feel a sense of horror watching this? I think somthing like this could be made into a pretty lit horror vr, trying to run away from *something* but nothing makes sense.
JackBond1234 said they'd love to see a horror game like this being sold as gnerative AI when really it's all hard-coded and gets eerie and fuck-y fast. I get the appeal!
And I share your sense of dread aswell: It's so foreboding not knowing where you'll end up just after a few blocks, or the thing in front of you shifting and morhping even in scale and distance right as you are staring at it. That was pretty scary! I thought the end was especially creepy, with a horse(??) morphing into a giant blob morhping into nothing but shade... It reminds me of eldritch horror somehow -- now that has some nice potential!! ^^ :D
Fortunately the endgame for Nvidia is neural rendering and they'll probably have consumer GPUs capable of it in maybe the next 10 years or so. Unfortunately they'll probably cost thousands of dollars.
@@someusername1872 In 10 years? Unless inflation really goes off the rail, it'll probably be ~500 for a decent one that can run it.
Like the backrooms 😂
@@someusername1872always assume super-tech like that is 10 times as far away as they say it is. They’ve been saying “fusion energy in next 10 years” for the fast century
LSD Dream Simulator remake looking good.
Hell yeah! Glad people thought of that too.
i saw a comment about this elsewhere and it won't leave my head: i think the reason that it's cropped into a square, rather than the usual aspect ratio, is to cut off all the facecams from the training data
thats actually genius!! that has to be the reason
I was thinking a lot about this, and idk actually if they trained on public videos. They needed to capture the keyboard and cursor inputs as part of the training data, so I think it's likely the had people/bots playing MC and captured the data that way. It was TONS of data though, so they were probably all virtual machines. Also square is pretty standard for a lot of these test/basic models. They mentioned with their future hardware it could be served in 4k.
Just guessing though.
i wonder if they got permission for said training data. no way is all of this generation going to backfire. sarcasm, btw
Nah, the reason is because training these models with square inputs that are power of twos in size is much more compute efficient. It is very common to intentionally crop all training data into a square for computer vision models.
@@andybarcia4827 This is most likely to be the reason. Otherwise, if they trained on data like streamer gameplay, there would be things like donations, etc...
Oh cool so it's like a video game without all the structure and rules and cohesion
and running it for any amount of time accelerates the effects of climate change!
@@DumplingDave (still believes in climate change)
dude, it's ai. this is incredibly impressive. y'all just hating to hate
@@DumplingDavegive it a couple years and it'll become way more efficient.
@@aussieglizzyyou're probably catholic
4:21 Do you also hate it when you're trying to choose an item and the fabric of reality melts as you speak? 😤
happens all time, I hate that
And a pickaxe suddenly materializes in your inventory
yeah me too.... you *are* talking about IRL, right?
When i first saw this I though it was gonna have like 10 millions views or something. You’re very underrated
Much appreciated 🙏 stick around, it'll be fun.
It’s been one day
his content is interesting, and he is a good host (I just subscribed). I think it just needs a more attention grabbing pacing. like, he only said the name of this tool midway throught
but I like the way he presents things. it's so relaxing
Generative AI logic is so similar to the logic of human dreams it's kind of profound.
am i the only one that finds this really cute
@@delta0xlike its funny the way the AI thinks things are correct? yeah i agree
@@skeleton819 you do this too but youre not a newly invented form of life so its much less charming
@@delta0xAI isn’t a newly invented form of life though, it’s just some really fancy equations that usually spit out something statistically likely to look like Minecraft
Does anyone else remember "deep dream?" That was one of the first generative AI publications Google made where they ran the ML model for image search backwards to debug it. It produced insanely trippy output.
bro a fully playable ai generated video game when ai media is still kinda fairly new being made this soon was not on my bingo card.
Truly surprised myself
@@JarrenRocks same
I wouldn't really say fully playable...the items,blocks and stuff in your hotbar just appear and dissapear on a whim,and the inventory and ui really just don't work.
Its still kinda playable but not really
@@Damian-cilr2you can play it in real time and without 1 frame per 10 seconds frame rate. This is huge. And resource cost is low, since he is playing it in the browser
@@Cubic_cat Nearly all the work is done on their servers. The resource cost of running AI models is very high. That is why Nvidia turned into a mutli trillion dollar company overnight, cause they were wise enough to invest in CUDA cores over the years.
The word "video-game" is suddenly x10 more accurate
Minecraft:
The Traumatic Brain Injury Update
The internal brain bleeding update
What’s scary is how close this feels to the progression of human dreaming.
Playing it made my brain think it was dreaming. Afterwards I kept having this subconscious expectation that websites should be morphing and changing when I'm not paying too much attention, kept feeling low level surprise that nothing was memlting or morphing, and kept having to remind myself that that is normal.
Pretty damned trippy.
This is low key unsettling ngl
babe wake up.
Jarren is playing minecraft again. It was a 8 year drought, but he is back.
I was about to start a gaming channel again after playing this.
As someone who has of course, NEVER done LSD, I can confirm this is pretty much the most accurate simulation of what that actually feels like.
It feels like a dream after playing MC for hours
Edit: bro i just realized that the views aren't past over a million damn
Keep it up bro you're so cool 🙏
Thanks 🙏
Minecraft Dementia Simulator edition
I think it's kinda lit, ngl
I think the solution to this is to inspect the game state and embed it into the model as parameters
For example, encode the inventory into a set of "bits" and embed it into the model, in this way there will be a persistent memory of the owned items regardless of the context interval
Similarly other information can be embeded too like HP, Hunger bar, XYZ coordinates and world seed (idk if that would make any difference), even if it's embeded as a "bias" that would kinda steer the model one way or another
Right, I've seen pseudo-text RPG games working in SillyTavern. You get the internal logic and variables that are stored in the context and update according to the events.
Maybe in the future, with a long enough context, you could store raw data for the time of day, nearby mobs, dropped items, biomes, blocks, the code that moves the player... oh wait
Hi
You've invented the concept of "variables"
Y'all are so close to reinventing videogame source code that it hurts, and it's burning forests
@@ferociousfeind8538 yeah that's true, but its hard do make dementia simulation in deterministic computing
And besides, I think it's a fun idea to "merge" models trained in different games and see what happens
For example if you merge Minecraft model with a Vintage Story model
Or if you merge a Doom with Halo, what kind of crazy "LSD" simulations would result from this?
I don't think this works as a product either, but it's an interesting research project
@@ferociousfeind8538ai bros reinvent stuff that already exists every few weeks but way less efficient, they’ve made trains but bad at least 40 times already
It's like someone uploaded their dream to the internet and made it into a game
This concept would make for a great horror game. Present it as a generative AI experience, but it's actually manually coded, and have the hallucinations become eerie and seem intentional and dark.
I was just thinking something like this. Like with doors you go thru and then you go back thru but it leads somewhere else. And like intentional datamoshing.
There is no real way you can make something much like this without using AI. Unless every single instance of a hallucination is done manually, but that would be nothing like this given how few there could reasonably be.
thats gay
@@whwhwhhwhhhwhdldkjdsnsjsks6544 You could do this very easily without AI. Just write a shader to give the fuzzy visual look, and have stuff change when you're not looking at it
You can still use code to randomise stuff to billions of permutations
@@caut5618 oh, I remember you, you were put into a coma in 2005 and must've just woke up. You're not gonna believe it, but it's been almost 20 years! there was a stock market crash, and then a recession, and then the US elected its first black president, and then everyone thought the world was gonna end, and then a gorilla got shot or something, and then the US elected a washed up reality star as president, and then there was this big pandemic that a lot of people thought was fake, and then- oh, yeah, and using "gay" as an insult - totally lame now. I know, never would've guessed it!
the thing i found the most interesting is that if you are in a dark place and you right-click the ground with your pickaxe in hand, it will place down a torch. it's like it doesn't know what a torch actually is, but it knows that people usually place torches when it's dark.
PhoenixSC needs to see this
He's gonna be devastated that you can't use commands
He already has a videp out where he plays this lol
This is a really interesting concept but I don't think it could ever be more than a gimmick in an actual video game. I agree with others saying it could work in a horror style game, but the incoherence would have to be balanced with monents of consistency to actually sell the vibe. Maybe if a character is walking through an otherwise fully designed game with good mechanics, etc, and the character has a breakdown or nightmare scene that works like this, sure. But even then, a lot of the compelling elements in something like that are story moments like the void of their dream revealing the character's dead sibling or something.
Idk, I just feel like once you're increasing the context window to ridiculous amounts and coding in UI overlays whateberver, at a certain point you might as well just make an actual game, with good gameplay feel, story, and level design. This will mostly just give us a distilled/generic version of games we already have with some trippy haze around it.
no you don’t understand, this is a glimpse into the FUTURE of interactive AI experiences!!! this will totally replace all video games and every game dev will be jobless by 2026!!!🤓🤓🤓
@@dogonit1936 people unironically think that and that's both hilarious and sad - imagine thinking it would be better to offload creativity onto a machine...................................................
@@sc0pe103 you see, if we automate all the creative, interesting, fulfilling and/or fun kinds of work, that will finally give us time to fulfill humanity's true purpose: dying in the lithium mines so the last human can be a Martian quadrillionaire
It'd be so cool if there's a way to put a permanent object in there somewhere so multiplayer and monsters would make much more sensible
You articulated my thoughts exactly. This is very cool as a Weird Thing™ you get to check out, but AI generated games have as much artistic value as AI generated images. And for me, it matters a lot if I'm having that "conversation" with a real developer or just with a machine.
I love this
I hope it doesnt go away like how the crazy ai art did
I loved those images that looked familiar, but you couldn't name anything in them
picking a thumbnail was so hard because every frame looks kinda real, but the morphing is the weird part
i loved those tooo
This project is way too unique, makes you think about what games will be in the future.
Watched a couple of these vids so far. Yours was my favorite because youre genuinley excited about it and go into some of the technical stuff, making it interesting to watch
@@UEatPoo I really appreciate that, I’m more in the tech space today, but used to do let’s plays, so it was just this weird coincidental, crossover event. I’ll definitely have more game related content, but not sure if I’ll get into a habit of gameplay like this again
at 3:30 you have a bunch of water, but go to a new map to "get some water in here". Maybe you're an AI hallucinating as well?
Imagine an AI being excited about an AI being able to generate a game for it to play
I blame it on recording. It’s harder to use 100% of your brain cells when you have a camera on
its like dreaming
I like just running around in circles and falling into clouds
Minecraft inception.
But less coherent and consistent
This is how AI dreams.
We’re all gonna be in trouble when AI wakes up.
Nothing like dreaming
The weird shifting of the textures is really similar to hallucinations off psilocybin and that's really fascinating
I was going to say that about LSD. Done it only once but this freaked me out because i had a very bad trip. It’s scary when you’re disoriented like this.
I tripped after the AI boom and I was like "what the hell why is everything AI generated" while I was coming up
As someone who's planning on trying psilocybin soon, this is an interesting insight
@@TruTrolldier Its astounding how easy it is to forget how disorienting psychedelic experiences are. Playing this pulled up some long forgotten memories.
@@Notanothercrayon Don't fight the trip and you'll be good.
I like the idea of storing the inventory outside of the model. Maybe they could also code something to keep track of blocks outside your FoV. And maybe also some check to make you fall when you're in the air, kinda like a mini physics engine and... oh, wait...
You’re right though. The simulation part is more interesting for the times when you want to fill in the gaps, rather than the whole game.
I’d like to see what a mix would look like.
This opens space for a completely new game genre. The AI generated game, one of which the gameplay is going to be completely new every time it is played. Almost like procedural generated game worlds like No Man's Sky or Minecraft, but maybe with the entire gameplay entirely different every time.
I don't think that this is the future of games due to the [totally unpredictable gameplay], but it's still cool research-wise and it may give people insight into how transformers and large models work, manipulate data, and store information.
Minecraft with dementia:
7:47 that cliffhanger was closest presentation of dream endings
16:42 close your inventory and turn around, some mysterious evil dark blob appears behind you, you try to investigate it and then the game crashes
this is a real life creepypasta
I got freaked out pretty fast by that blob
Damn, I saw something like this in my first "simulation". Something dark was standing 6-10 blocks away from me when I turned around on the last seconds of simulation. But it was probably a morphed darkness in some cave that was pretty far away from me.
@@Аноним-п1ж i tried it myself earlier (at 2 am woo what a good idea) and had my inventory opened for an extended period of time, i finally close it and i'm in a monochromatic hellscape that consists of nothing but lines of two colors, with a single small strip of red on the ground
i look down at it and i'm now in ACTUAL hell; the nether
It doesn't crash. It is timed. Look top right
That black blob is a horse
This is a visual representation of what is described in Eldritch Horror. Not the monsters but the contrasting natures of a given thing.
I would describe eldritch horror as being able to know something which you can’t understand but I do still totally see where you’re coming from. Actually half way through writing this I realized seeing something you think you understand then It changing completely could totally be interpreted as that.
The new Minecraft lobotomy update looks great
This is.. the single most insane thing I have ever seen in my life
*Context Window* is a _CRITICAL_ term. Crazy stuff. Thanks for this video!
This was incredibly entertaining to watch, no idea you had only 1k subs
13:43 HOW DID YOU NOT SEE THE SHEEP!
I know I only saw it while editing ☹️ if I saw it while playing I would have chased it
AI simulator
I love that it instantly transforms into a sand block
He was so determined to keep the village in frame in order to see if mobs were in! :)
what i think of whenever i see projects like this is how much less fun these projects will become the "better" they get. The entire charm of AI generated stuff for me is seeing them devolve into nonsense (like that ai generated beer ad). When it gets better it just becomes a video like any other.
yeah theres a charm to the weirdness i miss with flux that i could get with weird interleaved SD prompts
i predict it will be less of an issue with such a directly jnteractive thing though
and also when we have weights and code available we can start messing with the inference in fun ways
This is the closest we are getting to living a literal dream
So, this is the worst AI generative gaming is going to be and it’s already mind-blowing
Bro what are you on this looks like actual garbage. You can’t even play for more than five minutes and everything disappears so you can’t make any progress in the game. Are you actually tweaking rn?
Think of this like the Atari days of video games. square moving on screen, pong and all that.
Just like that was “bad” visually but impressive technically, this is similar. This is “bad” visually, but impressive technically.
And these demos set the stage for things to come.
@@elijahwallis956 it's a very cool experience I don't know why you feel the need to put down people who are excited for something
@@elijahwallis956 12 year old has to have his input because unfortunately there's no laws regarding children leaving stupid nonsensical comments online. Who let this kid have a computer? This is absolutely mind blowing
curious how it could eventually get applied to the creation of a new game tho if it needs to be trained on an existing one
This is like that 4D miner game but even more trippy
This somehow manages to be more bizarre than 4D hyper geometry
That's my dreams, clourised and visualized
It's like when you play till 3am, eat a ton of candy, and then try to sleep -> you see this
@JarrenRocks or play until 3am and then do maths until 4 am
Bro dreams in monochrome
It's like a dream. You're going somewhere to get a thing to bring back to someone, but on your journey to get the thing the world changes around you and you have trouble getting what you were looking for and even if you do, getting back to where you were is impossible. A dream is just a long free association with little to no ability to back trace through the "narrative".
This AI stuff is really teaching us a lot about ourselves.
Most people think this looks like a dream but to be honest i feel like this looks more like the tv in your head on NMDA antagonists
The gameplay seems like an uncontrolled lucid dream
No idea if it’s just me but, A.I incoherence just makes me itch, it’s such a disturbing feeling. Things like this, and just A.I generated videos in general make me feel this weird, crawling sensation
I feel that for sure. At the end of the vid there was a big black blob that was disconcerting, I backed up from it at first. It makes me nervous to see what's next sometimes.
@@JarrenRocks I used to seriously struggle with paranoia and it just has this hallucinogenic feeling to it lol
After playing this for a while I had to let my trust in my sense of object permanence build back up, this is insane
I feel like we're watching the matrix be built around us
What could humanity do about it?
9:35 etched is the company they partenered with (I believe). ASIC means application specific integrated circuit. it can allow you better performance than cpus and gpus because its designed specifically to do the specific calculations u want at an hardware level, hence why they could run their model in 4K
Yeah I realized when editing that ASIC was the type of hardware in this case. Good to note, I'll update my pinned tab with that info too.
It explains why the performance is so much better than the AI generated Doom gameplay I've seen before.
The giants drink from enders game is gonna be real.
AYYYY
You look at the time; look again and it is different but you don't know either time. Just if it is forward or back. You push a light swich but no matter how hard you push, it won't move. The person you were with has changed. The light switch is now a pull chord type but still won't pull. You check the time again, and it is later; but you can't say when. Or if it was before you began. You read your notepad for advice and you know what it says, but you can't see the words.
You look between the words and see smaller words written in the sides of the words written. The closer you look you notice a pattern. Like the fibanarchi sequence but with interlocking shapesyet with a distinct real 'word', yet you can't see the word.
.you wake up
Wait. Isn't this just how the brain works? Just more connections having a more/less correct version of reality?
Like, did y'all know that? The reality you see is a predictive algorithm simulation in the brain, and the brain only uses inputs when it's wrong. It would take the brain an impossible amount of energy to calculate using ALL inputs EVERY SECOND, so the brain evolved to have a more/less simulation of the reality.
Like color, for example, their is no such thing as color. It's an evaluationary trait that's only in the brain that we evolved to be able to distinguish stuff around us.
Color being a figment is not the same as the brain being a predictive algorithm like the model in this video. Your brain constantly takes in concrete input and definitely doesn't make up as much baseless nonsense as the AI model
@@ZerosiiniFIN I mean, but like. If the ai had more connections as in the same amount as the brain, would it not be the same? Our brains have like what? Billions of connections or something like that?
That's only the simplest explanation of the theory. Being basically a predictive algorithm simulation, or more of a "controlled hallucination". Look up: Why reality is a "Controlled hallucination"
Posted by Kyle Hill
and then you can probably do more indepth research of such topic
@budderman3rd Humans hallucinating is based on actual, concrete, current input. This predictive AI has been trained on terabytes of data and only uses that and the most recent prediction to form the next one. There is no new information other than the player's inout which is not more than a couple of bytes of data.
There are some similarities, but I fail to see how your comparison is accurate.
@@ZerosiiniFIN Did you even try to watch the video dude?
@@ZerosiiniFIN Our reality is basically only in our brains based on such theory. It's a more complex predictive algorithm, that doesn't even use inputs directly.
Honestly that would be a very interesting game mechanic.
Imagine a game that changes in the directions you look.
For example you go forward and things are normal, you turn and everything starts to change and barely anything is similar. The twist being walking backwards without looking back keeps the old design of the land loaded.
So... aslong as you move straight backwards or forward it stays the same, but once you start turning it all changes.
That's kinda what 4D miner is like
@jme7046 not necessarily. 4D miner is the same all around, you just simply move through 4d space as the game trys to simulate, nothing is technically changing. You just have way more angles of perspective.
But visually... maybe a little bit. I'm talking about completely changing the landscape for the most part, with the only 2 things being the same being things you can see directly and things you cannot see whatsoever.
Basically moving through space by turning while potentially trying to figure out a path to a point in said space.
For example they could turn and find themselfes with the dot being way below them, so they look for something cave like , look at it closely and turn, causing them to suddenly appear in a cave like environment with the dot suddenly being above them or on the same level.
It would be quite the brainteaser, hehe.
one of the few appropriate uses of ai in art. simulating a concoction of schizophrenia, alzheimers, and various other psychological conditions.
this... kinda _is_ art in how absolutely abstract an experience it is.
@@isawadelapradera6490 exactly. its like how dreams feel when not lucid
For a sandbox game this tech could be super interesting. But I can't imagine it would ever work for any other kind of game that would have a fixed endpoint. Do I have that right?
VR port gonna go crazy
oh shit for real didn't even think of that lol. But actually probably not possible for a while because there would need to be perfect consistancy between both eye positions. And if there is not it would give that weird effect when you look at two different scenes with both eyes
@@SpicyMelonYTthat is literally the least of the concerns of what it would take to implement this for VR. Please just stop endorsing the creation of AI slop.
@@bernatrosello4375 you're a bad person
@@SpicyMelonYT no, I'm a game developer. And I'm tired of seeing the humanity being stripped away from so many aspects of art, programming, writing, hell even games now.
I've worked on VR games and know the inner workings behind them, the technology is simply not there yet.
@@bernatrosello4375 I also make games. And the humanity is not being stripped away because these exist. Its because of the lack of laws keeping human jobs. Humanity is still very much there. We like making art, we like making tech. Humans rock.
And whats the point of your last statement? You say it like because its not there yet then whats the point? That same damn argument can be made about EVERY SINGLE CREATION EVER. Hell even a game you work on isn't finished until its finished. But you were already imagining what it would look like finished, what it could do, how it can feel. Its there same thing here dude. And you draw this arbitrary line in the sand because you don't like it. That's bad.
You know how many artist hated photoshop way back when? And now those people who use it hate the next this which is AI. Video games are fun, they are creative, they give you experiences, and they make money. None of these thing stop happening with AI nor should they.
It's a little bit like a high dimension game. When your rotating and everything changes it could be seen as bug, because it hallucinates. Or you could see it as feature, where you actually move in the forth dimension and that's why your perspective is completely changing
Imagine if, locked away deep in some generated space, there were a key that would help you progress in the game. Your job would be to hallucinate your way there.
This is really promising for future developments in both AI and video games. A major issue with price to performance in video games and hardware has been that the clock speed of processors (GPUs, CPUs) has stopped increasing. Instead, they’re generally going “wider” nowadays, meaning that not all classical software is able to scale to use it.
If you think about many game engines, they can’t necessarily use all the cores in a modern CPU to increase the frame rate, for instance.
But AI scales remarkably well. In fact, you don’t even need all the processors to be in a single device. You could imagine pre-computing slow moving components in a remote device, and computing fast latency sensitive elements in the device closest to the display, for instance. Imagine not being locked into, say, a single expensive GPU, and being able to combine two cheaper ones from the used market like we used to be able to do with SLI / Crossfire…Or being able to choose to run on CPU, or choosing to run on an NPU, or whatever else have you.
The cool thing is that once you get something like this Minecraft demo, it’s actually not that much more technically demanding to do, say, “realistic” graphics (and I mean actually realistic, not video-gamey realistic), or to do any art style you’d like to do. You could also imagine doing more realistic, larger scale fluid simulations, for instance, or crazy physics simulations that are actually realistic…And for essentially “free” without really much more processing.
Exactly. If we eventually have a model trained on physics, objects, and interactions, it's possible it could become a completely new type of game engine. Similar to how ChatGPT is like the ultimate text AI, this could be the ultimate spacial AI. Not to mention incorporating this with generative video like Sora or the recent Mochi1 by Genmo.
Diffusion tech is still pretty unoptimized. It'll probably become less resource-intensive in several years.
i think id rather have a game that runs on an engine than an interactive mp4 file
@@dogonit1936 tbh I’m tryinna have both
I cant wait for this technology to advance enough that actual playable games can be made that utilize it fully. One concept I can see these hallucinations being perfect for is a sequel to LSD Dream Simulator.
eventually we're going to have a model that can simulate an entire operating system as if you were on it. With websites and working pages and programs lol could be wild. The direction this stuff is going is really wild.
I remember hearing that if you ask ChatGPT to do that it can with reasonable-ish accuracy. You can type shell command and it will simulate them.
sounds like an amazing idea 😂 i open up a new chrome tab and all the files on my desktop disappear forever
Why would you want that over an actual operating system that takes exponentially less computation to run and is more accurate? Imagine an OS that just hallucinates files lmao
I really doubt we’re actually going to have that, at least not within our lifetimes. That would be so resource intensive it would just be worthless
i had a four hour inescapable lucid nightmare after watching this video before bed... thanks
What would be crazy is if someone used a camera and turned real-life footage into a game.
what is the potential use case for this supposed to be? you say you are excited but i dont get whats exciting, i get this is just a prototype and things will improve but towards what? what are we improving towards? how does this become anything?
Good question. I mention in a few other comments, but, things I could see happening from this off the top of my head:
- a generate anything you want game (like the ultimate Scribblenauts)
- new games that haven't existed before that could only be playable like this (assuming there were more constraints and goals etc)
- pause a moment in a movie, and suddenly it's a playable game like GTA or whatever
- super performative games by streaming low-res and using a trained model to upres (NVIDIA already does this a bit)
- maybe I want to make a mod, I define different settings "low gravity, floating blocks" and it could simulate that/convert the generation into an actual mod.
If LLMs are the "text and thought", tools with video/gameplay like this can become the "vision and physics". There are probably some better ones I haven't though of.
@@JarrenRocks hmm these all sound not that great... i spose they were just what you thought of off the top of your head but honestly most of this just looks like giving up control or things we can already do or in the case of the movie idea just horrid. oh well hopefully this will be used for good unlike the llms and other image to text ai stuff that just fills the digital world with slop
so when we're dreaming it's just our brain putting things where it think it should be based off of previous input?
Yeah, because that's its job in the waking world, too
Not necessarily.
If your dreams always look like this then there's something wrong in your brain
Dreams can be very coherent at times
@@luis-sophus-8227 dude have you read literally any other comment
@@delta0x Well then that's weird.
I know that people with clairvoyance have far more solid and structure dreams than the average.
Disordered dreams mean disordered self (not referring to medical term), I think that's what is going on here.
@@luis-sophus-8227 everyone has clairvoyance. sometimes i just dont feel like putting a 100% accurate and realistic world in my brain during a time reserved for being away from physicality. its fantasy hours for the little delta
Wait I’m so confused. It’s obviously generating the visuals, but how are things actually changing in the game as well? Is there a mod that goes with it or something to actually change the world? How?
It’s not connected to real minecraft at all, it’s like an AI image generator or chatbot trained on gameplay and the controls for the game.
@ We don’t have that kind of technology yet. Something like that will probably take at least another 5 years. This thing has to be fake. Not faked by you, but like the website being misleading to what’s actually going on here.
@@JoshyRB that’s the fun part, this is a research paper, so it’s highlighting a new technology. It’s not a commercial product - you can expect that in 5 years something will be commercially available.
@@JoshyRB that’s the fun part, this is a research paper, so it’s highlighting a new technology. It’s not a commercial product - you can expect that in 5 years something will be commercially available.
@@JarrenRocks Well I’m sorry but I don’t believe that this doesn’t have Minecraft running in the background somehow.
This gives me violent vertigo. Like, I got a migraine watching that gameplay. That was horrendous beyond measure.
Imagine a horror game like this where the AI is constantly trying to mess with you
Babe, wake up!
You're dreaming about Minecraft again!!
This is incredible! I love it. :D
It gives me the same sense of space itself breaking as I had when I first played Portal.
why is the website so pretentious at some points oml
It's basically a research paper, so it's being written towards other people in the research field. I just so happen to read those papers and love playing Minecraft 😅
tech bros
This is an awesome example of how ai work, its a non-sentient pattern module. We actually reconstructed our subconscious how amazing
Can you beat minecraft with dementia?
I think training it on 360 video would help with the environment disappearing by just rotating the camera and during "gameplay" just only display a portion of the frame buffer
Stuff like this should remain experimental.
It's cool don't get me wrong, but this should never be commercialized
How would it be commercialized when its just worst than the original product.
this will be google stadia in 2034
Why
"Ai is bad womp womp"
@@mr.crowjo_3933yes but unironically
I imagine the context window could be way bigger if not only the inventory was hardcoded, but the rendering engine also, because then you wouldn’t have to store visuals inside the model. But it is probably really hard to get training data for such a hybrid set up because while there is a lot of video on RUclips, collecting data for a hybrid model would require recording the game engine instead of the visuals.
Game engine already hallucinates.
And does it way better because its been hardcoded.
Ai will not reach this level until skynet. Food for thought.
It this technology succeds it will take valuable jobs from already struggling developers and artists, if it fails, we're stuck with inconsistent messes of games that nobody will end up playing.
Ultimately the tech is set up to fail, but even if it wasn't, this business model is horrible
Some of that is probably true. But also more people will become game developers and create new things that never existed before.
Even successful it'll never replace commercial products, you can't copyright A.I generated content and it wouldn't be allowed to train on works without permission.
@hatad321 I agree, I was only talking about the worst case scenario for everyone
I believe the next step of this technology is kind of, incorporating it into the game engine to give it all the context it needs whenever it needs it. Without using context windows for positions and whatnot, you'd likely need to give things tags to tell the AI what everything in a scene is, and with that information it can make some cool stuff, perhaps like Photorealistic Minecraft or even other games, without all the weird hallucinating. Since the AI wouldn't be using its context windows on figuring out what the scene should look like, more could be used to make the scene more consistent with artist vision and previous shots. Interesting stuff, this AI video generation
H. P. Minecraft
This feel like some sort of analog horror
We're going in the wrong direction for humanity...
okay grandpa lets get you back to bed
This is some Alice In Wonderland type shit! I can’t wait ‘til someone makes a mod to mimic these lack-of-rules in actual Minecraft!
If this is perfected, then you could play the most graphic-intensive games on any computer!
except you'll be using even more power since it'll be running off servers instead.
@@Dr_Pickle I’m pretty sure it would still be more efficient for the servers because generative AI is a lot easier to run then a hyper realistic game where it actually has to be rendered
@@tuxedocat3620 no it isn't. It takes one computer to generate graphics. It takes entire servers to generate an image.
Hey nice video! Been awhile since the days of… was it SMB? I can’t remember it’s been so long.
Hey! Good to see a familiar name! Yeah probably 8-12 years 😅