NEW Open Source AI Video (Multi-Consistent Characters + 30 Second Videos + More)
HTML-код
- Опубликовано: 3 май 2024
- Check it out here:
storydiffusion.github.io/
📧Join my newsletter to stay updated
delightfuldesign.eo.page/w7tf5
---
👨🏫Check out my AI courses:
www.udemy.com/user/samson-vow...
---
💼Business inquiries
samsonvowles@gmail.com
---
--- My top resources:
📖 Grab My AI Secrets! Dive into my handpicked prompts e-book.
delightfuldesign.eo.page/6sw6r
🎨 Unleash Your Creativity! Experience Adobe’s cutting-edge AI tools.
tinyurl.com/2sj2kwk6
🤖 Boost Your Efficiency! Automate with ClickUp.
tinyurl.com/4p9yxew5
🎤 Find Your AI Voice! Clone your voice with precision.
elevenlabs.io/?via=b38559
🎥 Revamp Your Content! Edit shorts effortlessly with AI.
www.opus.pro/?via=7fcbf8
✍️ Supercharge Your Writing! Generate content the smart way.
www.jasper.ai/?fpr=samson
📹 AI-Powered Videos! Elevate your content with Kaiber.
kaiber.ai/?via=samson
📢 Ads Made Easy! Create compelling ads in a flash.
free-trial.adcreative.ai/2yui...
👤 Stand Out Online! Craft your unique AI avatar.
www.synthesia.io/?via=delight...
🎬 Level Up Your Channel! Get coached by AI on RUclips growth.
vidiq.com/samsonvowles
🖥️ Present Like a Pro! AI-driven slides for impactful presentations.
slidesai.io/?via=samuel-vowles 
Наука
7:28 You got me at multiple consistent characters in scenes. It’s really important for storytelling to be able to have multiple consistent characters interacting with each other.
Oh yes. Definitely
I make music in a band, and I create music videos for our songs on the platforms. I have used AI as a collaborator on a song, and I use scenes created by AI as best I can, with what I can find. Your channel keeps me on the cutting edge of Music and Video Creation.
I am very excited at the idea of me putting myself and my other band members in some AI videos. Thank you for this channel.
the tech would be great to pre-vis shots before an actual shoot or better yet render the whole movie to perfect the flow before shooting begins
For what i read in their paper, their attention method work through batch generation, video length is limited by memory and that is probably why resolution is so "low".
This is the first time I seen multiple character interact with each other
Yeah.. we gonna need that for smut vids
I was thinking of using it to make realistic fight scenes for my superhero comics , but you do you 😄😄
@@Mightyflynn77 I was thinking of using it to make realistic fight scenes for my superhero comics , but you do you 😀😀
@dwainmorris7854 will this cartoon feature tentacled creatures? I would buy that
I still need to see it generate something. Hoping the gentleman in the vid gives it a shot. I don't know where to start.
When I see glitches such as the ball going through the metal rim I can't help but to think what this technology can do someday for special effects that are closer to realism than CGI while also being simpler to accomplish.
man i love your channel and your videos, thanks so much for all this updates and news, you rock!
thanks!
8:19 The ability to use this AI tool for creating comics (and graphic novels) would be amazing.
Or animating existing comics… I can see it already, no need for Hollywood adaptations when you can simply plug in a comic and have it create a authentic 1-1 movie based on any book or comic / manga…
@@sssurrealI would love this
there is already 1 comic based AI out there, though I don't remember it being too good.
@@jamesesparza6893 yes I remember, it's on huggingface, not that good though
@@jamesesparza6893 good for suggestive storyboards though?
looks great and will only improve over time cant wait to see what the future brings!
How well does it perform with dynamic action like people fighting or playing sports etc?
Absolutely extraordinary. AI video is developing faster than we can keep up with it.
long ago, around 2007, I remember watching something on Discovery channel, a special. The person who was being filmed said these words.
In the future, actors will no longer be used because movies will be created without them.
We're just about there.
Good or bad, it has its applications.
in 1992 when I was in gr 7 I made a speech about this in school :D
@@beofonemind it seems we're all making the connection.
no way to download and take advantage of this model on iPad's DrawThings app?
Is there any video generation feature released yet? I couldn't find an option to generate videos in the github reference
I’ll have to try this one. Thank you for the videos ❤️
it dosnt work
What is the software?
tbh my hopes are still on LTX. It is an awesome platform already but certainly has to learn alot.
Time will tell how well it´ll turn out, but I´m rather positive at moment.
Very nice. Will try to use this in my ai music videos.
I'm trying to test a video in the github page, but I keep getting an "ERROR" in the render section a few seconds after... Although this has happened without an image reference, I'm not sure if I fully understand what they mean by "𝘪𝘧 𝘺𝘰𝘶 𝘢𝘥𝘥 𝘵𝘩𝘦 𝘙𝘦𝘧-𝘐𝘮𝘢𝘨𝘦, 𝘮𝘢𝘬𝘪𝘯𝘨 𝘴𝘶𝘳𝘦 𝘵𝘰 𝘧𝘰𝘭𝘭𝘰𝘸 𝘵𝘩𝘦 𝘤𝘭𝘢𝘴𝘴 𝘸𝘰𝘳𝘥 𝘺𝘰𝘶 𝘸𝘢𝘯𝘵 𝘵𝘰 𝘤𝘶𝘴𝘵𝘰𝘮𝘪𝘻𝘦 𝘸𝘪𝘵𝘩 𝘵𝘩𝘦 𝘵𝘳𝘪𝘨𝘨𝘦𝘳 𝘸𝘰𝘳𝘥: 𝘪𝘮𝘨, 𝘴𝘶𝘤𝘩 𝘢𝘴: 𝘮𝘢𝘯 𝘪𝘮𝘨 𝘰𝘳 𝘸𝘰𝘮𝘢𝘯 𝘪𝘮𝘨 𝘰𝘳 𝘨𝘪𝘳𝘭 𝘪𝘮𝘨" I suppose you have your image subject and just put the letters img right after that word in the prompt? Any idea how to make the thing work? Tried in 2 browsers as well, and still getting that Error in red....
Thank you for this video!! Very useful to me :) You explain very well.
Thank you Samson once again an amazing video...
My pleasure!
I tested the demo page, and i have only images when generating.... not a video, have you got it to work?
This looks great; AI creating video content is a trend for the future.
When can I use this?
Anyone know how to install this locally?
Thanks you. This is really cool. Now all we need is a tutorial on how to run this locally.
no idea how to run it. any idea?
I’m here for an open source Ai 👏🏾 but any indication what it’s trained on?
Is there just a paper? How can we run the code?
Hmm... I can't get it to work. It just keeps saying error.
I try it on Hugging face, on white tiger with armor, and there is no consistency of the armor, what do I do wrong ? this is far from this video
Open Source is still very underrated but things like these demonstrate the potential in AI.
13:01 that's not a scene in England...it's an astoundingly accurate rendition of a view probably across Princes Street up towards the mound in Edinburgh, Scotland. I've seen that view often. Details like the unusual lamp and recognisable buildings are astoundingly accurate, actually. I nearly fell off my seat when I saw it!
Good grief the bus is even the 900 city link Edinburgh to Glasgow bus and the text on it is legible!
That's because it's just an illegal scape of real, likely copyrighted video
@robertruffo2134 nonsense. Try learning how this stuff works. There is not a compression algorithm known to man that could compress the volume of information in the training data down to the size of the download. The model learns by training on data not dissimilarly to how a human does. It doesn't have a database with the information in it. Only weights and biases representing what it learnt from studying the training data.
When people say things like you say:
1. I feel for them. Sucks that this affects livelihoods.
2. But it's clear you don't actually understand how this works.
@@iaincampbell4422 Dude, I know how this works. I even worked on earlier versions of this kind of system. If you think they learn "the way a human does" you have drunk many liters of tech hype Kool-Aid. Also, yes they do "keep" training data. Have a look at some recent lawsuits.
This is crazy bro
hey Samson i love your videos... I was wanting to ask you a couple of questions if you have time to answer...1 What is the AI video generator out there that anyone can just use that has the longest videos? and 2 When do you think Sora will be ready for people to use? like how long will it be before just anybody to use it ? i know you may not know the answer to #2 but maybe you can give me your best educated guess at it ... thankyou so much for making these videos for people like myself
I'd say that the answer will depend on when you return to this video to look at the replies :) At the incredibly fast pace of current development, the answer will vary from _week_ to _week_ ...
@@GwynethLlewelyn thank you I appreciate that
Maybe I'm missing something. But all the stuff I've seen from Sora recently and in this video looks way better to me.
Leon Ding leaked a lot of Google AI technology to the P R C
Anyone knows how to get a consistent background using Gradio or ComfyUI ?
I KNEW that wolves spawned at different spots in the woods!
There had to be a reason they put it in so many games!
amazing cheers
Opensource, sounds more than good. Will give it a testrun now
How did it go?
@@user-ug1rq5qz4e They just released a comic strip generator, but no videos in the script.
AI is going scarily fast. Mainly in algorithm optimization. We're getting ultra powerful models on consumer hardware. Insane stuff This is going to go out of control
Video related code is not yet released. Only makes images so far.
can you make locall install turorial?
Woah. Exciting times. Things are improving so fast now. I can handle some of the glitch if i get more continuity and length 👍
So exciting to witness humanity having itself robbed of creativity after 40000 years of making art...
@@Zareh_Abrahamian Ok, luddite.
@@EnderViBrittania I love technology. I've been using computers for almost 35 years, I've done computer/internet programming for several years. This is different: typing "write me a symphony in the style of Beethoven" and the machine pooping out symphonies, where the person typing cannot play a note, is the very end of human creativity which will turn humans into cucumbers. Think what you wish.
@@Zareh_AbrahamianAnd now you can create your own full length movie just as you imagine it, isn’t that cool?
@@9thebear Typing a few words and "creating" "your own" full length movie _just as_ you imagine it? Where is the creative process? Where is the artist in all that? This is why I am saying AI will destroy humanity by depriving it of creativity, turning humans into cucumbers.
I hope they add a landscape 16:9 aspect ratio, so we can create clips for AI movies.
I haven't tried this tool yet, but I'm thinking about using keywords like "Wide shot", "Long shot", "Panoramic", maybe even "Fisheye" - then cropping it, then AI upscaling.
*have AI generate, not _create_
@@Zareh_Abrahamian It still fits within the definition of ‘create’. It’s bringing to life content that does not exist anywhere else on the internet.
@@JohnSmith-gt3be "Created" by the machine, having stolen centuries long blood, sweat and tears hard work of creative people, making billions for AI companies, rendering all artists dead or living irrelevant. That is some life and content we can suck on!
@@Zareh_Abrahamian I think it’s an amazing idea. When the horsing industry was destroyed by the automobile industry the world became a much better place. I guess you’re just on the side of the horses today.
I have tried to make this work using the demo, It never does though. I am not sure about what I am doing wrong. I always get an error.
Same
Yea it’s broken for everyone on the demo face I think
same here
It has always worked for me. Click on one of the examples at the bottom and it will pre-fill the fields, then click generate. It will work. Then, you can edit the fields to fit your needs. I have no idea how to make a movie with enough frames for fluid movement, however.
@@redleader7988 yeah i've been chipping away at it and exploring it locally and the functionality to generate video doesn't seem to be present, unless i'm missing something
honestly asking, but has anyone ever made one the huggingface demos make run? I mean of any ai animation tool made by any bytedance group. Up until now no colab or demo page ever worked same in this case.
2:24 a glitch spotted on her eyes!
it dosnt work
Dude how do I install locally? Would you consider doing a setup tutorial please?
I have some stuff but it's not stable diffusion or story diffusion as my graphics card popped. 😢 But I do have easy instructions on installing local models with any hardware ect.
I can't testify to the video stuff. However, I tried the "comic book" (think graphic novel) stuff on 2024-MAY-09. Here's the problem. The system ignores your "story beat" prompts and just generates five different versions of your "character" prompt. Now, that's not a bad thing because you can then take that into Photoshop and move everything around into place. However, this is not a one-stop shop for graphic novel generation. Second, if you say "speaking", the character is not speaking. If you say "knock on door", the photo looks like a selfie. If you choose "photographic" you'll still get comic book style. Also, it does not apply the images to a "page". So if you ask for a 1024x512 page, you'll get five 1024x512 panels and you'll have to build your own page in Photoshop.
Unlike a Stable Diffusion site that will give you four images per press of the button, this site will give you the same quantity of images but make sure the uniforms are consistent. It seems as if the system uses text-to-image to make a baseline and then runs that through an image-to-image to get a final result.
So far, there are a lot of promises but very few victories. We'll see.
I don’t think Sora AI is out to the public yet but we will see it’s great to have good competition though
I've tried it today and I would just say it's ok but far from what the demo has shown. It still suffers from the typical Stable Diffusion problems which is prompt understanding and complex scenes. In its current state, I don't think it is useable.
It is an improvement from what we had. Hopefully future updates could fix some of the obvious issues.
Thanks so much for this quick feedback. You saved us a lot!
how did you try it? users on reddit say there is no repo no weights, maybe there will no be release
@@artemedutech There is a huggingface demo you can try.
What are the vram requirements though?
GPU >20 GB
Wow this amazing! I have more faith independent developers more than Sora.
so when will there be a full movie Christopher Reeve Superman v.s. Bizarro?
Apakah wajah yang dibuat Sora mengambil dari real person?
Nice vid. You need to do this stuff for a living.
I'm waiting for the day when we can scan in the pages of Japanese manga and also load in a episode or two of the related anime series have ai figure out the characters the voices and the music and create all new episodes based on the manga.
It seems like this isn't actually open source yet (the video generating part)... Am I correct in this understanding?
This definitely seems to have only the comic generation part open source right now - the video generation scripts are not included. Same with the demo; comics only right now.
Hmmm, thus far it draws the same character ok, but does not draw even basic story panel ideas for my explorer on a strange planet comic, so like 'leaving ship', 'discovering ancient ruins', just give the same character in same pose with slightly different backgrounds.
That's because, like all AI, it's likely mostly a scam
what kind of computer do you need to run this at home?
GPU >20 GB
Impressive stuff! But it also frustrates me that some new A.I. papers present all their examples in a square video. No one wants square video's, they should give examples in 16:9. I'm hoping for a platform that combines different features, like adding the audio (speech and singing) from a file to a video. Everything is so scattered right now, technologies need to merge into a basic set of video and audio features. Hard to say what we'll all be able to generate in one year's time.
You can upscale the video and then crop the upscaled video in any video editor. Gpus work with power of two square grids. It's why textures in video games are all squares. The reason you can even have resolutions larger than 512 and smaller than 1024 is because the gpu can split the extra pixels into smaller power of 2 square grid quadrants. Ai diffusion can't do it all. It can do a lot of the ground work but if you want to make real money you need to learn to traverse from pure ai generated to ai assisted artwork. Where the ai generated work is a template for a human edited full piece.
Couldn't agree more!
Embarking on a journey of storytelling and video exploration, and VideoGPT seamlessly integrated into my process, amplifying the professionalism of my content.
Not bad at all!!!..
The only problem is that I have tried this thing several times and it always says _"Error."_
So, this doesn't count as an AI video generator to me until I can actually GENERATE some videos.
I _desperately_ hope it will never happen...
@@Zareh_Abrahamian Lol, deluded if you think that.
@@EnderViBrittania I said _desperately_ (in italics), seems you already need AI to chew for you.
2:25 scared the crap out of me
nice share
How do i download it and install it please/ I tried but it didn't work - There was no.exe file or anything to install in the folder i got and extracted from github.... how the heck does this work?
I only wish that the term *create* when using AI would be replaced by *having AI generate* whatever is labelled AI "art".
Actually, the bear isn't as consistent as you claim. Note his chest at 5:05-08 compared to 5:09 where his chest has a vanilla inset.
I desperately hope it stays that way.
An Advertising company can literally spend pennies with this breakthrough
Real question: Do you have access to Sora or Story Diffusion? If not, how are you drawing any of these conclusions?
I think they are just speculative conclusions.
I love you bro
Getting better and better. New storyboard less, actor less movies soon on the way.
- [00:00](ruclips.net/video/jZWRENqCl6I/видео.html) 🎥 Overview of Story Diffusion Model
- Introduces "Story Diffusion," an open-source AI video model.
- Highlights the model's ability to maintain character consistency and adhere to reality and physics in videos.
- Discusses the model's advancement in character consistency, including facial, clothing, and body type consistency.
- [01:03](ruclips.net/video/jZWRENqCl6I/видео.html) 🖼️ Creation of AI Comics using Story Diffusion
- Demonstrates the generation of AI Comics with consistent characters and movement.
- Explains the process of creating sequential images and animating them for comic generation.
- Discusses the potential of AI Comics production enabled by Story Diffusion.
- [02:56](ruclips.net/video/jZWRENqCl6I/видео.html) 🤔 Comparison with Existing AI Video Generators
- Contrasts the capabilities of Story Diffusion with other AI video generators.
- Highlights the realism, consistency, and fluidity achieved by Story Diffusion.
- Discusses limitations and minor imperfections observed in Story Diffusion's output.
- [07:11](ruclips.net/video/jZWRENqCl6I/видео.html) 💻 Accessibility and Usage of Story Diffusion
- Addresses the accessibility of Story Diffusion, available as open-source but lacking a usable interface.
- Describes options for accessing Story Diffusion, including downloading/installing or running online on a Cloud Server.
- Discusses the significance of Story Diffusion's multiple character inclusion feature for scene consistency.
- [09:45](ruclips.net/video/jZWRENqCl6I/видео.html) 🧠 Mechanisms Behind Story Diffusion Model
- Explains the technical mechanisms employed by Story Diffusion, including consistent self-attention and story splitting.
- Describes how consistent self-attention ensures visual coherence across generated images.
- Discusses the process of story splitting and motion prediction in creating narrative sequences.
- [14:48](ruclips.net/video/jZWRENqCl6I/видео.html) 📈 Evolution of AI Video Generation
- Reflects on the advancements in AI video generation enabled by Story Diffusion.
- Emphasizes the potential for creating realistic and cohesive scenes with AI-generated videos.
- Encourages exploration of Story Diffusion's capabilities and contributions to the field of AI video production.
Thank you sir! Very helpful!!
@@OpenEscapeGaming you are very welcome.
It is crazy
Bro you made so much ai video software video I don't know which one to watch
So is it available for use now or??
... all of the physics and context consistency issues gets solved by an inevitable linkup between something like unreal engine and AI ... to a large degree these crude iterations aren't worth getting into too deeply because a year from now they'll be seen as something like mid-journey 1 ...
My heart is breaking for visual artists, genuinely. I used to wish for the advancement of technology, now I regret for what I wished. I'm super impressed with all of this, but genuinely, I say sincerely, this scared me and fills me with existential dread. It seems like we're gaining something amazing, but deep down I know we're losing something we won't get back. God help us all.
Awesome :)
Thanks 😁
why is so hard for AI to keep the person same through out the whole video ? It's like it's not working on same method as other video creation softwares
It doesn't work
6:09 Sora still looks at least one generation farther .... but beeing open Source makes Story Diffusion very very interesting!
what about music?
Guys plz inform about tool which r coming soon, not like Sora and LTX, which God only knows when they gona released
In the demo clip with the black girl and headphones her necklace disappears as she moves.
There's still hope (however tiny) that this monster will never wake.
Google VEO Is the king
What is this pleasure. Could it be made. Digital Renaissance here in my stay.
I am so happy. My Ai-Asian isekai desire in this reality.
Phenomenal
Until they add mouths that move with voice (either within or addon or after effect), there won't be movies as we understand them.
how can we earn from this skill?
SORA KILLER!
trash channel.
Doesnt work… 😢
The Demo seems to be cooked at this point, no matter what I do it results in an error.
Ok so it's not just me then.
thanks for the vid, website is down/overload by now :D
What about the cannabis leaf in the middle of the book lol😂
say what?
@@aisamsonreal 8:21 look in the center of the comic strip, there is a book open, with a cannabis leaf coming out of it 😅
Is there censorship of fight scenes and figures holding weapons
If it gets any better just like Sora, the public will never get access to it. Look how long Sora has been out and still to this day it's Not available to the public!
this would be much cooler if it actually worked...
I desperately hope it will fail...