Real time Stable Diffusion in TouchDesigner

Поделиться
HTML-код
  • Опубликовано: 30 сен 2024

Комментарии • 94

  • @oakland439
    @oakland439 27 дней назад +2

    does this work with MacOS Silicone?

    • @touchdesigner2967
      @touchdesigner2967 12 дней назад

      I was wondering the same thing...

    • @lematchafrappe
      @lematchafrappe 10 дней назад

      I was able to run streamdiffusion by it self, works great

    • @lematchafrappe
      @lematchafrappe 9 дней назад +2

      Okay. So basically this utilizes a tensorrt engine to render in Touchdesinger. This needs an Nvidia card so no for a mac.

  • @jeremietoussaint4085
    @jeremietoussaint4085 20 дней назад +2

    I got this error when i click on Make Engine "UnboundLocalError: cannot access local variable 'pipe' where it is not associated with a value"

  • @천재영-l1f
    @천재영-l1f 3 месяца назад +1

    Nevertheless I got finished engine preparation and selected the file which webui includes, it doesnt work.. I tried to press load engine and refresh engine several times, but it dosent work.. how to resolve this :(

  • @zukabaazov8014
    @zukabaazov8014 3 дня назад

    where i prerating models?

  • @MohamedAli-nb9qk
    @MohamedAli-nb9qk Месяц назад +1

    THIS video talks about the portable version and NOT the manual installation. If you try following this installation for the manual installation, you are going to face errors related to models and engines not being found, for example "cannot access local variable 'pipe' where it is not associated with a value", because the paths for the portable version and the manual version are slightly different. That's why I recommend using the portable version

    • @grayfover
      @grayfover Месяц назад

      Hi, I've downloaded the portable version when i use the webui.bat, should I still set the python location to my local file? thanks

  • @prictor
    @prictor 3 месяца назад +2

    hi, i followed every step and there are no erros, but in touchdesigner when i hit pulse on "load engine" nothing happens, can anybody help me?

    • @VJSCHOOL
      @VJSCHOOL  3 месяца назад

      Create issue on GitHub or Discord and share full log.

    • @m.s.kostin
      @m.s.kostin 7 дней назад

      Did you find the solution? Same issue

  • @burakuville
    @burakuville 3 месяца назад +2

    amazing job explaining the entire process...

  • @alessandrosaracino5463
    @alessandrosaracino5463 3 месяца назад +2

    this is great! lean and clean, thank you!

  • @王梓
    @王梓 Месяц назад

    excuse me, i follow every step you said, but when i click the webui button, it was going to fail, would you tell me why?
    here are the flows:
    Downloading StreamDiffusion...
    Cloning into 'StreamDiffusion'...
    remote: Enumerating objects: 2735, done.
    remote: Counting objects: 100% (1115/1115), done.
    remote: Compressing objects: 100% (294/294), done.
    error: RPC failed; curl 92 HTTP/2 stream 5 was not closed cleanly: CANCEL (err 8)
    error: 7789 bytes of body are still expected
    fetch-pack: unexpected disconnect while reading sideband packet
    fatal: early EOF
    fatal: fetch-pack: invalid index-pack output
    Failed to download StreamDiffusion

  • @Remvisuals
    @Remvisuals 3 месяца назад +2

    instnaly fast and so nice to be in 512, congrats on this one, cant wait for more features!

    • @VJSCHOOL
      @VJSCHOOL  3 месяца назад +1

      Custom resolutions already available, but keep in mind, that it will affect FPS.

    • @Remvisuals
      @Remvisuals 3 месяца назад

      @@VJSCHOOL I tried building a model with different res and more steps, but it wouldn't finish...
      Super impressed with the speed, it's still super jittery and "stabley", but that's just animdif in general I geuss, excited to try out control net!!

  • @channelbolabaru
    @channelbolabaru Месяц назад

    OSError: [WinError 126] The specified module could not be found. Error loading "C:\Users\PC ABU-ABU\Desktop\TouchDiffusion-main\TouchDiffusion-main\.venv\Lib\site-packages\torch\lib\fbgemm.dll" or one of its dependencies.
    No launch file found
    please help

  • @eurria
    @eurria Месяц назад

    Hello, first of all congratulations on the tremendous work
    I have been using the DSD patch for a long time without any problems, but now I have a constant error that I cannot solve
    As soon as I open the project it tells me that I have a problem with the line 4 of the TouchDiffusionExt operator that does not find the Torch module within the project...

  • @arshinazizi
    @arshinazizi 18 дней назад

    Thanks for sharing your knowledge , i appreciate it 🎉

  • @lematchafrappe
    @lematchafrappe 10 дней назад

    Thanks for making this free, much love

  • @pocong9867
    @pocong9867 Месяц назад

    I have tried it.. but say try experimenting with my video camera will it not work?

  • @hppdms
    @hppdms 2 месяца назад

    Thank you ! I made it work but it just creates the image only when i press load engine button and its a stable image. How can i make it live =?

  • @tadaizm
    @tadaizm 22 дня назад

    норм

  • @GIANNIANER
    @GIANNIANER Месяц назад

    thank you so much! great tutorial. is it possible to use other kind of
    models?

  • @苏传宇-o8q
    @苏传宇-o8q 3 месяца назад +1

    Very interesting, but when I clicked load engine in touchdesigner, I got an error like "cannot access local variable 'pipe' where it is not associated with a value". What is the problem?

    • @VJSCHOOL
      @VJSCHOOL  3 месяца назад

      That’s mean either model or engine can’t be found

    • @yanyu813
      @yanyu813 2 месяца назад

      Have you solved it?

    • @VJSCHOOL
      @VJSCHOOL  2 месяца назад

      @@yanyu813 download portable version on Github and try with it.

    • @StanislavDrca
      @StanislavDrca 14 дней назад

      @@VJSCHOOL Tried the portable as well as the manual version. Same error. It may be because Runway shut down their SD1.5 repositories on huggingface?
      Posted the full log on GitHub Issue.
      Thanks!

  • @MaxiCrows
    @MaxiCrows Месяц назад

    its posible with 8gvram?

  • @MichelleCarrasco
    @MichelleCarrasco 2 месяца назад

    Hellllooooo, I just arrive untill 4:05 and this happen : NameError: name 'StreamDiffusionWrapper' is not defined
    Any idea how to fix this ?

    • @VJSCHOOL
      @VJSCHOOL  2 месяца назад

      There is portable version on GitHub, better to try with it

  • @pocong9867
    @pocong9867 Месяц назад

    HOW MUCH FPS YOU GET MY FRIEND

  • @TheDciel1
    @TheDciel1 3 месяца назад

    Git is installed.
    Creating .venv directory...
    The filename, directory name, or volume label syntax is incorrect.
    Failed to create virtual environment.
    What is the problem?

    • @ogunkayikci7751
      @ogunkayikci7751 3 месяца назад

      chech where the python with "where python" command and copy paste in to webui.bat like he said

  • @mahmoud0800
    @mahmoud0800 2 месяца назад

    thats amazing but i got an ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (tensorrt) please help!

    • @mahmoud0800
      @mahmoud0800 2 месяца назад

      "No module named 'tensorrt'"

  • @marlus
    @marlus 3 месяца назад

    Thanks for this great solution! Amazing performance on 4070 12GB! SD Turbo and 1 sample step. I will test making other engine now. 3 sample steps were slow on 4070.

  • @SatSitSat
    @SatSitSat 3 месяца назад

    I don't understand how it's posssible to have multiple frames per second with 20 steps. Why don't we use TensorRT on normal UI's like Automatic or Comfy ? Sorry in advance for having such a vague question but i find this quite amazing haha.

    • @VJSCHOOL
      @VJSCHOOL  3 месяца назад

      Real-time can be achieved with Turbo models and Acceleration Lora, on low steps like 1-4 with TensorRT. TouchDesigner allow to copy data from and to GPU, that decrease latency.

  • @michaelkollar6299
    @michaelkollar6299 2 месяца назад

    the webui.bat did not create any folders (venv, streamdiffusion, models, engines) but tells me they already exist after relaunching, what am I doing wrong?

    • @dylanroscover939
      @dylanroscover939 Месяц назад

      the .venv folder is hidden so make sure you have Show Hidden Folders turned on in Windows Explorer. It didn't create the models/engines directories for me either, I just copied and pasted them manually from the StreamDiffusion folder once it was generated and that did the trick.

  • @detailsstudio
    @detailsstudio 3 месяца назад +1

    Thanks bro

  • @dleon9781
    @dleon9781 3 месяца назад

    hi ,i have followed your tutorial and every step went well .But in TouchDesigner,the log shows :cannot access local variable 'pipe' where it is not associated with a value.I don't know how did it happened

    • @VJSCHOOL
      @VJSCHOOL  3 месяца назад

      Create issue on GitHub or Discord with full log

  • @leonidkrykhtin
    @leonidkrykhtin 3 месяца назад

    How to increase CFG to 7-10 instead of 1?
    The quality of generations is very low at 1.

    • @VJSCHOOL
      @VJSCHOOL  3 месяца назад

      You should try to increase sampling steps (batch size)

  • @placebo_yue
    @placebo_yue 3 месяца назад

    does it have to be python 311? i have 310 and i don't wanna break all my other installations to try this...

    • @VJSCHOOL
      @VJSCHOOL  3 месяца назад

      1) everything installed in venv, so it not affecting main Python env.
      2) if you want to try with Python 3.10, then provide path to it in webui & use TouchDesigner version before 2023

  • @cayetanojozami4067
    @cayetanojozami4067 3 месяца назад +1

    Amazing

  • @alexandraganiari706
    @alexandraganiari706 2 месяца назад

    I get this error "Error installing packages: torch==2.1.0, Failed to install TensorRT" during the installation process. Any ideas how to fix this?

    • @VJSCHOOL
      @VJSCHOOL  2 месяца назад

      Try with portable version

    • @alexandraganiari706
      @alexandraganiari706 2 месяца назад

      @@VJSCHOOL I fixed this error but again, when I click on the "make engine" button, It does not work and a simple "error" appears without any info about it.

  • @kafidov
    @kafidov 3 месяца назад

    As I understand it, this miracle cannot be run under Mac?

    • @VJSCHOOL
      @VJSCHOOL  3 месяца назад

      Only PC & Windows

  • @alexdoho
    @alexdoho 3 месяца назад

    What's the difference between this and streamdiffusion?

    • @VJSCHOOL
      @VJSCHOOL  3 месяца назад +1

      Explained in first 3 min of video
      It’s near x2 faster

  • @N92601
    @N92601 2 месяца назад

    amazing fast speeds would you know why using my null of the render tops of my td projects as input isnt working? im getting flashing white and black with some cat images but noise top works normally, denoising value dosent change this would we have to repeat the webui model proceses with higher steps seems like it could be that or a different problem if so how should i do that thanks!

    • @N92601
      @N92601 2 месяца назад

      seems the same problem with a default moveifileintop

    • @N92601
      @N92601 2 месяца назад

      I figured out I had to reinitalize pulse touchddiffusion folder and possibly clear log too I did both then reload engine

  • @developmentmotion4765
    @developmentmotion4765 3 месяца назад

    amazing... this is realy faster,
    can we put lora and vae to the folder?
    what lora and vae for the best creation

    • @developmentmotion4765
      @developmentmotion4765 3 месяца назад

      and why icant change the seed ?

    • @VJSCHOOL
      @VJSCHOOL  3 месяца назад

      Tiny VAE already baked in for best performance. Loras not supported yet.

    • @VJSCHOOL
      @VJSCHOOL  3 месяца назад

      Seed can be changed in TouchDesigner with seed parameter

  • @ed6280
    @ed6280 2 месяца назад

    anybody test 4080 laptop fps?

    • @VJSCHOOL
      @VJSCHOOL  2 месяца назад

      4070 Laptop - 24 FPS with SD Turbo 512x512

  • @DenizKurt-q1k
    @DenizKurt-q1k 2 месяца назад

    Hi! Does it also work with Mac?

    • @VJSCHOOL
      @VJSCHOOL  2 месяца назад

      Only NVIDIA GPU

  • @th3curator
    @th3curator 2 месяца назад

    I am getting an error: name 'StreamDiffusionWrapper' is not defined. Any ideas?

    • @carloslavina9779
      @carloslavina9779 2 месяца назад

      same over here :( did you solve it?

    • @VJSCHOOL
      @VJSCHOOL  2 месяца назад

      Try portable version

    • @alexandraganiari706
      @alexandraganiari706 Месяц назад

      any progress with portable version? I can not download it

  • @PeterBowlesify
    @PeterBowlesify 3 месяца назад

    Cool! Can it work with controlnet?

    • @VJSCHOOL
      @VJSCHOOL  2 месяца назад

      No support for now

  • @OrpheusMusicOffice
    @OrpheusMusicOffice 2 месяца назад

    how about mac~ pls❤

    • @VJSCHOOL
      @VJSCHOOL  2 месяца назад

      Only NVIDIA GPU

  • @SwoleKitchen
    @SwoleKitchen 2 месяца назад

    Can you do this with AMD Ryzen Somehow?

    • @VJSCHOOL
      @VJSCHOOL  2 месяца назад

      For now there is no way

    • @SwoleKitchen
      @SwoleKitchen 2 месяца назад +1

      @@VJSCHOOL yeah I got it to run stable diffusion but it was quite slow, and then I couldn’t figure out how to put it in touch designer, I’ll just bite the bullet return this computer and get one with an nvidia videographics card

    • @Tanya_Mist
      @Tanya_Mist 2 месяца назад

      @@SwoleKitchen yep 😄

  • @SwoleKitchen
    @SwoleKitchen 3 месяца назад

    I really need a computer to do this. is there a way I could offload that GPU onto Google Collab?

    • @VJSCHOOL
      @VJSCHOOL  3 месяца назад

      TouchDiffusion component required TouchDesigner, so it can’t run in Google Collab. You can try with original StreamDiffusion repo.

    • @SwoleKitchen
      @SwoleKitchen 3 месяца назад

      @@VJSCHOOL - Sigh guess i just need better hardware.

    • @SwoleKitchen
      @SwoleKitchen 3 месяца назад

      @@VJSCHOOLsweet I’ll try that

    • @SwoleKitchen
      @SwoleKitchen Месяц назад

      @@VJSCHOOL I got a new computer with a 3060 Ti =D

  • @galaxyfrog-dot-org
    @galaxyfrog-dot-org 2 месяца назад

    Faster than stream diffusion?