dotsimulate
dotsimulate
  • Видео 12
  • Просмотров 139 866
LiveTranscribe for TouchDesigner [ new TOX + setup ]
To access the operator, visit www.patreon.com/dotsimulate
Links from the video + info:
Mac:
How to install Homebrew on Mac - docs.brew.sh/Installation / video ruclips.net/video/IWJKRmFLn-g/видео.html
Once installed, run this in terminal:brew install python@3.11
Windows:
Cuda 11.8 or cuda 12.1 installed.
CUDNN Archive - developer.nvidia.com/rdp/cudnn-archive
( More info on CUDNN - docs.nvidia.com/deeplearning/cudnn/latest/installation/windows.html )
Python 3.11.9 (windows) - www.python.org/downloads/release/python-3119/
Git for windows - git-scm.com/download/win
AssemblyAI - www.assemblyai.com/
OpenAI's Whisper original release - openai.com/index/whisper/
Real time whisper:
github.com/collabora/Whisper...
Просмотров: 834

Видео

ControlNet in TouchDesigner - StreamDiffusionTD 0.2.2
Просмотров 4,4 тыс.21 день назад
Install / Setup Video - ruclips.net/video/F2JSWAKex4k/видео.html StreamDiffusionTD TOX available here - www.patreon.com/dotsimulate 🟣 Discord link can be found in the About page of the operator. Thank you for your support. 0:00 - Intro 0:44 - What is ControlNet? 2:08 - Types of ControlNet models 4:25 - DotUpdater [ new ] 5:45 - ControlNet settings in StreamDiffusionTD 7:55 - Demonstrating edge ...
StreamDiffusionTD 0.2.0 [ Installation + Setup ]
Просмотров 6 тыс.Месяц назад
StreamDiffusionTD TOX available here - www.patreon.com/dotsimulate Helpful Hints: CMD window disappearing instantly? - In Settings 2, there is a Debug toggle. If it is due to an import error, then try hitting Install 2 pulse again and look for red or yellow text as it checks/installs the libraries. 🟣 Discord link can be found in the About page of the operator. Thank you for your support. This o...
ChatTD - Setup / Intro
Просмотров 2,3 тыс.4 месяца назад
This video was posted in March but I figured there is no reason to have it unlisted. ChatTD is currently available on my patreon for the 'DotExplorer' tier. To access my TouchDesigner tools and our community for feedback, support, and more, join on Patreon - www.patreon.com/dotsimulate Thank you to all of my supporters for making it possible for me to focus more on building workflows/tools for ...
ComfyTD Deep Dive
Просмотров 15 тыс.7 месяцев назад
ComfyTD operator available here - www.patreon.com/dotsimulate This video is a thorough walkthrough of ComfyTD, detailing my entire approach to working with the operator for integrating ComfyUI workfows into TouchDesigner. It is less of a step by step guide and more of a masterclass for how to approach using the operator. I hope you enjoy. Thank you for your support. 00:00 - Video Intro 01:05 - ...
Real-time diffusion in TouchDesigner - StreamdiffusionTD Setup + Install + Settings
Просмотров 85 тыс.8 месяцев назад
Important ! Read description for Installation Tips ! This uses TouchDesigner 2023 and will not work in TouchDesigner 2022. Currently only working using windows nvidia graphics card. StreamDiffusionTD TOX available here -www.patreon.com/dotsimulate StreamDiffusion github - github.com/cumulo-autumn/StreamDiffusion BEFORE Installing - make sure to have: Python 3.10 (make sure Python is added to Pa...
Never Before Seen NYC Rat Documentary
Просмотров 1,3 тыс.Год назад
A short documentary about rats. Videos are AI generated with the Modelscope base model with a custom Touchdesigner GUI. Voice is made with fakeyou.com. Script written by chatgpt and me. Upscaled with Topaz Edited with DaVinci Resolve For access to many Touchdesigner tools for building systems with generative AI - www.patreon.com/dotsimulate
Text2video Looks - Zeroscope XL
Просмотров 20 тыс.Год назад
The models used to create these clips are: huggingface.co/cerspense/zeroscope_v2_567w huggingface.co/cerspense/zeroscope_v2_dark_30x448x256 huggingface.co/cerspense/zeroscope_v2_XL Each of these clips were generated at a lower resolution and then upscaled using the XL model with vid2vid. clips were generated using the text2video extension API of automatic1111 - github.com/kabachuha/sd-webui-tex...
Musicgen Experiments (1 Hour of AI Generated Music)
Просмотров 1,2 тыс.Год назад
Several Musicgen / Audiocraft outputs all generated and visualized in Touchdesigner. The MusicGen operator and visualizer are available via my Patreon: patreon.com/dotsimulate
Flux + Form: Concrete Coral - Ambient AI Generated Animations
Просмотров 2,2 тыс.Год назад
A generative animation created using Touchdesigner and Stable Diffusion (automatic1111). Inspired by recursive growth experiments in Houdini. To get early access to my custom tools / interactive approach to creating and developing with Stable Diffusion, join my Patreon. Patreon: www.patreon.com/user?u=80175670 #stablediffusion #touchdesigner #automatic1111 #ai

Комментарии

  • @yvann.mp4
    @yvann.mp4 4 часа назад

    thanks !!

  • @yvann.mp4
    @yvann.mp4 3 дня назад

    amazing, video thanks !!

  • @indrasmirror
    @indrasmirror 4 дня назад

    This would be cool in conjunction with Streamdiffusion and controlnet, like if there is a way to display only the most current sentence or words. Could see some cool use cases with that :)

  • @fe3elix
    @fe3elix 5 дней назад

    I am having trouble starting the server. It just loops the lines INFO:root:Sent server ready INFO:root:self.running.is_set(): True

    • @dotsimulate
      @dotsimulate 5 дней назад

      @@fe3elix make sure to set the audio device or plug in a chip with audio data as well. The print outs aren’t ideal for whisper but it should change once the audio data is being sent to the server.

    • @fe3elix
      @fe3elix 4 дня назад

      @@dotsimulate tried both to no result, after looking into the tox it seems the script chop is bypassing the audio input somehow! will try setting it up again once i get back home

    • @dotsimulate
      @dotsimulate 3 дня назад

      @@fe3elix if possible, send me a screenshot or recording on discord so I can see what you mean. I think I might have an idea

  • @emmaforgues2419
    @emmaforgues2419 5 дней назад

    Thanks for your work! It's so good. I have an Alienware m15 r7a with: RTX 3080 gpu, 12th Gen Intel(R) Core(TM) i7 2.30 GHz, 32G RAM and a x64-based processor Do you think she can handle it? Video out needs to run at 60 fps and be 1080x1920. What do you think? Thank u so much <3

  • @Flor-ly5cg
    @Flor-ly5cg 7 дней назад

    Hi, thank you for this fantastic tutorial and new SD 2.0. I find that ingeneral are very low quality images specially when representing people. Do you have a tutorial for getting more resolution, and better faces, people in general?. Some of my friends said missing Triton causes the generate result to be blurry at sometimes.

  • @YongMumu
    @YongMumu 7 дней назад

    The version of Python is difficult to adjust, can you explain it in detail, I downloaded your example but it can't run and it shows 3.7 > 3.8 error rollout. I'm running 3.7 now, I tried to download 3.11, changed the environment variables, queried the version to 3.11 but continued to run it still with the same error.

    • @dotsimulate
      @dotsimulate 7 дней назад

      I’m not sure what you mean by version of python is difficult to adjust. For recent version of TD, it uses python 3.11 instead of 3.7. If you have python 3.11.9 installed (does need to be separate from TD python), then when you load the operator it should find that installation and use that to create the environment and install the libraries. I saw your other message on discord as well and followed up there too

    • @YongMumu
      @YongMumu 6 дней назад

      @@dotsimulate I don't know why the virtual environment where 3.7 is installed is never changed. I removed the files and clicked them out, and I was able to download 3.11 on the load when I entered it

  • @jkoolish
    @jkoolish 9 дней назад

    Hi, when I hit "start stream" I get this error in my terminal window: raise LocalEntryNotFoundError( huggingface_hub.errors.LocalEntryNotFoundError: Cannot find an appropriate cached snapshot folder for the specified revision on the local disk and outgoing traffic has been disabled. To enable repo look-ups and downloads online, pass 'local_files_only=False' as input.

  • @neilhilken
    @neilhilken 11 дней назад

    Incredible, thank you so much!

  • @hggjihfrt
    @hggjihfrt 13 дней назад

    hi, curious about where can see your live portrait content appearin in te end of this video:)

    • @dotsimulate
      @dotsimulate 13 дней назад

      @@hggjihfrt yes! Working on making that more functional + TD friendly. It’s working nice and fast but i want to switch out insightface model for mediapipe and drive it with Torin’s mediapipe TD tox. Hopping to find the time to finish that soon.

    • @hggjihfrt
      @hggjihfrt 11 дней назад

      @@dotsimulate wowww!! very excited about that!! hope can see it soon!!!

  • @hggjihfrt
    @hggjihfrt 13 дней назад

    you are great...!

  • @ShahanButt-n8q
    @ShahanButt-n8q 13 дней назад

    with this, will be able to input live feed from a camera and generate an output?

  • @Pleunpleun
    @Pleunpleun 14 дней назад

    Thank you so much! The work you do is heavily appreciated

  • @ibrahim47x
    @ibrahim47x 16 дней назад

    literally pioneering the future of event entertainment

  • @ashleighhewitt3399
    @ashleighhewitt3399 17 дней назад

    Will this ever be available for AMD graphic card users? :)

  • @33JBC
    @33JBC 18 дней назад

    How do you access the prompt located at 9:44? I can't get it to change to "reference" ruclips.net/video/F2JSWAKex4k/видео.htmlsi=upI5D-ytOoGG-5Jl&t=584

    • @Flor-ly5cg
      @Flor-ly5cg 6 дней назад

      use: op('prompt').par.text and make sure you rename the text operator "prompt"

    • @dotsimulate
      @dotsimulate 3 часа назад

      @@33JBC seems like the edit might have skipped showing that! You want to click and drag the text par from the text1 to the prompt parameter. The final reference on the parameter should be like op(‘text1’).par.text

  • @dleon9781
    @dleon9781 18 дней назад

    Amazing!

  • @bonvoyagevisualmapping749
    @bonvoyagevisualmapping749 18 дней назад

    How can I increase fps to improve flow?

  • @小Q-k7l
    @小Q-k7l 20 дней назад

    视觉vj中国

  • @hggjihfrt
    @hggjihfrt 21 день назад

    i feel confused, text2 image works for me, but image2image it turns black...

  • @cecisoloaga
    @cecisoloaga 21 день назад

    thank you!! this is amazing!!!!! :)

  • @DenizKurt-q1k
    @DenizKurt-q1k 21 день назад

    Hi! Does this work for Mac? I'm still waiting on that :') Thank you! <3

  • @__pawpaw__
    @__pawpaw__ 23 дня назад

    Any chance you'd share your pipeline python code? Would love to see how you're connecting all these TD controls to StreamDiffusion. Thanks for this!

    • @dotsimulate
      @dotsimulate 3 часа назад

      @@__pawpaw__ yes! All of it is inside the operator and can be explored. Most of the connection between python server and TD is shared memory and osc.

  • @ZixiZhang-p5l
    @ZixiZhang-p5l 23 дня назад

    Impressive! Does this new version support the lora model like 0.1?

    • @dotsimulate
      @dotsimulate 21 день назад

      Yep! And you can swap models / change the weight on the fly (with a slight delay when loading models)

    • @ZixiZhang-p5l
      @ZixiZhang-p5l 20 дней назад

      @@dotsimulateThx a lot! I've successfully done the model-swapping, the result is great!

  • @ewowoi
    @ewowoi 24 дня назад

    we Indonesian, have the worst time since we a lot of us can't be patreon due to broken banking software. i want to support dotsimulate and download the StreamDiffusionTD

    • @dotsimulate
      @dotsimulate 21 день назад

      hey, shoot me a dm on instagram !

    • @ewowoi
      @ewowoi 21 день назад

      @@dotsimulate already did !

    • @ewowoi
      @ewowoi 21 день назад

      @@dotsimulate already did with the same profile pic

  • @adrianmenaceri889
    @adrianmenaceri889 24 дня назад

    Thanks for sharing your knowledge with the community, you're a legend!

  • @cabelnam
    @cabelnam 25 дней назад

    was looking forward to an early update)) thank you for it!

  • @nime1575
    @nime1575 25 дней назад

    amazing work! The amazing art this implementation enables is endless.

  • @MikeFlex
    @MikeFlex 25 дней назад

    absolute madlad

  • @veryVANYA
    @veryVANYA 25 дней назад

    lfgooooooooo

  • @RetropunkAI
    @RetropunkAI 25 дней назад

    this might be THE BEST description of how ControlNet works...LFG

  • @PurzBeats
    @PurzBeats 25 дней назад

    WICKED

  • @robindelange
    @robindelange 25 дней назад

    Great, a tutorial! We're working on a live AI installation using your awesome TD work at this very moment! ControlNet and V2V batching seems to make a huge improvement

  • @pikimimmi
    @pikimimmi 25 дней назад

    does this work on mac

  • @mariaisabellisita8673
    @mariaisabellisita8673 26 дней назад

    hey Dot, does it have an AMD GPU version? where can I skip cuda test? it is necessary?

  • @chriswaters1480
    @chriswaters1480 28 дней назад

    Incredible work & such a useable setup, thanks for sharing! I am running into this error alot when trying to experiment with other resolutions / ratios: "ValueError: cannot reshape array of size 786432 into shape (720,1280,3)". Any advice greatly appreciated

  • @rajnaikgaonkar3910
    @rajnaikgaonkar3910 28 дней назад

    Is there any other way to buy the tox? Maybe yt membership? I'm from India and due to some rules patreon keeps declining all my payment methods.🥲 Tried 4 different cards, even paypal don't work.

  • @rajnaikgaonkar3910
    @rajnaikgaonkar3910 Месяц назад

    Can we get a controlnet tutorial next 🙏

    • @dotsimulate
      @dotsimulate Месяц назад

      @@rajnaikgaonkar3910 yes!

  • @TheodoreDu
    @TheodoreDu Месяц назад

    Hi, is that possible to run stream diffusion on GCP? cuz I see it's so slow to run it on Mac

  • @mamanche
    @mamanche Месяц назад

    Superb improvements! would you mind helping your mac users with an installation guide for mac, please? I've downloaded the TOX but I'm unable to make it run in my M1 Max MBP. Thanks in advance :D

  • @shellyzheng2674
    @shellyzheng2674 Месяц назад

    Hiii, thank you very much for the tutorial, I would like to ask is there any tutorial for mac streamdiffusion?

  • @王梓
    @王梓 Месяц назад

    excuse me when i clicked the "download streamdiffusion", it's said that no internet connection detected, but all of those wlan settings are correct, could you help me? i can't go to the next step due to that

    • @edeechen2689
      @edeechen2689 29 дней назад

      Hi I've got the same problem here. It is said that if you are in china mainland(forgive my presumption), you can set your vpn to global mode and try again. (it didn't work for me though😥)

  • @Flor-ly5cg
    @Flor-ly5cg Месяц назад

    It works great but the resolution and details are very poor, you don't get good faces

  • @dleon9781
    @dleon9781 Месяц назад

    super awsome!

  • @Saif-lv8cl
    @Saif-lv8cl Месяц назад

    Hi, I cant download the StreamDiffusionTD TOX, it says I have to upgrade my subscription

    • @dotsimulate
      @dotsimulate Месяц назад

      I just double checked. Should be available for all paid tiers.

    • @Saif-lv8cl
      @Saif-lv8cl Месяц назад

      @dotsimulate thanks, because I'm not a member. I thought it's free

  • @Ripplinghub
    @Ripplinghub Месяц назад

    Does this restrict the resolution under 516? Because I can not add a bigger output.

    • @augustineleudar
      @augustineleudar Месяц назад

      you can do bigger outputs of you don’t use tensorrt etc

  • @WhySoBroke
    @WhySoBroke Месяц назад

    MIDI support would be epic!!,

    • @augustineleudar
      @augustineleudar Месяц назад

      you can do that already in touch and just connect whichever parameter you like in sd

  • @spiritform111
    @spiritform111 Месяц назад

    cool... this is the error i get tho, after click on start stream. OSError: Error no file named pytorch_model.bin, model.safetensors, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory C:\Users\babee\.cache\huggingface\hub\models--stabilityai--sd-turbo\snapshots\b261bac6fd2cf515557d5d0707481eafa0485ec2\text_encoder. Failed to load model due to: Error no file named pytorch_model.bin, model.safetensors, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory C:\Users\babee\.cache\huggingface\hub\models--stabilityai--sd-turbo\snapshots\b261bac6fd2cf515557d5d0707481eafa0485ec2\text_encoder. the set hugging face cache is on... do i need to manually download the models?

    • @spiritform111
      @spiritform111 Месяц назад

      following the exact video instructions.

    • @spiritform111
      @spiritform111 Месяц назад

      i think i may have figured it out... hugging face model downloader. missed that step!

    • @spiritform111
      @spiritform111 Месяц назад

      nope.

    • @dotsimulate
      @dotsimulate Месяц назад

      @@spiritform111 It is easiest to look at these issues on discord. I've got a forum setup for you to post screenshots or error messages. I'm happy to follow up there if you make a post.

    • @spiritform111
      @spiritform111 Месяц назад

      @@dotsimulate ok, thank you!

  • @turtlefbi
    @turtlefbi Месяц назад

    Love where this is going!

  • @ibrahim47x
    @ibrahim47x Месяц назад

    you’re an angel in human form, Dot. Thank you!!!