Thanks for the plugin! For anyone having issues with the MediaPipe tox loading forever, the last version of 2022 (.35320) works for me on an M1 mac. I just avoided having to upgrade my commercial license for another $300 over this.
wow! I just get to know about MediaPipe - thank you so much for creating this and making this tools more accessible for individuals like me who have visual ideas but are not as technical as one can be.
Thanks for sharing such a helpful video! I never thought TouchDesigner could do this, and it's truly inspiring. Your video also has shown exactly what I've been looking for - how to superimpose TouchDesigner output to OBS.. Your effort is greatly appreciated!
Same. was having issues with spoutcam. it worked briefly at first but with a strange aspect ratio. then was showing noise. opened up the spoutpanel and spoutsettings and just randomly tinkered around until things got going
When looking for channel names(like around @20:37 & earlier) you can also just add a SELECT and click the black triangle next to "Channel Name" for a full list.
Hi, I hope you can help me. I just download the mediapipe plugin, so I open the main project MediaPipe TouchDesigner.toe then I Select my main camera and it doesn't show any image, but when i use a videodevice_in it automatically use my main camera. Therefore almost every subsequent node show me some kind of error Im using a macbook air M1, so, the main camera is a facetime camera. Also, when I drag and drop some of the .tox at an empty Touchdesigner project it just doesnt drop at the workspace. Any idea if Im the one thats doing something wrong or someting else happening?
I tried to do the obs set up but for some reason my virtual cam on the media pipe node gives me a black screen. I have tried various resolutions in obs preferences, the input video, my camera's resolution, and 1280x720 but it isn't working. Any recommendations?
This has been an amazing plugin to work with! I was wondering if there's a simple way to get the finger point tracking coordinates to reset to the centre of the screen when they are not visible? Currently they go to 0,0 in the bottom left so it constantly pulls anything I have mapped to it to this point if it can't see the intended finger. Thanks!
How is it that you are able to have both the mediapipe node and video device in run at the same time? I can only ever have one working at a time because my webcam doesn't like it when multiple things try to use it at once. Is there any workaround?
Fantastic plugin-I think I can pack up my old Leap Motion Controllers now. Question: Is there a way to permanently have h1 be the left hand/h2 as the right? My experience now means that whichever hand is detected first becomes h1, regardless of whether it's my left or right hand. Specifically referring to the normalized data from Output 1 in the hand tracking component. Appreciate any advice on this, as well as the awesome tool.
Looks amazing, but i could not get it to work, i get an error that resolutions is up to 720p. Tried putting the resolution down but still no working. Working on a MAc pro with sonoma TD 2022.33910
what is the best way to use the face mask and replace it with a custom face model, IE make the face track onto a scary ghost or something fun instead of the base White mask. Trying to figure out the best way
Hi guys, first of all I wanted to thank you for this great tool! I am working on a project where, using my hands, I reconstruct images as if I were painting them and I wanted to ask you a question. Is it possible to remove the neck part and keep only the hand part when working using output 1 of the image segmentation 21:33 (the body skin colour one)? In the preview both hands, and part of the arms, are coloured pink/purple but as all the skin is detected the neck, which always remains uncovered, is also detected, ruining the effect. Thanks again and congratulations on everything!
Hey, this is a super great plugin. I have a question though, There's no way to keep a consistent left hand and right hand. Everytime I remove one hand from the frame, the remaining hand becomes Hand_1. This makes it impossible for me to create a controller using 2 hands :(. Does this have an easy fix or is this functionality coming in the future? :)
thank you for your work it's real fun to use this and a lot of possibilities. I have one question: would it be possible to use the mediapipe plugin in td with an Infrared webcam?
Love the plugin! For some reason the latest version (0.32) shows the facePoints chop as an x,y,z chop (with green, yellow and red) as opposed to the time-slice style chop in the video. Any idea how to convert to the latter? Thanks!
how are you getting the z position of your fingertip from a non-3D camera? ty for sharing, this looks incredible - I've never had such a powerful tox up & running to quickly. Only issue was I had to download the latest TD.
Excellent tutorial and plugin. Very well made. I'm trying to apply the face feature points with a 3d model to animate the face according to the user. Any tips on how best to accomplish this? I keep running into blockers or dead ends.
Thank you so much! Such a game changer! Is there a way to out put the wire frame in a top? Trying to just out put stick man figure without the source video underneath it. Again, thank you!
Hello.. does this work on TD non-commercial as well? I subscribed to your patreon and downloaded some TD sketches.. none of my webcams seem to work (just blank white page)and I am wondering whether it is the TD version! mediapipe downloaded from your Github has the same issue. My MacOS is 14.0 Sonoma. Thanks in advance.
Hey what version of TD are you using. You don't need a commercial license. It should run on the latest version of TD, but loading into much older version won't work
hello thank you so much for the plugin!!! I just have a problem of saving the file. Everytime I save my file and exit, when I open the file again, all the work I've put disappears.. if the file is not saved in the same folder of the toxes folder, the work is still there but it says the file is having a trouble loading components(probably looking for the toxes files) How can I solve this? :(
Hey if you've lost any progress just put the tox files in a folder called toxes that's in the same directory as your .toe file and re-open your .toe file. In the future there's a toggle you can turn off on all the mediapip tox files in the parameter page under Common called "Reload .tox on Start"
@@blankensmithing omg thank you so much!!! I tried turning the toggle off from mediapipe.tox file and still had the same problem, I had to actually turn off all the toggles from rest of the tox files in the tox folder and convert them to tox files instead of toe files to make it work!! Thank you so much💗💗💗
Thanks so much for this amazing upgrade to working with mediapipe in TD, I was wondering if there is any benefit to using smaller resolutions than 1280 and if the spout method is the only way to achieve that downsizing? With the previous workflow of mediapipe I was able to get good hand coordinates at resolutions as low as 240x135 with an increase of roughly 8-10 fps. Although I am totally unfamiliar with the webbrowserCOMP and how/where it gets the webcam feed. Thanks
Hey! yeah the models run off a very small resolution feed, so there's no need to use a large video format. The only way to do it at the moment through spout or obs, which would add an additional delay
cool Torin, thanks a lot 4 da work work work. , do you know if i connect 3 webcam to TD, can i use mediaPipe, using each one to track their onw user's hand?
Hey, thanks! Yes you can just duplicate the MediaPipe component, make sure to change the Websocket Port and the MediaPipe Port to something different than the original component and you can select a different web cam. You can do this with however many webcams you have. I'd recommend turning off all the models you're not using for speed
Hi! I have a problem, when i open a patch all connection from MediaPipe base in disconnected. And i have red circles with cross. How i can connect them again?
Hey please follow the steps here: ruclips.net/video/Cx4Ellaj6kk/видео.htmlsi=jzo-JCOVxF89jOpS&t=139 so I can help debug. Make sure to attach screen shots. If you see a red x on a component, hover over it and include a screenshot of the text that's displayed in the error message
@@blankensmithingThank you for you help! Now it works good on my Apple laptop, but on PC it stucks in loading Media Mapper when i connect usb webcamera.
Thank you so much for your plugin!!! I have a question about the bound box data, there is an "angle" from the mediapipe face bounding box value list, but in my project, it always shows "0".
@@blankensmithing thank you for your immediate response! I'm looking for a face detector. I wanted to get the face/head rotation value, and then I saw an "angle" property, but it always shows"0".
This is my first time working with a plugin on TD, I followed the steps like you said, and referred to the github page but it doesnt load up media pipe for some reason, I selected my Webcam, checked it as well but MediaPipe doesnt finish loading, what should I do?
Hey please follow the steps here: ruclips.net/video/Cx4Ellaj6kk/видео.htmlsi=jzo-JCOVxF89jOpS&t=139 so I can help debug. Make sure to attach screen shots. If you see a red x on a component, hover over it and include a screenshot of the text that's displayed in the error message. Make sure to list what version of TouchDesigner you're running as well
@@blankensmithing Hi, so I was able to figure out the problem and now it works just fine. Excited to create with it. For anyone facing a similar issue, I used OBS studio instead of spoutcam for the webcam stream [PC] . In the video input of Mediapipe I used OBS Camera (which was scaled to 720p) and thats how it started to work. I created a new scene in OBS and simply added my webcam to the scene. Keeping OBS open I restarted TD and I could see "OBS Camera" as an input in the drop down. Hope this helps anyone facing this issue. Cant wait to get started with this plugin finally:) Thanks to the creators again!!!
Hey Torin! Thanks for the great video and plugin! I was wondering if there is a way to track multiple people at once. especially with the pose tracking? or is this not possible with MediaPipe?
Hey at the moment Media Pipe only has pose tracking for 1 person. It does look like multi-person pose is on the way soon though since their web based demo has multiple person detection(those examples are usually head of what they release to the public).
@@blankensmithing ok great thanks for your reply! Diffrent question: How can i use the Plugin with a higher resolution? I want to use the video output in 1080p but i can not use my Camera Input in a Video Device In TOP when i allready selected it in the MediaPipe Container.
@@user-uq9cn3wo8e The ML model scales down the webcam size, so using a higher resolution won't give better results. If you need to use a higher res cam you can use Spout Cam, or Syphon w/ OBS to change the resolution of your device and pipe it back into TD as a virtual webcam
Amazing! This is huge. How can I get Syphon to show up in the cameras dropdown? I'm using someone's proprietary OpenFrameworks Syphon App to get a KinectV2 working on my M1 Mac. I only get the video feeds tho so no CHOP data. Incorporating this mediapipe plugin into my workflow would nearly complete finally having a fully functioning kinectv2 in touchdesigner on apple silicon!
Hi, thank you for this beautiful project, but I am having an issue, why the MediaPipe box is always on Loading mode, and it print an image of Loading. I tried reseting the process but still doesn't work for me. Can you help me? Thank you!
Oh wow, just stumbled upon this. As I just recently used a #LeapMotion in #TouchDesigner it seems like #MediaPipe provides a great replacement as no dedicated hardware is needed anymore.
![[DOKKAN BATTLE] Worldwide Campaign Announcement Video Part 2!](http://i.ytimg.com/vi/JpT5Voak6WA/mqdefault.jpg)
Thanks for the plugin! For anyone having issues with the MediaPipe tox loading forever, the last version of 2022 (.35320) works for me on an M1 mac. I just avoided having to upgrade my commercial license for another $300 over this.
o my, brilliant tracking without special hardware. great explanations.
wow! I just get to know about MediaPipe - thank you so much for creating this and making this tools more accessible for individuals like me who have visual ideas but are not as technical as one can be.
Thanks! that manes a lot. Excited to see what you create with it!
Thanks for sharing such a helpful video! I never thought TouchDesigner could do this, and it's truly inspiring. Your video also has shown exactly what I've been looking for - how to superimpose TouchDesigner output to OBS.. Your effort is greatly appreciated!
350£ kinect 2 is good for the bin!
still useful for pointcloud projects
this is phenomenal!!... i was about the tackle the same thing and wishing to find time to port everything to GPU.. now we have it.. amazing.
This is truely amazing, thanks for all the hardwork
Wow this is incredible! I am really happy it also works for Mac computers!
he is using a mac
Sick sick sick. Am rolling into a project that might benefit from your great work. Perfect timing
i would be really grateful to understand the spoutcam for windows in depth for the mediapipeline plugin. Thank you
Same. was having issues with spoutcam. it worked briefly at first but with a strange aspect ratio. then was showing noise. opened up the spoutpanel and spoutsettings and just randomly tinkered around until things got going
WOW, I will be off grid now for some time to figure all of this out. Thank you. So helpful.
Thanks a lot for making this project public! it helps a lot for people that is starting
When looking for channel names(like around @20:37 & earlier) you can also just add a SELECT and click the black triangle next to "Channel Name" for a full list.
This plugin is amazing, I hope you keep updating it. thank you so much for sharing it!!!
Hi, I hope you can help me.
I just download the mediapipe plugin, so I open the main project MediaPipe TouchDesigner.toe then I Select my main camera and it doesn't show any image, but when i use a videodevice_in it automatically use my main camera.
Therefore almost every subsequent node show me some kind of error
Im using a macbook air M1, so, the main camera is a facetime camera.
Also, when I drag and drop some of the .tox at an empty Touchdesigner project it just doesnt drop at the workspace. Any idea if Im the one thats doing something wrong or someting else happening?
This is unbelievable, thank you guys!
Incredible content ! Verry smooth, everything is working perfectly ! (Macbookpro 2021 M1pro, lastest TD version)
More than amazing ! Thank you so much
This is amazing, thank you both for making this!
you are the real master bro ! keep doing it !
you two are so friking awsome! thanks a lot !
I tried to do the obs set up but for some reason my virtual cam on the media pipe node gives me a black screen. I have tried various resolutions in obs preferences, the input video, my camera's resolution, and 1280x720 but it isn't working. Any recommendations?
This is incredible! Thank you so much
Amazing, thanks for your hard work on this!
This is incredible! Thanks for publishing it ❤
We appreciate you!
you guys are amazing thank you so much!!!
This has been an amazing plugin to work with! I was wondering if there's a simple way to get the finger point tracking coordinates to reset to the centre of the screen when they are not visible? Currently they go to 0,0 in the bottom left so it constantly pulls anything I have mapped to it to this point if it can't see the intended finger. Thanks!
太谢谢惹！！！！！好棒好棒的组件！
😁
Thanks for publishing it
How is it that you are able to have both the mediapipe node and video device in run at the same time? I can only ever have one working at a time because my webcam doesn't like it when multiple things try to use it at once. Is there any workaround?
hmmmm the mediapipe component just keeps saying "loading" i don't think i did anything different than the tutorial, any thoughts on troubleshooting?
It's amazing! Thank you
looks cool! thanks for sharing!
WOW!
incredible
Thank you. 감사합니다.
Fantastic plugin-I think I can pack up my old Leap Motion Controllers now. Question: Is there a way to permanently have h1 be the left hand/h2 as the right? My experience now means that whichever hand is detected first becomes h1, regardless of whether it's my left or right hand. Specifically referring to the normalized data from Output 1 in the hand tracking component. Appreciate any advice on this, as well as the awesome tool.
Why does mediapipe report an error when I use it?
Hi! I got a problem, only loading and media pipe logo, can't detect the face in the camera, what is the solution?
Thanks! i have a question!
can i add my hand gestures? (not something that already exists like Closed Fist, Open palm, Point Up ... )
COOOOOOOOOL!!
Looks amazing, but i could not get it to work, i get an error that resolutions is up to 720p. Tried putting the resolution down but still no working. Working on a MAc pro with sonoma TD 2022.33910
Hey, thanks! Question: so there is no way to keep up tp date with git pull atm? So I will have to manually download updates?
what is the best way to use the face mask and replace it with a custom face model, IE make the face track onto a scary ghost or something fun instead of the base White mask. Trying to figure out the best way
Did you find a solution?
Hi guys, first of all I wanted to thank you for this great tool! I am working on a project where, using my hands, I reconstruct images as if I were painting them and I wanted to ask you a question.
Is it possible to remove the neck part and keep only the hand part when working using output 1 of the image segmentation 21:33 (the body skin colour one)?
In the preview both hands, and part of the arms, are coloured pink/purple but as all the skin is detected the neck, which always remains uncovered, is also detected, ruining the effect. Thanks again and congratulations on everything!
Hey, this is a super great plugin. I have a question though, There's no way to keep a consistent left hand and right hand. Everytime I remove one hand from the frame, the remaining hand becomes Hand_1. This makes it impossible for me to create a controller using 2 hands :(. Does this have an easy fix or is this functionality coming in the future? :)
thank you for your work it's real fun to use this and a lot of possibilities. I have one question: would it be possible to use the mediapipe plugin in td with an Infrared webcam?
Love the plugin! For some reason the latest version (0.32) shows the facePoints chop as an x,y,z chop (with green, yellow and red) as opposed to the time-slice style chop in the video. Any idea how to convert to the latter? Thanks!
u can try to use analyze CHOP i thought
@@kuanfuweimu Perfect, thank you so much!
Hey guys! Anyone knows if there is a workaround to use Mediapipe TD for vertical video / stream?
how are you getting the z position of your fingertip from a non-3D camera? ty for sharing, this looks incredible - I've never had such a powerful tox up & running to quickly. Only issue was I had to download the latest TD.
I am trying to use it but it gets extremely laggy and fps drops on camera even I have 3090 gpu and I9 cpu, can anyone help?
Try turning off the models you're not using. It's not intended to have all of them running.
nice job
Excellent tutorial and plugin. Very well made. I'm trying to apply the face feature points with a 3d model to animate the face according to the user. Any tips on how best to accomplish this? I keep running into blockers or dead ends.
Awesome tutorial Torin!
Hey! Thank you for this. So great! Is there a way to upload videos instead of using the webcam?
Awesome. How is the performance of this vs the kinect?
Thank you so much! Such a game changer! Is there a way to out put the wire frame in a top? Trying to just out put stick man figure without the source video underneath it. Again, thank you!
Hello.. does this work on TD non-commercial as well? I subscribed to your patreon and downloaded some TD sketches.. none of my webcams seem to work (just blank white page)and I am wondering whether it is the TD version! mediapipe downloaded from your Github has the same issue. My MacOS is 14.0 Sonoma. Thanks in advance.
Hey what version of TD are you using. You don't need a commercial license. It should run on the latest version of TD, but loading into much older version won't work
@@blankensmithing it's the most recent version. I downloaded it this week.
Can you connect cameras like arri?
hello thank you so much for the plugin!!! I just have a problem of saving the file. Everytime I save my file and exit, when I open the file again, all the work I've put disappears.. if the file is not saved in the same folder of the toxes folder, the work is still there but it says the file is having a trouble loading components(probably looking for the toxes files) How can I solve this? :(
Hey if you've lost any progress just put the tox files in a folder called toxes that's in the same directory as your .toe file and re-open your .toe file. In the future there's a toggle you can turn off on all the mediapip tox files in the parameter page under Common called "Reload .tox on Start"
@@blankensmithing omg thank you so much!!! I tried turning the toggle off from mediapipe.tox file and still had the same problem, I had to actually turn off all the toggles from rest of the tox files in the tox folder and convert them to tox files instead of toe files to make it work!! Thank you so much💗💗💗
Thanks so much for this amazing upgrade to working with mediapipe in TD, I was wondering if there is any benefit to using smaller resolutions than 1280 and if the spout method is the only way to achieve that downsizing? With the previous workflow of mediapipe I was able to get good hand coordinates at resolutions as low as 240x135 with an increase of roughly 8-10 fps. Although I am totally unfamiliar with the webbrowserCOMP and how/where it gets the webcam feed. Thanks
Hey! yeah the models run off a very small resolution feed, so there's no need to use a large video format. The only way to do it at the moment through spout or obs, which would add an additional delay
cool Torin, thanks a lot 4 da work work work. , do you know if i connect 3 webcam to TD, can i use mediaPipe, using each one to track their onw user's hand?
Hey, thanks! Yes you can just duplicate the MediaPipe component, make sure to change the Websocket Port and the MediaPipe Port to something different than the original component and you can select a different web cam. You can do this with however many webcams you have. I'd recommend turning off all the models you're not using for speed
Hi! I have a problem, when i open a patch all connection from MediaPipe base in disconnected. And i have red circles with cross. How i can connect them again?
Hey please follow the steps here: ruclips.net/video/Cx4Ellaj6kk/видео.htmlsi=jzo-JCOVxF89jOpS&t=139 so I can help debug. Make sure to attach screen shots. If you see a red x on a component, hover over it and include a screenshot of the text that's displayed in the error message
@@blankensmithingThank you for you help! Now it works good on my Apple laptop, but on PC it stucks in loading Media Mapper when i connect usb webcamera.
I resized usb camera resolution in OBS to 720p and it works now! Thank you very much!✨💜
Thank you so much for your plugin!!! I have a question about the bound box data, there is an "angle" from the mediapipe face bounding box value list, but in my project, it always shows "0".
Hey! which model are you looking at? Is it the face_detector, object_tracking, face_tracking component?
@@blankensmithing thank you for your immediate response! I'm looking for a face detector. I wanted to get the face/head rotation value, and then I saw an "angle" property, but it always shows"0".
Hi, I just would like to follow up on this if I did some wrong setting. Thank you so much🙏
This is my first time working with a plugin on TD, I followed the steps like you said, and referred to the github page but it doesnt load up media pipe for some reason, I selected my Webcam, checked it as well but MediaPipe doesnt finish loading, what should I do?
I was facing a similar issue. On downloading the current version of TD (2022.33910), it worked well!
Hey please follow the steps here: ruclips.net/video/Cx4Ellaj6kk/видео.htmlsi=jzo-JCOVxF89jOpS&t=139 so I can help debug. Make sure to attach screen shots. If you see a red x on a component, hover over it and include a screenshot of the text that's displayed in the error message. Make sure to list what version of TouchDesigner you're running as well
@@blankensmithing Hi, so I was able to figure out the problem and now it works just fine. Excited to create with it. For anyone facing a similar issue, I used OBS studio instead of spoutcam for the webcam stream [PC] . In the video input of Mediapipe I used OBS Camera (which was scaled to 720p) and thats how it started to work. I created a new scene in OBS and simply added my webcam to the scene. Keeping OBS open I restarted TD and I could see "OBS Camera" as an input in the drop down. Hope this helps anyone facing this issue. Cant wait to get started with this plugin finally:) Thanks to the creators again!!!
@amanpathak2338 Hey! I'm glad you got it working :) Thanks for the tips so we can pass it along to other people
Hey Torin! Thanks for the great video and plugin!
I was wondering if there is a way to track multiple people at once. especially with the pose tracking? or is this not possible with MediaPipe?
Hey at the moment Media Pipe only has pose tracking for 1 person. It does look like multi-person pose is on the way soon though since their web based demo has multiple person detection(those examples are usually head of what they release to the public).
@@blankensmithing ok great thanks for your reply!
Diffrent question:
How can i use the Plugin with a higher resolution? I want to use the video output in 1080p but i can not use my Camera Input in a Video Device In TOP when i allready selected it in the MediaPipe Container.
@@user-uq9cn3wo8e The ML model scales down the webcam size, so using a higher resolution won't give better results. If you need to use a higher res cam you can use Spout Cam, or Syphon w/ OBS to change the resolution of your device and pipe it back into TD as a virtual webcam
I feel like your intro gets louder every vid and its a slow prank :D
haha It's always the same volume, but may have to do this now :)
Another option is NDI in NDI out..
Amazing! This is huge. How can I get Syphon to show up in the cameras dropdown? I'm using someone's proprietary OpenFrameworks Syphon App to get a KinectV2 working on my M1 Mac. I only get the video feeds tho so no CHOP data. Incorporating this mediapipe plugin into my workflow would nearly complete finally having a fully functioning kinectv2 in touchdesigner on apple silicon!
never mind! Using ndi.
Hi, thank you for this beautiful project, but I am having an issue, why the MediaPipe box is always on Loading mode, and it print an image of Loading. I tried reseting the process but still doesn't work for me. Can you help me? Thank you!
Oh wow, just stumbled upon this. As I just recently used a #LeapMotion in #TouchDesigner it seems like #MediaPipe provides a great replacement as no dedicated hardware is needed anymore.
thank you so so so much for the amazing works!!!