@@impactframes running into "AniPortraitRun AniPortraitLoader" failed to load. When I go to install custom nodes, I get this "The custom node DB is currently being updated, and updates to custom nodes are being checked for. NOTE: Update only checks for extensions that have been fetched."
I couldn’t get this workflow to workout due too technical hurdles, but in the mean time I have networked with some comfyui artists in LA and hopefully will have an update for you here soon.
@@PSA04 I am curious what you have been cooking. I was in LA last week for the ComfyUI leadership council with people from all over the world but also met many great artist an developers from the local scene and everyone was super welcoming and amazing can't wait to go again
Can I use this to animate just the face on a existing video instead of a still like wav2lip? If not do you have other recommendations better than wav2lip?
Microsoft has announced a solution for creating avatars where you can control the angle of the face in 3D and emotions. In this regard, he has only presented the paper. How could you influence emotions in the final result of AniPortrait? The only thing I can think of is through the reference video. I will appreciate your appreciation on both points.
Yes, they have trained just a better version of this, basically incorporated face expressions blinks, higher resolution and better consistency. I made a custom node implementation of dreamtalker in ComfyUI there is many presets that control the head movement the only thing is some of the movements are too abrupt sometimes and lacking eye blinks made it look robotic but at least it renders much faster than MuseV. Again MuseV is slow for the video creation part while dreamtalk has prerender those. I think is going to be a long time before ms release vasa1 and if they do it is going to be API.
Thanks. I always get this error unfortunately. Do you know, what this means? "Error occurred when executing AniPortraitLoader: Error while deserializing header: HeaderTooSmall File "C:\StableDiffusion\COMFYUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\StableDiffusion\COMFYUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\StableDiffusion\COMFYUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\StableDiffusion\COMFYUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AniPortrait odes.py", line 112, in run a2m_model = Audio2MeshModel(audio_infer_config['a2m_model']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\StableDiffusion\COMFYUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AniPortrait\src\audio_models\model.py", line 27, in __init__ self.audio_encoder = Wav2Vec2Model.from_pretrained(model_path, local_files_only=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\StableDiffusion\COMFYUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\transformers\modeling_utils.py", line 3456, in from_pretrained with safe_open(resolved_archive_file, framework="pt") as f: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^"
Hello and thanks for the video. I tried to install everything, but when I import the workflow, I get two red nodes with 'undefined' written on them. I would like to point out that during the installation of the dependencies, I get this error: 'ERROR: Could not find a version that satisfies the requirement mediapipe.' Could this be the cause? Thank you
Do you have the latest version? If you do you might need to pip install requirements.txt it needs pip install huggingface_hub for that node specifically
@@hyunseungkim538 I just did it a few hours ago and it worked on a new computer are you sure you have the right repo sometimes you have to restart ComfyUI github.com/if-ai/ComfyUI-IF_AI_tools
I am sorry I was looking but I didn't save the workflow but I didn't make anything more than adding the image resize and image crop nodes to make both images 512x512 this nodes are from cubiq essentials node pack the int error you have is maybe because you have two different frame speeds on the kijai nodes int and float constants this is the repo for the image crop node github.com/cubiq/ComfyUI_essentials
You show two different workflows. Where can we download the second one along with the source videos & images? The second at 8:01 one has video: ruclips.net/video/T-B6xJRG6fQ/видео.html Unfortunately wf.json has a missing Encoder/Decoder error. "DonwEncoderBlock2D does not exist"
I am sorry Iwas looking but I didn't save the workflow but I didn't make anything more than adding the image resize and image crop nodes to make both images 512x512 this nodes are from cubiq essentials node pack the int error you have is maybe because you have two different frame speeds on the kijai nodes int and float constants this is the repo for the image crop node github.com/cubiq/ComfyUI_essentials
@@impactframes Thx for the quick reply. Solved many errors but at the end it did not work. Comfyui is super messy. I'll try NerdyRodent's installation.
@@impactframes That one from NerdyRofent with conda env did not work either. Folder structure for models seems to be correct but still gives mismatch errors. So AniPortrait seems to be badly documented for now. An installation script needed.
Excited to give this a shot! Thanks
Is a bit slow but it works I wonder if a better implementation comes along
Me too, this looks awesome!
Totally going to combo this up with some meta human unreal animation. Getting that out in the universe. New gaming and visual styles.
Yes that is a great 💡
@@impactframes running into "AniPortraitRun
AniPortraitLoader" failed to load. When I go to install custom nodes, I get this "The custom node DB is currently being updated, and updates to custom nodes are being checked for.
NOTE: Update only checks for extensions that have been fetched."
Very interested in hearing how your MetaHuman tests came out..
I couldn’t get this workflow to workout due too technical hurdles, but in the mean time I have networked with some comfyui artists in LA and hopefully will have an update for you here soon.
@@PSA04 I am curious what you have been cooking. I was in LA last week for the ComfyUI leadership council with people from all over the world but also met many great artist an developers from the local scene and everyone was super welcoming and amazing can't wait to go again
Can I use this to animate just the face on a existing video instead of a still like wav2lip? If not do you have other recommendations better than wav2lip?
Microsoft has announced a solution for creating avatars where you can control the angle of the face in 3D and emotions. In this regard, he has only presented the paper.
How could you influence emotions in the final result of AniPortrait? The only thing I can think of is through the reference video. I will appreciate your appreciation on both points.
Yes, they have trained just a better version of this, basically incorporated face expressions blinks, higher resolution and better consistency. I made a custom node implementation of dreamtalker in ComfyUI there is many presets that control the head movement the only thing is some of the movements are too abrupt sometimes and lacking eye blinks made it look robotic but at least it renders much faster than MuseV. Again MuseV is slow for the video creation part while dreamtalk has prerender those. I think is going to be a long time before ms release vasa1 and if they do it is going to be API.
Greattt , bro
Thank you 🙂
🔥🔥🔥
Thanks. I always get this error unfortunately. Do you know, what this means?
"Error occurred when executing AniPortraitLoader:
Error while deserializing header: HeaderTooSmall
File "C:\StableDiffusion\COMFYUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\StableDiffusion\COMFYUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\StableDiffusion\COMFYUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\StableDiffusion\COMFYUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AniPortrait
odes.py", line 112, in run
a2m_model = Audio2MeshModel(audio_infer_config['a2m_model'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\StableDiffusion\COMFYUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AniPortrait\src\audio_models\model.py", line 27, in __init__
self.audio_encoder = Wav2Vec2Model.from_pretrained(model_path, local_files_only=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\StableDiffusion\COMFYUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\transformers\modeling_utils.py", line 3456, in from_pretrained
with safe_open(resolved_archive_file, framework="pt") as f:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^"
Hello and thanks for the video. I tried to install everything, but when I import the workflow, I get two red nodes with 'undefined' written on them. I would like to point out that during the installation of the dependencies, I get this error: 'ERROR: Could not find a version that satisfies the requirement mediapipe.' Could this be the cause? Thank you
May I ask how I can get the huggleface download. Somehow, I can not see under the menu on ImpactFrames, any idea?
Do you have the latest version? If you do you might need to pip install requirements.txt it needs pip install huggingface_hub for that node specifically
@@impactframes thanks for reply in fact, I did clone and run requirement as well. but somehow, it does not show the options.
@@hyunseungkim538 I just did it a few hours ago and it worked on a new computer are you sure you have the right repo sometimes you have to restart ComfyUI github.com/if-ai/ComfyUI-IF_AI_tools
@@impactframes Huge Thanks and will do that~! it may be related somehow not got updated on my ends. I will try to new cut for sure! appreciate it!
I am sorry I was looking but I didn't save the workflow but I didn't make anything more than adding the image resize and image crop nodes to make both images 512x512 this nodes are from cubiq essentials node pack the int error you have is maybe because you have two different frame speeds on the kijai nodes int and float constants this is the repo for the image crop node github.com/cubiq/ComfyUI_essentials
You show two different workflows.
Where can we download the second one along with the source videos & images?
The second at 8:01 one has video: ruclips.net/video/T-B6xJRG6fQ/видео.html
Unfortunately wf.json has a missing Encoder/Decoder error.
"DonwEncoderBlock2D does not exist"
I am sorry Iwas looking but I didn't save the workflow but I didn't make anything more than adding the image resize and image crop nodes to make both images 512x512 this nodes are from cubiq essentials node pack the int error you have is maybe because you have two different frame speeds on the kijai nodes int and float constants this is the repo for the image crop node github.com/cubiq/ComfyUI_essentials
@@impactframes Thx for the quick reply. Solved many errors but at the end it did not work. Comfyui is super messy. I'll try NerdyRodent's installation.
@@user-uk9ls yes might be better for now
@@impactframes That one from NerdyRofent with conda env did not work either. Folder structure for models seems to be correct but still gives mismatch errors. So AniPortrait seems to be badly documented for now. An installation script needed.