EVERY Facial MoCap Shape-Key (In 3 Minutes!!)
HTML-код
- Опубликовано: 12 янв 2025
- Alright, here are ALL 52 shape-keys you'll need in order to use phone based Facial Motion Capture-!! (In less than 3 minutes!)
Facial MoCap Promo Project:
• BTSmash - Butter (3D A...
ArKit Apple iphone Shape-Key Official Documentation:
developer.appl...
PolyWink Main Page:
www.polywink.com
If you enjoyed this video, I have a small 1$ Member perk available for anyone who would like to help us save up for better tools to improve the content here like Hardware, Software, Environments, Animations, and special effects :)
(Just click the "Join" Button next to "Like"!)
/ royalskies
Twitter at: / theroyalskies
Blender RIGGING & ANIMATION SERIES:
• Blender 2.82 : Charact...
Intro To Unity Programming FULL Series:
• Introduction To Game P...
Anime Shader Tutorial Series: • Blender Anime Shading ...
Blender MODELING SERIES: • ENTIRE Low Poly Blende...
If you're a gamer, please check out my new game on steam! It took over 3 years to create and has thousands of hours and heart put into it :)
store.steampow...
Or if you just wanna play something casual, you can download my phone game to test your reaction time and see if you get into the top 10!
play.google.co...
As always, thank you so much for watching, please have a fantastic day, and see you around!
Royal Skies -
#3D#Animation#SmashBros
-------------------------------
Polywink RUclips Channel:
• 3D Facial Animation Ri...
Copyright Disclaimer: Under Section 107 of the Copyright Act 1976, allowance is made for "fair use" for purposes such as criticism, comment, news reporting, teaching, scholarship, and research. Fair use is a use permitted by copyright statute that might otherwise be infringing. Non-profit, educational or personal use tips the balance in favor of fair use.
It took me forever to organize and record this video, but hopefully it will save you a lot of wasted time - In case you forget or just need a specific Shape-Key, here are the time stamps for each:
1-eyeBlinkLeft 0:27
2-eyeLookDownLeft 0:32
3-eyeLookInLeft 0:34
4-eyeLookOutLeft 0:38
5-eyeLookUpLeft 0:40
6-eyeSquintLeft 0:43
7-eyeWideLeft 0:46
8-eyeBlinkRight 0:49
9-eyeLookDownRight 0:50
10-eyeLookInRight 0:54
11-eyeLookOutRight 0:56
12-eyeLookUpRight 0:59
13-eyeSquintRight 1:01
14-eyeWideRight 1:04
15-jawForward 1:06
16-jawLeft 1:15
17-jawRight 1:17
18-jawOpen 1:20
19-mouthClose 1:22
20-mouthFunnel 1:39
21-mouthPucker 1:41
22-mouthRight 1:43
23-mouthLeft 1:46
24-mouthSmileLeft 1:48
25-mouthSmileRight 1:51
26-mouthFrownRight 1:53
27-mouthFrownLeft 1:55
28-mouthDimpleLeft 1:58
29-mouthDimpleRight 2:00
30-mouthStretchLeft 2:03
31-mouthStretchRight 2:05
32-mouthRollLower 2:08
33-mouthRollUpper 2:20
34-mouthShrugLower 2:22
35-mouthShrugUpper 2:31
36-mouthPressLeft 2:33
37-mouthPressRight 2:35
38-mouthLowerDownLeft 2:37
39-mouthLowerDownRight 2:40
40-mouthUpperUpLeft 2:42
41-mouthUpperUpRight 2:45
42-browDownLeft 2:47
43-browDownRight 2:49
44-browInnerUp 2:52
45-browOuterUpLeft 2:54
46-browOuterUpRight 2:56
47-cheekPuff 2:58
48-cheekSquintLeft 3:03
49-cheekSquintRight 3:05
50-noseSneerLeft 3:08
51-noseSneerRight 3:11
52-tongueOut 3:13
This demonstration is in Maya, but the keys should look exactly the same in Blender - The Polywink website seems to be down for the time being, but I have a feeling it will be up again sometime soon. Either way, here are all the keys you'll need for phone based facial motion capture -
Thanks for mentioning us!
man you're just ... thank you for all your videos seriously
This has become my go to reference whenever I make shape keys for ARKit
Faceit is a blender addon that will make all the ARKit blend shapes + other types for abt 100$ on the blender market. Plus, if you’ve got one of those new iPhones with the depth cameras, I believe you can record your own facial mocap with it as well.
literally my favorite tutorial channel and it’s not even close
haha bro i could listen to you read shape keys all day. your voice and accent is uniquely awesome!
RoyalSkies: Sorry this vid was super long.
Video: *Barely 4 minutes*
Thanks!
You are my hero…for the next 20 minutes, at least! An ESSENTIAL video. Thanks 🙏☕️🎩🎩🎩
Early team yay!!
Hopefully I can buy a mocap soon 😭
OMG this information is pure gold!
I wonder if anybody has written a modifier that reads for any 'left' blendshapes and automatically makes mirror 'right' duplicates. It would make processes like this a little easier, especially if you're going back and forth with testing.
No need. Just CTRL-C your bones that you posed and then select the other bones (or just CTRL-A select all to be quick) and CTRL-SHIFT-V to paste the mirror opposite pose.
It possible to retarget facial mocap animation from base head with shapekeys to custom bone based rig, so it possible to have mocap from shapekeys, but with full advantage and flexibility of the custom rig, without adding shapekeys to existing rig
You can have the shape keys control bones instead yes. You can also have the best of both worlds and have shape keys AND still be able to adjust them with a rig on top of the shape keys -
Amazing as always!
This will be very helpful with the new VRC update coming out!
Your effort is appreciated...well done !
Man I tell you what. I sure like your videos.
Thank you for creating this video.
Question, how do you handle the teeth? Do they have a jawOpen shape key and stay together for the rest? You can kind of move your lips independently from the jaw. My character's teeth are parented to the head bone so I don't see what would be moving them with the ARkit setup
That looks like a lot of work to be done with multiple characters... How do I transfer those shape keys from one head to another?
Not quite an expert on this, but I think you could use a single base mesh for all characters and just shape the vertices to your character's proportions.
Make sure your mesh is in the same location as the other, you can use transfer shape key feature but there are a few plugins you can install which will help you. Usually you will have to select one, then select w shift for the other and the plugins will have a button that allows you to import the shape keys from one then transfer to another. The one I use is called "Shape Key Transfer"
Omniverse's Audio2Face does it automagically for you. Input your mesh, it transfer topology and the shape keys.
Thanks for the tips everybody. :)
Question for part of this, if I have eye bones that control the way the eyes are looking do I still also need to make blendshapes for eyes looking up/down?
You didn't need to use Ren Amamiya-san fir the mocap demonstration
Is the mouth close shape key necessary? I'm modeling an anthro character and I can't make that shape key without stretching the mesh a ton.
Truly awesome! Is there something familiar with the body too? Or don't you shape key the body?
How tf do you pump this specific video just as I struggle with ARKit
how in the hell did I miss this video for months? I've been looking for this one
That's a lot of shape keys
a good thing is separate Mouth Funnel and pucker left and right sides.. because .. well we made separated, and encrease possible expressions too...
basically just make it then duplicate and separate left and right....
i don´t know if this order is specific to a software.. but.. well ust put idea here.
41/52 shape keys maded; 1st of 10 characters....
Not all heroes wear capes!
Thank you!!! You are the man!
I really need this model to reference my shape keys but the sample download gives me a realistic avatar with 157 blends instead of 52
Someone needs to place a beat on it, it sounds like music lol. Thanks for this video tho, very helpful!
Do i need all the shape keys for live link face Use? I have less complex char, it s not even humanoid and i dont rly need so many expressions but i d like to record the animationsn with live link face. So... again... do i need all 52 morhp targets? Cool vid
Ren my beloved
truly the hero we needed
sorry, but it was hard to not laugh when u started going through the list 😂😅
I agree
I'm testing a mocap app with this shapekeys for VRM and I was wondering what many of them were supposed to be. Thanks! Now I know some of them, but I still find many from the mouth a little confusing. Anyways, do you think that all of them are necesary for anime style characters?
Honestly, for anime characters, if all you had were the eye keys (first 14) and then jaw open, you could probably get away with just those lol
If the VRM is from Vroid Studio. You can use HanaTool. It automatically makes all 54 shapekeys for Perfect Sync.
You can use Waidayo (an app for Iphone) to send the mocap data directly to Unity, using VMC (Virtual Motion Capture Protocol).
ruclips.net/video/DunqgLrUfpI/видео.html
The Hana tool is a Unity Asset, that you can buy on Booth.pm
Somewhere in the documentation of Hana Tool there is a link to github, to download a VRM model with all 54 shapekeys. As an example.
@@MrFrostCA Hum, my model is made in Blender, but that may be useful if someday I take commissions, thanks! I tried the faceit addon for Blender, but the models from VRoid are very messy to work with outside VRoid itself.
@@TheRoyalSkies Jaja yes, you can even skip the eye direction ones with some trackers! Also I noticed that it looks like there isn't any shapekeys for closing the teeth. Is it because they aren't tracked?
Thank awesome video
hey royal skies I have joined maya zbrush and max users posting hate comments on blender vids have you thought of making a video on why these people hate blender
Ok are those arkit shape keys? And how to mocap?
Is it normal that the eyelid clips middle through blinking, when I use the fbx in blender?
Would this work with the OSC protocol?
i just could not watch the whole video, it was way too long. So I searched for a video that is not that long, for 1 hour straight. In the end, I came back and forced myself to watch this 4 min long video 😩
jokes aside, really nice video
Great video! Question, let's say the character is a duplicate from another character but I changed the hair, clothes, changed body slightly, etc. I tried to copy the rig over with shapekeys from the original character and while it did copy over from a different Blender file, I tried transferring weight paint but it wasn't working out. Am I doing anything wrong? I assume I'll have to make a rig from scratch because I modified the character too much despite the character being a clone from another character I made from scratch in Blender. It's for my Senior year animation project so I am doing my best. And unlike having access to a mocap suit/iPhone facial mocap, I don't have either of those (I have a Samsung phone) so I guess I am animating by hand (the hard way).
Definitely DON'T make a rig from scratch just because the weight paint doesn't work - Just change the proportions of your old rig to fit the new character, then in edit mode, try to use the lattice modifier to stretch your old body into the shape of your new body > THEN try weight paint transfer, and if that doesn't work, just fix the weight painting with the smooth tool from last video. But, don't recreate the rig lol
@@TheRoyalSkies Thank you for the advice! I will try again. 👍 I wanted to say your videos has been helping me so much as an animation major. I learned a lot from you more than my 4 years of college, and this is my last year before finishing college. I appreciate your videos and you are amazing, keep it up!
@@paolaanimator I appreciate the kind words, thank you, and no problem, it's been my pleasure - I wish you the very best of luck on your last year in college :)
@@TheRoyalSkies hell, the lattice advice is incredible! I've never thought of that
So a free system to capture facial mocap with a webcam and your face coming up next?
How do you control the eyes to be part of the shapekey even if they are a different object? (I'm just sculpting not rigging to get the shapekeys)
best way would prabably be using vertext groups, as in join the eyes in with the main head but select them all and assign them to a vertex group named right eye or left eye or whatever you want, then when doing the shape keys you can quickly select the eye from the rest of the mesh to edit it seperately in edit mode without having to individually select all the vertexes again and again and again which is great for eyes as it's just a simple rotate tool to get those shape keys done, though for more complex stuff you can go for the more tedias route of doing all of the shape keys shown in this amazing video on the seperated objects then join them together and the shapekey data is automatically is joined also, which may be helpful for more complex deformations of seperate objects.
How would you add these in blender Sensei!!!
Is the website down again? Or has it been down for 6 months? I'm mad that I haven't gotten to this video a lot sooner when the site was available!!! 😥
I saw it up a few days ago, but it seems to be down now
Ya I want to make a 3d VTuber from scratch with blender so this will probably really help
Could you combine the wiggle and spring Bones in a model? like one part of the model is wiggle Bones and one is spring Bones
is there an add-on that just generates all the shape keys for you?
Feels weird how faces don't have a tool to do this by now.
Technically, there is an add-on called 'FaceIt' for $100, however in my personal experience, the time you spend fixing all the things it gets wrong, is usually more than the time it takes to just do them manually in the first place.You can try it though if you want, you might have better luck with it than me -
Can we get faceit addon for free ?
Anyone got any ideas on y mesh stretches for animations in unity but in blender it looks fine and dandy. Transformations applied.
That happened to me for a while, turned out the problem was I didn't reset my normals > open your face in edit mode > go to face mode > 'a' to select all > mesh > normals > 'reset vectors' > then export it to Unity, it should be smooth. Otherwise, I'd check your shader and light sources -
Ah I managed to figure it out after an hour or so. I think it was because my controllers had deform checked or i think that was the prob. I messed with the settings for a while n exported a few times n it worked out in the end. Thanks anyways
and again thank you
sorry if this vid is super long 😅
Hi royal, this isnt related to mocap or much of the video really but I was wondering if u would personally help me out with a model of mine that I want to rig because I was impressed by your blenda series but I’m really confused with mine because I’ve got a lot of armor on my model and I don’t know where even to start because of it, I understand if you can’t or don’t want to you seem like a very busy person
ps: if anyone else could help in this comment section I would really appreciate it
If it's your first time rigging something I'd advice you to keep it simple. Don't use driver's or any correcting bone as they are for fixing very specific problems. You should even skip the bones of the fingers because they are too much. Parent the armature to the mesh with automatic weighs, see how it works and try some weigh painting. Once you feel more confident try more advanced stuff as drivers and remember that the values on the drivers may depend on the scale of your model.
Gracias video bme sirve muchisimo
Odd, these shape keys avoid skin crinkles and crevices. I wonder why that is.
Yes
i have come to the sudden realization as to why they charge so much for expressions
My gf also makes that jigglypuff face when she's mad. I can't say she's immature, she's rarely that mad, but she makes the jigglypuff face and ever since I pointed it out to her she hasn't made that face to me.
This guy 0_0