Видео

Unity. uLipSync.

Clearly my person life is creeping into this alternate AI reality.

Captured with iphone

​@@brycelynchwhich

Sequencer

That was with a 2080ti running on Linux.

Yes, it’s just a demo. SwiftUI and RealityKit using webhooks to send commands to coffee machine.

If you pause the video at the beginning you can see the skeleton hierarchy. It requires unbinding meshes, combining head and body skeletons under same root, reskinning to new skeleton (copy skin weight back).

It's been awhile since I looked into this... but I think the issue that I was running into was getting mocopi to compile for android. I don't believe Sony's unreal plugin will allow this. There might be a way to do it in windows... but probably not worth the effort. Depending on the level of fidelity you need... the new full body movement SDK from meta is probably the way to go. If you're comfortable building from source, it's straightforward to setup and supports everything you are looking for. The Q3 solves the realtime body a little better than the pro but the pro has the facial tracking. Trade offs everywhere. 🤪

Not sure what you mean by your post... but the VHS source was captured using a Aja Kona card and ~200 frames were extracted and then processed using the official gaussian-splatting reference implementation available through GitHub. I trained the data set for 30k iterations on a 4090. The splat was then edited down using the UnityGaussianSplatting toolset (available on Github) and A-Frame handled the immersive webXR render to headset.

@@brycelynchwhat kind of webxr system did you use?

It did for me. :)

Great. Would really appreciate if you made a video of how mocopi handles sitting down, standing up and moving around it seems to do great but sitting down is a mystery🪑@@brycelynch

Sure can! I'd imagine there are a lot of examples of people doing exactly that. :)

This is headset through immersive web browser. WebXR. aframe.

Body and face is livelink. Real-time not recorded. A couple cameras switched using touch OSC. NDI is streaming live video feed into game and OBS is recording game output.

I used a gs plugin on marketplace and imported to ue5. It creates a Niagara particle system to render the splat and I migrated that to UEFN.

Yeah, just an internal (unreleased) project. :)

@@brycelynchas someone who loves playing with puppets and is just getting into vr… it looks great! I like the way the puppets can be customized too. I wish there was a link for “buy beta version” 😊 I found this video looking for a vr puppet app. I wish it was available!

Seconded! I'm working on a similar implementation - got to the point where things are rendering, but of course everything is flipped... >.< Headache inducing...

Hard to now why that's happening, guessing that the orientation of the splat doesn't align with your world coordinates. I edited this splat using the UnityGaussianSplatting tool on Github and you can bake out modified world transforms. I did this to place the splat at the world origin, I also parented it to an invisible cube in A-Frame (that had collision) so I could pick it up. When you parent you could try changing the rotation if it's not oriented correctly.

Here is the original test. If you view source you in your browser you can see everything that is going on. xr.chrisyoung.com/content/test2/

This is a one off demo I built for myself. I used VRE Plugin for Unreal to handle interactions

@@brycelynch nice. I’ve always thought it would be dope to have a AR/VR library with real readable books. I wonder if Apple will do that with the Vision Pro and Apple Books 🤔

Unreal Engine. Oculus source build with passthrough. VRE Plugin for interactions.

sweet! Just starting Exploration and PrePro on a game, I really like this mechanic @@brycelynch