Hybrid 3D Engine: Meshes, Animations, Gaussian Splats with Neural Rendering

Currently, the predominant usage of AI-generated assets is as imports into standard game engines. However, the gap between importable assets and full-scale simulated environments is still large and requires complex programmability involving physics, lighting, mesh interactions, etc. To this end, we have created a new real-time diffusion-based rendering engine that synthesizes the complex input conditions from our 3D system. This approach bridges the visual fidelity gap of virtual environments while grounding them in 3D representations from first principles. We believe that these hybrid neural networks will allow users to iterate, create, and navigate breathtaking worlds with fine-grained control, and will be a central component of creating AI that can experiment, plan, and act.
Access now: www.csm.ai/