Jointly Generating Multi-view Consistent PBR Textures using Collaborative Control

Shimon Vainer, Konstantin Kutsy, Dante De Nigris, Ciara, Slava Elizarov, Simon Donné

Jointly Generating Multi-view Consistent PBR Textures using Collaborative Control: 5 upvotes on Hugging Face Daily Papers, #40 of 49 papers on 2024-10-10. Day-by-day upvote history.

Multi-view consistency remains a challenge for image diffusion models. Even within the Text-to-Texture problem, where perfect geometric correspondences are known a priori, many methods fail to yield aligned predictions across views, necessitating non-trivial fusion methods to incorporate the results onto the original mesh. We explore this issue for a Collaborative Control workflow specifically in PBR Text-to-Texture. Collaborative Control directly models PBR image probability distributions, including normal bump maps; to our knowledge, the only diffusion model to directly output full PBR stacks. We discuss the design decisions involved in making this model multi-view consistent, and demonstrate the effectiveness of our approach in ablation studies, as well as practical applications.

Paper page on Hugging Face · arXiv

Data: hysts-bot-data/daily-papers-stats and the Daily Papers API. Open data: tardellirs/paper-pulse-data. Sister project: Model Pulse, the download history of every model on the Hub.