LDM3D-VR: Latent Diffusion Model for 3D VR

gabriela ben melech, Diana Wofk, Estelle Aflalo, Shaoyen Tseng, zhipeng cai, Michael Paulitsch, Vasudev Lal

LDM3D-VR: Latent Diffusion Model for 3D VR: 9 upvotes on Hugging Face Daily Papers, #10 of 16 papers on 2023-11-07. Day-by-day upvote history.

Latent diffusion models have proven to be state-of-the-art in the creation and manipulation of visual outputs. However, as far as we know, the generation of depth maps jointly with RGB is still limited. We introduce LDM3D-VR, a suite of diffusion models targeting virtual reality development that includes LDM3D-pano and LDM3D-SR. These models enable the generation of panoramic RGBD based on textual prompts and the upscaling of low-resolution inputs to high-resolution RGBD, respectively. Our models are fine-tuned from existing pretrained models on datasets containing panoramic/high-resolution RGB images, depth maps and captions. Both models are evaluated in comparison to existing related methods.

Paper page on Hugging Face · arXiv

Data: hysts-bot-data/daily-papers-stats and the Daily Papers API. Open data: tardellirs/paper-pulse-data. Sister project: Model Pulse, the download history of every model on the Hub.