UniGeo: Taming Video Diffusion for Unified Consistent Geometry Estimation

Arxiv 2025

Yang-Tian Sun¹,

Xin Yu¹,

Zehuan Huang²,

Yi-Hua Huang¹,

Yuan-Chen Guo³,

Ziyi Yang¹,

Yan-Pei Cao³,

Xiaojuan Qi¹

¹The University of Hong Kong, ²Beihang University, ³VAST

arXiv Code

Huggingface (coming soon)

TL, DR: We propose a method for joint geometry estimation (specifically surface normals and coordinates) from multi-view images and video clips within a unified space, by leveraging the inter-frame correspondence prior embedded in the video diffusion model.

Motivation

(a) Pre-trained video diffusion models capture accurate inter-frame correspondence (the same patches in different frames highlight in the attention maps)
(b) The correspondence can be specified by applying identical positional encodings onto different frames
(c) Geometric properties within a shared global coordinate system naturally exhibit alignment across frames.