arXiv subjects

Haoyu Guo

Publications and source records attributed to Haoyu Guo.

At least 37 records · Page 2Linked to original sources

arXiv2025

BRIDGE -- Building Reinforcement-Learning Depth-to-Image Data Generation Engine for Monocular Depth Estimation

Monocular Depth Estimation (MDE) is a foundational task for computer vision. Traditional methods are limited by data scarcity and quality, hindering their robustness. To overcome this, we propose BRIDGE, an RL-optimized depth-to-image (D2I) generation framework that synthesizes over 20M realistic and geometrically accurate RGB images, each intrinsically paired with its ground truth depth, from diverse source depth maps. Then we train our depth estimation model on this dataset, employing a hybrid supervision strategy that integrates teacher pseudo-labels with ground truth depth for comprehensive and robust training. This innovative data generation and training paradigm enables BRIDGE to achieve breakthroughs in scale and domain diversity, consistently outperforming existing state-of-the-art approaches quantitatively and in complex scene detail capture, thereby fostering general and robust depth features. Code and models are available at https://dingning-liu.github.io/bridge.github.io/.

cs.CV↗

arXiv2025

OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling

The field of 4D world modeling - aiming to jointly capture spatial geometry and temporal dynamics - has witnessed remarkable progress in recent years, driven by advances in large-scale generative models and multimodal learning. However, the development of truly general 4D world models remains fundamentally constrained by the availability of high-quality data. Existing datasets and benchmarks often lack the dynamic complexity, multi-domain diversity, and spatial-temporal annotations required to support key tasks such as 4D geometric reconstruction, future prediction, and camera-control video generation. To address this gap, we introduce OmniWorld, a large-scale, multi-domain, multi-modal dataset specifically designed for 4D world modeling. OmniWorld consists of a newly collected OmniWorld-Game dataset and several curated public datasets spanning diverse domains. Compared with existing synthetic datasets, OmniWorld-Game provides richer modality coverage, larger scale, and more realistic dynamic interactions. Based on this dataset, we establish a challenging benchmark that exposes the limitations of current state-of-the-art (SOTA) approaches in modeling complex 4D environments. Moreover, fine-tuning existing SOTA methods on OmniWorld leads to significant performance gains across 4D reconstruction and video generation tasks, strongly validating OmniWorld as a powerful resource for training and evaluation. We envision OmniWorld as a catalyst for accelerating the development of general-purpose 4D world models, ultimately advancing machines' holistic understanding of the physical world.

cs.CV↗

arXiv2025

WinT3R: Window-Based Streaming Reconstruction with Camera Token Pool

We present WinT3R, a feed-forward reconstruction model capable of online prediction of precise camera poses and high-quality point maps. Previous methods suffer from a trade-off between reconstruction quality and real-time performance. To address this, we first introduce a sliding window mechanism that ensures sufficient information exchange among frames within the window, thereby improving the quality of geometric predictions without large computation. In addition, we leverage a compact representation of cameras and maintain a global camera token pool, which enhances the reliability of camera pose estimation without sacrificing efficiency. These designs enable WinT3R to achieve state-of-the-art performance in terms of online reconstruction quality, camera pose estimation, and reconstruction speed, as validated by extensive experiments on diverse datasets. Code and model are publicly available at https://github.com/LiZizun/WinT3R.

cs.CV↗

arXiv2025

StreetCrafter: Street View Synthesis with Controllable Video Diffusion Models

This paper aims to tackle the problem of photorealistic view synthesis from vehicle sensor data. Recent advancements in neural scene representation have achieved notable success in rendering high-quality autonomous driving scenes, but the performance significantly degrades as the viewpoint deviates from the training trajectory. To mitigate this problem, we introduce StreetCrafter, a novel controllable video diffusion model that utilizes LiDAR point cloud renderings as pixel-level conditions, which fully exploits the generative prior for novel view synthesis, while preserving precise camera control. Moreover, the utilization of pixel-level LiDAR conditions allows us to make accurate pixel-level edits to target scenes. In addition, the generative prior of StreetCrafter can be effectively incorporated into dynamic scene representations to achieve real-time rendering. Experiments on Waymo Open Dataset and PandaSet demonstrate that our model enables flexible control over viewpoint changes, enlarging the view synthesis regions for satisfying rendering, which outperforms existing methods.

cs.CV↗

arXiv2025

Precise Action-to-Video Generation Through Visual Action Prompts

We present visual action prompts, a unified action representation for action-to-video generation of complex high-DoF interactions while maintaining transferable visual dynamics across domains. Action-driven video generation faces a precision-generality trade-off: existing methods using text, primitive actions, or coarse masks offer generality but lack precision, while agent-centric action signals provide precision at the cost of cross-domain transferability. To balance action precision and dynamic transferability, we propose to "render" actions into precise visual prompts as domain-agnostic representations that preserve both geometric precision and cross-domain adaptability for complex actions; specifically, we choose visual skeletons for their generality and accessibility. We propose robust pipelines to construct skeletons from two interaction-rich data sources - human-object interactions (HOI) and dexterous robotic manipulation - enabling cross-domain training of action-driven generative models. By integrating visual skeletons into pretrained video generation models via lightweight fine-tuning, we enable precise action control of complex interaction while preserving the learning of cross-domain dynamics. Experiments on EgoVid, RT-1 and DROID demonstrate the effectiveness of our proposed approach. Project page: https://zju3dv.github.io/VAP/.

cs.CV↗

arXiv2025

Thermal Hall response of an abelian chiral spin liquid at finite temperatures

Thermal Hall transport has emerged as a valuable tool for probing the fractionalized excitations in chiral quantum spin liquids. Observing quantized thermal Hall response, expected at temperatures below the spectral gap, has been challenging and controversial. The finite temperature behavior, especially in the quantum critical regime above the spectral gap, can provide useful signatures of the underlying topological order. In this context, we study the spin-$1/2$ Heisenberg antiferromagnet on a kagome lattice that is believed to be a U$(1)$ Dirac spin liquid over a wide intermediate energy range. Scalar spin chirality perturbations turn this into a gapped abelian chiral spin liquid (CSL) with semionic topological order. Using a recently developed large-$N$ technique [Guo et al., Phys. Rev. B 101, 195126 (2020)], we obtain explicit expressions for the thermal Hall conductivity $κ_{xy}$ at finite temperatures taking into account both matter and gauge fluctuations. At low temperatures below the spectral gap, the quantized thermal Hall response agrees with that expected from conformal field theory and gravitational anomaly arguments. Our main finding is that in a large temperature window spanning the spectral gap and the Curie temperature scales where quantum critical fluctuations dominate, $κ_{xy}/T$ obeys a power-law with logarithmic corrections. Our analysis also provides a route to understanding the thermal Hall response at higher temperatures in the quantum critical regime.

cond-mat.str-el↗

arXiv2025

Dynamical Freezing in Exactly Solvable Models of Driven Chaotic Quantum Dots

The late-time equilibrium behavior of generic interacting models is determined by the coupled hydrodynamic equations associated with the globally conserved quantities. In the presence of an external time-dependent drive, non-integrable systems typically thermalize to an effectively infinite-temperature state, losing all memory of their initial states. However, in the presence of a large time-periodic Floquet drive, there exist special points in phase-space where the strongly interacting system develops approximate {\it emergent} conservation laws. Here we present results for an exactly solvable model of two coupled chaotic quantum dots with multiple orbitals interacting via random two and four-fermion interactions in the presence of a Floquet drive. We analyze the phenomenology of dynamically generated freezing using a combination of exact diagonalization, and field-theoretic analysis in the limit of a large number of electronic orbitals. The model displays universal freezing behavior irrespective of whether the theory is averaged over the disorder configurations or not. We present explicit computations for the growth of many-body chaos and entanglement entropy, which demonstrates the long-lived coherence associated with the interacting degrees of freedom even at late-times at the dynamically frozen points. We also compute the slow timescale that controls relaxation away from exact freezing in a high-frequency expansion.

cond-mat.str-el↗

arXiv2025

Field theory of monitored, interacting fermion dynamics with charge conservation

Measurement-induced phase transitions (MIPTs) in monitored quantum dynamics are non-equilibrium phase transitions between quantum-chaotic (volume-law entangled) and entanglement-suppressed, area-law phases. We reveal how monitored dynamics are situated within the framework of general far-from-equilibrium, quantum condensed-matter physics. Measurement-induced heating effects scramble the distribution function in generic (interacting) monitored fermion systems, which enables a simplified symmetry-based description of the dynamics. We demonstrate the equivalence of the Keldysh technique with the conventional Statistical-Mechanics Model for circuits, resulting from a doubled Hilbert-space (Choi-Jamiołkowski) mapping. We illustrate this using the monitored dynamics of interacting fermions with a conserved charge, deriving a unified effective field theory that captures all phases and phase transitions. The non-interacting counterpart in 1D space only has an area-law phase, with no MIPT. This was explained via an effective non-linear sigma model replica field theory possessing a very large symmetry. We show that other phases and phase transitions emerge when the replica symmetry is reduced by interactions. The reduced symmetry combines a replica permutation symmetry and charge-conservation within each replica. The former and its spontaneous breaking govern the MIPT, which can be recognized via a separatrix in the renormalization group flow. The replica-resolved charge conservation dictates the ``charge-sharpening" transition between two kinds of dynamics, where the global charge information is either hidden or reconstructible from the measurements. The field theory explains why the charge-sharpening transition should occur only in the volume-law phase. Our framework provides a template for other classes of MIPTs and situates these within the arena of non-equilibrium condensed matter physics.

cond-mat.stat-mech↗

arXiv2025

Multi-view Reconstruction via SfM-guided Monocular Depth Estimation

In this paper, we present a new method for multi-view geometric reconstruction. In recent years, large vision models have rapidly developed, performing excellently across various tasks and demonstrating remarkable generalization capabilities. Some works use large vision models for monocular depth estimation, which have been applied to facilitate multi-view reconstruction tasks in an indirect manner. Due to the ambiguity of the monocular depth estimation task, the estimated depth values are usually not accurate enough, limiting their utility in aiding multi-view reconstruction. We propose to incorporate SfM information, a strong multi-view prior, into the depth estimation process, thus enhancing the quality of depth prediction and enabling their direct application in multi-view geometric reconstruction. Experimental results on public real-world datasets show that our method significantly improves the quality of depth estimation compared to previous monocular depth estimation works. Additionally, we evaluate the reconstruction quality of our approach in various types of scenes including indoor, streetscape, and aerial views, surpassing state-of-the-art MVS methods. The code and supplementary materials are available at https://zju3dv.github.io/murre/ .

cs.CV↗

arXiv2025

Arresting Quantum Chaos Dynamically in Transmon Arrays

Ergodic quantum many-body systems evolving under unitary time dynamics typically lose memory of their initial state via information scrambling. Here we consider a paradigmatic translationally invariant many-body Hamiltonian of interacting bosons -- a Josephson junction array in the transmon regime -- in the presence of a strong Floquet drive. Generically, such a time-dependent drive is expected to heat the system to an effectively infinite temperature, featureless state in the late-time limit. However, using numerical exact-diagonalization we find evidence of special ratios of the drive amplitude and frequency where the system develops {\it emergent} conservation laws, and {\it approximate} integrability. Remarkably, at these same set of points, the Lyapunov exponent associated with the semi-classical dynamics for the coupled many-body equations of motion drops by orders of magnitude, arresting the growth of chaos. We supplement our numerical results with an analytical Floquet-Magnus expansion that includes higher-order corrections, and capture the slow dynamics that controls decay away from exact freezing.

cond-mat.str-el↗

arXiv2025

Automatic Annotation Augmentation Boosts Translation between Molecules and Natural Language

Recent advancements in AI for biological research focus on integrating molecular data with natural language to accelerate drug discovery. However, the scarcity of high-quality annotations limits progress in this area. This paper introduces LA$^3$, a Language-based Automatic Annotation Augmentation framework that leverages large language models to augment existing datasets, thereby improving AI training. We demonstrate the effectiveness of LA$^3$ by creating an enhanced dataset, LaChEBI-20, where we systematically rewrite the annotations of molecules from an established dataset. These rewritten annotations preserve essential molecular information while providing more varied sentence structures and vocabulary. Using LaChEBI-20, we train LaMolT5 based on a benchmark architecture to learn the mapping between molecular representations and augmented annotations. Experimental results on text-based *de novo* molecule generation and molecule captioning demonstrate that LaMolT5 outperforms state-of-the-art models. Notably, incorporating LA$^3$ leads to improvements of up to 301% over the benchmark architecture. Furthermore, we validate the effectiveness of LA$^3$ notable applications in *image*, *text* and *graph* tasks, affirming its versatility and utility.

cs.LG↗

arXiv2024

Strange metal and superconductor in the two-dimensional Yukawa-Sachdev-Ye-Kitaev model

The two-dimensional Yukawa-Sachdev-Ye-Kitaev (2d-YSYK) model provides a universal theory of quantum phase transitions in metals in the presence of quenched random spatial fluctuations in the local position of the quantum critical point. It has a Fermi surface coupled to a scalar field by spatially random Yukawa interactions. We present full numerical solutions of a self-consistent disorder averaged analysis of the 2d-YSYK model in both the normal and superconducting states, obtaining electronic spectral functions, frequency-dependent conductivity, and superfluid stiffness. Our results reproduce key aspects of observations in the cuprates as analyzed by Michon et al. (arXiv:2205.04030). We also find a regime of increasing zero temperature superfluid stiffness with decreasing superconducting critical temperature, as is observed in bulk cuprates.

cond-mat.str-el↗

arXiv2024

Fluctuation Spectrum of Critical Fermi Surfaces

We investigate the low-energy effective theory of a Fermi surface coupled to an Ising-nematic quantum critical point in (2+1) spacetime dimensions with translation symmetry. We formulate the system using the large $N$ Yukawa-SYK model, whose saddle point is described by the Migdal-Eliashberg equations. The low-energy physics can be revealed by studying the Gaussian fluctuation spectrum around the saddle point, which is generated by the Bethe-Salpeter kernel $K_\text{BS}$. Based on the Ward identities, we propose an inner product on the space of two point functions, which reveals a large number of soft modes of $K_\text{BS}$. These soft modes parameterize deformation of the Fermi surface, and their fluctuation eigenvalues describe their decay rates. We analytically compute these eigenvalues for a circular Fermi surface, and we discover the odd-parity modes to be parametrically longer-lived than the even-parity modes, due to the kinematic constraint of fermions scattering on a convex FS. The sign of the eigenvalues signals an instability of the Ising-nematic quantum critical point at zero temperature for a convex Fermi surface. At finite temperature, the system can be stabilized by thermal fluctuations of the critical boson. We derive an effective action that describes the soft-mode dynamics, and it leads to a linearized Boltzmann equation, where the real part of the soft-mode eigenvalues can be interpreted as the collision rates. The structure of the effective action is similar to the theory of linear bosonization of a Fermi surface. As an application, we investigate the hydrodynamic transport of non-Fermi liquid. Analyzing the Boltzmann equation, we obtain a conventional hydrodynamic transport regime and a tomographic transport regime. In both regimes, the conductance of the system in finite geometry can be a sharp indicator for the soft-mode dynamics and non-Fermi liquid physics.

cond-mat.str-el↗

arXiv2024

SAM-guided Graph Cut for 3D Instance Segmentation

This paper addresses the challenge of 3D instance segmentation by simultaneously leveraging 3D geometric and multi-view image information. Many previous works have applied deep learning techniques to 3D point clouds for instance segmentation. However, these methods often failed to generalize to various types of scenes due to the scarcity and low-diversity of labeled 3D point cloud data. Some recent works have attempted to lift 2D instance segmentations to 3D within a bottom-up framework. The inconsistency in 2D instance segmentations among views can substantially degrade the performance of 3D segmentation. In this work, we introduce a novel 3D-to-2D query framework to effectively exploit 2D segmentation models for 3D instance segmentation. Specifically, we pre-segment the scene into several superpoints in 3D, formulating the task into a graph cut problem. The superpoint graph is constructed based on 2D segmentation models, where node features are obtained from multi-view image features and edge weights are computed based on multi-view segmentation results, enabling the better generalization ability. To process the graph, we train a graph neural network using pseudo 3D labels from 2D segmentation models. Experimental results on the ScanNet, ScanNet++ and KITTI-360 datasets demonstrate that our method achieves robust segmentation performance and can generalize across different types of scenes. Our project page is available at https://zju3dv.github.io/sam_graph.

cs.CV↗

arXiv2024

Quantitative measurement of viscosity in two-dimensional electron fluids

Electron hydrodynamics is an emerging framework that describes dynamics of interacting electron systems as conventional fluids. While evidence for hydrodynamic-like transport is reported in a variety of two-dimensional materials, precise quantitative measurement of the core parameter, electron viscosity, remains challenging. In this work, we demonstrate that magnetoresistance in Corbino-shaped graphene devices offers a simultaneous Ohmmeter/viscosometer, allowing us to disentangle the individual Ohmic and viscous contributions to the transport response, even in the mixed flow regime. Most surprising, we find that in both monolayer and bilayer graphene, the effective electron-electron scattering rate scales linearly with temperature, at odds with the expected $T$-squared dependence expected from conventional Fermi liquid theory, but consistent with a recently identified tomographic flow regime, which was theoretically conjectured to be generic for two-dimensional charged fluids.

cond-mat.mes-hall↗

arXiv2024

Cyclotron resonance and quantum oscillations of critical Fermi surfaces

Kohn's theorem places strong constraints on the cyclotron response of Fermi liquids. Recent observations of a doping dependence in the cyclotron mass of La$_{2-x}$Sr$_x$CuO$_4$ (Legros et al., Phys. Rev. B 106, 195110 (2022)) are therefore surprising because the cyclotron mass can only be renormalized by large momentum umklapp interactions which are not expected to vary significantly with doping. We show that a version of Kohn's theorem continues to apply to disorder-free non-Fermi liquids with a critical boson near zero momentum. However, marginal Fermi liquids arising from a spatially random Yukawa coupling between the electrons and bosons do give rise to significant corrections to the cyclotron mass which we compute. This is the same theory which yields linear-in-temperature resistivity and other properties of strange metals at zero fields (Patel et al., Science 381, 790 (2023)).

cond-mat.str-el↗

arXiv2023

Is the Migdal-Eliashberg Theory for 2+1D Critical Fermi Surface Stable?

We diagnose the stability of the Migdal-Eliashberg theory for a Fermi surface coupled to a gapless boson in 2+1 dimensions. We provide a scheme for diagonalizing the Bethe-Salpeter ladder when small-angle scattering mediated by the boson plays a dominant role. We found a large number of soft modes which correspond to shape fluctuations of the Fermi surface, and these shape deformations follow a diffusion-like dynamics on the Fermi surface. Surprisingly, the odd-parity deformations of a convex Fermi surface becomes unstable near the non-Fermi liquid regime of the Ising-Nematic quantum critical point and our finding calls for revisit of the Migdal-Eliashberg framework. The implication of the Bethe-Salpeter eigenvalues in transport will be discussed in the companion paper [H.Guo,arXiv:2311.03458].

cond-mat.str-el↗

arXiv2023

Fluctuation Spectrum of 2+1D Critical Fermi Surface and its Application to Optical Conductivity and Hydrodynamics

We extend the kinetic operator formalism developed in the companion paper [H.Guo,arXiv:2311.03455] to study the general eigenvalues of the fluctuation normal modes. We apply the formalism to calculate the optical conductivity of a critical Fermi surface near the Ising-Nematic quantum critical point. We find that the conductivity is the sum of multiple conduction channels including both the soft and non-soft eigenvectors of the kinetic operator, and therefore it is not appropriate to interpret the optical conductivity using extended Drude formula for momentum conserved systems. We also show that the propagation of the FS soft modes is governed by a Boltzmann equation from which hydrodynamics emerges. We calculate the viscosity and it shows clear signature of the non-Fermi liquid physics.

cond-mat.str-el↗