arXiv subjects

Zihua Liu

Publications and source records attributed to Zihua Liu.

18 recordsLinked to original sources

arXiv2026

StereoSplat+: Feed-Forward Stereo Gaussian Splatting with Diffusion-Assisted Progressive Inference

Recent advances in 3D Gaussian Splatting (3DGS) have enabled high-quality, render-ready scene representations for novel-view synthesis. However, most existing 3DGS pipelines rely on multi-view observations (or non-causal access to future frames) to achieve sufficient coverage, which is often unavailable in on-device robotics and AR settings where sensing is restricted to a single stereo rig. Recovering a high-quality 3DGS scene from one stereo observation, therefore, remains challenging due to occlusions, limited field of view, and missing geometry. We present StereoSplat+, a diffusion-enhanced feed-forward framework that enables causal reconstruction from a single stereo pair. Our method builds on two key components. First, we propose StereoSplat, an input-invariant feed-forward 3D Gaussian estimator that takes a variable number of posed stereo pairs as input and predicts high-quality 3D Gaussians. StereoSplat fuses complementary geometry cues via a cost-volume branch and a triplane-based 3D volume branch and leverages continuous pose encoding to generalize across view counts and camera configurations. Second, since multiple posed stereo pairs are typically unavailable at inference time, we introduce a diffusion-enhanced one-shot progressive inference scheme called StereoSplat+: starting from one stereo pair, we render novel stereo views from the predicted 3DGS, refine them with a one-step diffusion enhancer, and feed them back as additional inputs to update the 3DGS. Experiments on the KITTI-360 dataset show that StereoSplat+ improves novel-view rendering quality and geometry accuracy, especially in occluded regions and under strong view extrapolation, outperforming recent feed-forward 3DGS baselines.

cs.CV↗

arXiv2025

VSRD++: Autolabeling for 3D Object Detection via Instance-Aware Volumetric Silhouette Rendering

Monocular 3D object detection is a fundamental yet challenging task in 3D scene understanding. Existing approaches heavily depend on supervised learning with extensive 3D annotations, which are often acquired from LiDAR point clouds through labor-intensive labeling processes. To tackle this problem, we propose VSRD++, a novel weakly supervised framework for monocular 3D object detection that eliminates the reliance on 3D annotations and leverages neural-field-based volumetric rendering with weak 2D supervision. VSRD++ consists of a two-stage pipeline: multi-view 3D autolabeling and subsequent monocular 3D detector training. In the multi-view autolabeling stage, object surfaces are represented as signed distance fields (SDFs) and rendered as instance masks via the proposed instance-aware volumetric silhouette rendering. To optimize 3D bounding boxes, we decompose each instance's SDF into a cuboid SDF and a residual distance field (RDF) that captures deviations from the cuboid. To address the geometry inconsistency commonly observed in volume rendering methods applied to dynamic objects, we model the dynamic objects by including velocity into bounding box attributes as well as assigning confidence to each pseudo-label. Moreover, we also employ a 3D attribute initialization module to initialize the dynamic bounding box parameters. In the monocular 3D object detection phase, the optimized 3D bounding boxes serve as pseudo labels for training monocular 3D object detectors. Extensive experiments on the KITTI-360 dataset demonstrate that VSRD++ significantly outperforms existing weakly supervised approaches for monocular 3D object detection on both static and dynamic scenes. Code is available at https://github.com/Magicboomliu/VSRD_plus_plus

cs.CV↗

arXiv2025

Noise-tolerant correlated coincidence imaging based on super-correlated light at 1550 nm

Single-photon-level imaging at 1550 nm is a key driver for significant advancements in the next-generation laser detection technology. This cutting-edge approach plays a vital role in space ranging, target recognition, and three-dimensional remote sensing. However, it has faced severe challenges such as insufficient noise-tolerant performance. Here, we introduced noise-tolerant correlated coincidence imaging (CCI) based on super-correlated light. The light source, generated through nonlinear interaction between a pulsed laser and a photonic crystal fiber, exhibits a broader power-law photon number probability distribution and extremely strong photon correlation (with second-order correlation function $g^{(2)}(0)$ up to 18,166). Our noise-tolerant CCI can resist random environmental noise up to 100,000 times stronger than the echo signal photons. Super-correlated light offers an exceptionally strong noise tolerance for single-photon-level imaging in extreme environments with intense noise, paving the way for the future development of extremely sensitive light detection.

physics.optics↗

arXiv2025

DMS:Diffusion-Based Multi-Baseline Stereo Generation for Improving Self-Supervised Depth Estimation

While supervised stereo matching and monocular depth estimation have advanced significantly with learning-based algorithms, self-supervised methods using stereo images as supervision signals have received relatively less focus and require further investigation. A primary challenge arises from ambiguity introduced during photometric reconstruction, particularly due to missing corresponding pixels in ill-posed regions of the target view, such as occlusions and out-of-frame areas. To address this and establish explicit photometric correspondences, we propose DMS, a model-agnostic approach that utilizes geometric priors from diffusion models to synthesize novel views along the epipolar direction, guided by directional prompts. Specifically, we finetune a Stable Diffusion model to simulate perspectives at key positions: left-left view shifted from the left camera, right-right view shifted from the right camera, along with an additional novel view between the left and right cameras. These synthesized views supplement occluded pixels, enabling explicit photometric reconstruction. Our proposed DMS is a cost-free, ''plug-and-play'' method that seamlessly enhances self-supervised stereo matching and monocular depth estimation, and relies solely on unlabeled stereo image pairs for both training and synthesizing. Extensive experiments demonstrate the effectiveness of our approach, with up to 35% outlier reduction and state-of-the-art performance across multiple benchmark datasets.

cs.CV↗

arXiv2025

Super-bunching light with giant high-order correlations and extreme multi-photon events

Non-classical light sources emitting bundles of N-photons with strong correlation represent versatile resources of interdisciplinary importance with applications ranging from fundamental tests of quantum mechanics to quantum information processing. Yet, high-order correlations, gN(0),quantifying photon correlation, are still limited to hundreds. Here, we report the generation of a super-bunching light source in photonic crystal fiber with g2(0) reaching 5.86*104 and g5(0) up to 2.72*108, through measuring its photon number probability distributions. under giant g2(0) values, the super-bunching light source presents upturned-tail photon distributions and ubiquitous extreme multi-photon events, where 31 photons from a single light pulse at a mean of 1.99*10-4 photons per pulse have been determined. The probability of this extreme event has been enhanced by 10139 folds compared to a coherent laser with Poissonian distribution. By varying the power of the pumping laser, both photon number distributions and corresponding high-order correlations of this light source can be substantially tailored from Poissonian to super-bunching distributions. These phenomena are attributed to the synchronized nonlinear interactions in photonic crystal fibers pumping by bright squeezed light, and the theoretical simulations agree well with the experimental results. Our research showcases the ability to achieve non-classical light sources with giant high-order correlations and extreme multi-photon events, paving the way for high-order correlation imaging, extreme nonlinear optical effects, quantum information processing, and exploring light-matter interactions with multi-photon physics.

quant-ph↗

arXiv2025

TDM: Temporally-Consistent Diffusion Model for All-in-One Real-World Video Restoration

In this paper, we propose the first diffusion-based all-in-one video restoration method that utilizes the power of a pre-trained Stable Diffusion and a fine-tuned ControlNet. Our method can restore various types of video degradation with a single unified model, overcoming the limitation of standard methods that require specific models for each restoration task. Our contributions include an efficient training strategy with Task Prompt Guidance (TPG) for diverse restoration tasks, an inference strategy that combines Denoising Diffusion Implicit Models~(DDIM) inversion with a novel Sliding Window Cross-Frame Attention (SW-CFA) mechanism for enhanced content preservation and temporal consistency, and a scalable pipeline that makes our method all-in-one to adapt to different video restoration tasks. Through extensive experiments on five video restoration tasks, we demonstrate the superiority of our method in generalization capability to real-world videos and temporal consistency preservation over existing state-of-the-art methods. Our method advances the video restoration task by providing a unified solution that enhances video quality across multiple applications.

cs.CV↗

arXiv2024

Disparity Estimation Using a Quad-Pixel Sensor

A quad-pixel (QP) sensor is increasingly integrated into commercial mobile cameras. The QP sensor has a unit of 2$\times$2 four photodiodes under a single microlens, generating multi-directional phase shifting when out-focus blurs occur. Similar to a dual-pixel (DP) sensor, the phase shifting can be regarded as stereo disparity and utilized for depth estimation. Based on this, we propose a QP disparity estimation network (QPDNet), which exploits abundant QP information by fusing vertical and horizontal stereo-matching correlations for effective disparity estimation. We also present a synthetic pipeline to generate a training dataset from an existing RGB-Depth dataset. Experimental results demonstrate that our QPDNet outperforms state-of-the-art stereo and DP methods. Our code and synthetic dataset are available at https://github.com/Zhuofeng-Wu/QPDNet.

cs.CV↗

arXiv2024

VSRD: Instance-Aware Volumetric Silhouette Rendering for Weakly Supervised 3D Object Detection

Monocular 3D object detection poses a significant challenge in 3D scene understanding due to its inherently ill-posed nature in monocular depth estimation. Existing methods heavily rely on supervised learning using abundant 3D labels, typically obtained through expensive and labor-intensive annotation on LiDAR point clouds. To tackle this problem, we propose a novel weakly supervised 3D object detection framework named VSRD (Volumetric Silhouette Rendering for Detection) to train 3D object detectors without any 3D supervision but only weak 2D supervision. VSRD consists of multi-view 3D auto-labeling and subsequent training of monocular 3D object detectors using the pseudo labels generated in the auto-labeling stage. In the auto-labeling stage, we represent the surface of each instance as a signed distance field (SDF) and render its silhouette as an instance mask through our proposed instance-aware volumetric silhouette rendering. To directly optimize the 3D bounding boxes through rendering, we decompose the SDF of each instance into the SDF of a cuboid and the residual distance field (RDF) that represents the residual from the cuboid. This mechanism enables us to optimize the 3D bounding boxes in an end-to-end manner by comparing the rendered instance masks with the ground truth instance masks. The optimized 3D bounding boxes serve as effective training data for 3D object detection. We conduct extensive experiments on the KITTI-360 dataset, demonstrating that our method outperforms the existing weakly supervised 3D object detection methods. The code is available at https://github.com/skmhrk1209/VSRD.

cs.CV↗

arXiv2024

CFDNet: A Generalizable Foggy Stereo Matching Network with Contrastive Feature Distillation

Stereo matching under foggy scenes remains a challenging task since the scattering effect degrades the visibility and results in less distinctive features for dense correspondence matching. While some previous learning-based methods integrated a physical scattering function for simultaneous stereo-matching and dehazing, simply removing fog might not aid depth estimation because the fog itself can provide crucial depth cues. In this work, we introduce a framework based on contrastive feature distillation (CFD). This strategy combines feature distillation from merged clean-fog features with contrastive learning, ensuring balanced dependence on fog depth hints and clean matching features. This framework helps to enhance model generalization across both clean and foggy environments. Comprehensive experiments on synthetic and real-world datasets affirm the superior strength and adaptability of our method.

cs.CV↗

arXiv2024

Digging Into Normal Incorporated Stereo Matching

Despite the remarkable progress facilitated by learning-based stereo-matching algorithms, disparity estimation in low-texture, occluded, and bordered regions still remains a bottleneck that limits the performance. To tackle these challenges, geometric guidance like plane information is necessary as it provides intuitive guidance about disparity consistency and affinity similarity. In this paper, we propose a normal incorporated joint learning framework consisting of two specific modules named non-local disparity propagation(NDP) and affinity-aware residual learning(ARL). The estimated normal map is first utilized for calculating a non-local affinity matrix and a non-local offset to perform spatial propagation at the disparity level. To enhance geometric consistency, especially in low-texture regions, the estimated normal map is then leveraged to calculate a local affinity matrix, providing the residual learning with information about where the correction should refer and thus improving the residual learning efficiency. Extensive experiments on several public datasets including Scene Flow, KITTI 2015, and Middlebury 2014 validate the effectiveness of our proposed method. By the time we finished this work, our approach ranked 1st for stereo matching across foreground pixels on the KITTI 2015 dataset and 3rd on the Scene Flow dataset among all the published works.

cs.CV↗

arXiv2023

Global Occlusion-Aware Transformer for Robust Stereo Matching

Despite the remarkable progress facilitated by learning-based stereo-matching algorithms, the performance in the ill-conditioned regions, such as the occluded regions, remains a bottleneck. Due to the limited receptive field, existing CNN-based methods struggle to handle these ill-conditioned regions effectively. To address this issue, this paper introduces a novel attention-based stereo-matching network called Global Occlusion-Aware Transformer (GOAT) to exploit long-range dependency and occlusion-awareness global context for disparity estimation. In the GOAT architecture, a parallel disparity and occlusion estimation module PDO is proposed to estimate the initial disparity map and the occlusion mask using a parallel attention mechanism. To further enhance the disparity estimates in the occluded regions, an occlusion-aware global aggregation module (OGA) is proposed. This module aims to refine the disparity in the occluded regions by leveraging restricted global correlation within the focus scope of the occluded areas. Extensive experiments were conducted on several public benchmark datasets including SceneFlow, KITTI 2015, and Middlebury. The results show that the proposed GOAT demonstrates outstanding performance among all benchmarks, particularly in the occluded regions.

cs.CV↗

arXiv2023

Structural dynamics of a model of amorphous silicon

We perform extensive simulations and systematic statistical analyses of the structural dynamics of amorphous silicon. The simulations follow the dynamics introduced by Wooten, Winer and Weaire: the energy is obtained with the Keating potential, and the dynamics consists of bond transpositions proposed at random locations and accepted with the Metropolis acceptance ratio. The structural quantities we track are the variations in time of the lateral lengths ($L_x$,$L_y$,$L_z$) of the cuboid simulation cell. We transform these quantities into the volume $V$ and two aspect ratios $B_1$ and $B_2$. Our analysis reveals that at short times, the mean squared displacement (MSD) for all of them exhibits normal diffusion. At longer times, they cross over to anomalous diffusion, with a temperature-dependent anomalous exponent $α<1$. We analyze our findings in the light of two standard models in statistical physics that feature anomalous dynamics, $viz.$, continuous time random walker (CTRW) and fractional Brownian motion (fBm). We obtain the distribution of waiting times, and find that the data are consistent with a stretched-exponential decay. We also show that the three quantities, $V$, $B_1$ and $B_2$ exhibit negative velocity autocorrelation functions. These observations together suggest that the dynamics of the material belong to the fBm class.

cond-mat.soft↗

arXiv2023

Domain coarsening in polycrystalline graphene

Graphene is a two-dimensional carbon material which exhibits exceptional properties, making it highly suitable for a wide range of applications. Practical graphene fabrication often yields a polycrystalline structure with many inherent defects, which significantly influence its performance. In this study, we utilize a Monte Carlo approach based on the optimized Wooten, Winer and Weaire (WWW) algorithm to simulate the crystalline domain coarsening process of polycrystalline graphene. Our sample configurations show excellent agreement with experimental data. We conduct statistical analyses of the bond and angle distribution, temporal evolution of the defect distribution, and spatial correlation of the lattice orientation that follows a stretched exponential distribution. Furthermore, we thoroughly investigate the diffusion behavior of defects and find that the changes in domain size follow a power-law distribution. We briefly discuss the possible connections of these results to (and differences from) domain growth processes in other statistical models, such as the Ising dynamics. We also examine the impact of buckling of polycrystalline graphene on the crystallization rate under substrate effects. Our findings may offer valuable guidance and insights for both theoretical investigations and experimental advancements.

cond-mat.dis-nn↗

arXiv2023

Critical dynamical behavior of the Ising model

We investigate the dynamical critical behavior of the two- and three-dimensional Ising model with Glauber dynamics in equilibrium. In contrast to the usual standing, we focus on the mean-squared deviation of the magnetization $M$, MSD$_M$, as a function of time, as well as on the autocorrelation function of $M$. These two functions are distinct but closely related. We find that MSD$_M$ features a first crossover at time $τ_1 \sim L^{z_{1}}$, from ordinary diffusion with MSD$_M$ $\sim t$, to anomalous diffusion with MSD$_M$ $\sim t^α$. Purely on numerical grounds, we obtain the values $z_1=0.45(5)$ and $α=0.752(5)$ for the two-dimensional Ising ferromagnet. Related to this, the magnetization autocorrelation function crosses over from an exponential decay to a stretched-exponential decay. At later times, we find a second crossover at time $τ_2 \sim L^{z_{2}}$. Here, MSD$_M$ saturates to its late-time value $\sim L^{2+γ/ν}$, while the autocorrelation function crosses over from stretched-exponential decay to simple exponential one. We also confirm numerically the value $z_{2}=2.1665(12)$, earlier reported as the single dynamic exponent. Continuity of MSD$_M$ requires that $α(z_{2}-z_{1})=γ/ν-z_1$. We speculate that $z_{1} = 1/2$ and $α= 3/4$, values that indeed lead to the expected $z_{2} = 13/6$ result. A complementary analysis for the three-dimensional Ising model provides the estimates $z_{1} = 1.35(2)$, $α=0.90(2)$, and $z_{2} = 2.032(3)$. While $z_{2}$ has attracted significant attention in the literature, we argue that for all practical purposes $z_{1}$ is more important, as it determines the number of statistically independent measurements during a long simulation.

cond-mat.stat-mech↗

arXiv2022

Structural dynamics of polycrystalline graphene

The exceptional properties of the two-dimensional material graphene make it attractive for multiple functional applications, whose large-area samples are typically polycrystalline. Here, we study the mechanical properties of graphene in computer simulations and connect these to the experimentally relevant mechanical properties. In particular, we study the fluctuations in the lateral dimensions of the periodic simulation cell. We show that over short time scales, both the area A and the aspect ratio B of the rectangular periodic box show diffusive behavior under zero external field during dynamical evolution, with diffusion coefficients DA and DB that are related to each other. At longer times, fluctuations in A are bounded, while those in B are not. This makes the direct determination of DB much more accurate, from which DA can then be derived indirectly. We then show that the dynamic behavior of polycrystalline graphene under external forces can also be derived from DA and DB via the Nernst-Einstein relation. Additionally, we study how the diffusion coefficients depend on structural properties of the polycrystalline graphene, in particular, the density of defects.

cond-mat.mtrl-sci↗

arXiv2020

A Dataset and Baselines for Visual Question Answering on Art

Answering questions related to art pieces (paintings) is a difficult task, as it implies the understanding of not only the visual information that is shown in the picture, but also the contextual knowledge that is acquired through the study of the history of art. In this work, we introduce our first attempt towards building a new dataset, coined AQUA (Art QUestion Answering). The question-answer (QA) pairs are automatically generated using state-of-the-art question generation methods based on paintings and comments provided in an existing art understanding dataset. The QA pairs are cleansed by crowdsourcing workers with respect to their grammatical correctness, answerability, and answers' correctness. Our dataset inherently consists of visual (painting-based) and knowledge (comment-based) questions. We also present a two-branch model as baseline, where the visual and knowledge questions are handled independently. We extensively compare our baseline model against the state-of-the-art models for question answering, and we provide a comprehensive study about the challenges and potential future directions for visual question answering on art.

cs.CV↗

arXiv2019

SURREAL-System: Fully-Integrated Stack for Distributed Deep Reinforcement Learning

We present an overview of SURREAL-System, a reproducible, flexible, and scalable framework for distributed reinforcement learning (RL). The framework consists of a stack of four layers: Provisioner, Orchestrator, Protocol, and Algorithms. The Provisioner abstracts away the machine hardware and node pools across different cloud providers. The Orchestrator provides a unified interface for scheduling and deploying distributed algorithms by high-level description, which is capable of deploying to a wide range of hardware from a personal laptop to full-fledged cloud clusters. The Protocol provides network communication primitives optimized for RL. Finally, the SURREAL algorithms, such as Proximal Policy Optimization (PPO) and Evolution Strategies (ES), can easily scale to 1000s of CPU cores and 100s of GPUs. The learning performances of our distributed algorithms establish new state-of-the-art on OpenAI Gym and Robotics Suites tasks.

cs.LG↗

arXiv2019

Question Answering via Web Extracted Tables and Pipelined Models

In this paper, we describe a dataset and baseline result for a question answering that utilizes web tables. It contains commonly asked questions on the web and their corresponding answers found in tables on websites. Our dataset is novel in that every question is paired with a table of a different signature. In particular, the dataset contains two classes of tables: entity-instance tables and the key-value tables. Each QA instance comprises a table of either kind, a natural language question, and a corresponding structured SQL query. We build our model by dividing question answering into several tasks, including table retrieval and question element classification, and conduct experiments to measure the performance of each task. We extract various features specific to each task and compose a full pipeline which constructs the SQL query from its parts. Our work provides qualitative results and error analysis for each task, and identifies in detail the reasoning required to generate SQL expressions from natural language questions. This analysis of reasoning informs future models based on neural machine learning.

cs.CL↗