arXiv subjects

Haipeng Li

Publications and source records attributed to Haipeng Li.

At least 37 records · Page 2Linked to original sources

arXiv2025

Estimating 2D Camera Motion with Hybrid Motion Basis

Estimating 2D camera motion is a fundamental computer vision task that models the projection of 3D camera movements onto the 2D image plane. Current methods rely on either homography-based approaches, limited to planar scenes, or meshflow techniques that use grid-based local homographies but struggle with complex non-linear transformations. A key insight of our work is that combining flow fields from different homographies creates motion patterns that cannot be represented by any single homography. We introduce CamFlow, a novel framework that represents camera motion using hybrid motion bases: physical bases derived from camera geometry and stochastic bases for complex scenarios. Our approach includes a hybrid probabilistic loss function based on the Laplace distribution that enhances training robustness. For evaluation, we create a new benchmark by masking dynamic objects in existing optical flow datasets to isolate pure camera motion. Experiments show CamFlow outperforms state-of-the-art methods across diverse scenarios, demonstrating superior robustness and generalization in zero-shot settings. Code and datasets are available at our project page: https://lhaippp.github.io/CamFlow/.

cs.CV↗

arXiv2025

The symbolic partition with generalized Koopman analysis

Symbolic dynamics serves as a crucial tool in the study of chaotic systems, prompting extensive research into various methods for symbolic partitioning. The limitations of these methods are heuristic and empirical for the partition the multivariate chaotic state space. Notably, the use of operator theory in partitioning the multivariable chaotic series into precise symbolic cells has been underexplored. In this paper, we propose a novel symbolic partition method, referred to as Koopman Analysis(KA) method, exploiting Koopman operator theory to address the symbolic partition, especially multivariate chaotic time series. We map the chaotic time series into the basis functions to obtain the approximate representation of the Koopman operator.Then we transpose the Koopman approximate matrix and subsequently perform spectral decomposition to obtain the Koopman left eigenfunctions. We apply KA method to one-dimensional unimodal chaotic map to obtain Koopman left eigenfunctions. Then we find some particular eigenfunctions whose eigenvalues are zero, some of which can be used to identify the symbolic boundary of region composed of chaotic series in that the oscillation coincides with the subregion where the theoretical symbolic boundary is located. We refer to the function as the Valid Left Eigenfunction with Zero(VLEZ). Based on the number of oscillations, we further classify VLEZ into two categories. Then, we modify the KA method applicable to chaotic localized subregion and further propose the Generalized Koopman Analysis (GKA) method. The KA method can be also applied to the multimodal maps, multivariate chaotic maps and hyperchaotic maps and their noisy version. The current work can be well further expand to higher dimensional and more complex time series due to its interpretability and availability.

nlin.CD↗

arXiv2025

Coding-Prior Guided Diffusion Network for Video Deblurring

While recent video deblurring methods have advanced significantly, they often overlook two valuable prior information: (1) motion vectors (MVs) and coding residuals (CRs) from video codecs, which provide efficient inter-frame alignment cues, and (2) the rich real-world knowledge embedded in pre-trained diffusion generative models. We present CPGDNet, a novel two-stage framework that effectively leverages both coding priors and generative diffusion priors for high-quality deblurring. First, our coding-prior feature propagation (CPFP) module utilizes MVs for efficient frame alignment and CRs to generate attention masks, addressing motion inaccuracies and texture variations. Second, a coding-prior controlled generation (CPC) module network integrates coding priors into a pretrained diffusion model, guiding it to enhance critical regions and synthesize realistic details. Experiments demonstrate our method achieves state-of-the-art perceptual quality with up to 30% improvement in IQA metrics. Both the code and the codingprior-augmented dataset will be open-sourced.

cs.CV↗

arXiv2025

CodingHomo: Bootstrapping Deep Homography With Video Coding

Homography estimation is a fundamental task in computer vision with applications in diverse fields. Recent advances in deep learning have improved homography estimation, particularly with unsupervised learning approaches, offering increased robustness and generalizability. However, accurately predicting homography, especially in complex motions, remains a challenge. In response, this work introduces a novel method leveraging video coding, particularly by harnessing inherent motion vectors (MVs) present in videos. We present CodingHomo, an unsupervised framework for homography estimation. Our framework features a Mask-Guided Fusion (MGF) module that identifies and utilizes beneficial features among the MVs, thereby enhancing the accuracy of homography prediction. Additionally, the Mask-Guided Homography Estimation (MGHE) module is presented for eliminating undesired features in the coarse-to-fine homography refinement process. CodingHomo outperforms existing state-of-the-art unsupervised methods, delivering good robustness and generalizability. The code and dataset are available at: \href{github}{https://github.com/liuyike422/CodingHomo

cs.CV↗

arXiv2025

Synthetic-to-Real Self-supervised Robust Depth Estimation via Learning with Motion and Structure Priors

Self-supervised depth estimation from monocular cameras in diverse outdoor conditions, such as daytime, rain, and nighttime, is challenging due to the difficulty of learning universal representations and the severe lack of labeled real-world adverse data. Previous methods either rely on synthetic inputs and pseudo-depth labels or directly apply daytime strategies to adverse conditions, resulting in suboptimal results. In this paper, we present the first synthetic-to-real robust depth estimation framework, incorporating motion and structure priors to capture real-world knowledge effectively. In the synthetic adaptation, we transfer motion-structure knowledge inside cost volumes for better robust representation, using a frozen daytime model to train a depth estimator in synthetic adverse conditions. In the innovative real adaptation, which targets to fix synthetic-real gaps, models trained earlier identify the weather-insensitive regions with a designed consistency-reweighting strategy to emphasize valid pseudo-labels. We introduce a new regularization by gathering explicit depth distributions to constrain the model when facing real-world data. Experiments show that our method outperforms the state-of-the-art across diverse conditions in multi-frame and single-frame evaluations. We achieve improvements of 7.5% and 4.3% in AbsRel and RMSE on average for nuScenes and Robotcar datasets (daytime, nighttime, rain). In zero-shot evaluation of DrivingStereo (rain, fog), our method generalizes better than the previous ones.

cs.CV↗

arXiv2025

Urban Sensing Using Existing Fiber-Optic Networks

The analysis of urban seismic signals offers valuable insights into urban environments and society. Yet, accurate detection and localization of seismic sources on a city-wide scale with conventional seismographic network is unavailable due to the prohibitive costs of ultra-dense seismic arrays required for imaging high-frequency anthropogenic sources. Here, we leverage existing fiber-optic networks as a distributed acoustic sensing system to accurately locate urban seismic sources and estimate how their intensity varies over time. By repurposing a 50-kilometer telecommunication fiber into an ultra-dense seismic array, we generate spatiotemporal maps of seismic source power (SSP) across San Jose, California. Our approach overcomes the proximity limitations of urban seismic sensing, enabling accurate localization of remote seismic sources generated by urban activities, such as traffic, construction, and school operations. We also show strong correlations between SSP values and environmental noise levels, as well as various persistent urban features, including land use patterns and demographics.

physics.geo-ph↗

arXiv2025

Single Image Rolling Shutter Removal with Diffusion Models

We present RS-Diffusion, the first Diffusion Models-based method for single-frame Rolling Shutter (RS) correction. RS artifacts compromise visual quality of frames due to the row-wise exposure of CMOS sensors. Most previous methods have focused on multi-frame approaches, using temporal information from consecutive frames for the motion rectification. However, few approaches address the more challenging but important single frame RS correction. In this work, we present an ``image-to-motion" framework via diffusion techniques, with a designed patch-attention module. In addition, we present the RS-Real dataset, comprised of captured RS frames alongside their corresponding Global Shutter (GS) ground-truth pairs. The GS frames are corrected from the RS ones, guided by the corresponding Inertial Measurement Unit (IMU) gyroscope data acquired during capture. Experiments show that RS-Diffusion surpasses previous single-frame RS methods, demonstrates the potential of diffusion-based approaches, and provides a valuable dataset for further research.

cs.CV↗

arXiv2025

HybridReg: Robust 3D Point Cloud Registration with Hybrid Motions

Scene-level point cloud registration is very challenging when considering dynamic foregrounds. Existing indoor datasets mostly assume rigid motions, so the trained models cannot robustly handle scenes with non-rigid motions. On the other hand, non-rigid datasets are mainly object-level, so the trained models cannot generalize well to complex scenes. This paper presents HybridReg, a new approach to 3D point cloud registration, learning uncertainty mask to account for hybrid motions: rigid for backgrounds and non-rigid/rigid for instance-level foregrounds. First, we build a scene-level 3D registration dataset, namely HybridMatch, designed specifically with strategies to arrange diverse deforming foregrounds in a controllable manner. Second, we account for different motion types and formulate a mask-learning module to alleviate the interference of deforming outliers. Third, we exploit a simple yet effective negative log-likelihood loss to adopt uncertainty to guide the feature extraction and correlation computation. To our best knowledge, HybridReg is the first work that exploits hybrid motions for robust point cloud registration. Extensive experiments show HybridReg's strengths, leading it to achieve state-of-the-art performance on both widely-used indoor and outdoor datasets.

cs.CV↗

arXiv2025

Daily Groundwater Monitoring Using Vehicle-DAS Elastic Full-waveform Inversion

Understanding groundwater dynamics is critical for sustainable water management, particularly as climate extremes intensify. However, the resolutions of existing subsurface observational tools are still inadequate for detailed aquifer monitoring and imaging. We introduce an innovative technique for groundwater monitoring using time-lapse full-waveform inversion, leveraging fiber-optic cables as seismic sensors and vehicular traffic as repetitive seismic sources. Over a two-year period along Sandhill Road, California, this approach captures detailed spatiotemporal S-wave velocity variations, revealing a 2.9% reduction corresponding to a 9.0-meter groundwater table rise after atmospheric-river storms in Water Year 2023. Notably, this approach enables the high-resolution daily analysis of rapid aquifer responses. We observe spatially inhomogeneous velocity changes, with less reduction beneath impervious paved zones than under grassy areas, underscoring the impact of urbanization on the natural recharge of aquifers. Our findings highlight the potential of Vehicle-DAS FWI for high-resolution daily monitoring and quantitative spatiotemporal characterizations of groundwater systems.

physics.geo-ph↗

arXiv2024

Characterizing Vehicle-Induced Distributed Acoustic Sensing Signals for Accurate Urban Near-Surface Imaging

Continuous seismic monitoring of the near-surface structure is crucial for urban infrastructure safety, aiding in the detection of sinkholes, subsidence, and other seismic hazards. Utilizing existing telecommunication optical fibers as Distributed Acoustic Sensing (DAS) systems offers a cost-effective method for creating dense seismic arrays in urban areas. DAS leverages roadside fiber-optic cables to record vehicle-induced surface waves for near-surface imaging. However, the influence of roadway vehicle characteristics on their induced surface waves and the resulting imaging of near-surface structures is poorly understood. We investigate surface waves generated by vehicles of varying weights and speeds to provide insights into accurate and efficient near-surface characterization. We first classify vehicles into light, mid-weight, and heavy based on the maximum amplitudes of quasi-static DAS records. Vehicles are also classified by their traveling speed using their arrival times at DAS channels. To investigate how vehicle characteristics influence the induced surface waves, we extract phase velocity dispersion and invert the subsurface structure for each vehicle class by retrieving virtual shot gathers (VSGs). Our results reveal that heavy vehicles produce higher signal-to-noise ratio surface waves, and a sevenfold increase in vehicle weight can reduce uncertainties in phase velocity measurements from dispersion spectra by up to 3X. Thus, data from heavy vehicles better constrain structures at greater depths. Additionally, with driving speeds ranging from 5 to 30 meters per second in our study, differences in the dispersion curves due to vehicle speed are less pronounced than those due to vehicle weight. Our results suggest judiciously selecting and processing surface wave signals from certain vehicle types can improve the quality of near-surface imaging in urban environments.

physics.geo-ph↗

arXiv2024

Automatic Differentiation-based Full Waveform Inversion with Flexible Workflows

Full waveform inversion (FWI) is able to construct high-resolution subsurface models by iteratively minimizing discrepancies between observed and simulated seismic data. However, its implementation can be rather involved for complex wave equations, objective functions, or regularization. Recently, automatic differentiation (AD) has proven to be effective in simplifying solutions of various inverse problems, including FWI. In this study, we present an open-source AD-based FWI framework (ADFWI), which is designed to simplify the design, development, and evaluation of novel approaches in FWI with flexibility. The AD-based framework not only includes forword modeling and associated gradient computations for wave equations in various types of media from isotropic acoustic to vertically or horizontally transverse isotropic elastic, but also incorporates a suite of objective functions, regularization techniques, and optimization algorithms. By leveraging state-of-the-art AD, objective functions such as soft dynamic time warping and Wasserstein distance, which are difficult to apply in traditional FWI are also easily integrated into ADFWI. In addition, ADFWI is integrated with deep learning for implicit model reparameterization via neural networks, which not only introduces learned regularization but also allows rapid estimation of uncertainty through dropout. To manage high memory demands in large-scale inversion associated with AD, the proposed framework adopts strategies such as mini-batch and checkpointing. Through comprehensive evaluations, we demonstrate the novelty, practicality and robustness of ADFWI, which can be used to address challenges in FWI and as a workbench for prompt experiments and the development of new inversion strategies.

cs.LG↗

arXiv2024

Cross-Domain Sequential Recommendation via Neural Process

Cross-Domain Sequential Recommendation (CDSR) is a hot topic in sequence-based user interest modeling, which aims at utilizing a single model to predict the next items for different domains. To tackle the CDSR, many methods are focused on domain overlapped users' behaviors fitting, which heavily relies on the same user's different-domain item sequences collaborating signals to capture the synergy of cross-domain item-item correlation. Indeed, these overlapped users occupy a small fraction of the entire user set only, which introduces a strong assumption that the small group of domain overlapped users is enough to represent all domain user behavior characteristics. However, intuitively, such a suggestion is biased, and the insufficient learning paradigm in non-overlapped users will inevitably limit model performance. Further, it is not trivial to model non-overlapped user behaviors in CDSR because there are no other domain behaviors to collaborate with, which causes the observed single-domain users' behavior sequences to be hard to contribute to cross-domain knowledge mining. Considering such a phenomenon, we raise a challenging and unexplored question: How to unleash the potential of non-overlapped users' behaviors to empower CDSR?

cs.IR↗

arXiv2024

PointRegGPT: Boosting 3D Point Cloud Registration using Generative Point-Cloud Pairs for Training

Data plays a crucial role in training learning-based methods for 3D point cloud registration. However, the real-world dataset is expensive to build, while rendering-based synthetic data suffers from domain gaps. In this work, we present PointRegGPT, boosting 3D point cloud registration using generative point-cloud pairs for training. Given a single depth map, we first apply a random camera motion to re-project it into a target depth map. Converting them to point clouds gives a training pair. To enhance the data realism, we formulate a generative model as a depth inpainting diffusion to process the target depth map with the re-projected source depth map as the condition. Also, we design a depth correction module to alleviate artifacts caused by point penetration during the re-projection. To our knowledge, this is the first generative approach that explores realistic data generation for indoor point cloud registration. When equipped with our approach, several recent algorithms can improve their performance significantly and achieve SOTA consistently on two common benchmarks. The code and dataset will be released on https://github.com/Chen-Suyi/PointRegGPT.

cs.CV↗

arXiv2024

Mitigating Interpretation Bias in Rock Records with Large Language Models: Insights from Paleoenvironmental Analysis

The reconstruction of Earth's history faces significant challenges due to the nonunique interpretations often derived from rock records. The problem has long been recognized but there are no systematic solutions in practice. This study introduces an innovative approach that leverages Large Language Models (LLMs) along with retrieval augmented generation and real-time search capabilities to counteract interpretation biases, thereby enhancing the accuracy and reliability of geological analyses. By applying this framework to sedimentology and paleogeography, we demonstrate its effectiveness in mitigating interpretations biases through the generation and evaluation of multiple hypotheses for the same data, which can effectively reduce human bias. Our research illuminates the transformative potential of LLMs in refining paleoenvironmental studies and extends their applicability across various sub-disciplines of Earth sciences, enabling a deeper and more accurate depiction of Earth's evolution.

physics.geo-ph↗

arXiv2024

RecDiffusion: Rectangling for Image Stitching with Diffusion Models

Image stitching from different captures often results in non-rectangular boundaries, which is often considered unappealing. To solve non-rectangular boundaries, current solutions involve cropping, which discards image content, inpainting, which can introduce unrelated content, or warping, which can distort non-linear features and introduce artifacts. To overcome these issues, we introduce a novel diffusion-based learning framework, \textbf{RecDiffusion}, for image stitching rectangling. This framework combines Motion Diffusion Models (MDM) to generate motion fields, effectively transitioning from the stitched image's irregular borders to a geometrically corrected intermediary. Followed by Content Diffusion Models (CDM) for image detail refinement. Notably, our sampling process utilizes a weighted map to identify regions needing correction during each iteration of CDM. Our RecDiffusion ensures geometric accuracy and overall visual appeal, surpassing all previous methods in both quantitative and qualitative measures when evaluated on public benchmarks. Code is released at https://github.com/lhaippp/RecDiffusion.

cs.CV↗

arXiv2024

HandBooster: Boosting 3D Hand-Mesh Reconstruction by Conditional Synthesis and Sampling of Hand-Object Interactions

Reconstructing 3D hand mesh robustly from a single image is very challenging, due to the lack of diversity in existing real-world datasets. While data synthesis helps relieve the issue, the syn-to-real gap still hinders its usage. In this work, we present HandBooster, a new approach to uplift the data diversity and boost the 3D hand-mesh reconstruction performance by training a conditional generative space on hand-object interactions and purposely sampling the space to synthesize effective data samples. First, we construct versatile content-aware conditions to guide a diffusion model to produce realistic images with diverse hand appearances, poses, views, and backgrounds; favorably, accurate 3D annotations are obtained for free. Then, we design a novel condition creator based on our similarity-aware distribution sampling strategies to deliberately find novel and realistic interaction poses that are distinctive from the training set. Equipped with our method, several baselines can be significantly improved beyond the SOTA on the HO3D and DexYCB benchmarks. Our code will be released on https://github.com/hxwork/HandBooster_Pytorch.

cs.CV↗

arXiv2023

Ab Initio Studies on Interactions in K$_3$C$_{60}$ under High Pressure

Fullerene solids doped with alkali metals (A$_3$C$_{60}$, A = K, Rb, Cs) exhibit a superconducting transition temperature ($T_c$) as high as 40 K, and their unconventional superconducting properties have been a subject of debate. With application of high pressure on K$_3$C$_{60}$ and Rb$_3$C$_{60}$, the experiments demonstrate the decrease of $T_c$. In this paper, we focus on K$_3$C$_{60}$ and derive the structure of K$_3$C$_{60}$ under different pressures based on first-principles calculations, exploring the trends of Coulomb interactions at various pressures. By utilizing the Maximally Localized Wannier function approach, Constrained Density Functional Perturbation Theory (cDFPT), and Constrained Random Phase Approximation (cRPA), we construct a microscopic low-energy model near the Fermi level. Our results strongly indicate that, in the K$_3$C$_{60}$ system, as pressure increases, the effect of phonons is the key to intraorbital electron pairing. There is a dominance of the phonon-driven superconducting mechanism at high pressure.

cond-mat.supr-con↗

arXiv2023

Supervised Homography Learning with Realistic Dataset Generation

In this paper, we propose an iterative framework, which consists of two phases: a generation phase and a training phase, to generate realistic training data and yield a supervised homography network. In the generation phase, given an unlabeled image pair, we utilize the pre-estimated dominant plane masks and homography of the pair, along with another sampled homography that serves as ground truth to generate a new labeled training pair with realistic motion. In the training phase, the generated data is used to train the supervised homography network, in which the training data is refined via a content consistency module and a quality assessment module. Once an iteration is finished, the trained network is used in the next data generation phase to update the pre-estimated homography. Through such an iterative strategy, the quality of the dataset and the performance of the network can be gradually and simultaneously improved. Experimental results show that our method achieves state-of-the-art performance and existing supervised methods can be also improved based on the generated dataset. Code and dataset are available at https://github.com/JianghaiSCU/RealSH.

cs.CV↗