arXiv subjects

Hao Zhang

Publications and source records attributed to Hao Zhang.

At least 595 records · Page 33Linked to original sources

arXiv2024

CRAYM: Neural Field Optimization via Camera RAY Matching

We introduce camera ray matching (CRAYM) into the joint optimization of camera poses and neural fields from multi-view images. The optimized field, referred to as a feature volume, can be "probed" by the camera rays for novel view synthesis (NVS) and 3D geometry reconstruction. One key reason for matching camera rays, instead of pixels as in prior works, is that the camera rays can be parameterized by the feature volume to carry both geometric and photometric information. Multi-view consistencies involving the camera rays and scene rendering can be naturally integrated into the joint optimization and network training, to impose physically meaningful constraints to improve the final quality of both the geometric reconstruction and photorealistic rendering. We formulate our per-ray optimization and matched ray coherence by focusing on camera rays passing through keypoints in the input images to elevate both the efficiency and accuracy of scene correspondences. Accumulated ray features along the feature volume provide a means to discount the coherence constraint amid erroneous ray matching. We demonstrate the effectiveness of CRAYM for both NVS and geometry reconstruction, over dense- or sparse-view settings, with qualitative and quantitative comparisons to state-of-the-art alternatives.

cs.CV↗

arXiv2024

Simultaneous two-dimensional velocity and distance measurements based on laser triangulation

Laser triangulation sensors are widely used in industry for surface inspection due to simple setup, micron precision and low cost. Conventional laser triangulation methods only enable axial distance measurement limiting further applications, and their lateral resolution is limited by surface microstructure. For overcoming these issues, based on the geometric optics we propose novel theoretical models and methods to achieve lateral velocity measurement. Moreover, a novel axial distance measurement method using edge detection is presented, which can increase the lateral resolution by the order of one magnitude. The performance of the proposed methods are validated through simultaneous orthogonal velocity and distance measurements on a moving established metal specimen, showing the relative error and relative uncertainty can reach 10^{-4}. The versatility of this multi degree of freedom measurement method paves the way for its broad application across all laser triangulation systems. Therefore, this simultaneous two-dimensional velocity and distance sensing approach can propel advancements in dynamic behavior discipline, including but not limited to motion mechanology and fluid mechanics.

physics.ins-det↗

arXiv2024

Cross-modal Medical Image Generation Based on Pyramid Convolutional Attention Network

The integration of multimodal medical imaging can provide complementary and comprehensive information for the diagnosis of Alzheimer's disease (AD). However, in clinical practice, since positron emission tomography (PET) is often missing, multimodal images might be incomplete. To address this problem, we propose a method that can efficiently utilize structural magnetic resonance imaging (sMRI) image information to generate high-quality PET images. Our generation model efficiently utilizes pyramid convolution combined with channel attention mechanism to extract multi-scale local features in sMRI, and injects global correlation information into these features using self-attention mechanism to ensure the restoration of the generated PET image on local texture and global structure. Additionally, we introduce additional loss functions to guide the generation model in producing higher-quality PET images. Through experiments conducted on publicly available ADNI databases, the generated images outperform previous research methods in various performance indicators (average absolute error: 0.0194, peak signal-to-noise ratio: 29.65, structural similarity: 0.9486) and are close to real images. In promoting AD diagnosis, the generated images combined with their corresponding sMRI also showed excellent performance in AD diagnosis tasks (classification accuracy: 94.21 %), and outperformed previous research methods of the same type. The experimental results demonstrate that our method outperforms other competing methods in quantitative metrics, qualitative visualization, and evaluation criteria.

cs.CE↗

arXiv2024

Block Modulating Video Compression: An Ultra Low Complexity Image Compression Encoder for Resource Limited Platforms

We consider the image and video compression on resource limited platforms. An ultra low-cost image encoder, named Block Modulating Video Compression (BMVC) with an encoding complexity ${\cal O}(1)$ is proposed to be implemented on mobile platforms with low consumption of power and computation resources. We also develop two types of BMVC decoders, implemented by deep neural networks. The first BMVC decoder is based on the Plug-and-Play (PnP) algorithm, which is flexible to different compression ratios. And the second decoder is a memory efficient end-to-end convolutional neural network, which aims for real-time decoding. Extensive results on the high definition images and videos demonstrate the superior performance of the proposed codec and the robustness against bit quantization.

eess.IV↗

arXiv2024

Why Anticipatory Sensing Matters in Commercial ACC Systems under Cut-In Scenarios: A Perspective from Stochastic Safety Analysis

This study presents an analytical solution for the vehicle state evolution of Adaptive Cruise Control (ACC) systems under cut-in scenarios, incorporating sensing delays and anticipation using the Lambert W function. The theoretical analysis demonstrates that the vehicle state evolution and the corresponding safety of ACC in cut-in situations are influenced by multiple factors, including the original leading vehicle's state, the initial conditions of the cut-in vehicle, subsequent cut-in maneuvers, sensing delays, and the ACC's anticipation capabilities. To quantitatively assess these influences, a series of numerical experiments were conducted to perform a stochastic safety analysis of ACC systems, accounting for embedded sensing delays and anticipation, using empirically calibrated control parameters from real-world data. The experiments revealed that the impact of sensing delays on ACC is multifaceted. Specifically, sensing delays negatively affect ACC stability, with the severity increasing as the delay lengthens. Furthermore, collision risk in cut-in scenarios becomes more significant with sensing delays, particularly when the cut-in vehicle is slower than the following vehicle and when cut-ins are aggressive. However, anticipation plays a crucial role in mitigating these risks. Even with a 0.6-second anticipation, collision risk can be reduced by 91% in highly adverse scenarios. Finally, both sensing delays and anticipation have effects that intensify with their duration. An anticipation period of 2 seconds effectively ensures safety in aggressive cut-in conditions, even in the presence of sensing delays.

eess.SY↗

arXiv2024

On the boundedness and Schatten class property of noncommutative martingale paraproducts and operator-valued commutators

We study the Schatten class membership of semicommutative martingale paraproducts and use the transference method to describe Schatten class membership of purely noncommutative martingale paraproducts, especially for CAR algebras and $\mathop{\otimes}\limits_{k=1}^{\infty}\mathbb{M}_{d}$ in terms of martingale Besov spaces. Using Hytönen's dyadic martingale technique, we also obtain sufficient conditions on the Schatten class membership and the boundedness of operator-valued commutators involving general singular integral operators. We establish the complex median method, which is applicable to complex-valued functions. We apply it to get the optimal necessary conditions on the Schatten class membership of operator-valued commutators associated with non-degenerate kernels in Hytönen's sense. This resolves the problem on the characterization of Schatten class membership of operator-valued commutators. Our results are new even in the scalar case. Our new approach is built on Hytönen's dyadic martingale technique and the complex median method. Compared with all the previous ones, this new one is more powerful in several aspects: $(a)$ it permits us to deal with more general singular integral operators with little smoothness; $(b)$ it allows us to deal with commutators with complex-valued kernels; $(c)$ it goes much further beyond the scalar case and can be applied to the semicommutative setting. By a weak-factorization type decomposition, we get some necessary but not optimal conditions on the boundedness of operator-valued commutators. In addition, we give a new proof of the boundedness of commutators still involving general singular integral operators concerning $BMO$ spaces in the commutative setting.

math.FA↗

arXiv2024

Quantum K Whitney relations for partial flag varieties

In a recent paper, we stated conjectural presentations for the equivariant quantum K ring of partial flag varieties, motivated by physics considerations. In this companion paper, we analyze these presentations mathematically. We start by proving a Nakayama type result for quantum K theory: if the conjectured set of relations deforms a complete set of relations of the classical K theory ring, then they must form a complete set of relations for the quantum K ring. We prove the conjectured presentation in the case of the incidence varieties, and we show that if a quantum K divisor axiom holds (as conjectured by Buch and Mihalcea), then the conjectured presentation also holds for the complete flag variety. Finally, we briefly revisit the change of variables relating the mathematics and physics presentations.

math.AG↗

arXiv2024

Multipolar Skyrmion Crystals in Non-Kramers Doublet Systems

We study the Kondo lattice model of multipolar magnetic moments interacting with conduction electrons on a triangular lattice. Bond-dependent electron hoppings induce a compass-like anisotropy in the effective Ruderman-Kittel-Kasuya-Yosida interaction between multipolar moments. This unique anisotropy stabilizes multipolar skyrmion crystals at zero magnetic field. In a unit cell, the skyrmion fractionalizes into meron composites subject to the conservation of total topological charge. Diverse multipolar phases in the phase diagram give rise to novel spontaneous Hall response of conduction electrons.

cond-mat.str-el↗

arXiv2024

Boundedness of operator-valued commutators involving martingale paraproducts

Let $1<p<\infty$. We show the boundedness of operator-valued commutators $[π_a,M_b]$ on the noncommutative $L_p(L_\infty(\mathbb{R})\otimes \mathcal{M})$ for any von Neumann algebra $\mathcal{M}$, where $π_a$ is the $d$-adic martingale paraproduct with symbol $a\in BMO^d(\mathbb{R})$ and $M_b$ is the noncommutative left multiplication operator with $b\in BMO^d_\mathcal{M}(\mathbb{R})$. Besides, we consider the extrapolation property of semicommutative $d$-adic martingale paraproducts in terms of the $BMO^d_\mathcal{M}(\mathbb{R})$ space.

math.OA↗

arXiv2024

On the best constants of the noncommutative Littlewood-Paley-Stein inequalities

Let $1 0}$ be a noncommutative symmetric diffusion semigroup on a semifinite von Neumann algebra $\mathcal{M}$, and let $\{P_t\}_{t>0}$ be its associated subordinated Poisson semigroup. The celebrated noncommutative Littlewood-Paley-Stein inequality asserts that for any $x\in L_p(\mathcal{M})$, \begin{equation*} α_p^{-1}\|x\|_{p}\le \|x\|_{p,P}\le β_p \|x\|_{p}, \end{equation*} where $\|\cdot\|_{p,P}$ is the $L_p(\mathcal{M})$-norm of square functions associated with $\{P_t\}_{t>0}$, and $α_p, β_p$ are the best constants only depending on $p$. We show that as $p\to \infty$, $$ β_p\lesssim p, $$ and $p$ is the optimal possible order of $β_p$ as well. We also obtain some lower and upper bounds of $α_p$ and $β_p$ in the other cases.

math.OA↗

arXiv2024

Complex median method and Schatten class membership of commutators

This article is devoted to the study of the Schatten class membership of commutators involving singular integral operators. We utilize martingale paraproducts and Hytönen's dyadic martingale technique to obtain sufficient conditions on the weak-type and strong-type Schatten class membership of commutators in terms of Sobolev spaces and Besov spaces respectively. We also establish the complex median method, which is applicable to complex-valued functions. We apply it to get the optimal necessary conditions on the weak-type and strong-type Schatten class membership of commutators associated with non-degenerate kernels. This resolves the problem on the characterization of the weak-type and strong-type Schatten class membership of commutators. Our new approach is built on Hytönen's dyadic martingale technique and the complex median method. Compared with all the previous ones, this new one is more powerful in several aspects: $(a)$ it permits us to deal with more general singular integral operators with little smoothness; $(b)$ it allows us to deal with commutators with complex-valued kernels; $(c)$ it turns out to be powerful enough to deal with the weak-type and strong-type Schatten class of commutators in a universal way.

math.FA↗

arXiv2024

When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System

Artificial intelligence (AI) has significantly advanced Earth sciences, yet its full potential in to comprehensively modeling Earth's complex dynamics remains unrealized. Geoscience foundation models (GFMs) emerge as a paradigm-shifting solution, integrating extensive cross-disciplinary data to enhance the simulation and understanding of Earth system dynamics. These data-centric AI models extract insights from petabytes of structured and unstructured data, effectively addressing the complexities of Earth systems that traditional models struggle to capture. The unique strengths of GFMs include flexible task specification, diverse input-output capabilities, and multi-modal knowledge representation, enabling analyses that surpass those of individual data sources or traditional AI methods. This review not only highlights the key advantages of GFMs, but also presents essential techniques for their construction, with a focus on transformers, pre-training, and adaptation strategies. Subsequently, we examine recent advancements in GFMs, including large language models, vision models, and vision-language models, particularly emphasizing the potential applications in remote sensing. Additionally, the review concludes with a comprehensive analysis of the challenges and future trends in GFMs, addressing five critical aspects: data integration, model complexity, uncertainty quantification, interdisciplinary collaboration, and concerns related to privacy, trust, and security. This review offers a comprehensive overview of emerging geoscientific research paradigms, emphasizing the untapped opportunities at the intersection of advanced AI techniques and geoscience. It examines major methodologies, showcases advances in large-scale models, and discusses the challenges and prospects that will shape the future landscape of GFMs.

cs.AI↗

arXiv2024

Cross-Domain Pre-training with Language Models for Transferable Time Series Representations

Advancements in self-supervised pre-training (SSL) have significantly advanced the field of learning transferable time series representations, which can be very useful in enhancing the downstream task. Despite being effective, most existing works struggle to achieve cross-domain SSL pre-training, missing valuable opportunities to integrate patterns and features from different domains. The main challenge lies in the significant differences in the characteristics of time-series data across different domains, such as variations in the number of channels and temporal resolution scales. To address this challenge, we propose CrossTimeNet, a novel cross-domain SSL learning framework to learn transferable knowledge from various domains to largely benefit the target downstream task. One of the key characteristics of CrossTimeNet is the newly designed time series tokenization module, which could effectively convert the raw time series into a sequence of discrete tokens based on a reconstruction optimization process. Besides, we highlight that predicting a high proportion of corrupted tokens can be very helpful for extracting informative patterns across different domains during SSL pre-training, which has been largely overlooked in past years. Furthermore, unlike previous works, our work treats the pre-training language model (PLM) as the initialization of the encoder network, investigating the feasibility of transferring the knowledge learned by the PLM to the time series area. Through these efforts, the path to cross-domain pre-training of a generic time series model can be effectively paved. We conduct extensive experiments in a real-world scenario across various time series classification domains. The experimental results clearly confirm CrossTimeNet's superior performance.

cs.LG↗

arXiv2024

Linear and Nonlinear Coupling of Light in Twin-Resonators with Kerr Nonlinearity

Nonlinear effects in microresonators are efficient building blocks for all-optical computing and telecom systems. With the latest advances in microfabrication, coupled microresonators are used in a rapidly growing number of applications. In this work, we investigate the coupling between twin-resonators in the presence of Kerr-nonlinearity. We use an experimental setup with controllable coupling between two high-Q resonators and discuss the effects caused by the simultaneous presence of linear and non-linear coupling between the optical fields. Linear-coupling-induced mode splitting is observed at low input powers, with the controllable coupling leading to a tunable mode splitting. At high input powers, the hybridized resonances show spontaneous symmetry breaking (SSB) effects, in which the optical power is unevenly distributed between the resonators. Our experimental results are supported by a detailed theoretical model of nonlinear twin-resonators. With the recent interest in coupled resonator systems for neuromorphic computing, quantum systems, and optical frequency comb generation, our work provides important insights into the behavior of these systems at high circulating powers.

physics.optics↗

arXiv2024

Text-DiFuse: An Interactive Multi-Modal Image Fusion Framework based on Text-modulated Diffusion Model

Existing multi-modal image fusion methods fail to address the compound degradations presented in source images, resulting in fusion images plagued by noise, color bias, improper exposure, \textit{etc}. Additionally, these methods often overlook the specificity of foreground objects, weakening the salience of the objects of interest within the fused images. To address these challenges, this study proposes a novel interactive multi-modal image fusion framework based on the text-modulated diffusion model, called Text-DiFuse. First, this framework integrates feature-level information integration into the diffusion process, allowing adaptive degradation removal and multi-modal information fusion. This is the first attempt to deeply and explicitly embed information fusion within the diffusion process, effectively addressing compound degradation in image fusion. Second, by embedding the combination of the text and zero-shot location model into the diffusion fusion process, a text-controlled fusion re-modulation strategy is developed. This enables user-customized text control to improve fusion performance and highlight foreground objects in the fused images. Extensive experiments on diverse public datasets show that our Text-DiFuse achieves state-of-the-art fusion performance across various scenarios with complex degradation. Moreover, the semantic segmentation experiment validates the significant enhancement in semantic performance achieved by our text-controlled fusion re-modulation strategy. The code is publicly available at https://github.com/Leiii-Cao/Text-DiFuse.

cs.CV↗

arXiv2024

Unsupervised Modality Adaptation with Text-to-Image Diffusion Models for Semantic Segmentation

Despite their success, unsupervised domain adaptation methods for semantic segmentation primarily focus on adaptation between image domains and do not utilize other abundant visual modalities like depth, infrared and event. This limitation hinders their performance and restricts their application in real-world multimodal scenarios. To address this issue, we propose Modality Adaptation with text-to-image Diffusion Models (MADM) for semantic segmentation task which utilizes text-to-image diffusion models pre-trained on extensive image-text pairs to enhance the model's cross-modality capabilities. Specifically, MADM comprises two key complementary components to tackle major challenges. First, due to the large modality gap, using one modal data to generate pseudo labels for another modality suffers from a significant drop in accuracy. To address this, MADM designs diffusion-based pseudo-label generation which adds latent noise to stabilize pseudo-labels and enhance label accuracy. Second, to overcome the limitations of latent low-resolution features in diffusion models, MADM introduces the label palette and latent regression which converts one-hot encoded labels into the RGB form by palette and regresses them in the latent space, thus ensuring the pre-trained decoder for up-sampling to obtain fine-grained features. Extensive experimental results demonstrate that MADM achieves state-of-the-art adaptation performance across various modality tasks, including images to depth, infrared, and event modalities. We open-source our code and models at https://github.com/XiaRho/MADM.

cs.CV↗

arXiv2024

DiffusionVel: Multi-Information Integrated Velocity Inversion Using Generative Diffusion Models

Full waveform inversion (FWI) is capable of reconstructing subsurface properties with high resolution from seismic data. However, conventional FWI faces challenges such as cycle-skipping and high computational costs. Recently, deep learning method has emerged as a promising solution for efficient velocity estimation. We develop DiffusionVel, a data-driven technique based on the state-of-the-art generative diffusion models (GDMs) with integration of multiple information including seismic data, background velocity, geological knowledge, and well logs. We use two separate conditional GDMs, namely the seismic-data GDM and the well-log GDM, and an unconditional GDM, i.e., the geology-oriented GDM, to adapt the generated velocity model to the constraints of seismic data, well logs, and prior geological knowledge, respectively. Besides, the background velocity can be incorporated into the generated velocity model with a low-pass filter. The generation of these GDM are then combined together with a weighted summation in the sampling process. We can flexibly control the constraints from each information by adjusting the weighting factors. We make a comprehensive comparison between the proposed DiffusionVel and three previously-developed methods including conventional FWI, InversionNet, and VelocityGAN by using the OpenFWI datasets and the Hess VTI model example. The test results demonstrate that the proposed DiffusionVel method predicts the velocity model reasonably by integrating multiple information effectively.

physics.geo-ph↗

arXiv2024

LLaVA-OneVision: Easy Visual Task Transfer

We present LLaVA-OneVision, a family of open large multimodal models (LMMs) developed by consolidating our insights into data, models, and visual representations in the LLaVA-NeXT blog series. Our experimental results demonstrate that LLaVA-OneVision is the first single model that can simultaneously push the performance boundaries of open LMMs in three important computer vision scenarios: single-image, multi-image, and video scenarios. Importantly, the design of LLaVA-OneVision allows strong transfer learning across different modalities/scenarios, yielding new emerging capabilities. In particular, strong video understanding and cross-scenario capabilities are demonstrated through task transfer from images to videos.

cs.CV↗