arXiv subjects

Xiaowen Zhang

Publications and source records attributed to Xiaowen Zhang.

At least 37 records · Page 2Linked to original sources

arXiv2026

Bootstrapping MLLM for Weakly-Supervised Class-Agnostic Object Counting

Object counting is a fundamental task in computer vision, with broad applicability in many real-world scenarios. Fully-supervised counting methods require costly point-level annotations per object. Few weakly-supervised methods leverage only image-level object counts as supervision and achieve fairly promising results. They are, however, often limited to counting a single category, e.g. person. In this paper, we propose WS-COC, the first MLLM-driven weakly-supervised framework for class-agnostic object counting. Instead of directly fine-tuning MLLMs to predict object counts, which can be challenging due to the modality gap, we incorporate three simple yet effective strategies to bootstrap the counting paradigm in both training and testing: First, a divide-and-discern dialogue tuning strategy is proposed to guide the MLLM to determine whether the object count falls within a specific range and progressively break down the range through multi-round dialogue. Second, a compare-and-rank count optimization strategy is introduced to train the MLLM to optimize the relative ranking of multiple images according to their object counts. Third, a global-and-local counting enhancement strategy aggregates and fuses local and global count predictions to improve counting performance in dense scenes. Extensive experiments on FSC-147, CARPK, PUCPR+, and ShanghaiTech show that WS-COC matches or even surpasses many state-of-art fully-supervised methods while significantly reducing annotation costs. Code is available at https://github.com/viscom-tongji/WS-COC.

cs.CV↗

arXiv2026

STVG-R1: Incentivizing Instance-Level Reasoning and Grounding in Videos via Reinforcement Learning

In vision-language models (VLMs), misalignment between textual descriptions and visual coordinates often induces hallucinations. This issue becomes particularly severe in dense prediction tasks such as spatial-temporal video grounding (STVG). Prior approaches typically focus on enhancing visual-textual alignment or attaching auxiliary decoders. However, these strategies inevitably introduce additional trainable modules, leading to significant annotation costs and computational overhead. In this work, we propose a novel visual prompting paradigm that avoids the difficult problem of aligning coordinates across modalities. Specifically, we reformulate per-frame coordinate prediction as a compact instance-level identification problem by assigning each object a unique, temporally consistent ID. These IDs are embedded into the video as visual prompts, providing explicit and interpretable inputs to the VLMs. Furthermore, we introduce STVG-R1, the first reinforcement learning framework for STVG, which employs a task-driven reward to jointly optimize temporal accuracy, spatial consistency, and structural format regularization. Extensive experiments on six benchmarks demonstrate the effectiveness of our approach. STVG-R1 surpasses the baseline Qwen2.5-VL-7B by a remarkable margin of 20.9% on m_IoU on the HCSTVG-v2 benchmark, establishing a new state of the art (SOTA). Surprisingly, STVG-R1 also exhibits strong zero-shot generalization to multi-object referring video object segmentation tasks, achieving a SOTA 47.3% J&F on MeViS.

cs.CV↗

arXiv2025

Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMs

Vision language models (VLMs) have achieved impressive performance across a variety of computer vision tasks. However, the multimodal reasoning capability has not been fully explored in existing models. In this paper, we propose a Chain-of-Focus (CoF) method that allows VLMs to perform adaptive focusing and zooming in on key image regions based on obtained visual cues and the given questions, achieving efficient multimodal reasoning. To enable this CoF capability, we present a two-stage training pipeline, including supervised fine-tuning (SFT) and reinforcement learning (RL). In the SFT stage, we construct the MM-CoF dataset, comprising 3K samples derived from a visual agent designed to adaptively identify key regions to solve visual tasks with different image resolutions and questions. We use MM-CoF to fine-tune the Qwen2.5-VL model for cold start. In the RL stage, we leverage the outcome accuracies and formats as rewards to update the Qwen2.5-VL model, enabling further refining the search and reasoning strategy of models without human priors. Our model achieves significant improvements on multiple benchmarks. On the V* benchmark that requires strong visual reasoning capability, our model outperforms existing VLMs by 5% among 8 image resolutions ranging from 224 to 4K, demonstrating the effectiveness of the proposed CoF method and facilitating the more efficient deployment of VLMs in practical applications.

cs.CV↗

arXiv2025

Longwave-transparent low-emissivity material

Low emissivity (low-e) materials are crucial for conserving thermal energy in buildings, cold chain logistics and transportation by minimizing unwanted radiative heat loss or gain. However, their metallic nature intrinsically causes severe longwave attenuation, hindering their broad applications. Here, we introduce, for the first time, an all-dielectric longwave-transparent low-emissivity material (LLM) with ultra-broadband, high transmittance spanning 9 orders of magnitude, from terahertz to kilohertz frequencies. This meter-scale LLM not only achieves energy savings of up to 41.1% over commercial white paint and 10.2% over traditional low-e materials, but also unlocks various fundamentally new capabilities including high-speed wireless communication in energy-efficient buildings, wireless energy transfer with radiative thermal insulation, as well as non-invasive terahertz security screening and radio frequency identification in cold chain logistics. Our approach represents a new photonic solution towards carbon neutrality and smart city development, paving the way for a more sustainable and interconnected future.

physics.optics↗

arXiv2025

An AI super-resolution field emulator for cosmological hydrodynamics: the Lyman-α forest

We extend our super-resolution and emulation framework for cosmological dark matter simulations to include hydrodynamics. We present a two-stage deep learning model to emulate high-resolution (HR-HydroSim) baryonic fields from low-resolution (LR-HydroSim) simulations at redshift $z = 3$. The method takes as inputs an LR-HydroSim and the high-resolution initial conditions (HR-HydroICs). First, the model stochastically generates high-resolution baryonic fields from the LR-HydroSim. Second, a deterministic emulator refines these fields using HR-HydroICs to reconstruct small-scale structures including displacement, velocity, internal energy, and gas/star classification. Trained on paired low- and high-resolution simulations produced with \texttt{MP-Gadget}, the model captures small-scale structures of the intergalactic medium and %Lyman-$α$ forest observables down to the 100 kpc pressure smoothing scale relevant to the Lyman-$α$ forest. The model achieves subpercent error for overdensity, temperature, velocity, and optical depth fields, a mean relative error of 1.07\% in the large-scale flux power spectrum ($k < 3 \times 10^{-2}\ \mathrm{s/km}$), and less than 10\% error in the flux probability distribution function. Notably, the two-stage model reduces the compute time by a factor of $\sim$450 compared to full smoothed particle hydrodynamics at the same resolution. This work demonstrates the potential of this framework as a powerful and efficient tool for generating high-resolution fields offering fast and accurate alternatives to traditional cosmological hydrodynamic simulations and enabling large-volume mock datasets for next-generation cosmological surveys.

astro-ph.CO↗

arXiv2025

Bridging Literature and the Universe Via A Multi-Agent Large Language Model System

As cosmological simulations and their associated software become increasingly complex, physicists face the challenge of searching through vast amounts of literature and user manuals to extract simulation parameters from dense academic papers, each using different models and formats. Translating these parameters into executable scripts remains a time-consuming and error-prone process. To improve efficiency in physics research and accelerate the cosmological simulation process, we introduce SimAgents, a multi-agent system designed to automate both parameter configuration from the literature and preliminary analysis for cosmology research. SimAgents is powered by specialized LLM agents capable of physics reasoning, simulation software validation, and tool execution. These agents collaborate through structured communication, ensuring that extracted parameters are physically meaningful, internally consistent, and software-compliant. We also construct a cosmological parameter extraction evaluation dataset by collecting over 40 simulations in published papers from Arxiv and leading journals that cover diverse simulation types. Experiments on the dataset demonstrate a strong performance of SimAgents, highlighting its effectiveness and potential to accelerate scientific research for physicists. Our demonstration video is available at: https://youtu.be/w1zLpm_CaWA. The complete system and dataset are publicly available at https://github.com/xwzhang98/SimAgents.

astro-ph.IM↗

arXiv2025

Text-promptable Object Counting via Quantity Awareness Enhancement

Recent advances in large vision-language models (VLMs) have shown remarkable progress in solving the text-promptable object counting problem. Representative methods typically specify text prompts with object category information in images. This however is insufficient for training the model to accurately distinguish the number of objects in the counting task. To this end, we propose QUANet, which introduces novel quantity-oriented text prompts with a vision-text quantity alignment loss to enhance the model's quantity awareness. Moreover, we propose a dual-stream adaptive counting decoder consisting of a Transformer stream, a CNN stream, and a number of Transformer-to-CNN enhancement adapters (T2C-adapters) for density map prediction. The T2C-adapters facilitate the effective knowledge communication and aggregation between the Transformer and CNN streams. A cross-stream quantity ranking loss is proposed in the end to optimize the ranking orders of predictions from the two streams. Extensive experiments on standard benchmarks such as FSC-147, CARPK, PUCPR+, and ShanghaiTech demonstrate our model's strong generalizability for zero-shot class-agnostic counting. Code is available at https://github.com/viscom-tongji/QUANet

cs.CV↗

arXiv2025

LightVA: Lightweight Visual Analytics with LLM Agent-Based Task Planning and Execution

Visual analytics (VA) requires analysts to iteratively propose analysis tasks based on observations and execute tasks by creating visualizations and interactive exploration to gain insights. This process demands skills in programming, data processing, and visualization tools, highlighting the need for a more intelligent, streamlined VA approach. Large language models (LLMs) have recently been developed as agents to handle various tasks with dynamic planning and tool-using capabilities, offering the potential to enhance the efficiency and versatility of VA. We propose LightVA, a lightweight VA framework that supports task decomposition, data analysis, and interactive exploration through human-agent collaboration. Our method is designed to help users progressively translate high-level analytical goals into low-level tasks, producing visualizations and deriving insights. Specifically, we introduce an LLM agent-based task planning and execution strategy, employing a recursive process involving a planner, executor, and controller. The planner is responsible for recommending and decomposing tasks, the executor handles task execution, including data analysis, visualization generation and multi-view composition, and the controller coordinates the interaction between the planner and executor. Building on the framework, we develop a system with a hybrid user interface that includes a task flow diagram for monitoring and managing the task planning process, a visualization panel for interactive data exploration, and a chat view for guiding the model through natural language instructions. We examine the effectiveness of our method through a usage scenario and an expert study.

cs.HC↗

arXiv2025

Resolving Phonons in Superconductor Bi2Sr2CaCu2O8+δ at Sub-Unit-Cell Resolution

The role of phonons in cuprates remains controversial, with their complex lattice structure complicating the investigation. Here, we identify phonon modes originating from charge reservoir and superconducting layers of Bi2Sr2CaCu2O8+δ using sub-unit-cell resolved electron energy loss spectroscopy in a scanning transmission electron microscope. We reveal several phonon modes exhibiting layer-specific localization: ~78 meV in-plane modes localized in the CuO2 planes, ~42 meV hybrid (in-plane and out-of-plane) modes in the CuO2 planes, and ~38 meV hybrid modes in the BiO layers. We also observe a periodic modulation of phonon frequencies induced by the superstructure, spatially correlating with the superconducting gap variation. Our findings offer new insights into the electron-phonon coupling in cuprates, fostering deeper exploration of the microscopic linkage between lattice dynamics and superconductivity.

cond-mat.supr-con↗

arXiv2025

A Fan-type condition involving bipartite independence number for hamiltonicity in graphs

The bipartite independence number of a graph $G$, denoted by $\widetildeα(G)$, is defined as the smallest integer $q$ for which there exist positive integers $s$ and $t$ with $s + t = q + 1$, such that for any two disjoint subsets $A, B \subseteq V(G)$ with $|A| = s$ and $|B| = t$, there exists an edge between $A$ and $B$. In this paper, we prove that for a 2-connected graph $G$ of order at least three, if $\max\{d_G(x), d_G(y)\} \ge \widetildeα(G)$ for every pair of nonadjacent vertices $x, y$ at distance two, then $G$ is hamiltonian. Moreover, we prove that if $G$ is 3-connected and $\max\{d_G(x), d_G(y)\} \ge \widetildeα(G)+1$ for every pair of nonadjacent vertices $x, y$ at distance two, then $G$ is hamiltonian-connected. Our results generalize the recent work by Li and Liu.

math.CO↗

arXiv2025

TTRD3: Texture Transfer Residual Denoising Dual Diffusion Model for Remote Sensing Image Super-Resolution

Remote Sensing Image Super-Resolution (RSISR) reconstructs high-resolution (HR) remote sensing images from low-resolution inputs to support fine-grained ground object interpretation. Existing methods face three key challenges: (1) Difficulty in extracting multi-scale features from spatially heterogeneous RS scenes, (2) Limited prior information causing semantic inconsistency in reconstructions, and (3) Trade-off imbalance between geometric accuracy and visual quality. To address these issues, we propose the Texture Transfer Residual Denoising Dual Diffusion Model (TTRD3) with three innovations: First, a Multi-scale Feature Aggregation Block (MFAB) employing parallel heterogeneous convolutional kernels for multi-scale feature extraction. Second, a Sparse Texture Transfer Guidance (STTG) module that transfers HR texture priors from reference images of similar scenes. Third, a Residual Denoising Dual Diffusion Model (RDDM) framework combining residual diffusion for deterministic reconstruction and noise diffusion for diverse generation. Experiments on multi-source RS datasets demonstrate TTRD3's superiority over state-of-the-art methods, achieving 1.43% LPIPS improvement and 3.67% FID enhancement compared to best-performing baselines. Code/model: https://github.com/LED-666/TTRD3.

cs.CV↗

arXiv2025

AI-assisted super-resolution cosmological simulations IV: An emulator for deterministic realizations

Super-resolution (SR) models in cosmological simulations use deep learning (DL) to rapidly enhance low-resolution (LR) runs with statistically correct fine details. These models preserves large-scale structures by conditioning on an LR version of the simulation. On smaller scales, the generative process is inherently stochastic, producing multiple possible SR realizations with distinct small-scale structures. Validation of reconstructed SR runs from LR simulations requires ensuring that specific statistics of interest are accurately reproduced by comparing SR outputs with target high resolution (HR) runs. In this study, we develop an emulator designed to reproduce the small-scale structures of target HR simulation with high fidelity. By processing an SR realization alongside the high-resolution initial condition (HRIC), we transform the SR output to emulate the result of a full simulation with that HRIC. By comparing various metrics, from visualization to individual halo measurements, we demonstrate that the emulated SR runs closely align with the target HR simulation, even at length scales an order of magnitude smaller than the corresponding LR run. These results show the potential of this method for efficiently generating accurate simulations and mock observations for large galaxy surveys.

astro-ph.CO↗

arXiv2025

Fantastic Multi-Task Gradient Updates and How to Find Them In a Cone

Balancing competing objectives remains a fundamental challenge in multi-task learning (MTL), primarily due to conflicting gradients across individual tasks. A common solution relies on computing a dynamic gradient update vector that balances competing tasks as optimization progresses. Building on this idea, we propose ConicGrad, a principled, scalable, and robust MTL approach formulated as a constrained optimization problem. Our method introduces an angular constraint to dynamically regulate gradient update directions, confining them within a cone centered on the reference gradient of the overall objective. By balancing task-specific gradients without over-constraining their direction or magnitude, ConicGrad effectively resolves inter-task gradient conflicts. Moreover, our framework ensures computational efficiency and scalability to high-dimensional parameter spaces. We conduct extensive experiments on standard supervised learning and reinforcement learning MTL benchmarks, and demonstrate that ConicGrad achieves state-of-the-art performance across diverse tasks.

cs.LG↗

arXiv2024

Precise structure and polarization determination of Hf0.5Zr0.5O2 with electron ptychography

Hf0.5Zr0.5O2 (HZO) is a promising candidate for next generation ferroelectric memories and transistors. However, its ferroelectricity origin is still under debate due to the complex of its phase and microstructure in practical samples. In this study, we investigate the atomic structure of substrate-free HZO freestanding film with multislice electron ptychography, for which the ultra-high space resolution (up to ~25 pm) and capability to simultaneously image the cation and oxygen allow us to precisely determine the intrinsic atomic structures of different phases and reveal subtle changes among them. We clarify that the orthorhombic phase is ferroelectric with spontaneous polarization ~34{\pm}4 μC/cm2 (corresponding to 56{\pm}6 pm in displacement) that is accurately measured through statistical analysis. Significant polarization suppression is observed near the grain boundary, while no distinguishable structural changes are detected near the 180° ferroelectric domain walls. Through the direct oxygen imaging of orthorhombic phase from the [111] zone axis, we quantify a substantial number of oxygen vacancies with a preferential distribution, which influences the polarization direction and strength. These findings provide fundamentals for HZO research, and thus lay a foundation for the design of high-performance ferroelectric devices.

cond-mat.mtrl-sci↗

arXiv2024

Coloring some $(P_6,C_4)$-free graphs with $Δ-1$ colors

The Borodin-Kostochka Conjecture states that for a graph $G$, if $Δ(G)\geq9$, then $χ(G)\leq\max\{Δ(G)-1,ω(G)\}$. We use $P_t$ and $C_t$ to denote a path and a cycle on $t$ vertices, respectively. Let $C=v_1v_2v_3v_4v_5v_1$ be an induced $C_5$. A {\em $C_5^+$} is a graph obtained from $C$ by adding a $C_3=xyzx$ and a $P_2=t_1t_2$ such that (1) $x$ and $y$ are both exactly adjacent to $v_1,v_2,v_3$ in $V(C)$, $z$ is exactly adjacent to $v_2$ in $V(C)$, $t_1$ is exactly adjacent to $v_4,v_5$ in $V(C)$ and $t_2$ is exactly adjacent to $v_1,v_4,v_5$ in $V(C)$, (2) $t_1$ is exactly adjacent to $z$ in $\{x,y,z\}$ and $t_2$ has no neighbors in $\{x,y,z\}$. In this paper, we show that the Borodin-Kostochka Conjecture holds for ($P_6,C_4,H$)-free graphs, where $H\in \{K_7,C_5^+\}$. This generalizes some results of Gupta and Pradhan in \cite{GP21,GP24}.

math.CO↗

arXiv2024

SeNM-VAE: Semi-Supervised Noise Modeling with Hierarchical Variational Autoencoder

The data bottleneck has emerged as a fundamental challenge in learning based image restoration methods. Researchers have attempted to generate synthesized training data using paired or unpaired samples to address this challenge. This study proposes SeNM-VAE, a semi-supervised noise modeling method that leverages both paired and unpaired datasets to generate realistic degraded data. Our approach is based on modeling the conditional distribution of degraded and clean images with a specially designed graphical model. Under the variational inference framework, we develop an objective function for handling both paired and unpaired data. We employ our method to generate paired training samples for real-world image denoising and super-resolution tasks. Our approach excels in the quality of synthetic degraded images compared to other unpaired and paired noise modeling methods. Furthermore, our approach demonstrates remarkable performance in downstream image restoration tasks, even with limited paired data. With more paired data, our method achieves the best performance on the SIDD dataset.

cs.CV↗

arXiv2024

Epsilon*: Privacy Metric for Machine Learning Models

We introduce Epsilon*, a new privacy metric for measuring the privacy risk of a single model instance prior to, during, or after deployment of privacy mitigation strategies. The metric requires only black-box access to model predictions, does not require training data re-sampling or model re-training, and can be used to measure the privacy risk of models not trained with differential privacy. Epsilon* is a function of true positive and false positive rates in a hypothesis test used by an adversary in a membership inference attack. We distinguish between quantifying the privacy loss of a trained model instance, which we refer to as empirical privacy, and quantifying the privacy loss of the training mechanism which produces this model instance. Existing approaches in the privacy auditing literature provide lower bounds for the latter, while our metric provides an empirical lower bound for the former by relying on an ($ε$, $δ$)-type of quantification of the privacy of the trained model instance. We establish a relationship between these lower bounds and show how to implement Epsilon* to avoid numerical and noise amplification instability. We further show in experiments on benchmark public data sets that Epsilon* is sensitive to privacy risk mitigation by training with differential privacy (DP), where the value of Epsilon* is reduced by up to 800% compared to the Epsilon* values of non-DP trained baseline models. This metric allows privacy auditors to be independent of model owners, and enables visualizing the privacy-utility landscape to make informed decisions regarding the trade-offs between model privacy and utility.

cs.LG↗

arXiv2023

Atomic-scale mechanism of enhanced electron-phonon coupling at the interface of MgB$_2$ thin film

In this study, we explore the heterointerface of MgB$_2$ film on SiC substrate at atomic scale using electron microscopy and spectroscopy. We detect ~1 nm MgO between MgB$_2$ and SiC. Atomic-level electron energy loss spectra (EELS) show MgB$_2$-E2g mode splitting and softening near the MgB$_2$/MgO interface. Orbital-resolved core-level EELS link the phonon softening to in-plane boron-atom electron states' changes. Ab initio calculations confirm this softening enhances electron-phonon coupling at the interface. Our findings highlight interface engineering's potential for superconductivity enhancement.

cond-mat.supr-con↗