arXiv subjects

Weijie Zhang

Publications and source records attributed to Weijie Zhang.

14 recordsLinked to original sources

arXiv2026

Warp-Geo: Differentiable Geometry Representation for Dynamic-Boundary Simulation and Shape Optimization

Modern inverse problems and adaptive simulations require efficient and differentiable representations of complex geometries that can be updated frequently for gradient-based optimization. Existing methods cannot simultaneously achieve both. We present Warp-Geo, a differentiable GPU-accelerated framework for reconstructing signed distance fields and evaluating surface normals from point clouds of complex 3D geometries. Warp-Geo combines uniform grids for GPU parallelism with implicit differentiation through the Poisson solve, enabling frequent geometry recomputation while maintaining full differentiability. We demonstrate Warp-Geo on forward SDF reconstruction, dynamic fluid--structure interaction with moving boundaries, and inverse shape optimization via automatic differentiation with gradient validation. GPU scalability analysis confirms practical feasibility for frequent geometry updates. The framework, implemented in Warp and JAX, enables seamless integration with differentiable physics solvers for end-to-end shape optimization.

physics.comp-ph↗

arXiv2026

Zoom In, Reason Out: Efficient Far-field Anomaly Detection in Expressway Surveillance Videos via Focused VLM Reasoning Guided by Bayesian Inference

Expressway video anomaly detection is important for traffic safety, but remains challenging across diverse scenes, particularly for far-field vehicles with subtle abnormal motion. Vision-Language Models (VLMs) provide strong semantic reasoning capabilities, yet processing full frames can dilute evidence from distant targets and introduce substantial computational overhead. To address these challenges, we propose VIBES, an asynchronous framework that uses Bayesian inference to guide focused VLM reasoning. Specifically, an online kinematics-guided Bayesian inference module continuously estimates a context-dependent normal-motion distribution from vehicle trajectories and updates its probabilistic boundaries. Deviations from these boundaries produce asynchronous triggers that localize candidate anomalies in time and space. Instead of processing continuous full-frame video, the VLM reasons only over selected frames and localized visual regions associated with the triggers, reducing irrelevant visual content and unnecessary inference. Extensive experiments show that VIBES improves far-field anomaly detection and semantic interpretation while achieving real-time processing efficiency across diverse expressway conditions.

cs.CV↗

arXiv2026

Balancing Multimodal Learning through Label Space Reshaping

Multimodal learning often suffers from modality imbalance, where modalities that converge faster dominate optimization while others remain undertrained. Existing approaches typically mitigate this issue by strengthening the weak modality or adjusting optimization gradients. However, such strategies mainly compensate for optimization rate discrepancies, often at the expense of the strong modality's optimization capacity, without analyzing how these discrepancies arise at the modality level. Based on theoretical insights and empirical observations, we argue that the discrepancy of learning pace arises from differences in the mapping difficulty between modality-specific feature space and the shared label space. To address this issue, we propose Balanced Multimodal Label Reshaping (BMLR), the first method that promotes multimodal balance from the label-side design. BMLR reshapes the cross-modal label space to equalize mapping difficulty across modalities, thereby facilitating modality interaction and injecting richer inter-class information into each modality. Extensive experiments across multiple architectures demonstrate that BMLR consistently improves multimodal performance and exhibits strong compatibility with diverse model designs. The source code will be released soon.

cs.LG↗

arXiv2026

STK-Adapter: Incorporating Evolving Graph and Event Chain for Temporal Knowledge Graph Extrapolation

Temporal Knowledge Graph (TKG) extrapolation aims to predict future events based on historical facts. Recent studies have attempted to enhance TKG extrapolation by integrating TKG's evolving structural representations and textual event chains into Large Language Models (LLMs). Yet, two main challenges limit these approaches: (1) The loss of essential spatial-temporal information due to shallow alignment between TKG's graph evolving structural representation and the LLM's semantic space, and (2) the progressive dilution of the TKG's evolving structural features during LLM fine-tuning. To address these challenges, we propose the Spatial-Temporal Knowledge Adapter (STK-Adapter), which flexibly integrates the evolving graph encoder and the LLM to facilitate TKG reasoning. In STK-Adapter, a Spatial-Temporal MoE is designed to capture spatial structures and temporal patterns inherent in TKGs. An Event-Aware MoE is employed to model intricate temporal semantics dependencies within event chains. In addition, a Cross-Modality Alignment MoE is proposed to facilitate deep cross-modality alignment by TKG-guided attention experts. Extensive experiments on benchmark datasets demonstrate that STK-Adapter significantly outperforms state-of-the-art methods and exhibits strong generalization capabilities in cross-dataset task. The code is available at https://github.com/Zhaoshuyuan0246/STK-Adapter.

cs.IR↗

arXiv2026

OmniEarth: A Benchmark for Evaluating Vision-Language Models in Geospatial Tasks

Vision-Language Models (VLMs) have demonstrated effective perception and reasoning capabilities on general-domain tasks, leading to growing interest in their application to Earth observation. However, a systematic benchmark for comprehensively evaluating remote sensing vision-language models (RSVLMs) remains lacking. To address this gap, we introduce OmniEarth, a benchmark for evaluating RSVLMs under realistic Earth observation scenarios. OmniEarth organizes tasks along three capability dimensions: perception, reasoning, and robustness. It defines 28 fine-grained tasks covering multi-source sensing data and diverse geospatial contexts. The benchmark supports two task formulations: multiple-choice VQA and open-ended VQA. The latter includes pure text outputs for captioning tasks, bounding box outputs for visual grounding tasks, and mask outputs for segmentation tasks. To reduce linguistic bias and examine whether model predictions rely on visual evidence, OmniEarth adopts a blind test protocol and a quintuple semantic consistency requirement. OmniEarth includes 9,275 carefully quality-controlled images, including proprietary satellite imagery from Jilin-1 (JL-1), along with 44,210 manually verified instructions. We conduct a systematic evaluation of contrastive learning-based models, general closed-source and open-source VLMs, as well as RSVLMs. Results show that existing VLMs still struggle with geospatially complex tasks, revealing clear gaps that need to be addressed for remote sensing applications. OmniEarth is publicly available at https://huggingface.co/datasets/sjeeudd/OmniEarth.

cs.CV↗

arXiv2026

DIA-CLIP: a universal representation learning framework for zero-shot DIA proteomics

Data-independent acquisition mass spectrometry (DIA-MS) has established itself as a cornerstone of proteomic profiling and large-scale systems biology, offering unparalleled depth and reproducibility. Current DIA analysis frameworks, however, require semi-supervised training within each run for peptide-spectrum match (PSM) re-scoring. This approach is prone to overfitting and lacks generalizability across diverse species and experimental conditions. Here, we present DIA-CLIP, a pre-trained model shifting the DIA analysis paradigm from semi-supervised training to universal cross-modal representation learning. By integrating dual-encoder contrastive learning framework with encoder-decoder architecture, DIA-CLIP establishes a unified cross-modal representation for peptides and corresponding spectral features, achieving high-precision, zero-shot PSM inference. Extensive evaluations across diverse benchmarks demonstrate that DIA-CLIP consistently outperforms state-of-the-art tools, yielding up to a 45% increase in protein identification while achieving a 12% reduction in entrapment identifications. Moreover, DIA-CLIP holds immense potential for diverse practical applications, such as single-cell and spatial proteomics, where its enhanced identification depth facilitates the discovery of novel biomarkers and the elucidates of intricate cellular mechanisms.

cs.LG↗

arXiv2025

Nexus: An Omni-Perceptive And -Interactive Model for Language, Audio, And Vision

This work proposes an industry-level omni-modal large language model (LLM) pipeline that integrates auditory, visual, and linguistic modalities to overcome challenges such as limited tri-modal datasets, high computational costs, and complex feature alignments. Our pipeline consists of three main components: First, a modular framework enabling flexible configuration of various encoder-LLM-decoder architectures. Second, a lightweight training strategy that pre-trains audio-language alignment on the state-of-the-art vision-language model Qwen2.5-VL, thus avoiding the costly pre-training of vision-specific modalities. Third, an audio synthesis pipeline that generates high-quality audio-text data from diverse real-world scenarios, supporting applications such as Automatic Speech Recognition and Speech-to-Speech chat. To this end, we introduce an industry-level omni-modal LLM, Nexus. Extensive experiments validate the efficacy of our pipeline, yielding the following key findings:(1) In the visual understanding task, Nexus exhibits superior performance compared with its backbone model - Qwen2.5-VL-7B, validating the efficiency of our training strategy. (2) Within the English Spoken Question-Answering task, the model achieves better accuracy than the same-period competitor (i.e, MiniCPM-o2.6-7B) in the LLaMA Q. benchmark. (3) In our real-world ASR testset, Nexus achieves outstanding performance, indicating its robustness in real scenarios. (4) In the Speech-to-Text Translation task, our model outperforms Qwen2-Audio-Instruct-7B. (5) In the Text-to-Speech task, based on pretrained vocoder (e.g., Fishspeech1.4 or CosyVoice2.0), Nexus is comparable to its backbone vocoder on Seed-TTS benchmark. (6) An in-depth analysis of tri-modal alignment reveals that incorporating the audio modality enhances representational alignment between vision and language.

cs.MM↗

arXiv2025

Minibeam-pLATTICE: A novel proton LATTICE modality using minibeams

Purpose: LATTICE, a form of spatially fractionated radiation therapy that delivers high-dose peaks and low-dose valleys within the target, has been clinically utilized for treating bulky tumors. However, its application to small-to-medium-sized target remains challenging due to beam size limitations. To address this challenge, this work proposes a novel proton LATTICE (pLATTICE) modality using minibeams, namely minibeam-pLATTICE, that extends LATTICE approach for small-to-medium targets. Methods: Three minibeam-pLATTICE methods are introduced. (1) M0: a fixed minibeam orientation for all beam angles; (2) M1: alternated minibeam orientations, for consecutive beam angles; (3) M2: multiple minibeam orientations for each beam angle. For each minibeam-pLATTICE method, an optimization problem is formulated to optimize dose uniformity in target peaks and valleys, as well as dose-volume-histogram-based objectives. This problem is solved using iterative convex relaxation and alternating direction method of multipliers. Results: Three minibeam-pLATTICE methods are validated to demonstrate the feasibility of minibeam-pLATTICE for head-and-neck cases. The advantages of this modality over conventional beam (CONV) pLATTICE are evaluated by comparing peak-to-valley dose ratio (PVDR) and dose delivered to organs at risk (OAR). All three minibeam-pLATTICE modalities achieved improved plan quality compared to CONV, with M2 yielding the best results. For example, in terms of PVDR, M2=5.89, compared to CONV=4.13, M0=4.87 and M1=4.7. Conclusion: A novel minibeam-pLATTICE modality is proposed that generates lattice dose patterns for small-to-medium targets, which are not achievable with conventional pLATTICE due to beam size limitations.

physics.med-ph↗

arXiv2025

pUniFind: a unified large pre-trained deep learning model pushing the limit of mass spectra interpretation

Deep learning has advanced mass spectrometry data interpretation, yet most models remain feature extractors rather than unified scoring frameworks. We present pUniFind, the first large-scale multimodal pre-trained model in proteomics that integrates end-to-end peptide-spectrum scoring with open, zero-shot de novo sequencing. Trained on over 100 million open search-derived spectra, pUniFind aligns spectral and peptide modalities via cross modality prediction and outperforms traditional engines across diverse datasets, particularly achieving a 42.6 percent increase in the number of identified peptides in immunopeptidomics. Supporting over 1,300 modifications, pUniFind identifies 60 percent more PSMs than existing de novo methods despite a 300-fold larger search space. A deep learning based quality control module further recovers 38.5 percent additional peptides including 1,891 mapped to the genome but absent from reference proteomes while preserving full fragment ion coverage. These results establish a unified, scalable deep learning framework for proteomic analysis, offering improved sensitivity, modification coverage, and interpretability.

cs.LG↗

arXiv2025

A fully flexible joint lattice position and dose optimization method for LATTICE therapy

Lattice radiotherapy (LATTICE) is a form of spatially fractionated radiation therapy (SFRT) designed to deliver high doses to tumor regions while sparing surrounding tissues. Traditional LATTICE uses rigid vertex patterns, limiting adaptability for irregular tumors or those near critical organs. This study introduces a novel planning method with flexible vertex placement and joint optimization of vertex positions and dose distribution, enhancing treatment precision. The method integrates vertex positioning with other treatment variables within a constrained optimization framework, allowing dynamic adjustments. Results showed that plans generated with the new method (NEW) demonstrated superior or comparable quality to conventional LATTICE plans, with improvements in the optimization objective and peak-to-valley dose ratio (PVDR). This approach offers significant improvements in target dose conformity and OAR sparing, providing an enhanced LATTICE technique.

physics.med-ph↗

arXiv2025

A Proton Treatment Planning Method for Combining FLASH and Spatially Fractionated Radiation Therapy to Enhance Normal Tissue Protection

Background: FLASH radiation therapy (FLASH-RT) uses ultra-high dose rates to induce the FLASH effect, enhancing normal tissue sparing. In proton Bragg peak FLASH-RT, this effect is confined to high-dose regions near the target at deep tissue levels. In contrast, Spatially Fractionated Radiation Therapy (SFRT) creates alternating high- and low-dose regions with high peak-to-valley dose ratios (PVDR), sparing tissues at shallow-to-intermediate depths. Purpose: This study investigates a novel proton modality (SFRT-FLASH) that synergizes FLASH-RT and SFRT to enhance normal tissue protection across all depths. Methods: Two SFRT techniques are integrated with FLASH-RT: proton GRID therapy (pGRID) with conventional beam sizes and proton minibeam radiation therapy (pMBRT) with submillimeter beams. These are implemented as pGRID-FLASH (SB-FLASH) and minibeam-FLASH (MB-FLASH), respectively. The pGRID technique uses a scissor-beam (SB) method to achieve uniform target coverage. To meet FLASH dose (5 Gy) and dose-rate (40 Gy/s) thresholds, a single-field uniform-dose-per-fraction strategy is used. Dose and dose-rate constraints are jointly optimized, including a CTV1cm structure (a 1 cm ring around the CTV) for each field. Results: Across four clinical cases, MB-FLASH and SB-FLASH plans were benchmarked against conventional (CONV), FLASH-RT (FLASH), pMBRT (MB), and pGRID (SB) plans. SFRT-FLASH achieved high FLASH effect coverage (~60-80% in CTV1cm) while preserving PVDR (~2.5-7) at shallow-to-intermediate depths. Conclusions: We present a proton treatment planning approach that combines the FLASH effect at depth with high PVDR near the surface, enhancing normal tissue protection and advancing proton therapy.

physics.med-ph↗

arXiv2025

A mixed integer programming approach to minibeam aperture optimization for multi-collimator proton minibeam radiotherapy

Background: Multi-collimator proton minibeam radiation therapy (MC-pMBRT) has recently emerged as a versatile technique for dose shaping, enabling peak-valley dose patterns in organs-at-risk (OAR) while maintaining a uniform dose distribution in tumor. MC-pMBRT leverages a set of generic multi-slit collimators (MSC) with varying center-to-center distances. However, the current method for minibeam aperture optimization (MAO), i.e., the selection of MSC per beam angle, is manual and heuristic, resulting in computational inefficiencies and no guarantee of optimality. This work introduces a novel mixed integer programming (MIP) approach to MAO for optimizing MC-pMBRT plan quality. Methods: The proposed MIP approach jointly optimizes dose distributions, peak-to-valley dose ratio (PVDR), and selects the optimal set of MSC per beam angle. The optimization problem includes decision variables for MSC selection per beam angle and spot weights. The proposed MIP approach is a two-step process: Step1: the binary variables are optimally determined to select MSC for each beam angle; Step 2: the continuous variables are solved to determine the spot weights. Both steps utilize iterative convex relaxation and the alternating direction method of multipliers to solve the problems. Results: The proposed MIP method for MAO (MIP-MAO) was validated against the conventional heuristic method (CONV) for MC-pMBRT treatment planning. Results indicate that MIP-MAO enhances the conformity index (CI) for the target and improves PVDR for OAR. For instance, in a head-and-neck case, CI improved from 0.61 (CONV) to 0.70 (MIP-MAO); in an abdomen case, CI improved from 0.78 (CONV) to 0.83 (MIP-MAO). Additionally, MIP-MAO reduced mean doses in the body and OAR. Conclusions: A novel MIP approach for MAO in MC-pMBRT is presented, showing demonstrated improvements in plan quality and PVDR compared to the heuristic method.

physics.med-ph↗

arXiv2022

Energy conserving and well-balanced discontinuous Galerkin methods for the Euler-Poisson equations in spherical symmetry

This paper presents high-order Runge-Kutta (RK) discontinuous Galerkin methods for the Euler-Poisson equations in spherical symmetry. The scheme can preserve a general polytropic equilibrium state and achieve total energy conservation up to machine precision with carefully designed spatial and temporal discretizations. To achieve the well-balanced property, the numerical solutions are decomposed into equilibrium and fluctuation components which are treated differently in the source term approximation. One non-trivial challenge encountered in the procedure is the complexity of the equilibrium state, which is governed by the Lane-Emden equation. For total energy conservation, we present second- and third-order RK time discretization, where different source term approximations are introduced in each stage of the RK method to ensure the conservation of total energy. A carefully designed slope limiter for spherical symmetry is also introduced to eliminate oscillations near discontinuities while maintaining the well-balanced and total-energy-conserving properties. Extensive numerical examples -- including a toy model of stellar core-collapse with a phenomenological equation of state that results in core-bounce and shock formation -- are provided to demonstrate the desired properties of the proposed methods, including the well-balanced property, high-order accuracy, shock capturing capability, and total energy conservation.

math.NA↗

arXiv2021

A Knowledge-Grounded Dialog System Based on Pre-Trained Language Models

We present a knowledge-grounded dialog system developed for the ninth Dialog System Technology Challenge (DSTC9) Track 1 - Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access. We leverage transfer learning with existing language models to accomplish the tasks in this challenge track. Specifically, we divided the task into four sub-tasks and fine-tuned several Transformer models on each of the sub-tasks. We made additional changes that yielded gains in both performance and efficiency, including the combination of the model with traditional entity-matching techniques, and the addition of a pointer network to the output layer of the language model.

cs.CL↗