arXiv subjects

Weiran Li

Publications and source records attributed to Weiran Li.

9 recordsLinked to original sources

arXiv2025

DM$^3$T: Harmonizing Modalities via Diffusion for Multi-Object Tracking

Multi-object tracking (MOT) is a fundamental task in computer vision with critical applications in autonomous driving and robotics. Multimodal MOT that integrates visible light and thermal infrared information is particularly essential for robust autonomous driving systems. However, effectively fusing these heterogeneous modalities is challenging. Simple strategies like concatenation or addition often fail to bridge the significant non-linear distribution gap between their feature representations, which can lead to modality conflicts and degrade tracking accuracy. Drawing inspiration from the connection between multimodal MOT and the iterative refinement in diffusion models, this paper proposes DM$^3$T, a novel framework that reformulates multimodal fusion as an iterative feature alignment process to generate accurate and temporally coherent object trajectories. Our approach performs iterative cross-modal harmonization through a proposed Cross-Modal Diffusion Fusion (C-MDF) module. In this process, features from both modalities provide mutual guidance, iteratively projecting them onto a shared, consistent feature manifold. This enables the learning of complementary information and achieves deeper fusion compared to conventional methods. Additionally, we introduce a plug-and-play Diffusion Refiner (DR) to enhance and refine the unified feature representation. To further improve tracking robustness, we design a Hierarchical Tracker that adaptively handles confidence estimation. DM$^3$T unifies object detection, state estimation, and data association into a comprehensive online tracking framework without complex post-processing. Extensive experiments on the VT-MOT benchmark demonstrate that our method achieves 41.7 HOTA, representing a 1.54% relative improvement over existing state-of-the-art methods. The code and models are available at https://vranlee.github.io/DM-3-T/.

cs.CV↗

arXiv2025

From Points to Clouds: Learning Robust Semantic Distributions for Multi-modal Prompts

Multimodal Prompt Learning (MPL) has emerged as a pivotal technique for adapting large-scale Visual Language Models (VLMs). However, current MPL methods are fundamentally limited by their optimization of a single, static point representation. This paradigm is inherently brittle, leads to overfitting on base classes, and generalizes poorly to novel or ambiguous categories. We challenge this point paradigm, proposing that robust generalization requires learning a semantic cloud (i.e., a distribution over the embedding space). To achieve this, we introduce Points-to-Clouds (P2C), a novel framework inspired by diffusion models that reframes prompt learning as a dynamic denoising task. At the core of P2C is a dual denoising mechanism: a Dynamic Prompt Denoising (DPD) mechanism perturbs text prompts with sophisticated, annealed noise to learn a smoother semantic landscape, while an auxiliary V-L Mapper denoising loss re-tasks the mapper as a denoising autoencoder. This forces the mapper to reconstruct clean visual prompts from noisy text inputs, ensuring robust cross-modal alignment. Extensive experiments across 11 datasets demonstrate that P2C consistently outperforms strong baselines. On the base-to-novel generalization benchmark, our method achieves a Harmonic Mean of 79.7%, representing a relative improvement of 1.4% over the baseline. The code and models are available at https://vranlee.github.io/P2C/.

cs.CV↗

arXiv2025

SAIL-Embedding Technical Report: Omni-modal Embedding Foundation Model

Multimodal embedding models aim to yield informative unified representations that empower diverse cross-modal tasks. Despite promising developments in the evolution from CLIP-based dual-tower architectures to large vision-language models, prior works still face unavoidable challenges in real-world applications and business scenarios, such as the limited modality support, unstable training mechanisms, and industrial domain gaps. In this work, we introduce SAIL-Embedding, an omni-modal embedding foundation model that addresses these issues through tailored training strategies and architectural design. In the optimization procedure, we propose a multi-stage training scheme to boost the multifaceted effectiveness of representation learning. Specifically, the content-aware progressive training aims to enhance the model's adaptability to diverse downstream tasks and master enriched cross-modal proficiency. The collaboration-aware recommendation enhancement training further adapts multimodal representations for recommendation scenarios by distilling knowledge from sequence-to-item and ID-to-item embeddings while mining user historical interests. Concurrently, we develop the stochastic specialization and dataset-driven pattern matching to strengthen model training flexibility and generalizability. Experimental results show that SAIL-Embedding achieves SOTA performance compared to other methods in different retrieval tasks. In online experiments across various real-world scenarios integrated with our model, we observe a significant increase in Lifetime (LT), which is a crucial indicator for the recommendation experience. For instance, the model delivers the 7-day LT gain of +0.5% in the Douyin-Selected scenario. For the Douyin feed rank model, the match features produced by SAIL-Embedding yield a +0.1% AUC gain.

cs.IR↗

arXiv2025

When Trackers Date Fish: A Benchmark and Framework for Underwater Multiple Fish Tracking

Multiple object tracking (MOT) technology has made significant progress in terrestrial applications, but underwater tracking scenarios remain underexplored despite their importance to marine ecology and aquaculture. In this paper, we present Multiple Fish Tracking Dataset 2025 (MFT25), a comprehensive dataset specifically designed for underwater multiple fish tracking, featuring 15 diverse video sequences with 408,578 meticulously annotated bounding boxes across 48,066 frames. Our dataset captures various underwater environments, fish species, and challenging conditions including occlusions, similar appearances, and erratic motion patterns. Additionally, we introduce Scale-aware and Unscented Tracker (SU-T), a specialized tracking framework featuring an Unscented Kalman Filter (UKF) optimized for non-linear swimming patterns of fish and a novel Fish-Intersection-over-Union (FishIoU) matching that accounts for the unique morphological characteristics of aquatic species. Extensive experiments demonstrate that our SU-T baseline achieves state-of-the-art performance on MFT25, with 34.1 HOTA and 44.6 IDF1, while revealing fundamental differences between fish tracking and terrestrial object tracking scenarios. The dataset and codes are released at https://vranlee.github.io/SU-T/.

cs.CV↗

arXiv2023

A Survey of Fish Tracking Techniques Based on Computer Vision

Fish tracking is a key technology for obtaining movement trajectories and identifying abnormal behavior. However, it faces considerable challenges, including occlusion, multi-scale tracking, and fish deformation. Notably, extant reviews have focused more on behavioral analysis rather than providing a comprehensive overview of computer vision-based fish tracking approaches. This paper presents a comprehensive review of the advancements of fish tracking technologies over the past seven years (2017-2023). It explores diverse fish tracking techniques with an emphasis on fundamental localization and tracking methods. Auxiliary plugins commonly integrated into fish tracking systems, such as underwater image enhancement and re-identification, are also examined. Additionally, this paper summarizes open-source datasets, evaluation metrics, challenges, and applications in fish tracking research. Finally, a comprehensive discussion offers insights and future directions for vision-based fish tracking techniques. We hope that our work could provide a partial reference in the development of fish tracking algorithms.

cs.CV↗

arXiv2014

Thermodynamics of Attractive and Repulsive Fermi Gases in Two Dimensions

We study the attractive and repulsive two-component Fermi gas with spin imbalance in two dimensions. Using a generalized $T$-matrix approximation, we examine the thermodynamic properties of both attractive and repulsive contact interacting Fermi gases. The interaction strength, which is characterized by the bound state energy $E_b=\hbar^2/m a_{2d}^2$ in vacuum, can be adjusted through a Feshbach resonance. We calculate the interaction energy, compressibility and spin susceptibility of the two branches of the Fermi gas. For the repulsive branch, we also find a critical strength of interaction $a_{2d}^{(c)}$ above which this metastable thermodynamic state becomes unstable. This critical value depends on the temperature and the spin imbalance (the "magnetization") of the system.

cond-mat.quant-gas↗

arXiv2014

Variational study of polarons in Bose-Einstein condensates

We use a class of variational wave functions to study the properties of an impurity in a Bose-Einstein condensate, i.e. the "Bose polaron". The impurity interacts with the condensate through a contact interaction, which can be tuned by a Feshbach resonance. We find a stable attractive polaron branch that evolves continuously across the resonance to a tight-binding diatomic molecule deep in the positive scattering length side. A repulsive polaron branch with finite lifetime is also observed and it becomes unstable as the interaction strength increases. The effective mass of the attractive polaron also changes smoothly across the resonance connecting the two well-understood limits deep on both sides.

cond-mat.quant-gas↗

arXiv2012

Alternative Route to Strong Interaction: Narrow Feshbach Resonance

We show that a narrow resonance produces strong interaction effects far beyond its width on the side of the resonance where the bound state has not been formed. This is due to a resonance structure of its phase shift, which shifts the phase of a large number of scattering states by $π$ before the bound state emerges. As a result, the magnitude of the interaction energy when approaching the resonance on the "upper" and "lower" branch from different side of the resonance is highly asymmetric, unlike their counter part in wide resonances. Measurements of these effects are experimentally feasible.

cond-mat.quant-gas↗

arXiv2012

Bose Gases Near Unitarity

We study the properties of strongly interacting Bose gases at the density and temperature regime when the three-body recombination rate is substantially reduced. In this regime, one can have a Bose gas with all particles in scattering states (i.e. the "upper branch") with little loss even at unitarity over the duration of the experiment. We show that because of bosonic enhancement, pair formation is shifted to the atomic side of the original resonance (where scattering length $a_s<0$), opposite to the fermionic case. In a trap, a repulsive Bose gas remains mechanically stable when brought across resonance to the atomic side until it reaches a critical scattering length $a_{s}^{\ast}<0$. For $a_s<a_{s}^{\ast}$, the density consists of a core of upper branch bosons surrounded by an outer layer of equilibrium phase. The conditions of low three-body recombination requires that the particle number $N<α(T/ω)^{5/2}$ in a harmonic trap with frequency $ω$, where $α$ is a constant.

cond-mat.quant-gas↗