arXiv subjects

Dong Yang

Publications and source records attributed to Dong Yang.

At least 55 records · Page 3Linked to original sources

arXiv2025

OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLM

Advancing machine intelligence requires developing the ability to perceive across multiple modalities, much as humans sense the world. We introduce OmniVinci, an initiative to build a strong, open-source, omni-modal LLM. We carefully study the design choices across model architecture and data curation. For model architecture, we present three key innovations: (i) OmniAlignNet for strengthening alignment between vision and audio embeddings in a shared omni-modal latent space; (ii) Temporal Embedding Grouping for capturing relative temporal alignment between vision and audio signals; and (iii) Constrained Rotary Time Embedding for encoding absolute temporal information in omni-modal embeddings. We introduce a curation and synthesis pipeline that generates 24M single-modal and omni-modal conversations. We find that modalities reinforce one another in both perception and reasoning. Our model, OmniVinci, outperforms Qwen2.5-Omni with +19.05 on DailyOmni (cross-modal understanding), +1.7 on MMAR (audio), and +3.9 on Video-MME (vision), while using just 0.2T training tokens - a 6 times reduction compared to Qwen2.5-Omni's 1.2T. We finally demonstrate omni-modal advantages in downstream applications spanning robotics, medical AI, and smart factory.

cs.CV↗

arXiv2025

Shallow Flow Matching for Coarse-to-Fine Text-to-Speech Synthesis

We propose Shallow Flow Matching (SFM), a novel mechanism that enhances flow matching (FM)-based text-to-speech (TTS) models within a coarse-to-fine generation paradigm. Unlike conventional FM modules, which use the coarse representations from the weak generator as conditions, SFM constructs intermediate states along the FM paths from these representations. During training, we introduce an orthogonal projection method to adaptively determine the temporal position of these states, and apply a principled construction strategy based on a single-segment piecewise flow. The SFM inference starts from the intermediate state rather than pure noise, thereby focusing computation on the latter stages of the FM paths. We integrate SFM into multiple TTS models with a lightweight SFM head. Experiments demonstrate that SFM yields consistent gains in speech naturalness across both objective and subjective evaluations, and significantly accelerates inference when using adaptive-step ODE solvers. Demo and codes are available at https://ydqmkkx.github.io/SFMDemo/.

eess.AS↗

arXiv2025

Better Tokens for Better 3D: Advancing Vision-Language Modeling in 3D Medical Imaging

Recent progress in vision-language modeling for 3D medical imaging has been fueled by large-scale computed tomography (CT) corpora with paired free-text reports, stronger architectures, and powerful pretrained models. This has enabled applications such as automated report generation and text-conditioned 3D image synthesis. Yet, current approaches struggle with high-resolution, long-sequence volumes: contrastive pretraining often yields vision encoders that are misaligned with clinical language, and slice-wise tokenization blurs fine anatomy, reducing diagnostic performance on downstream tasks. We introduce BTB3D (Better Tokens for Better 3D), a causal convolutional encoder-decoder that unifies 2D and 3D training and inference while producing compact, frequency-aware volumetric tokens. A three-stage training curriculum enables (i) local reconstruction, (ii) overlapping-window tiling, and (iii) long-context decoder refinement, during which the model learns from short slice excerpts yet generalizes to scans exceeding 300 slices without additional memory overhead. BTB3D sets a new state-of-the-art on two key tasks: it improves BLEU scores and increases clinical F1 by 40% over CT2Rep, CT-CHAT, and Merlin for report generation; and it reduces FID by 75% and halves FVD compared to GenerateCT and MedSyn for text-to-CT synthesis, producing anatomically consistent 512*512*241 volumes. These results confirm that precise three-dimensional tokenization, rather than larger language backbones alone, is essential for scalable vision-language modeling in 3D medical imaging. The codebase is available at: https://github.com/ibrahimethemhamamci/BTB3D

cs.CV↗

arXiv2025

Discrete Diffusion Models with MLLMs for Unified Medical Multimodal Generation

Recent advances in generative medical models are constrained by modality-specific scenarios that hinder the integration of complementary evidence from imaging, pathology, and clinical notes. This fragmentation limits their evolution into foundation models that can learn and reason across the full spectrum of biomedical data. We propose MeDiM, the first medical discrete diffusion model that learns shared distributions across modalities without modality-specific components. MeDiM unifies multiple generative tasks: translating between images and text, and jointly producing image-report pairs across domains in response to prompts. Built on a discrete diffusion framework, MeDiM bridges vision and language representations through a shared probabilistic space. To enable unified and flexible medical generation, we employ a multimodal large language model (MLLM) as the diffusion backbone, leveraging its prior knowledge and cross-modal reasoning. Two key designs are introduced: (1) removing the causal attention mask for bidirectional context, and (2) injecting continuous timestep embeddings for diffusion awareness. Experiments demonstrate high-fidelity medical generation (FID 16.60 on MIMIC-CXR and FID 24.19 on PathGen) and accurate report generation (METEOR 0.2650 and 0.2580). Jointly generated image-report pairs further enhance downstream performance (plus6.43 percent BLEU-1, plus18.57 percent BLEU-2, plus31.58 percent BLEU-3, plus4.80 percent METEOR), showing that MeDiM supports coherent and clinically grounded multimodal outputs.

cs.CV↗

arXiv2025

Does Using Counterfactual Help LLMs Explain Textual Importance in Classification?

Large language models (LLMs) are becoming useful in many domains due to their impressive abilities that arise from large training datasets and large model sizes. More recently, they have been shown to be very effective in textual classification tasks, motivating the need to explain the LLMs' decisions. Motivated by practical constrains where LLMs are black-boxed and LLM calls are expensive, we study how incorporating counterfactuals into LLM reasoning can affect the LLM's ability to identify the top words that have contributed to its classification decision. To this end, we introduce a framework called the decision changing rate that helps us quantify the importance of the top words in classification. Our experimental results show that using counterfactuals can be helpful.

cs.CL↗

arXiv2025

Flexible and Efficient Drift Detection without Labels

Machine learning models are being increasingly used to automate decisions in almost every domain, and ensuring the performance of these models is crucial for ensuring high quality machine learning enabled services. Ensuring concept drift is detected early is thus of the highest importance. A lot of research on concept drift has focused on the supervised case that assumes the true labels of supervised tasks are available immediately after making predictions. Controlling for false positives while monitoring the performance of predictive models used to make inference from extremely large datasets periodically, where the true labels are not instantly available, becomes extremely challenging. We propose a flexible and efficient concept drift detection algorithm that uses classical statistical process control in a label-less setting to accurately detect concept drifts. We show empirically that under computational constraints, our approach has better statistical power than previous known methods. Furthermore, we introduce a new semi-supervised drift detection framework to model the scenario of detecting drift (without labels) given prior detections, and show how our drift detection algorithm can be incorporated effectively into this framework. We demonstrate promising performance via numerical simulations.

stat.ML↗

arXiv2025

From objects finitely presented by a rigid object in a triangulated category to 2-term complexes

For a rigid object $M$ in an algebraic triangulated category $\mathcal{T}$, a functor pr$(M)\to\mathcal{H}^{[-1,0]}({\rm proj}\, A)$ is constructed, which essentially takes an object to its `presentation', where pr$(M)$ is the full subcategory of $\mathcal{T}$ of objects finitely presented by $M$, $A$ is the endomorphism algebra of $M$ and $\mathcal{H}^{[-1,0]}({\rm proj}\, A)$ is the homotopy category of complexes of finitely projective $A$-modules concentrated in degrees $-1$ and $0$. This functor is shown to be full and dense and its kernel is described. It detects isomorphisms, indecomposability and extriangles. In the Hom-finite case, it induces a bijection from the set of isomorphism classes of basic relative cluster-tilting objects of pr$(M)$ to that of basic silting complexs of $\mathcal{H}^{[-1,0]}({\rm proj}\, A)$, which commutes with mutations. These results are applied to cluster categories of self-injective quivers with potential to recover a theorem of Mizuno on the endomorphism algebras of certain 2-term silting complexes. As an interesting consequence of the main results, if $\mathcal{T}$ is a 2-Calabi--Yau triangulated category and $M$ is a cluster-tilting object such that $A$ is self-injective, then $\mathbb{P}$ is an equivalence, in particular, $\mathcal{H}^{[-1,0]}({\rm proj}\, A)$ admits a triangle structure. In the appendix by Iyama it is shown that for a finite-dimensional algebra $A$, if $\mathcal{H}^{[-1,0]}({\rm proj}\, A)$ admits a triangle structure, then $A$ is necessarily self-injective.

math.RT↗

arXiv2025

SpectMamba: Integrating Frequency and State Space Models for Enhanced Medical Image Detection

Abnormality detection in medical imaging is a critical task requiring both high efficiency and accuracy to support effective diagnosis. While convolutional neural networks (CNNs) and Transformer-based models are widely used, both face intrinsic challenges: CNNs have limited receptive fields, restricting their ability to capture broad contextual information, and Transformers encounter prohibitive computational costs when processing high-resolution medical images. Mamba, a recent innovation in natural language processing, has gained attention for its ability to process long sequences with linear complexity, offering a promising alternative. Building on this foundation, we present SpectMamba, the first Mamba-based architecture designed for medical image detection. A key component of SpectMamba is the Hybrid Spatial-Frequency Attention (HSFA) block, which separately learns high- and low-frequency features. This approach effectively mitigates the loss of high-frequency information caused by frequency bias and correlates frequency-domain features with spatial features, thereby enhancing the model's ability to capture global context. To further improve long-range dependencies, we propose the Visual State-Space Module (VSSM) and introduce a novel Hilbert Curve Scanning technique to strengthen spatial correlations and local dependencies, further optimizing the Mamba framework. Comprehensive experiments show that SpectMamba achieves state-of-the-art performance while being both effective and efficient across various medical image detection tasks.

cs.CV↗

arXiv2025

Speaker-Conditioned Phrase Break Prediction for Text-to-Speech with Phoneme-Level Pre-trained Language Model

This paper advances phrase break prediction (also known as phrasing) in multi-speaker text-to-speech (TTS) systems. We integrate speaker-specific features by leveraging speaker embeddings to enhance the performance of the phrasing model. We further demonstrate that these speaker embeddings can capture speaker-related characteristics solely from the phrasing task. Besides, we explore the potential of pre-trained speaker embeddings for unseen speakers through a few-shot adaptation method. Furthermore, we pioneer the application of phoneme-level pre-trained language models to this TTS front-end task, which significantly boosts the accuracy of the phrasing model. Our methods are rigorously assessed through both objective and subjective evaluations, demonstrating their effectiveness.

eess.AS↗

arXiv2025

Decoding the Single-peaked HI Spectra of Low Redshift Post-starburst Galaxies

Recent observations with the Five-hundred-meter Aperture Spherical Telescope (FAST) have revealed abundant reservoirs of neutral hydrogen (HI) in low redshift post-starburst galaxies (PSBs), raising the question of why star formation ceases rapidly in these systems. In this study, we present a detailed analysis of the shape of the integrated HI spectra of 67 PSBs. We find that PSBs exhibit significantly higher HI spectral concentration values ($K$) compared to a matched sample from xGASS, and are more comparable to those of starburst galaxies. By extending our analysis to spatially resolved HI data from THINGS and ATLAS$^{\rm 3D}$, we show that both centrally concentrated HI distributions and dynamically unsettled HI can effectively increase $K$, while non-axisymmetric structures only contribute to the scatter of the $K$ distribution. Distinguishing between central concentration and dynamically unsettled gas as the origin of high $K$ can be achieved by measuring the spectral asymmetry ($A_{\rm F}$), making the $K$-$A_{\rm F}$ plane a powerful diagnostic tool for identifying galaxies with unsettled HI using integrated spectra alone. Based on their location in the $K$-$A_{\rm F}$ plane, we find that most PSBs are not dominated by unsettled HI, but rather exhibit elevated central gas concentration. Both modes of gas redistribution in PSBs may eventually contribute to their quenching.

astro-ph.GA↗

arXiv2025

The Man Behind the Sound: Demystifying Audio Private Attribute Profiling via Multimodal Large Language Model Agents

Our research uncovers a novel privacy risk associated with multimodal large language models (MLLMs): the ability to infer sensitive personal attributes from audio data -- a technique we term audio private attribute profiling. This capability poses a significant threat, as audio can be covertly captured without direct interaction or visibility. Moreover, compared to images and text, audio carries unique characteristics, such as tone and pitch, which can be exploited for more detailed profiling. However, two key challenges exist in understanding MLLM-employed private attribute profiling from audio: (1) the lack of audio benchmark datasets with sensitive attribute annotations and (2) the limited ability of current MLLMs to infer such attributes directly from audio. To address these challenges, we introduce AP^2, an audio benchmark dataset that consists of two subsets collected and composed from real-world data, and both are annotated with sensitive attribute labels. Additionally, we propose Gifts, a hybrid multi-agent framework that leverages the complementary strengths of audio-language models (ALMs) and large language models (LLMs) to enhance inference capabilities. Gifts employs an LLM to guide the ALM in inferring sensitive attributes, then forensically analyzes and consolidates the ALM's inferences, overcoming severe hallucinations of existing ALMs in generating long-context responses. Our evaluations demonstrate that Gifts significantly outperforms baseline approaches in inferring sensitive attributes. Finally, we investigate model-level and data-level defense strategies to mitigate the risks of audio private attribute profiling. Our work validates the feasibility of audio-based privacy attacks using MLLMs, highlighting the need for robust defenses, and provides a dataset and framework to facilitate future research.

cs.CR↗

arXiv2025

Fast Algorithm for Moving Sound Source

Modern neural network-based speech processing systems usually need to have reverberation resistance, so the training of such systems requires a large amount of reverberation data. In the process of system training, it is now more inclined to use sampling static systems to simulate dynamic systems, or to supplement data through actually recorded data. However, this cannot fundamentally solve the problem of simulating motion data that conforms to physical laws. Aiming at the core issue of insufficient training data for speech enhancement models in moving scenarios, this paper proposes Yang's motion spatio-temporal sampling reconstruction theory to realize efficient simulation of motion continuous time-varying reverberation. This theory breaks through the limitations of the traditional static Image-Source Method (ISM) in time-varying systems. By decomposing the impulse response of the moving image source into two parts: linear time-invariant modulation and discrete time-varying fractional delay, a moving sound field model conforming to physical laws is established. Based on the band-limited characteristics of motion displacement, a hierarchical sampling strategy is proposed: high sampling rate is used for low-order images to retain details, and low sampling rate is used for high-order images to reduce computational complexity. A fast synthesis architecture is designed to realize real-time simulation. Experiments show that compared with the open-source models, the proposed theory can more accurately restore the amplitude and phase changes in moving scenarios, solving the industry problem of motion sound source data simulation, and providing high-quality dynamic training data for speech enhancement models.

eess.AS↗

arXiv2025

Relative singularity categories III: Cluster resolutions

We build foundations of an approach to study canonical forms of $2$-Calabi--Yau triangulated categories with cluster-tilting objects, using dg algebras and relative singularity categories. This is motivated by cluster theory, singularity categories, Wemyss's Homological Minimal Model Program and the relations between these topics.

math.RT↗

arXiv2025

Exploring Task Performance with Interpretable Models via Sparse Auto-Encoders

Large Language Models (LLMs) are traditionally viewed as black-box algorithms, therefore reducing trustworthiness and obscuring potential approaches to increasing performance on downstream tasks. In this work, we apply an effective LLM decomposition method using a dictionary-learning approach with sparse autoencoders. This helps extract monosemantic features from polysemantic LLM neurons. Remarkably, our work identifies model-internal misunderstanding, allowing the automatic reformulation of the prompts with additional annotations to improve the interpretation by LLMs. Moreover, this approach demonstrates a significant performance improvement in downstream tasks, such as mathematical reasoning and metaphor detection.

cs.CL↗

arXiv2025

Low redshift post-starburst galaxies host abundant HI reservoirs

Studying the gas content of post-starburst (PSB) galaxies can provide valuable clues regarding the process of fast quenching. Although previous works have studied the molecular gas content of PSBs, only a handful of HI measurements exist. Here, we present new Five hundred metre Aperture Spherical Telescope (FAST) 21cm observations of 44 PSBs, leading to 43 detections or sensitive upper limits of HI, which we combine with 25 archival measurements, for a total sample of 68 PSB MHI measurements. HI is detected in 57/68 galaxies, with HI masses ranging from MHI ~10^8.5 up to 10^10 Msun and gas fractions (fHI = MHI/M*) from a few percent up to almost 30 percent. Post-starbursts therefore retain ample atomic gas reservoirs, despite no longer forming stars. By comparing with a stellar mass-matched sample of star-forming galaxies in xGASS, we find that PSBs have, on average, gas fractions lower by ~0.2-0.4 dex, consistent with a mild reduction compared with their progenitor population. However, PSBs show a diversity of HI properties; about half have HI gas masses within the expected scatter of the star-forming population with the remaining 50 per cent up to a factor of 10 more gas-poor. Compared with galaxies in the green valley, about two thirds of PSBs have gas fractions within the expected range, with the remaining third up to a factor of 10 more gas-rich. Our results demonstrate that quenching in PSBs is not the result of wholesale removal of the atomic gas reservoir and that the population has atomic gas fractions that span the range from star-forming to green valley galaxies. We find no correlation between HI gas mass and time since burst; even galaxies a Gyr past their burst can remain HI-normal. The significant gas reservoirs remaining in many PSBs leaves open the possibility for future rekindling of star formation.

astro-ph.GA↗

arXiv2025

Multi-Knowledge-oriented Nighttime Haze Imaging Enhancer for Vision-driven Intelligent Systems

Salient object detection (SOD) plays a critical role in Intelligent Imaging, facilitating the detection and segmentation of key visual elements in an image. However, adverse imaging conditions such as haze during the day, low light, and haze at night severely degrade image quality and hinder reliable object detection in real-world scenarios. To address these challenges, we propose a multi-knowledge-oriented nighttime haze imaging enhancer (MKoIE), which integrates three tasks: daytime dehazing, low-light enhancement, and nighttime dehazing. The MKoIE incorporates two key innovative components: First, the network employs a task-oriented node learning mechanism to handle three specific degradation types: day-time haze, low light, and night-time haze conditions, with an embedded self-attention module enhancing its performance in nighttime imaging. In addition, multi-receptive field enhancement module that efficiently extracts multi-scale features through three parallel depthwise separable convolution branches with different dilation rates, capturing comprehensive spatial information with minimal computational overhead to meet the requirements of real-time imaging deployment. To ensure optimal image reconstruction quality and visual characteristics, we suggest a hybrid loss function. Extensive experiments on different types of weather/imaging conditions illustrate that MKoIE surpasses existing methods, enhancing the reliability, accuracy, and operational efficiency of intelligent imaging.

cs.CV↗

arXiv2025

Derived preprojective algebras and spherical twist functors

We study silting objects over derived preprojective algebras of acyclic quivers by giving a direct relationship between silting objects, spherical twist functors and mutations. Especially, for a Dynkin quiver, we establish a bijection between the elements of the braid group and the set of isomorphism classes of basic silting objects over the derived preprojective algebra.

math.RT↗

arXiv2025

The coexistence of possible magnetic and chiral rotation in $^{129}\mathrm{Cs}$ and $^{131}\mathrm{La}$: a microscopic investigation

A microscopic investigation of the rotational properties in $^{129}\mathrm{Cs}$ and $^{131}\mathrm{La}$ was carried out using the three-dimensional tilted axis cranking covariant density functional theory (3DTAC-CDFT). The calculations reveal the coexistence of magnetic and chiral rotation built on identical qusiparticle configurations $π{h}_{11/2}^{1}\otimes ν{h}_{11/2}^{-2}$ in $^{129}\mathrm{Cs}$ and $^{131}\mathrm{La}$, establishing a new type of shape coexistence. The calculations predict the deformation parameters for magnetic rotation in $^{129}\mathrm{Cs}$ ($β\approx0.23$, $γ\approx41^\circ$) and $^{131}\mathrm{La}$ ($β\approx0.25,$ $γ\approx42^\circ$), along with those for possible chiral rotation in $^{129}\mathrm{Cs}$ ($β\approx0.20,$ $γ\approx29^\circ$) and in $^{131}\mathrm{La}$ ($β\approx0.20,$ $γ\approx27^\circ$). The energy spectra, the relation between the spin and the rotational frequency, and the reduced $M1$ and $E2$ transition probabilities are obtained with the various configurations. The experimental characteristics of band B8 in $^{129}\mathrm{Cs}$ and band 13 in $^{131}\mathrm{La}$ are well reproduced. Moreover, a distinctive rotational mode transition is uncovered in this work, progressing from the principal-axis rotation to the planar rotation and finally to the chiral rotation, as the rotational frequency evolves through different regimes.

nucl-th↗