arXiv subjects

Jingyu Li

Publications and source records attributed to Jingyu Li.

At least 73 records · Page 4Linked to original sources

arXiv2025

Benchmarking Foundation Speech and Language Models for Alzheimer's Disease and Related Dementia Detection from Spontaneous Speech

Background: Alzheimer's disease and related dementias (ADRD) are progressive neurodegenerative conditions where early detection is vital for timely intervention and care. Spontaneous speech contains rich acoustic and linguistic markers that may serve as non-invasive biomarkers for cognitive decline. Foundation models, pre-trained on large-scale audio or text data, produce high-dimensional embeddings encoding contextual and acoustic features. Methods: We used the PREPARE Challenge dataset, which includes audio recordings from over 1,600 participants with three cognitive statuses: healthy control (HC), mild cognitive impairment (MCI), and Alzheimer's Disease (AD). We excluded non-English, non-spontaneous, or poor-quality recordings. The final dataset included 703 (59.13%) HC, 81 (6.81%) MCI, and 405 (34.06%) AD cases. We benchmarked a range of open-source foundation speech and language models to classify cognitive status into the three categories. Results: The Whisper-medium model achieved the highest performance among speech models (accuracy = 0.731, AUC = 0.802). Among language models, BERT with pause annotation performed best (accuracy = 0.662, AUC = 0.744). ADRD detection using state-of-the-art automatic speech recognition (ASR) model-generated audio embeddings outperformed others. Including non-semantic features like pause patterns consistently improved text-based classification. Conclusion: This study introduces a benchmarking framework using foundation models and a clinically relevant dataset. Acoustic-based approaches -- particularly ASR-derived embeddings -- demonstrate strong potential for scalable, non-invasive, and cost-effective early detection of ADRD.

cs.CL↗

arXiv2025

Label-efficient multi-organ segmentation with a diffusion model

Accurate segmentation of multiple organs in Computed Tomography (CT) images plays a vital role in computer-aided diagnosis systems. While various supervised learning approaches have been proposed recently, these methods heavily depend on a large amount of high-quality labeled data, which are expensive to obtain in practice. To address this challenge, we propose a label-efficient framework using knowledge transfer from a pre-trained diffusion model for CT multi-organ segmentation. Specifically, we first pre-train a denoising diffusion model on 207,029 unlabeled 2D CT slices to capture anatomical patterns. Then, the model backbone is transferred to the downstream multi-organ segmentation task, followed by fine-tuning with few labeled data. In fine-tuning, two fine-tuning strategies, linear classification and fine-tuning decoder, are employed to enhance segmentation performance while preserving learned representations. Quantitative results show that the pre-trained diffusion model is capable of generating diverse and realistic 256x256 CT images (Fréchet inception distance (FID): 11.32, spatial Fréchet inception distance (sFID): 46.93, F1-score: 73.1%). Compared to state-of-the-art methods for multi-organ segmentation, our method achieves competitive performance on the FLARE 2022 dataset, particularly in limited labeled data scenarios. After fine-tuning with 1% and 10% labeled data, our method achieves dice similarity coefficients (DSCs) of 71.56% and 78.51%, respectively. Remarkably, the method achieves a DSC score of 51.81% using only four labeled CT slices. These results demonstrate the efficacy of our approach in overcoming the limitations of supervised learning approaches that is highly dependent on large-scale labeled data.

cs.CV↗

arXiv2025

MiCEval: Unveiling Multimodal Chain of Thought's Quality via Image Description and Reasoning Steps

Multimodal Chain of Thought (MCoT) is a popular prompting strategy for improving the performance of multimodal large language models (MLLMs) across a range of complex reasoning tasks. Despite its popularity, there is a notable absence of automated methods for evaluating the quality of reasoning steps in MCoT. To address this gap, we propose Multimodal Chain-of-Thought Evaluation (MiCEval), a framework designed to assess the correctness of reasoning chains by evaluating the quality of both the description and each reasoning step. The evaluation of the description component focuses on the accuracy of the image descriptions, while the reasoning step evaluates the quality of each step as it is conditionally generated based on the preceding steps. MiCEval is built upon a fine-grained dataset with annotations that rate each step according to correctness, relevance, and informativeness. Extensive experiments on four state-of-the-art MLLMs show that step-wise evaluations using MiCEval align more closely with human judgments compared to existing methods based on cosine similarity or fine-tuning approaches. MiCEval datasets and code can be found in https://github.com/alenai97/MiCEval.

cs.CL↗

arXiv2025

UI-TARS: Pioneering Automated GUI Interaction with Native Agents

This paper introduces UI-TARS, a native GUI agent model that solely perceives the screenshots as input and performs human-like interactions (e.g., keyboard and mouse operations). Unlike prevailing agent frameworks that depend on heavily wrapped commercial models (e.g., GPT-4o) with expert-crafted prompts and workflows, UI-TARS is an end-to-end model that outperforms these sophisticated frameworks. Experiments demonstrate its superior performance: UI-TARS achieves SOTA performance in 10+ GUI agent benchmarks evaluating perception, grounding, and GUI task execution. Notably, in the OSWorld benchmark, UI-TARS achieves scores of 24.6 with 50 steps and 22.7 with 15 steps, outperforming Claude (22.0 and 14.9 respectively). In AndroidWorld, UI-TARS achieves 46.6, surpassing GPT-4o (34.5). UI-TARS incorporates several key innovations: (1) Enhanced Perception: leveraging a large-scale dataset of GUI screenshots for context-aware understanding of UI elements and precise captioning; (2) Unified Action Modeling, which standardizes actions into a unified space across platforms and achieves precise grounding and interaction through large-scale action traces; (3) System-2 Reasoning, which incorporates deliberate reasoning into multi-step decision making, involving multiple reasoning patterns such as task decomposition, reflection thinking, milestone recognition, etc. (4) Iterative Training with Reflective Online Traces, which addresses the data bottleneck by automatically collecting, filtering, and reflectively refining new interaction traces on hundreds of virtual machines. Through iterative training and reflection tuning, UI-TARS continuously learns from its mistakes and adapts to unforeseen situations with minimal human intervention. We also analyze the evolution path of GUI agents to guide the further development of this domain.

cs.AI↗

arXiv2024

Traveling waves to a logarithmic chemotaxis model with fast diffusion and singularities

This paper is concerned with a chemotaxis model with logarithmic sensitivity and fast diffusion, which possesses strong singularities for the sensitivity at zero-concentration of chemical signal, and for the diffusion at zero-population of cells, respectively. The main purpose is to show the existence of traveling waves connecting the singular zero-end-state, and particularly, to show the asymptotic stability of these traveling waves. The challenge of the problem is the interaction of two kinds of singularities involved in the model: one is the logarithmic singularity of the sensitivity; and the other is the power-law singularity of the diffusivity. To overcome the singularities for the wave stability, some new techniques of weighted energy method are introduced artfully. Numerical simulations are also carried out, which further confirm our theoretical stability results, in particular, the numerical results indicate that the effect of fast diffusion to the structure of traveling waves is essential, which causes the traveling waves much steeper like shock waves. This new phenomenon is a first observation.

math.AP↗

arXiv2024

Transition layers to chemotaxis-consumption models with volume-filling effect

We are interested in the dynamical behaviors of solutions to a parabolic-parabolic chemotaxis-consumption model with a volume-filling effect on a bounded interval, where the physical no-flux boundary condition for the bacteria and mixed Dirichlet-Neumann boundary condition for the oxygen are prescribed. By taking a continuity argument, we first show that the model admits a unique nonconstant steady state. Then we use Helly's compactness theorem to show that the asymptotic profile of steady state is a transition layer as the chemotactic coefficient goes to infinity. Finally, based on the energy method along with a cancellation structure of the model, we show that the steady state is nonlinearly stable under appropriate perturbations. Moreover, we do not need any assumption on the parameters in showing the stability of steady state.

math.AP↗

arXiv2024

FairSort: Learning to Fair Rank for Personalized Recommendations in Two-Sided Platforms

Traditional recommendation systems focus on maximizing user satisfaction by suggesting their favourite items. This user-centric approach may lead to unfair exposure distribution among the providers. On the contrary, a provider-centric design might become unfair to the users. Therefore, this paper proposes a re-ranking model FairSort to find a trade-off solution among user-side fairness, provider-side fairness, and personalized recommendations utility. Previous works habitually treat this issue as a knapsack problem, incorporating both-side fairness as constraints. In this paper, we adopt a novel perspective, treating each recommendation list as a runway rather than a knapsack. In this perspective, each item on the runway gains a velocity and runs within a specific time, achieving re-ranking for both-side fairness. Meanwhile, we ensure the Minimum Utility Guarantee for personalized recommendations by designing a Binary Search approach. This can provide more reliable recommendations compared to the conventional greedy strategy based on the knapsack problem. We further broaden the applicability of FairSort, designing two versions for online and offline recommendation scenarios. Theoretical analysis and extensive experiments on real-world datasets indicate that FairSort can ensure more reliable personalized recommendations while considering fairness for both the provider and user.

cs.GT↗

arXiv2024

Functional central limit theorems for spatial averages of the parabolic Anderson model with delta initial condition in dimension $d\geq 1$

Let $\{u(t,x)\}_{t>0,x\in{{\mathbb R}^{d}}}$ denote the solution to a $d$-dimensional parabolic Anderson model with delta initial condition and driven by a multiplicative noise that is white in time and has a spatially homogeneous covariance given by a nonnegative-definite measure $f$. Let $S_{N,t}:=N^{-d}\int_{{[0,N]}^d}{[U(t,x)-1]}{\rm d}x$ denote the spatial average on ${{\mathbb R}^{d}}$. We obtain various functional central limit theorems (CLTs) for spatial averages based on the quantitative analysis of $f$ and spatial dimension $d$. In particular, when $f$ is given by Riesz kernel, that is, $f({\rm x})={\Vert x \Vert}^{-β}{\rm d}x$, $β\in(0,2\wedge d)$, the functional CLT is also based on the index $β$.

math.PR↗

arXiv2024

Boundary spike-layer solutions of the singular Keller-Segel system: existence, profiles and stability

This paper is concerned with the boundary-layer solutions of the singular Keller-Segel model proposed by Keller-Segel (1971) in a multi-dimensional domain, where the zero-flux boundary condition is imposed to the cell while inhomogeneous Dirichlet boundary condition to the nutrient. The steady-state problem of the Keller-Segel system is reduced to a scalar Dirichlet nonlocal elliptic problem with singularity. Studying this nonlocal problem, we obtain the unique steady-state solution which possesses a boundary spike-layer profile as nutrient diffusion coefficient $\varepsilon>0$ tends to zero. When the domain is radially symmetric, we find the explicit expansion for the slope of boundary-layer profiles at the boundary and boundary-layer thickness in terms of the radius as $\varepsilon>0$ is small, which pinpoints how the boundary curvature affects the boundary-layer profile and thickness. Furthermore, we establish the nonlinear exponential stability of the boundary-layer steady-state solution for the radially symmetric domain. The main challenge encountered in the analysis is that the singularity will arise when the nutrient diffusion coefficient $\varepsilon>0$ is small for both stationary and time-dependent problems. By relegating the nonlocal steady-state problem to local problems and performing a delicate analysis using the barrier method and Fermi coordinates, we can obtain refined estimates for the solution of local steady-state problem near the boundary. This strategy finally helps us to find the asymptotic profile of the solution to the nonlocal problem as $\varepsilon \to 0$ so that the singularity is accurately captured and hence properly handled to achieve our results.

math.AP↗

arXiv2024

Dual-path Collaborative Generation Network for Emotional Video Captioning

Emotional Video Captioning is an emerging task that aims to describe factual content with the intrinsic emotions expressed in videos. The essential of the EVC task is to effectively perceive subtle and ambiguous visual emotional cues during the caption generation, which is neglected by the traditional video captioning. Existing emotional video captioning methods perceive global visual emotional cues at first, and then combine them with the video features to guide the emotional caption generation, which neglects two characteristics of the EVC task. Firstly, their methods neglect the dynamic subtle changes in the intrinsic emotions of the video, which makes it difficult to meet the needs of common scenes with diverse and changeable emotions. Secondly, as their methods incorporate emotional cues into each step, the guidance role of emotion is overemphasized, which makes factual content more or less ignored during generation. To this end, we propose a dual-path collaborative generation network, which dynamically perceives visual emotional cues evolutions while generating emotional captions by collaborative learning. Specifically, in the dynamic emotion perception path, we propose a dynamic emotion evolution module, which first aggregates visual features and historical caption features to summarize the global visual emotional cues, and then dynamically selects emotional cues required to be re-composed at each stage. Besides, in the adaptive caption generation path, to balance the description of factual content and emotional cues, we propose an emotion adaptive decoder. Thus, our methods can generate emotion-related words at the necessary time step, and our caption generation balances the guidance of factual content and emotional cues well. Extensive experiments on three challenging datasets demonstrate the superiority of our approach and each proposed module.

cs.CV↗

arXiv2024

Not all explicit cues help communicate: Pedestrians' perceptions, fixations, and decisions toward automated vehicles with varied appearance

Given pedestrians' vulnerability in road traffic, it remains unclear how novel AV appearances will impact pedestrians crossing behaviour. To address this gap, this study pioneers an investigation into the influence of AVs' exterior design, correlated with their kinematics, on pedestrians' road-crossing perception and decision-making. A video-based eye-tracking experimental study was conducted with 61 participants who responded to video stimuli depicting a manipulated vehicle approaching a predefined road-crossing location on an unsignalized, two-way road. The vehicle's kinematic pattern was manipulated into yielding and non-yielding, and its external appearances were varied across five types: with a human driver (as a conventional vehicle), with no driver (as an AV), with text-based identity indications, with roof radar sensors, with dynamic eHMIs adjusted to vehicle kinematics. Participants' perceived clarity, crossing initiation distance (CID), crossing decision time (CDT), and gaze behaviour, during interactions were recorded and reported. The results indicated that AVs' kinematic profiles play a dominant role in pedestrians' road-crossing decisions, supported by their subjective evaluations, CID, CDT, and gaze patterns during interactions. Moreover, the use of clear eHMI, such as dynamic pedestrian icons, reduced pedestrians' visual load, enhanced their perceptual clarity, expedited road-crossing decisions, and thereby improved overall crossing efficiency. However, it was found that both textual identity indications and roof radar sensors have no significant effect on pedestrians' decisions but negatively impact pedestrians' visual attention, as evidenced by heightened fixation counts and prolonged fixation durations, particularly under yielding conditions. Excessive visual and cognitive resource occupation suggests that not all explicit cues facilitate human-vehicle communication.

cs.HC↗

arXiv2024

DSNet: A Novel Way to Use Atrous Convolutions in Semantic Segmentation

Atrous convolutions are employed as a method to increase the receptive field in semantic segmentation tasks. However, in previous works of semantic segmentation, it was rarely employed in the shallow layers of the model. We revisit the design of atrous convolutions in modern convolutional neural networks (CNNs), and demonstrate that the concept of using large kernels to apply atrous convolutions could be a more powerful paradigm. We propose three guidelines to apply atrous convolutions more efficiently. Following these guidelines, we propose DSNet, a Dual-Branch CNN architecture, which incorporates atrous convolutions in the shallow layers of the model architecture, as well as pretraining the nearly entire encoder on ImageNet to achieve better performance. To demonstrate the effectiveness of our approach, our models achieve a new state-of-the-art trade-off between accuracy and speed on ADE20K, Cityscapes and BDD datasets. Specifically, DSNet achieves 40.0% mIOU with inference speed of 179.2 FPS on ADE20K, and 80.4% mIOU with speed of 81.9 FPS on Cityscapes. Source code and models are available at Github: https://github.com/takaniwa/DSNet.

cs.CV↗

arXiv2024

Nonlinear stability of shock profiles to Burgers' equation with critical fast diffusion and singularity

In this paper we propose the first framework to study Burgers' equation featuring critical fast diffusion in form of $u_t+f(u)_x = (\ln u)_{xx}$. The solution possesses a strong singularity when $u=0$ hence bringing technical challenges. The main purpose of this paper is to investigate the asymptotic stability of viscous shocks, particularly those with shock profiles vanishing at the far field $x=+\infty$. To overcome the singularity, we introduce some weight functions and show the nonlinear stability of shock profiles through the weighted energy method. Numerical simulations are also carried out in different cases of fast diffusion with singularity, which illustrate and confirm our theoretical results.

math.AP↗

arXiv2024

Creating Personalized Synthetic Voices from Articulation Impaired Speech Using Augmented Reconstruction Loss

This research is about the creation of personalized synthetic voices for head and neck cancer survivors. It is focused particularly on tongue cancer patients whose speech might exhibit severe articulation impairment. Our goal is to restore normal articulation in the synthesized speech, while maximally preserving the target speaker's individuality in terms of both the voice timbre and speaking style. This is formulated as a task of learning from noisy labels. We propose to augment the commonly used speech reconstruction loss with two additional terms. The first term constitutes a regularization loss that mitigates the impact of distorted articulation in the training speech. The second term is a consistency loss that encourages correct articulation in the generated speech. These additional loss terms are obtained from frame-level articulation scores of original and generated speech, which are derived using a separately trained phone classifier. Experimental results on a real case of tongue cancer patient confirm that the synthetic voice achieves comparable articulation quality to unimpaired natural speech, while effectively maintaining the target speaker's individuality. Audio samples are available at https://myspeechproject.github.io/ArticulationRepair/.

eess.AS↗

arXiv2023

Model Compression for DNN-based Speaker Verification Using Weight Quantization

DNN-based speaker verification (SV) models demonstrate significant performance at relatively high computation costs. Model compression can be applied to reduce the model size for lower resource consumption. The present study exploits weight quantization to compress two widely-used SV models, namely ECAPA-TDNN and ResNet. Experimental results on VoxCeleb show that weight quantization is effective for compressing SV models. The model size can be reduced multiple times without noticeable degradation in performance. Compression of ResNet shows more robust results than ECAPA-TDNN with lower-bitwidth quantization. Analysis of the layer weights suggests that the smooth weight distribution of ResNet may be related to its better robustness. The generalization ability of the quantized model is validated via a language-mismatched SV task. Furthermore, analysis by information probing reveals that the quantized models can retain most of the speaker-relevant knowledge learned by the original models.

eess.AS↗

arXiv2023

Efficient Black-Box Speaker Verification Model Adaptation with Reprogramming and Backend Learning

The development of deep neural networks (DNN) has significantly enhanced the performance of speaker verification (SV) systems in recent years. However, a critical issue that persists when applying DNN-based SV systems in practical applications is domain mismatch. To mitigate the performance degradation caused by the mismatch, domain adaptation becomes necessary. This paper introduces an approach to adapt DNN-based SV models by manipulating the learnable model inputs, inspired by the concept of adversarial reprogramming. The pre-trained SV model remains fixed and functions solely in the forward process, resembling a black-box model. A lightweight network is utilized to estimate the gradients for the learnable parameters at the input, which bypasses the gradient backpropagation through the black-box model. The reprogrammed output is processed by a two-layer backend learning module as the final adapted speaker embedding. The number of parameters involved in the gradient calculation is small in our design. With few additional parameters, the proposed method achieves both memory and parameter efficiency. The experiments are conducted in language mismatch scenarios. Using much less computation cost, the proposed method obtains close or superior performance to the fully finetuned models in our experiments, which demonstrates its effectiveness.

eess.AS↗

arXiv2023

Convergence of densities of spatial averages of the linear stochastic heat equation

Let $\{u(t,x)\}_{t>0,x\in{{\mathbb R}^{d}}}$ denote the solution to the linear (fractional) stochastic heat equation. We establish rates of convergence with respect to the uniform distance between the density of spatial averages of solution and the density of the standard normal distribution in some different scenarios. We first consider the case that $u_0\equiv1$, and the stochastic fractional heat equation is driven by a space-time white noise. When $α=2$ (parabolic Anderson model, PAM for short) and the stochastic heat equation is driven by colored noise in space, we present the rates of convergence respectively in the case that $u_0\equiv1$, $d\geq1$ and $u_0=δ_0$, $d=1$ under an additional condition $\hat f(\mathbb{R}^d)<\infty$. Our results are obtained by using a combination of the Malliavin calculus and Stein's method for normal approximations.

math.PR↗

arXiv2023

SOOD: Towards Semi-Supervised Oriented Object Detection

Semi-Supervised Object Detection (SSOD), aiming to explore unlabeled data for boosting object detectors, has become an active task in recent years. However, existing SSOD approaches mainly focus on horizontal objects, leaving multi-oriented objects that are common in aerial images unexplored. This paper proposes a novel Semi-supervised Oriented Object Detection model, termed SOOD, built upon the mainstream pseudo-labeling framework. Towards oriented objects in aerial scenes, we design two loss functions to provide better supervision. Focusing on the orientations of objects, the first loss regularizes the consistency between each pseudo-label-prediction pair (includes a prediction and its corresponding pseudo label) with adaptive weights based on their orientation gap. Focusing on the layout of an image, the second loss regularizes the similarity and explicitly builds the many-to-many relation between the sets of pseudo-labels and predictions. Such a global consistency constraint can further boost semi-supervised learning. Our experiments show that when trained with the two proposed losses, SOOD surpasses the state-of-the-art SSOD methods under various settings on the DOTA-v1.5 benchmark. The code will be available at https://github.com/HamPerdredes/SOOD.

cs.CV↗