arXiv subjects

Yu Xiao

Publications and source records attributed to Yu Xiao.

At least 55 records · Page 3Linked to original sources

arXiv2025

Hierarchical Motion Captioning Utilizing External Text Data Source

This paper introduces a novel approach to enhance existing motion captioning methods, which directly map representations of movement to high-level descriptive captions (e.g., ``a person doing jumping jacks"). The existing methods require motion data annotated with high-level descriptions (e.g., ``jumping jacks"). However, such data is rarely available in existing motion-text datasets, which additionally do not include low-level motion descriptions. To address this, we propose a two-step hierarchical approach. First, we employ large language models to create detailed descriptions corresponding to each high-level caption that appears in the motion-text datasets (e.g., ``jumping while synchronizing arm extensions with the opening and closing of legs" for ``jumping jacks"). These refined annotations are used to retrain motion-to-text models to produce captions with low-level details. Second, we introduce a pioneering retrieval-based mechanism. It aligns the detailed low-level captions with candidate high-level captions from additional text data sources, and combine them with motion features to fabricate precise high-level captions. Our methodology is distinctive in its ability to harness knowledge from external text sources to greatly increase motion captioning accuracy, especially for movements not covered in existing motion-text datasets. Experiments on three distinct motion-text datasets (HumanML3D, KIT, and BOTH57M) demonstrate that our method achieves an improvement in average performance (across BLEU-1, BLEU-4, CIDEr, and ROUGE-L) ranging from 6% to 50% compared to the state-of-the-art M2T-Interpretable.

cs.LG↗

arXiv2025

Exploring the Inquiry-Diagnosis Relationship with Advanced Patient Simulators

Recently, large language models have shown great potential to transform online medical consultation. Despite this, most research targets improving diagnostic accuracy with ample information, often overlooking the inquiry phase. Some studies try to evaluate or refine doctor models by using prompt-engineered patient agents. However, prompt engineering alone falls short in accurately simulating real patients. We need to explore new paradigms for patient simulation. Furthermore, the relationship between inquiry and diagnosis remains unexplored. This paper extracts dialogue strategies from real doctor-patient conversations to guide the training of a patient simulator. Our simulator shows higher anthropomorphism and lower hallucination rates, using dynamic dialogue strategies. This innovation offers a more accurate evaluation of diagnostic models and generates realistic synthetic data. We conduct extensive experiments on the relationship between inquiry and diagnosis, showing they adhere to Liebig's law: poor inquiry limits diagnosis effectiveness, regardless of diagnostic skill, and vice versa. The experiments also reveal substantial differences in inquiry performance among models. To delve into this phenomenon, the inquiry process is categorized into four distinct types. Analyzing the distribution of inquiries across these types helps explain the performance differences. The weights of our patient simulator are available https://github.com/PatientSimulator/PatientSimulator.

cs.CL↗

arXiv2025

Zero-shot Load Forecasting for Integrated Energy Systems: A Large Language Model-based Framework with Multi-task Learning

The growing penetration of renewable energy sources in power systems has increased the complexity and uncertainty of load forecasting, especially for integrated energy systems with multiple energy carriers. Traditional forecasting methods heavily rely on historical data and exhibit limited transferability across different scenarios, posing significant challenges for emerging applications in smart grids and energy internet. This paper proposes the TSLLM-Load Forecasting Mechanism, a novel zero-shot load forecasting framework based on large language models (LLMs) to address these challenges. The framework consists of three key components: a data preprocessing module that handles multi-source energy load data, a time series prompt generation module that bridges the semantic gap between energy data and LLMs through multi-task learning and similarity alignment, and a prediction module that leverages pre-trained LLMs for accurate forecasting. The framework's effectiveness was validated on a real-world dataset comprising load profiles from 20 Australian solar-powered households, demonstrating superior performance in both conventional and zero-shot scenarios. In conventional testing, our method achieved a Mean Squared Error (MSE) of 0.4163 and a Mean Absolute Error (MAE) of 0.3760, outperforming existing approaches by at least 8\%. In zero-shot prediction experiments across 19 households, the framework maintained consistent accuracy with a total MSE of 11.2712 and MAE of 7.6709, showing at least 12\% improvement over current methods. The results validate the framework's potential for accurate and transferable load forecasting in integrated energy systems, particularly beneficial for renewable energy integration and smart grid applications.

cs.LG↗

arXiv2024

Intelligent acceleration adaptive control of linear $2\times2$ hyperbolic PDE systems

Traditional approaches to stabilizing hyperbolic PDEs, such as PDE backstepping, often encounter challenges when dealing with high-dimensional or complex nonlinear problems. Their solutions require high computational and analytical costs. Recently, neural operators (NOs) for the backstepping design of first-order hyperbolic partial differential equations (PDEs) have been introduced, which rapidly generate gain kernel without requiring online numerical solution. In this paper we apply neural operators to a more complex class of $2\times2$ hyperbolic PDE systems for adaptive stability control. Once the NO has been well-trained offline on a sufficient training set obtained using a numerical solver, the kernel equation no longer needs to be solved again, thereby avoiding the high computational cost during online operations.Specifically, we introduce the deep operator network (DeepONet), a neural network framework, to learn the nonlinear operator of the system parameters to the kernel gain. The approximate backstepping kernel is obtained by utilizing the network after learning, instead of numerically solving the kernel equations in the form of PDEs, to further derive the approximate controller and the target system. We analyze the existence and approximation of DeepONet operators and provide stability and convergence proofs for the closed-loop systems with NOs. Finally, the effectiveness of the proposed NN-adaptive control scheme is verified by comparative simulation, which shows that the NN operator achieved up to three orders of magnitude faster compared to conventional PDE solvers, significantly improving real-time control performance.

math.AP↗

arXiv2024

Transformer-Based Approaches for Sensor-Based Human Activity Recognition: Opportunities and Challenges

Transformers have excelled in natural language processing and computer vision, paving their way to sensor-based Human Activity Recognition (HAR). Previous studies show that transformers outperform their counterparts exclusively when they harness abundant data or employ compute-intensive optimization algorithms. However, neither of these scenarios is viable in sensor-based HAR due to the scarcity of data in this field and the frequent need to perform training and inference on resource-constrained devices. Our extensive investigation into various implementations of transformer-based versus non-transformer-based HAR using wearable sensors, encompassing more than 500 experiments, corroborates these concerns. We observe that transformer-based solutions pose higher computational demands, consistently yield inferior performance, and experience significant performance degradation when quantized to accommodate resource-constrained devices. Additionally, transformers demonstrate lower robustness to adversarial attacks, posing a potential threat to user trust in HAR.

cs.LG↗

arXiv2024

Enhancing Motion Variation in Text-to-Motion Models via Pose and Video Conditioned Editing

Text-to-motion models that generate sequences of human poses from textual descriptions are garnering significant attention. However, due to data scarcity, the range of motions these models can produce is still limited. For instance, current text-to-motion models cannot generate a motion of kicking a football with the instep of the foot, since the training data only includes martial arts kicks. We propose a novel method that uses short video clips or images as conditions to modify existing basic motions. In this approach, the model's understanding of a kick serves as the prior, while the video or image of a football kick acts as the posterior, enabling the generation of the desired motion. By incorporating these additional modalities as conditions, our method can create motions not present in the training set, overcoming the limitations of text-motion datasets. A user study with 26 participants demonstrated that our approach produces unseen motions with realism comparable to commonly represented motions in text-motion datasets (e.g., HumanML3D), such as walking, running, squatting, and kicking.

cs.LG↗

arXiv2024

Cooperative Path Planning with Asynchronous Multiagent Reinforcement Learning

In this paper, we study the shortest path problem (SPP) with multiple source-destination pairs (MSD), namely MSD-SPP, to minimize average travel time of all shortest paths. The inherent traffic capacity limits within a road network contributes to the competition among vehicles. Multi-agent reinforcement learning (MARL) model cannot offer effective and efficient path planning cooperation due to the asynchronous decision making setting in MSD-SPP, where vehicles (a.k.a agents) cannot simultaneously complete routing actions in the previous time step. To tackle the efficiency issue, we propose to divide an entire road network into multiple sub-graphs and subsequently execute a two-stage process of inter-region and intra-region route planning. To address the asynchronous issue, in the proposed asyn-MARL framework, we first design a global state, which exploits a low-dimensional vector to implicitly represent the joint observations and actions of multi-agents. Then we develop a novel trajectory collection mechanism to decrease the redundancy in training trajectories. Additionally, we design a novel actor network to facilitate the cooperation among vehicles towards the same or close destinations and a reachability graph aimed at preventing infinite loops in routing paths. On both synthetic and real road networks, our evaluation result demonstrates that our approach outperforms state-of-the-art planning approaches.

cs.AI↗

arXiv2024

Spatial-temporal manipulations of visible nanosecond sub-pulse sequences in an actively Q-switched Pr:YLF laser

Pulsed visible lasers either by Q-switching or mode locking have been attracting intense attentions both in solid-state laser and fiber laser. Here, we report on the simultaneous manipulation of reconfigurable sub-pulse sequences and customizable high-order vortex beams in an actively Q-switched visible laser. On the one hand, pulse sequences with up to 4 sub-pulses could be generated and fully controlled by means of an acoustic-optic modulator driven by an arbitrary waveform generator. Both pulse number and pulse intensity can be manipulated through the programmable step-signal, which is also theoretically simulated through the rate equations. On the other hand, assisted by the off-axis pumping technique and the astigmatic mode conversion, the laser cavity could emit high-quality vortex beams carrying Laguerre-Gaussian modes up to 30th order. To the best of our knowledge, this is the most flexible active manipulations not only on the intensity distribution of the transverse modes but also on the temporal distribution of the pulse sequences in a visible laser. The versatile manipulating techniques in this work could be immediately implemented into all other solid-state lasers to obtain sub-pulse vortex beams, which may provide enhanced functionality and flexibility for a large range of laser systems.

physics.optics↗

arXiv2024

Well-posedness and Continuity Properties of the Fornberg_Whitham equation in Besov space $B^1_{\infty,1}(\R)$

For the Fornberg-Whitham equation, the local well-posedness in the critical Besov space $B_{p, 1}^{1+\frac{1}{p}}(\mathbb{R})$ with $1\leq p <\infty$ has been studied in (Guo, Nonlinear Anal. RWA., 2023). However, for the endpoint case $p=\infty$, whether it is locally well-posed or ill-posed in $B_{\infty, 1}^{1}(\mathbb{R})$ is still unknown. In this paper, we prove that the Fornberg-Whitham equation is well-posed in the critical Besov space $B_{\infty, 1}^{1}(\mathbb{R})$ with solutions depending continuously on initial data, which is different from that of the Camassa-Holm equation (Guo et al., J. Differ. Equ., 2022). In addition, we show that this dependence is sharp by showing that the solution map is not uniformly continuous on the initial data.

math.AP↗

arXiv2023

Beyond the Snapshot: Brain Tokenized Graph Transformer for Longitudinal Brain Functional Connectome Embedding

Under the framework of network-based neurodegeneration, brain functional connectome (FC)-based Graph Neural Networks (GNN) have emerged as a valuable tool for the diagnosis and prognosis of neurodegenerative diseases such as Alzheimer's disease (AD). However, these models are tailored for brain FC at a single time point instead of characterizing FC trajectory. Discerning how FC evolves with disease progression, particularly at the predementia stages such as cognitively normal individuals with amyloid deposition or individuals with mild cognitive impairment (MCI), is crucial for delineating disease spreading patterns and developing effective strategies to slow down or even halt disease advancement. In this work, we proposed the first interpretable framework for brain FC trajectory embedding with application to neurodegenerative disease diagnosis and prognosis, namely Brain Tokenized Graph Transformer (Brain TokenGT). It consists of two modules: 1) Graph Invariant and Variant Embedding (GIVE) for generation of node and spatio-temporal edge embeddings, which were tokenized for downstream processing; 2) Brain Informed Graph Transformer Readout (BIGTR) which augments previous tokens with trainable type identifiers and non-trainable node identifiers and feeds them into a standard transformer encoder to readout. We conducted extensive experiments on two public longitudinal fMRI datasets of the AD continuum for three tasks, including differentiating MCI from controls, predicting dementia conversion in MCI, and classification of amyloid positive or negative cognitively normal individuals. Based on brain FC trajectory, the proposed Brain TokenGT approach outperformed all the other benchmark models and at the same time provided excellent interpretability. The code is available at https://github.com/ZijianD/Brain-TokenGT.git

q-bio.NC↗

arXiv2023

An image segmentation algorithm based on multi-scale feature pyramid network

Medical image segmentation is particularly critical as a prerequisite for relevant quantitative analysis in the treatment of clinical diseases. For example, in clinical cervical cancer radiotherapy, after acquiring subabdominal MRI images, a fast and accurate image segmentation of organs and tumors in MRI images can optimize the clinical radiotherapy process, whereas traditional approaches use manual annotation by specialist doctors, which is time-consuming and laborious, therefore, automatic organ segmentation of subabdominal MRI images is a valuable research topic.

eess.IV↗

arXiv2023

Quantum bandit with amplitude amplification exploration in an adversarial environment

The rapid proliferation of learning systems in an arbitrarily changing environment mandates the need for managing tensions between exploration and exploitation. This work proposes a quantum-inspired bandit learning approach for the learning-and-adapting-based offloading problem where a client observes and learns the costs of each task offloaded to the candidate resource providers, e.g., fog nodes. In this approach, a new action update strategy and novel probabilistic action selection are adopted, provoked by the amplitude amplification and collapse postulate in quantum computation theory, respectively. We devise a locally linear mapping between a quantum-mechanical phase in a quantum domain, e.g., Grover-type search algorithm, and a distilled probability-magnitude in a value-based decision-making domain, e.g., adversarial multi-armed bandit algorithm. The proposed algorithm is generalized, via the devised mapping, for better learning weight adjustments on favourable/unfavourable actions and its effectiveness is verified via simulation.

quant-ph↗

arXiv2023

A repeated unknown game: Decentralized task offloading in vehicular fog computing

Offloading computation to nearby edge/fog computing nodes, including the ones carried by moving vehicles, e.g., vehicular fog nodes (VFN), has proved to be a promising approach for enabling low-latency and compute-intensive mobility applications, such as cooperative and autonomous driving. This work considers vehicular fog computing scenarios where the clients of computation offloading services try to minimize their own costs while deciding which VFNs to offload their tasks. We focus on decentralized multi-agent decision-making in a repeated unknown game where each agent, e.g., service client, can observe only its own action and realized cost. In other words, each agent is unaware of the game composition or even the existence of opponents. We apply a completely uncoupled learning rule to generalize the decentralized decision-making algorithm presented in \cite{Cho2021} for the multi-agent case. The multi-agent solution proposed in this work can capture the unknown offloading cost variations susceptive to resource congestion under an adversarial framework where each agent may take implicit cost estimation and suitable resource choice adapting to the dynamics associated with volatile supply and demand. According to the evaluation via simulation, this work reveals that such individual perturbations for robustness to uncertainty and adaptation to dynamicity ensure a certain level of optimality in terms of social welfare, e.g., converging the actual sequence of play with unknown and asymmetric attributes and lowering the correspondent cost in social welfare due to the self-interested behaviors of agents.

cs.MA↗

arXiv2022

Motley: Benchmarking Heterogeneity and Personalization in Federated Learning

Personalized federated learning considers learning models unique to each client in a heterogeneous network. The resulting client-specific models have been purported to improve metrics such as accuracy, fairness, and robustness in federated networks. However, despite a plethora of work in this area, it remains unclear: (1) which personalization techniques are most effective in various settings, and (2) how important personalization truly is for realistic federated applications. To better answer these questions, we propose Motley, a benchmark for personalized federated learning. Motley consists of a suite of cross-device and cross-silo federated datasets from varied problem domains, as well as thorough evaluation metrics for better understanding the possible impacts of personalization. We establish baselines on the benchmark by comparing a number of representative personalized federated learning methods. These initial results highlight strengths and weaknesses of existing approaches, and raise several open questions for the community. Motley aims to provide a reproducible means with which to advance developments in personalized and heterogeneity-aware federated learning, as well as the related areas of transfer learning, meta-learning, and multi-task learning.

cs.LG↗

arXiv2022

Anomalous transverse optic phonons in SnTe and PbTe -- revisited

We present a study of the soft transverse optic phonon mode in SnTe in comparison to the corresponding mode in PbTe using inelastic neutron scattering and ab-initio lattice dynamical calculations. In contrast to previous reports our calculations predict that the soft mode in SnTe features a strongly asymmetric spectral weight distribution qualitatively similar to that found in PbTe. Experimentally, we find that the overall width in energy of the phonon peaks is comparable in our neutron scattering spectra for SnTe and PbTe. We observe the well-known double-peak-like signature of the TO mode in PbTe even down to $T$ = 5 K questioning its proposed origin purely based on phonon-phonon scattering. The proximity to the incipient ferroelectric transition in PbTe likely plays an important role not included in current models.

cond-mat.mtrl-sci↗

arXiv2022

Automatic Map Update Using Dashcam Videos

Autonomous driving requires 3D maps that provide accurate and up-to-date information about semantic landmarks. Due to the wider availability and lower cost of cameras compared with laser scanners, vision-based mapping solutions, especially the ones using crowdsourced visual data, have attracted much attention from academia and industry. However, previous works have mainly focused on creating 3D point clouds, leaving automatic change detection as open issues. We propose in this paper a pipeline for initiating and updating 3D maps with dashcam videos, with a focus on automatic change detection based on comparison of metadata (e.g., the types and locations of traffic signs). To improve the performance of metadata generation, which depends on the accuracy of 3D object detection and localization, we introduce a novel deep learning-based pixel-wise 3D localization algorithm. The algorithm, trained directly with SfM point cloud data, can locate objects detected from 2D images in a 3D space with high accuracy by estimating not only depth from monocular images but also lateral and height distances. In addition, we also propose a point clustering and thresholding algorithm to improve the robustness of the system to errors. We have performed experiments on two distinct areas - a campus and a residential area - with different types of cameras, lighting, and weather conditions. The changes were detected with 85% and 100% accuracy in the campus and residential areas, respectively. The errors in the campus area were mainly due to traffic signs seen from a far distance to the vehicle and intended for pedestrians and cyclists only. We also conducted cause analysis of the detection and localization errors to measure the impact from the performance of the background technology in use.

cs.CV↗

arXiv2021

Learning-based decentralized offloading decision making in an adversarial environment

Vehicular fog computing (VFC) pushes the cloud computing capability to the distributed fog nodes at the edge of the Internet, enabling compute-intensive and latency-sensitive computing services for vehicles through task offloading. However, a heterogeneous mobility environment introduces uncertainties in terms of resource supply and demand, which are inevitable bottlenecks for the optimal offloading decision. Also, these uncertainties bring extra challenges to task offloading under the oblivious adversary attack and data privacy risks. In this article, we develop a new adversarial online learning algorithm with bandit feedback based on the adversarial multi-armed bandit theory, to enable scalable and low-complexity offloading decision making. Specifically, we focus on optimizing fog node selection with the aim of minimizing the offloading service costs in terms of delay and energy. The key is to implicitly tune the exploration bonus in the selection process and the assessment rules of the designed algorithm, taking into account volatile resource supply and demand. We theoretically prove that the input-size dependent selection rule allows to choose a suitable fog node without exploring the sub-optimal actions, and also an appropriate score patching rule allows to quickly adapt to evolving circumstances, which reduce variance and bias simultaneously, thereby achieving a better exploitation-exploration balance. Simulation results verify the effectiveness and robustness of the proposed algorithm.

cs.LG↗

arXiv2021

Non-uniform dependence on initial data for the generalized Camassa-Holm-Novikov equation in Besov space

Considered in this paper is the generalized Camassa-Holm-Novikov equation with high order nonlinearity, which unifies the Camassa-Holm and Novikov equations as special cases. We show that the solution map of generalized Camassa-Holm-Novikov equation is not uniformly continuous on the initial data in Besov spaces $B_{p, r}^s(\mathbb{R})$ with $s>\max\{1+\frac{1}{p}, \frac{3}{2}\}$, $1\leq p, r< \infty$ as well as in critical space $B_{2, 1}^{\frac{3}{2}}(\mathbb{R}).$ Our result covers and improves the previous work given by Li et al. \cite{Li 2020, 1Li 2020, Li 2021}(J. Differ. Equ. 269 (2020) 8686-8700; J. Math. Fluid Mech. 22 (2020) 4:50; J. Math. Fluid Mech., (2021) 23:36).

math.AP↗