arXiv subjects

Yixiao Wang

Publications and source records attributed to Yixiao Wang.

At least 55 records · Page 3Linked to original sources

arXiv2025

MobA: Multifaceted Memory-Enhanced Adaptive Planning for Efficient Mobile Task Automation

Existing Multimodal Large Language Model (MLLM)-based agents face significant challenges in handling complex GUI (Graphical User Interface) interactions on devices. These challenges arise from the dynamic and structured nature of GUI environments, which integrate text, images, and spatial relationships, as well as the variability in action spaces across different pages and tasks. To address these limitations, we propose MobA, a novel MLLM-based mobile assistant system. MobA introduces an adaptive planning module that incorporates a reflection mechanism for error recovery and dynamically adjusts plans to align with the real environment contexts and action module's execution capacity. Additionally, a multifaceted memory module provides comprehensive memory support to enhance adaptability and efficiency. We also present MobBench, a dataset designed for complex mobile interactions. Experimental results on MobBench and AndroidArena demonstrate MobA's ability to handle dynamic GUI environments and perform complex mobile tasks.

cs.MA↗

arXiv2025

Generalization Capability for Imitation Learning

Imitation learning holds the promise of equipping robots with versatile skills by learning from expert demonstrations. However, policies trained on finite datasets often struggle to generalize beyond the training distribution. In this work, we present a unified perspective on the generalization capability of imitation learning, grounded in both information theorey and data distribution property. We first show that the generalization gap can be upper bounded by (i) the conditional information bottleneck on intermediate representations and (ii) the mutual information between the model parameters and the training dataset. This characterization provides theoretical guidance for designing effective training strategies in imitation learning, particularly in determining whether to freeze, fine-tune, or train large pretrained encoders (e.g., vision-language models or vision foundation models) from scratch to achieve better generalization. Furthermore, we demonstrate that high conditional entropy from input to output induces a flatter likelihood landscape, thereby reducing the upper bound on the generalization gap. In addition, it shortens the stochastic gradient descent (SGD) escape time from sharp local minima, which may increase the likelihood of reaching global optima under fixed optimization budgets. These insights explain why imitation learning often exhibits limited generalization and underscore the importance of not only scaling the diversity of input data but also enriching the variability of output labels conditioned on the same input.

cs.LG↗

arXiv2025

Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Management

Multiple Low-Rank Adapters (Multi-LoRAs) are gaining popularity for task-specific Large Language Model (LLM) applications. For multi-LoRA serving, caching hot KV caches and LoRA adapters in high bandwidth memory of accelerations can improve inference performance. However, existing Multi-LoRA inference systems fail to optimize serving performance like Time-To-First-Toke (TTFT), neglecting usage dependencies when caching LoRAs and KVs. We therefore propose FASTLIBRA, a Multi-LoRA caching system to optimize the serving performance. FASTLIBRA comprises a dependency-aware cache manager and a performance-driven cache swapper. The cache manager maintains the usage dependencies between LoRAs and KV caches during the inference with a unified caching pool. The cache swapper determines the swap-in or out of LoRAs and KV caches based on a unified cost model, when the HBM is idle or busy, respectively. Experimental results show that ELORA reduces the TTFT by 63.4% on average, compared to state-of-the-art works.

cs.AR↗

arXiv2025

UoB-NLP at SemEval-2025 Task 11: Leveraging Adapters for Multilingual and Cross-Lingual Emotion Detection

Emotion detection in natural language processing is a challenging task due to the complexity of human emotions and linguistic diversity. While significant progress has been made in high-resource languages, emotion detection in low-resource languages remains underexplored. In this work, we address multilingual and cross-lingual emotion detection by leveraging adapter-based fine-tuning with multilingual pre-trained language models. Adapters introduce a small number of trainable parameters while keeping the pre-trained model weights fixed, offering a parameter-efficient approach to adaptation. We experiment with different adapter tuning strategies, including task-only adapters, target-language-ready task adapters, and language-family-based adapters. Our results show that target-language-ready task adapters achieve the best overall performance, particularly for low-resource African languages with our team ranking 7th for Tigrinya, and 8th for Kinyarwanda in Track A. In Track C, our system ranked 3rd for Amharic, and 4th for Oromo, Tigrinya, Kinyarwanda, Hausa, and Igbo. Our approach outperforms large language models in 11 languages and matches their performance in four others, despite our models having significantly fewer parameters. Furthermore, we find that adapter-based models retain cross-linguistic transfer capabilities while requiring fewer computational resources compared to full fine-tuning for each language.

cs.CL↗

arXiv2025

Adaptive Energy Regularization for Autonomous Gait Transition and Energy-Efficient Quadruped Locomotion

In reinforcement learning for legged robot locomotion, crafting effective reward strategies is crucial. Pre-defined gait patterns and complex reward systems are widely used to stabilize policy training. Drawing from the natural locomotion behaviors of humans and animals, which adapt their gaits to minimize energy consumption, we propose a simplified, energy-centric reward strategy to foster the development of energy-efficient locomotion across various speeds in quadruped robots. By implementing an adaptive energy reward function and adjusting the weights based on velocity, we demonstrate that our approach enables ANYmal-C and Unitree Go1 robots to autonomously select appropriate gaits, such as four-beat walking at lower speeds and trotting at higher speeds, resulting in improved energy efficiency and stable velocity tracking compared to previous methods using complex reward designs and prior gait knowledge. The effectiveness of our policy is validated through simulations in the IsaacGym simulation environment and on real robots, demonstrating its potential to facilitate stable and adaptive locomotion.

cs.RO↗

arXiv2025

Physics-Aware Robotic Palletization with Online Masking Inference

The efficient planning of stacking boxes, especially in the online setting where the sequence of item arrivals is unpredictable, remains a critical challenge in modern warehouse and logistics management. Existing solutions often address box size variations, but overlook their intrinsic and physical properties, such as density and rigidity, which are crucial for real-world applications. We use reinforcement learning (RL) to solve this problem by employing action space masking to direct the RL policy toward valid actions. Unlike previous methods that rely on heuristic stability assessments which are difficult to assess in physical scenarios, our framework utilizes online learning to dynamically train the action space mask, eliminating the need for manual heuristic design. Extensive experiments demonstrate that our proposed method outperforms existing state-of-the-arts. Furthermore, we deploy our learned task planner in a real-world robotic palletizer, validating its practical applicability in operational settings.

cs.RO↗

arXiv2025

Trimmed Mean for Partially Observed Functional Data

In recent years, partially observable functional data has gained significant attention in practical applications and has become the focus of increasing interest in the literature. In this thesis, we build upon the concept of data integration depth for partially observable functions, as proposed by Elias et al. (2023), and the trimmed-mean estimator method along with its consistency proof introduced by Fraiman and Muniz (2001) for completely observable functions. We introduce the concept of trimmed mean specifically for partially observable functional data. Additionally, we address several theoretical and practical issues, including a proof of the strong consistency of the proposed trimmed mean, and we provide a simulation study. The results demonstrate that our estimator outperforms the ordinary mean in terms of accuracy and robustness when applied to partially observable functional data.

math.ST↗

arXiv2024

Imagined Potential Games: A Framework for Simulating, Learning and Evaluating Interactive Behaviors

Interacting with human agents in complex scenarios presents a significant challenge for robotic navigation, particularly in environments that necessitate both collision avoidance and collaborative interaction, such as indoor spaces. Unlike static or predictably moving obstacles, human behavior is inherently complex and unpredictable, stemming from dynamic interactions with other agents. Existing simulation tools frequently fail to adequately model such reactive and collaborative behaviors, impeding the development and evaluation of robust social navigation strategies. This paper introduces a novel framework utilizing distributed potential games to simulate human-like interactions in highly interactive scenarios. Within this framework, each agent imagines a virtual cooperative game with others based on its estimation. We demonstrate this formulation can facilitate the generation of diverse and realistic interaction patterns in a configurable manner across various scenarios. Additionally, we have developed a gym-like environment leveraging our interactive agent model to facilitate the learning and evaluation of interactive navigation algorithms.

cs.RO↗

arXiv2024

Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learning

The increasing complexity of tasks in robotics demands efficient strategies for multitask and continual learning. Traditional models typically rely on a universal policy for all tasks, facing challenges such as high computational costs and catastrophic forgetting when learning new tasks. To address these issues, we introduce a sparse, reusable, and flexible policy, Sparse Diffusion Policy (SDP). By adopting Mixture of Experts (MoE) within a transformer-based diffusion policy, SDP selectively activates experts and skills, enabling efficient and task-specific learning without retraining the entire model. SDP not only reduces the burden of active parameters but also facilitates the seamless integration and reuse of experts across various tasks. Extensive experiments on diverse tasks in both simulations and real world show that SDP 1) excels in multitask scenarios with negligible increases in active parameters, 2) prevents forgetting in continual learning of new tasks, and 3) enables efficient task transfer, offering a promising solution for advanced robotic applications. Demos and codes can be found in https://forrest-110.github.io/sparse_diffusion_policy/.

cs.RO↗

arXiv2024

Joint Pedestrian Trajectory Prediction through Posterior Sampling

Joint pedestrian trajectory prediction has long grappled with the inherent unpredictability of human behaviors. Recent investigations employing variants of conditional diffusion models in trajectory prediction have exhibited notable success. Nevertheless, the heavy dependence on accurate historical data results in their vulnerability to noise disturbances and data incompleteness. To improve the robustness and reliability, we introduce the Guided Full Trajectory Diffuser (GFTD), a novel diffusion model framework that captures the joint full (historical and future) trajectory distribution. By learning from the full trajectory, GFTD can recover the noisy and missing data, hence improving the robustness. In addition, GFTD can adapt to data imperfections without additional training requirements, leveraging posterior sampling for reliable prediction and controllable generation. Our approach not only simplifies the prediction process but also enhances generalizability in scenarios with noise and incomplete inputs. Through rigorous experimental evaluation, GFTD exhibits superior performance in both trajectory prediction and controllable generation.

cs.RO↗

arXiv2024

Optimizing Diffusion Models for Joint Trajectory Prediction and Controllable Generation

Diffusion models are promising for joint trajectory prediction and controllable generation in autonomous driving, but they face challenges of inefficient inference steps and high computational demands. To tackle these challenges, we introduce Optimal Gaussian Diffusion (OGD) and Estimated Clean Manifold (ECM) Guidance. OGD optimizes the prior distribution for a small diffusion time $T$ and starts the reverse diffusion process from it. ECM directly injects guidance gradients to the estimated clean manifold, eliminating extensive gradient backpropagation throughout the network. Our methodology streamlines the generative process, enabling practical applications with reduced computational overhead. Experimental validation on the large-scale Argoverse 2 dataset demonstrates our approach's superior performance, offering a viable solution for computationally efficient, high-quality joint trajectory prediction and controllable generation for autonomous driving. Our project webpage is at https://yixiaowang7.github.io/OptTrajDiff_Page/.

cs.CV↗

arXiv2024

A WT-ResNet based fault diagnosis model for the urban rail train transmission system

This study presents a novel fault diagnosis model for urban rail transit systems based on Wavelet Transform Residual Neural Network (WT-ResNet). The model integrates the advantages of wavelet transform for feature extraction and ResNet for pattern recognition, offering enhanced diagnostic accuracy and robustness. Experimental results demonstrate the effectiveness of the proposed model in identifying faults in urban rail trains, paving the way for improved maintenance strategies and reduced downtime.

cs.IR↗

arXiv2024

Composition Vision-Language Understanding via Segment and Depth Anything Model

We introduce a pioneering unified library that leverages depth anything, segment anything models to augment neural comprehension in language-vision model zero-shot understanding. This library synergizes the capabilities of the Depth Anything Model (DAM), Segment Anything Model (SAM), and GPT-4V, enhancing multimodal tasks such as vision-question-answering (VQA) and composition reasoning. Through the fusion of segmentation and depth analysis at the symbolic instance level, our library provides nuanced inputs for language models, significantly advancing image interpretation. Validated across a spectrum of in-the-wild real-world images, our findings showcase progress in vision-language models through neural-symbolic integration. This novel approach melds visual and language analysis in an unprecedented manner. Overall, our library opens new directions for future research aimed at decoding the complexities of the real world through advanced multimodal technologies and our code is available at \url{https://github.com/AnthonyHuo/SAM-DAM-for-Compositional-Reasoning}.

cs.CV↗

arXiv2024

Comparison of the high-order Runge-Kutta discontinuous Galerkin method and gas-kinetic scheme for inviscid compressible flow simulations

The Runge--Kutta discontinuous Galerkin (RKDG) method is a high-order technique for addressing hyperbolic conservation laws, which has been refined over recent decades and is effective in handling shock discontinuities. Despite its advancements, the RKDG method faces challenges, such as stringent constraints on the explicit time-step size and reduced robustness when dealing with strong discontinuities. On the other hand, the Gas-Kinetic Scheme (GKS) based on a high-order gas evolution model also delivers significant accuracy and stability in solving hyperbolic conservation laws through refined spatial and temporal discretizations. Unlike RKDG, GKS allows for more flexible CFL number constraints and features an advanced flow evolution mechanism at cell interfaces. Additionally, GKS' compact spatial reconstruction enhances the accuracy of the method and its ability to capture stable strong discontinuities effectively. In this study, we conduct a thorough examination of the RKDG method using various numerical fluxes and the GKS method employing both compact and non-compact spatial reconstructions. Both methods are applied under the framework of explicit time discretization and are tested solely in inviscid scenarios. We will present numerous numerical tests and provide a comparative analysis of the outcomes derived from these two computational approaches.

math.NA↗

arXiv2023

Unsupervised CNN-Based DIC for 2D Displacement Measurement

Digital image correlation method is a non contact deformation measurement technique. Despite years of development, it is still difficult to solve the contradiction between calculation efficiency and seed point quantity.With the development of deep learning, the DIC algorithm based on deep learning provides a new solution for the problem of insufficient calculation efficiency in DIC.All supervised learning DIC methods requires a large set of high quality training set. However, obtaining such a dataset can be challenging and time consuming in generating ground truth. To fix the problem,we propose an unsupervised CNN Based DIC for 2D Displacement Measurement.The speckle image warp model is created to transform the target speckle image to the corresponding predicted reference speckle image by predicted 2D displacement map, the predicted reference speckle image is compared with the original reference speckle image to realize the unsupervised training of the CNN.The network's parameters are optimized using a composite loss function that incorporates both the Mean Squared Error and Pearson correlation coefficient.Our proposed method has a significant advantage of eliminating the need for extensive training data annotations. We conducted several experiments to demonstrate the validity and robustness of the proposed method. The experimental results demonstrate that our method can achieve can achieve accuracy comparable to previous supervised methods. The PyTorch code will be available at the following URL: https://github.com/fead1.

physics.ins-det↗

arXiv2023

UHRNet: A Deep Learning-Based Method for Accurate 3D Reconstruction from a Single Fringe-Pattern

The quick and accurate retrieval of an object height from a single fringe pattern in Fringe Projection Profilometry has been a topic of ongoing research. While a single shot fringe to depth CNN based method can restore height map directly from a single pattern, its accuracy is currently inferior to the traditional phase shifting technique. To improve this method's accuracy, we propose using a U shaped High resolution Network (UHRNet). The network uses UNet encoding and decoding structure as backbone, with Multi-Level convolution Block and High resolution Fusion Block applied to extract local features and global features. We also designed a compound loss function by combining Structural Similarity Index Measure Loss (SSIMLoss) function and chunked L2 loss function to improve 3D reconstruction details.We conducted several experiments to demonstrate the validity and robustness of our proposed method. A few experiments have been conducted to demonstrate the validity and robustness of the proposed method, The average RMSE of 3D reconstruction by our method is only 0.443(mm). which is 41.13% of the UNet method and 33.31% of Wang et al hNet method. Our experimental results show that our proposed method can increase the accuracy of 3D reconstruction from a single fringe pattern.

cs.CV↗

arXiv2023

Topological Floquet engineering using two frequencies in two dimensions

Using two-frequency driving in two dimensions opens up new possibilites for Floquet engineering, which range from controlling specific symmetries to tuning the properties of resonant gaps. In this work, we study two-band lattice models subject to two-tone Floquet driving and analyse the resulting effective Floquet bandstructures both numerically and analytically. On the one hand, we extend the methodology of Sandholzer et al. [10.1103/PhysRevResearch.4.013056] from one to two dimensions and find competing topological phases in a simple Bravais lattice when the two resonant drives at $1ω$ and $2ω$ interfere. On the other hand, we explore driving-induced symmetry breaking in the hexagonal lattice, in which the breaking of either inversion or time-reversal symmetry can be tuned independently via the Floquet modulation. Possible applications of our work include a simpler generation of topological bands for ultracold atoms, and the realisation of non-linear Hall effects as well as Haldane's parity anomaly in inversion-symmetric parent lattices.

cond-mat.quant-gas↗

arXiv2022

Absolute phase measurement method based on bidirectional coding patterns

The stair phase coding patterns have been widely used to determine the fringe order for phase unwrapping of the wrapped phase in 3D shape measurement. Although the special coding sequence algorithm can achieve with a large number of codewords, it needs current codeword and its adjacent codewords to jointly determine the fringe order. If any codeword of the grouped adjacent codewords is incorrectly recognized, it will result in false fringe order.Therefore, it is challenging to significantly increase the number of codewords. When it is necessary to simultaneously measure more than two isolated objects with large size differences, if the fringe frequency is high, the number of fringes on the smaller objects is too few to determine the fringe orders. On the other hand, if the fringe frequency is low, the image can not contain all isolated objects at the same time. To solve this problem, we propose an absolute phase measurement method based on bidirectional coding patterns. The wrapped phase of the object is obtained by fou step phase shifting patterns, and the fringe orders is obtained by two bidirectional coded patterns. When coding the bidirectional patterns, we code two groups of stair phase with different frequency along horizontal direction, which respectively represent local fringe order information and partition information, then, we alternately repeated the two groups of stair phase along the vertical direction to obtain the bidirectional coding patterns in the whole pattern. Each local fringe order information and the corresponding partition information in small region jointly determine the fringe order of each position in the wrapped phase. To verify the effectiveness of our method, we did simulations and three experiments. Simulation and experimental results show that our method is effective for complex surfaces and isolated objects with different sizes.

physics.optics↗