arXiv subjects

Wenxuan Ma

Publications and source records attributed to Wenxuan Ma.

At least 19 recordsLinked to original sources

arXiv2026

HiTac-WAM: A Hierarchical Tactile World Action Model for Contact-Rich Robot Manipulation

World action models jointly predict future visual observations and actions, whereas existing tactile-aware variants typically represent future touch as an image or latent stream without modeling the physical dependencies that organize tactile states hierarchically. We present HiTac-WAM, a hierarchical tactile world action model that forecasts a sequence of future tactile states for each candidate action chunk before execution. The forecast factorizes into contact state, a 3D deformation field, and slip risk, organized as a directed hierarchy in which each downstream stage is conditioned on stop-gradient signals from preceding stages. A directed attention mask allows tactile queries to attend to the video-action context of each candidate while preventing video and action queries from attending to tactile tokens. For planning, HiTac-WAM ranks candidate action chunks using tactile forecasts and task-progress estimates. For execution, the selected tactile forecast is retained as a reference; persistent discrepancies between predicted and observed tactile states trigger corrective replanning. HiTac-WAM achieves a mean contact F1 of 0.921; under matched training budgets, the directed hierarchy reduces 3D displacement L2 error by 17.6% relative to the deformation-only predictor and improves slip AUPRC by 60.4% relative to the slip-only predictor. Across chip grasping, blackboard erasing, and USB insertion, selection guided by the hierarchical forecasts increases the average real-robot success rate from 31.1% to 61.1%, while the full system attains 72.2%.

cs.RO↗

arXiv2026

FeelWorld: Visuo-Tactile World Model for Hierarchical Contact Prediction and Planning

Humans plan physical interactions by imagining the possible outcomes of candidate actions. However, existing visual world models primarily capture appearance dynamics while overlooking the tactile states that govern contact-rich interactions, potentially producing imagined futures that appear visually plausible but violate physical dynamics. We introduce FeelWorld, a hierarchical visuo-tactile world model that jointly predicts future visual latents and three tactile states. FeelWorld organizes these states hierarchically as contact state, a 3D tactile latent that encodes force-related information, and slip state. These states are jointly predicted by a shared latent dynamics model with explicit supervision. To prevent irrelevant tactile signals during free-space motion from degrading visual prediction, we introduce a contact-gated asymmetric attention mechanism that maintains a visual-only prediction pathway before contact and enables joint visuo-tactile dynamics prediction during contact. The model is further trained with autoregressive rollouts and context noise injection to improve robustness to compounding errors. The predicted contact and slip states also support contact-aware CEM planning. Experiments on chip grasping, fruit grasping, and USB insertion show that FeelWorld reduces 10-step LPIPS from 0.084 to 0.058 and maintains an LPIPS that is 61% lower than that of the visual baseline after an 80-step autoregressive rollout. FeelWorld also achieves an average zero-shot planning success rate of 81.7%, providing an effective approach for incorporating tactile sensing into world models.

cs.RO↗

arXiv2026

FG-CLTP: Fine-Grained Contrastive Language Tactile Pretraining for Robotic Manipulation

Recent advancements in integrating tactile sensing into vision-language-action (VLA) models have demonstrated transformative potential for robotic perception. However, existing tactile representations predominantly rely on qualitative descriptors (e.g., texture), neglecting quantitative contact states such as force magnitude, contact geometry, and principal axis orientation, which are indispensable for fine-grained manipulation. To bridge this gap, we propose FG-CLTP, a fine-grained contrastive language tactile pretraining framework. We first introduce a novel dataset comprising over 100k tactile 3D point cloud-language pairs that explicitly capture multidimensional contact states from the sensor's perspective. We then implement a discretized numerical tokenization mechanism to achieve quantitative-semantic alignment, effectively injecting explicit physical metrics into the multimodal feature space. The proposed FG-CLTP model yields a 95.9% classification accuracy and reduces the regression error (MAE) by 52.6% compared to state-of-the-art methods. Furthermore, the integration of 3D point cloud representations establishes a sensor-agnostic foundation with a minimal sim-to-real gap of 3.5%. Building upon this fine-grained representation, we develop a 3D tactile-language-action (3D-TLA) architecture driven by a flow matching policy to enable multimodal reasoning and control. Extensive experiments demonstrate that our framework significantly outperforms strong baselines in contact-rich manipulation tasks, providing a robust and generalizable foundation for tactile-language-action models.

cs.RO↗

arXiv2026

SIM-assisted Secure Mobile Communications via Enhanced Proximal Policy Optimization Algorithm

With the development of sixth-generation (6G) wireless communication networks, the security challenges are becoming increasingly prominent, especially for mobile users (MUs). As a promising solution, physical layer security (PLS) technology leverages the inherent characteristics of wireless channels to provide security assurance. Particularly, stacked intelligent metasurface (SIM) directly manipulates electromagnetic waves through their multilayer structures, offering significant potential for enhancing PLS performance in an energy efficient manner. Thus, in this work, we investigate an SIM-assisted secure communication system for MUs under the threat of an eavesdropper, addressing practical challenges such as channel uncertainty in mobile environments, multiple MU interference, and residual hardware impairments. Consequently, we formulate a joint power and phase shift optimization problem (JPPSOP), aiming at maximizing the achievable secrecy rate (ASR) of all MUs. Given the non-convexity and dynamic nature of this optimization problem, we propose an enhanced proximal policy optimization algorithm with a bidirectional long short-term memory mechanism, an off-policy data utilization mechanism, and a policy feedback mechanism (PPO-BOP). Through these mechanisms, the proposed algorithm can effectively capture short-term channel fading and long-term MU mobility, improve sample utilization efficiency, and enhance exploration capabilities. Extensive simulation results demonstrate that PPO-BOP significantly outperforms benchmark strategies and other deep reinforcement learning algorithms in terms of ASR.10.1109/TWC.2026.3658332

eess.SP↗

arXiv2025

Parsimonious Generative Machine Learning for Non-Gaussian Tail Modeling

The presence of non-Gaussian tails is a prevalent characteristic in many financial modeling scenarios, necessitating the use of complex non-Gaussian distributions such as the generalized beta of the second kind (GB2) and the skewed generalized $t$ (SGT). The approach we propose for modeling heavy-tailed data differs significantly from traditional methods. We utilize generative machine learning, which offers an entirely different paradigm for modeling distributions. A parsimonious nonlinear transformation is applied to a simple base random variable such as Gaussian. The parameters can be estimated effectively, and the theoretical heavy-tail properties are derived. Robust performance is observed with this approach when compared to traditional distributions. More importantly, this method is broadly useful for machine learning due to its mathematical elegance and numerical convenience.

stat.AP↗

arXiv2025

Generalized Singular Value Decompositions of Dual Quaternion Matrix Triplets

In signal processing and identification, generalized singular value decomposition (GSVD), related to a sequence of matrices in product/quotient form are essential numerical linear algebra tools. On behalf of the growing demand for efficient processing of coupled rotation-translation signals in modern engineering, we introduce the restricted SVD of a dual quaternion matrix triplet $(\boldsymbol{A},\boldsymbol{B},\boldsymbol{C})$ with $\boldsymbol{A}\in {\bf \mathbb{DQ}}^{m \times n}$, $\boldsymbol{B} \in {\bf \mathbb{DQ}}^{m \times p}$, $\boldsymbol{C} \in {\bf \mathbb{DQ}}^{q\times n}$, and the product-product SVD of a dual quaternion matrix triplet $(\boldsymbol{A},\boldsymbol{B},\boldsymbol{C})$ with $\boldsymbol{A}\in {\bf \mathbb{DQ}}^{m \times n}$, $\boldsymbol{B} \in {\bf \mathbb{DQ}}^{n \times p}$, $\boldsymbol{C} \in {\bf \mathbb{DQ}}^{p\times q}$. The two types of GSVDs represent a sophisticated matrix factorization that accounts for a given dual quaternion matrix in conjunction with two additional dual quaternion matrices. The decompositions can be conceptualized as an adaptation of the standard SVD, where the distinctive feature lies in the application of distinct inner products to the row and column spaces. Two examples are outlined to illustrate the feasibility of the decompositions.

math.NA↗

arXiv2025

Generalized singular value decompositions of dual quaternion matrices and beyond

In high-dimensional data processing and data analysis related to dual quaternion statistics, generalized singular value decomposition (GSVD) of a dual quaternion matrix pair is an essential numerical linear algebra tool for an elegant problem formulation and numerical implementation. In this paper, building upon the existing singular value decomposition (SVD) of a dual quaternion matrix, we put forward several types of GSVD of dual quaternion data matrices in accordance with their dimensions. Explicitly, for a given dual quaternion matrix pair $\{{\boldsymbol A}, {\boldsymbol B}\}$, if ${\boldsymbol A}$ and ${\boldsymbol B}$ have the same number of columns, we investigate two forms of their quotient-type SVD (DQGSVD) through different strategies, which can be selected to use in different scenarios. Three artificial examples are presented to illustrate the principle of the DQGSVD. Alternatively, if ${\boldsymbol A}$ and ${\boldsymbol B}$ have the same number of rows, we consider their canonical correlation decomposition (DQCCD). If ${\boldsymbol A}$ and ${\boldsymbol B}$ are consistent for dual quaternion matrix multiplication, we present their product-type SVD (DQPSVD). As a preparation, we also study the QR decomposition of a dual quaternion matrix based on the dual quaternion Householder transformation, and introduce the CS decomposition of an 2-by-2 blocked unitary dual quaternion matrix. Due to the peculiarity of containing dual part for dual quaternion matrices, the obtained series of GSVD of dual quaternion matrices dramatically distinguish from those in the real number field, the complex number field, and even the quaternion ring, but can be treated as an extension of them to some extent.

math.NA↗

arXiv2025

Emu3.5: Native Multimodal Models are World Learners

We introduce Emu3.5, a large-scale multimodal world model that natively predicts the next state across vision and language. Emu3.5 is pre-trained end-to-end with a unified next-token prediction objective on a corpus of vision-language interleaved data containing over 10 trillion tokens, primarily derived from sequential frames and transcripts of internet videos. The model naturally accepts interleaved vision-language inputs and generates interleaved vision-language outputs. Emu3.5 is further post-trained with large-scale reinforcement learning to enhance multimodal reasoning and generation. To improve inference efficiency, we propose Discrete Diffusion Adaptation (DiDA), which converts token-by-token decoding into bidirectional parallel prediction, accelerating per-image inference by about 20x without sacrificing performance. Emu3.5 exhibits strong native multimodal capabilities, including long-horizon vision-language generation, any-to-image (X2I) generation, and complex text-rich image generation. It also exhibits generalizable world-modeling abilities, enabling spatiotemporally consistent world exploration and open-world embodied manipulation across diverse scenarios and tasks. For comparison, Emu3.5 achieves performance comparable to Gemini 2.5 Flash Image (Nano Banana) on image generation and editing tasks and demonstrates superior results on a suite of interleaved generation tasks. We open-source Emu3.5 at https://github.com/baaivision/Emu3.5 to support community research.

cs.CV↗

arXiv2025

From Local Details to Global Context: Advancing Vision-Language Models with Attention-Based Selection

Pretrained vision-language models (VLMs), e.g., CLIP, demonstrate impressive zero-shot capabilities on downstream tasks. Prior research highlights the crucial role of visual augmentation techniques, like random cropping, in alignment with fine-grained class descriptions generated by large language models (LLMs), significantly enhancing zero-shot performance by incorporating multi-view information. However, the inherent randomness of these augmentations can inevitably introduce background artifacts and cause models to overly focus on local details, compromising global semantic understanding. To address these issues, we propose an \textbf{A}ttention-\textbf{B}ased \textbf{S}election (\textbf{ABS}) method from local details to global context, which applies attention-guided cropping in both raw images and feature space, supplement global semantic information through strategic feature selection. Additionally, we introduce a soft matching technique to effectively filter LLM descriptions for better alignment. \textbf{ABS} achieves state-of-the-art performance on out-of-distribution generalization and zero-shot classification tasks. Notably, \textbf{ABS} is training-free and even rivals few-shot and test-time adaptation methods. Our code is available at \href{https://github.com/BIT-DA/ABS}{\textcolor{darkgreen}{https://github.com/BIT-DA/ABS}}.

cs.CV↗

arXiv2025

CLTP: Contrastive Language-Tactile Pre-training for 3D Contact Geometry Understanding

Recent advancements in integrating tactile sensing with vision-language models (VLMs) have demonstrated remarkable potential for robotic multimodal perception. However, existing tactile descriptions remain limited to superficial attributes like texture, neglecting critical contact states essential for robotic manipulation. To bridge this gap, we propose CLTP, an intuitive and effective language tactile pretraining framework that aligns tactile 3D point clouds with natural language in various contact scenarios, thus enabling contact-state-aware tactile language understanding for contact-rich manipulation tasks. We first collect a novel dataset of 50k+ tactile 3D point cloud-language pairs, where descriptions explicitly capture multidimensional contact states (e.g., contact location, shape, and force) from the tactile sensor's perspective. CLTP leverages a pre-aligned and frozen vision-language feature space to bridge holistic textual and tactile modalities. Experiments validate its superiority in three downstream tasks: zero-shot 3D classification, contact state classification, and tactile 3D large language model (LLM) interaction. To the best of our knowledge, this is the first study to align tactile and language representations from the contact state perspective for manipulation tasks, providing great potential for tactile-language-action model learning. Code and datasets are open-sourced at https://sites.google.com/view/cltp/.

cs.RO↗

arXiv2025

Robust Decentralized Quantum Kernel Learning for Noisy and Adversarial Environment

This paper proposes a general decentralized framework for quantum kernel learning (QKL). It has robustness against quantum noise and can also be designed to defend adversarial information attacks forming a robust approach named RDQKL. We analyze the impact of noise on QKL and study the robustness of decentralized QKL to the noise. By integrating robust decentralized optimization techniques, our method is able to mitigate the impact of malicious data injections across multiple nodes. Experimental results demonstrate that our approach maintains high accuracy under noisy quantum operations and effectively counter adversarial modifications, offering a promising pathway towards the future practical, scalable and secure quantum machine learning (QML).

quant-ph↗

arXiv2024

Consensus-based Distributed Quantum Kernel Learning for Speech Recognition

This paper presents a Consensus-based Distributed Quantum Kernel Learning (CDQKL) framework aimed at improving speech recognition through distributed quantum computing.CDQKL addresses the challenges of scalability and data privacy in centralized quantum kernel learning. It does this by distributing computational tasks across quantum terminals, which are connected through classical channels. This approach enables the exchange of model parameters without sharing local training data, thereby maintaining data privacy and enhancing computational efficiency. Experimental evaluations on benchmark speech emotion recognition datasets demonstrate that CDQKL achieves competitive classification accuracy and scalability compared to centralized and local quantum kernel learning models. The distributed nature of CDQKL offers advantages in privacy preservation and computational efficiency, making it suitable for data-sensitive fields such as telecommunications, automotive, and finance. The findings suggest that CDQKL can effectively leverage distributed quantum computing for large-scale machine-learning tasks.

quant-ph↗

arXiv2024

Learning Modality Knowledge Alignment for Cross-Modality Transfer

Cross-modality transfer aims to leverage large pretrained models to complete tasks that may not belong to the modality of pretraining data. Existing works achieve certain success in extending classical finetuning to cross-modal scenarios, yet we still lack understanding about the influence of modality gap on the transfer. In this work, a series of experiments focusing on the source representation quality during transfer are conducted, revealing the connection between larger modality gap and lesser knowledge reuse which means ineffective transfer. We then formalize the gap as the knowledge misalignment between modalities using conditional distribution P(Y|X). Towards this problem, we present Modality kNowledge Alignment (MoNA), a meta-learning approach that learns target data transformation to reduce the modality knowledge discrepancy ahead of the transfer. Experiments show that out method enables better reuse of source modality knowledge in cross-modality transfer, which leads to improvements upon existing finetuning methods.

cs.CV↗

arXiv2024

Enhancing Cross-Modal Fine-Tuning with Gradually Intermediate Modality Generation

Large-scale pretrained models have proven immensely valuable in handling data-intensive modalities like text and image. However, fine-tuning these models for certain specialized modalities, such as protein sequence and cosmic ray, poses challenges due to the significant modality discrepancy and scarcity of labeled data. In this paper, we propose an end-to-end method, PaRe, to enhance cross-modal fine-tuning, aiming to transfer a large-scale pretrained model to various target modalities. PaRe employs a gating mechanism to select key patches from both source and target data. Through a modality-agnostic Patch Replacement scheme, these patches are preserved and combined to construct data-rich intermediate modalities ranging from easy to hard. By gradually intermediate modality generation, we can not only effectively bridge the modality gap to enhance stability and transferability of cross-modal fine-tuning, but also address the challenge of limited data in the target modality by leveraging enriched intermediate modality data. Compared with hand-designed, general-purpose, task-specific, and state-of-the-art cross-modal fine-tuning approaches, PaRe demonstrates superior performance across three challenging benchmarks, encompassing more than ten modalities.

cs.CV↗

arXiv2023

Language Semantic Graph Guided Data-Efficient Learning

Developing generalizable models that can effectively learn from limited data and with minimal reliance on human supervision is a significant objective within the machine learning community, particularly in the era of deep neural networks. Therefore, to achieve data-efficient learning, researchers typically explore approaches that can leverage more related or unlabeled data without necessitating additional manual labeling efforts, such as Semi-Supervised Learning (SSL), Transfer Learning (TL), and Data Augmentation (DA). SSL leverages unlabeled data in the training process, while TL enables the transfer of expertise from related data distributions. DA broadens the dataset by synthesizing new data from existing examples. However, the significance of additional knowledge contained within labels has been largely overlooked in research. In this paper, we propose a novel perspective on data efficiency that involves exploiting the semantic information contained in the labels of the available data. Specifically, we introduce a Language Semantic Graph (LSG) which is constructed from labels manifest as natural language descriptions. Upon this graph, an auxiliary graph neural network is trained to extract high-level semantic relations and then used to guide the training of the primary model, enabling more adequate utilization of label knowledge. Across image, video, and audio modalities, we utilize the LSG method in both TL and SSL scenarios and illustrate its versatility in significantly enhancing performance compared to other data-efficient learning approaches. Additionally, our in-depth analysis shows that the LSG method also expedites the training process.

cs.CV↗

arXiv2023

Improving Uncertainty Quantification of Variance Networks by Tree-Structured Learning

To improve the uncertainty quantification of variance networks, we propose a novel tree-structured local neural network model that partitions the feature space into multiple regions based on uncertainty heterogeneity. A tree is built upon giving the training data, whose leaf nodes represent different regions where region-specific neural networks are trained to predict both the mean and the variance for quantifying uncertainty. The proposed Uncertainty-Splitting Neural Regression Tree (USNRT) employs novel splitting criteria. At each node, a neural network is trained on the full data first, and a statistical test for the residuals is conducted to find the best split, corresponding to the two sub-regions with the most significant uncertainty heterogeneity between them. USNRT is computationally friendly because very few leaf nodes are sufficient and pruning is unnecessary. Furthermore, an ensemble version can be easily constructed to estimate the total uncertainty including the aleatory and epistemic. On extensive UCI datasets, USNRT or its ensemble shows superior performance compared to some recent popular methods for quantifying uncertainty with variances. Through comprehensive visualization and analysis, we uncover how USNRT works and show its merits, revealing that uncertainty heterogeneity does exist in many datasets and can be learned by USNRT.

cs.LG↗

arXiv2023

Ensemble Multi-Quantiles: Adaptively Flexible Distribution Prediction for Uncertainty Quantification

We propose a novel, succinct, and effective approach for distribution prediction to quantify uncertainty in machine learning. It incorporates adaptively flexible distribution prediction of $\mathbb{P}(\mathbf{y}|\mathbf{X}=x)$ in regression tasks. This conditional distribution's quantiles of probability levels spreading the interval $(0,1)$ are boosted by additive models which are designed by us with intuitions and interpretability. We seek an adaptive balance between the structural integrity and the flexibility for $\mathbb{P}(\mathbf{y}|\mathbf{X}=x)$, while Gaussian assumption results in a lack of flexibility for real data and highly flexible approaches (e.g., estimating the quantiles separately without a distribution structure) inevitably have drawbacks and may not lead to good generalization. This ensemble multi-quantiles approach called EMQ proposed by us is totally data-driven, and can gradually depart from Gaussian and discover the optimal conditional distribution in the boosting. On extensive regression tasks from UCI datasets, we show that EMQ achieves state-of-the-art performance comparing to many recent uncertainty quantification methods. Visualization results further illustrate the necessity and the merits of such an ensemble model.

cs.LG↗

arXiv2022

Making the Best of Both Worlds: A Domain-Oriented Transformer for Unsupervised Domain Adaptation

Extensive studies on Unsupervised Domain Adaptation (UDA) have propelled the deployment of deep learning from limited experimental datasets into real-world unconstrained domains. Most UDA approaches align features within a common embedding space and apply a shared classifier for target prediction. However, since a perfectly aligned feature space may not exist when the domain discrepancy is large, these methods suffer from two limitations. First, the coercive domain alignment deteriorates target domain discriminability due to lacking target label supervision. Second, the source-supervised classifier is inevitably biased to source data, thus it may underperform in target domain. To alleviate these issues, we propose to simultaneously conduct feature alignment in two individual spaces focusing on different domains, and create for each space a domain-oriented classifier tailored specifically for that domain. Specifically, we design a Domain-Oriented Transformer (DOT) that has two individual classification tokens to learn different domain-oriented representations, and two classifiers to preserve domain-wise discriminability. Theoretical guaranteed contrastive-based alignment and the source-guided pseudo-label refinement strategy are utilized to explore both domain-invariant and specific information. Comprehensive experiments validate that our method achieves state-of-the-art on several benchmarks.

cs.CV↗