arXiv subjects

Chao Wang

Publications and source records attributed to Chao Wang.

At least 253 records · Page 14Linked to original sources

arXiv2025

GOAT-SLM: A Spoken Language Model with Paralinguistic and Speaker Characteristic Awareness

Recent advances in end-to-end spoken language models (SLMs) have significantly improved the ability of AI systems to engage in natural spoken interactions. However, most existing models treat speech merely as a vehicle for linguistic content, often overlooking the rich paralinguistic and speaker characteristic cues embedded in human speech, such as dialect, age, emotion, and non-speech vocalizations. In this work, we introduce GOAT-SLM, a novel spoken language model with paralinguistic and speaker characteristic awareness, designed to extend spoken language modeling beyond text semantics. GOAT-SLM adopts a dual-modality head architecture that decouples linguistic modeling from acoustic realization, enabling robust language understanding while supporting expressive and adaptive speech generation. To enhance model efficiency and versatility, we propose a modular, staged training strategy that progressively aligns linguistic, paralinguistic, and speaker characteristic information using large-scale speech-text corpora. Experimental results on TELEVAL, a multi-dimensional evaluation benchmark, demonstrate that GOAT-SLM achieves well-balanced performance across both semantic and non-semantic tasks, and outperforms existing open-source models in handling emotion, dialectal variation, and age-sensitive interactions. This work highlights the importance of modeling beyond linguistic content and advances the development of more natural, adaptive, and socially aware spoken language systems.

cs.CL↗

arXiv2025

Hierarchical Learning-Enhanced MPC for Safe Crowd Navigation with Heterogeneous Constraints

In this paper, we propose a novel hierarchical framework for robot navigation in dynamic environments with heterogeneous constraints. Our approach leverages a graph neural network trained via reinforcement learning (RL) to efficiently estimate the robot's cost-to-go, formulated as local goal recommendations. A spatio-temporal path-searching module, which accounts for kinematic constraints, is then employed to generate a reference trajectory to facilitate solving the non-convex optimization problem used for explicit constraint enforcement. More importantly, we introduce an incremental action-masking mechanism and a privileged learning strategy, enabling end-to-end training of the proposed planner. Both simulation and real-world experiments demonstrate that the proposed method effectively addresses local planning in complex dynamic environments, achieving state-of-the-art (SOTA) performance. Compared with existing learning-optimization hybrid methods, our approach eliminates the dependency on high-fidelity simulation environments, offering significant advantages in computational efficiency and training scalability. The code will be released as open-source upon acceptance of the paper.

cs.RO↗

arXiv2025

A remark on the counterexample to the unknotting number conjecture

By using Snappy, M. Brittenham and S. Hermiller discovered a very surprising example that $u(7_1\#\overline{7_1})\leq 5<6=u(7_1)+u(\overline{7_1})$, where $7_1$ is the $(2,7)$-torus knot and $\overline{7_1}$ is its mirror image. Based on their work, we give a direct verification of this fact.

math.GT↗

arXiv2025

Generative Audio Language Modeling with Continuous-valued Tokens and Masked Next-Token Prediction

Autoregressive next-token prediction with the Transformer decoder has become a de facto standard in large language models (LLMs), achieving remarkable success in Natural Language Processing (NLP) at scale. Extending this paradigm to audio poses unique challenges due to its inherently continuous nature. We research audio generation with a causal language model (LM) without discrete tokens. We leverage token-wise diffusion to model the continuous distribution of the next continuous-valued token. Our approach delivers significant improvements over previous discrete solution, AudioGen, achieving 20% and 40% relative gains on AudioCaps in Frechet Audio Distance (FAD) and Kullback-Leibler (KL) divergence, respectively. Additionally, we propose a novel masked next-token prediction task that incorporates masked prediction into the causal LM framework. On AudioCaps, the innovation yields 41% and 33% relative FAD improvements over AudioGen Base (285M) and AudioGen Large (1B) models, respectively, and is on par with the state-of-the-art (SOTA) diffusion models. Furthermore, we achieve these results with significantly fewer parameters -- 193M for our Base and 462M for our Large models.

eess.AS↗

arXiv2025

X-Dancer: Expressive Music to Human Dance Video Generation

We present X-Dancer, a novel zero-shot music-driven image animation pipeline that creates diverse and long-range lifelike human dance videos from a single static image. As its core, we introduce a unified transformer-diffusion framework, featuring an autoregressive transformer model that synthesize extended and music-synchronized token sequences for 2D body, head and hands poses, which then guide a diffusion model to produce coherent and realistic dance video frames. Unlike traditional methods that primarily generate human motion in 3D, X-Dancer addresses data limitations and enhances scalability by modeling a wide spectrum of 2D dance motions, capturing their nuanced alignment with musical beats through readily available monocular videos. To achieve this, we first build a spatially compositional token representation from 2D human pose labels associated with keypoint confidences, encoding both large articulated body movements (e.g., upper and lower body) and fine-grained motions (e.g., head and hands). We then design a music-to-motion transformer model that autoregressively generates music-aligned dance pose token sequences, incorporating global attention to both musical style and prior motion context. Finally we leverage a diffusion backbone to animate the reference image with these synthesized pose tokens through AdaIN, forming a fully differentiable end-to-end framework. Experimental results demonstrate that X-Dancer is able to produce both diverse and characterized dance videos, substantially outperforming state-of-the-art methods in term of diversity, expressiveness and realism. Code and model will be available for research purposes.

cs.CV↗

arXiv2025

Seamless Interaction: Dyadic Audiovisual Motion Modeling and Large-Scale Dataset

Human communication involves a complex interplay of verbal and nonverbal signals, essential for conveying meaning and achieving interpersonal goals. To develop socially intelligent AI technologies, it is crucial to develop models that can both comprehend and generate dyadic behavioral dynamics. To this end, we introduce the Seamless Interaction Dataset, a large-scale collection of over 4,000 hours of face-to-face interaction footage from over 4,000 participants in diverse contexts. This dataset enables the development of AI technologies that understand dyadic embodied dynamics, unlocking breakthroughs in virtual agents, telepresence experiences, and multimodal content analysis tools. We also develop a suite of models that utilize the dataset to generate dyadic motion gestures and facial expressions aligned with human speech. These models can take as input both the speech and visual behavior of their interlocutors. We present a variant with speech from an LLM model and integrations with 2D and 3D rendering methods, bringing us closer to interactive virtual agents. Additionally, we describe controllable variants of our motion models that can adapt emotional responses and expressivity levels, as well as generating more semantically-relevant gestures. Finally, we discuss methods for assessing the quality of these dyadic motion models, which are demonstrating the potential for more intuitive and responsive human-AI interactions.

cs.CV↗

arXiv2025

CARMA: Context-Aware Situational Grounding of Human-Robot Group Interactions by Combining Vision-Language Models with Object and Action Recognition

We introduce CARMA, a system for situational grounding in human-robot group interactions. Effective collaboration in such group settings requires situational awareness based on a consistent representation of present persons and objects coupled with an episodic abstraction of events regarding actors and manipulated objects. This calls for a clear and consistent assignment of instances, ensuring that robots correctly recognize and track actors, objects, and their interactions over time. To achieve this, CARMA uniquely identifies physical instances of such entities in the real world and organizes them into grounded triplets of actors, objects, and actions. To validate our approach, we conducted three experiments, where multiple humans and a robot interact: collaborative pouring, handovers, and sorting. These scenarios allow the assessment of the system's capabilities as to role distinction, multi-actor awareness, and consistent instance identification. Our experiments demonstrate that the system can reliably generate accurate actor-action-object triplets, providing a structured and robust foundation for applications requiring spatiotemporal reasoning and situated decision-making in collaborative settings.

cs.RO↗

arXiv2025

Incompressible Euler limit from the Boltzmann equation with Maxwell reflection boundary condition in the half-space

In this paper, we rigorously justify the incompressible Euler limit of the Boltzmann equation with general Maxwell reflection boundary condition in the half-space. The accommodation coefficient $α\in (0,1]$ is assumed to be $O(1)$. Our construction of solutions includes the interior fluid part and Knudsen-Prandtl coupled boundary layers. The corresponding solutions to the nonlinear Euler and nonlinear Prandtl systems are taken to be shear flows. Due to the presence of the nonlinear Prandtl layer, the remainder equation loses one order normal derivative. The key technical novelty lies in employing the full conservation laws to convert this loss of the normal derivative into the loss of tangential spatial derivative, avoiding any loss of regularity in time. By working within an analytic $L^2 \mbox{-} L^\infty$ framework, we establish the uniform estimate on the remainder equations, thus justify the validity of the incompressible Euler limit from the Boltzmann equation for the shear flow case.

math.AP↗

arXiv2025

Mirror Eyes: Explainable Human-Robot Interaction at a Glance

The gaze of a person tends to reflect their interest. This work explores what happens when this statement is taken literally and applied to robots. Here we present a robot system that employs a moving robot head with a screen-based eye model that can direct the robot's gaze to points in physical space and present a reflection-like mirror image of the attended region on top of each eye. We conducted a user study with 33 participants, who were asked to instruct the robot to perform pick-and-place tasks, monitor the robot's task execution, and interrupt it in case of erroneous actions. Despite a deliberate lack of instructions about the role of the eyes and a very brief system exposure, participants felt more aware about the robot's information processing, detected erroneous actions earlier, and rated the user experience higher when eye-based mirroring was enabled compared to non-reflective eyes. These results suggest a beneficial and intuitive utilization of the introduced method in cooperative human-robot interaction.

cs.RO↗

arXiv2025

3DeepRep: 3D Deep Low-rank Tensor Representation for Hyperspectral Image Inpainting

Recent approaches based on transform-based tensor nuclear norm (TNN) have demonstrated notable effectiveness in hyperspectral image (HSI) inpainting by leveraging low-rank structures in latent representations. Recent developments incorporate deep transforms to improve low-rank tensor representation; however, existing approaches typically restrict the transform to the spectral mode, neglecting low-rank properties along other tensor modes. In this paper, we propose a novel 3-directional deep low-rank tensor representation (3DeepRep) model, which performs deep nonlinear transforms along all three modes of the HSI tensor. To enforce low-rankness, the model minimizes the nuclear norms of mode-i frontal slices in the corresponding latent space for each direction (i=1,2,3), forming a 3-directional TNN regularization. The outputs from the three directional branches are subsequently fused via a learnable aggregation module to produce the final result. An efficient gradient-based optimization algorithm is developed to solve the model in a self-supervised manner. Extensive experiments on real-world HSI datasets demonstrate that the proposed method achieves superior inpainting performance compared to existing state-of-the-art techniques, both qualitatively and quantitatively.

cs.CV↗

arXiv2025

Large Language Model Empowered Design of Fluid Antenna Systems: Challenges, Frameworks, and Case Studies for 6G

The Fluid Antenna System (FAS), which enables flexible Multiple-Input Multiple-Output (MIMO) communications, introduces new spatial degrees of freedom for next-generation wireless networks. Unlike traditional MIMO, FAS involves joint port selection and precoder design, a combinatorial NP-hard optimization problem. Moreover, fully leveraging FAS requires acquiring Channel State Information (CSI) across its ports, a challenge exacerbated by the system's near-continuous reconfigurability. These factors make traditional system design methods impractical for FAS due to nonconvexity and prohibitive computational complexity. While deep learning (DL)-based approaches have been proposed for MIMO optimization, their limited generalization and fitting capabilities render them suboptimal for FAS. In contrast, Large Language Models (LLMs) extend DL's capabilities by offering general-purpose adaptability, reasoning, and few-shot learning, thereby overcoming the limitations of task-specific, data-intensive models. This article presents a vision for LLM-driven FAS design, proposing a novel flexible communication framework. To demonstrate the potential, we examine LLM-enhanced FAS in multiuser scenarios, showcasing how LLMs can revolutionize FAS optimization.

cs.IT↗

arXiv2025

The bump-on-tail instability excited by energetic electrons in helicon plasma

This work explores for the first time bump-on-tail (BOT) instability excited by energetic electrons in helicon plasma. The Berk-Breizman model that developed for the wave-particle interaction and resulted instability in magnetic fusion is used. Details of the BOT instability are computed referring to typical helicon discharge conditions. Parameter studies are also conducted to reveal the effects of collisionality and energetic drive, to account for high-pressure and high-power senarios respectively. It is found that under the HXHM (high magnetic field helicon experiment) experimental parameters, the disturbed distribution function oscillates explosively at the initial stage of BOT instability excitation, and the wave frequency shift does not appear, i.e., the steady-state solution always exists under this mode. In the process of restoring stability, the exchange of energetic particles and wave energy is concurrent with the change of wave amplitude. As the Krook operator increases (i.e., from 0.1 to 1), the saturation level of the electric field and the instability enhance. Additionally, there have a bigger disturbance for the initial EEDF (electron energy distribution function) in high-power helicon devices, so that the energy exchange between waves and energetic particles is stronger as well. Moreover, BOT instability effects the density and flux of bulk plasma, and the flux increases with the Krook operator. The effect of BOT instability is one order of magnitude larger on rotating plasma than that on stationary plasma.These findings present a full picture of BOT instability in helicon plasma and are valuable to controlling it for efficient and safe applications, e.g., high-power space plasma propulsion and plasma material interactions using helicon source.

physics.plasm-ph↗

arXiv2025

MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention

We introduce MiniMax-M1, the world's first open-weight, large-scale hybrid-attention reasoning model. MiniMax-M1 is powered by a hybrid Mixture-of-Experts (MoE) architecture combined with a lightning attention mechanism. The model is developed based on our previous MiniMax-Text-01 model, which contains a total of 456 billion parameters with 45.9 billion parameters activated per token. The M1 model natively supports a context length of 1 million tokens, 8x the context size of DeepSeek R1. Furthermore, the lightning attention mechanism in MiniMax-M1 enables efficient scaling of test-time compute. These properties make M1 particularly suitable for complex tasks that require processing long inputs and thinking extensively. MiniMax-M1 is trained using large-scale reinforcement learning (RL) on diverse problems including sandbox-based, real-world software engineering environments. In addition to M1's inherent efficiency advantage for RL training, we propose CISPO, a novel RL algorithm to further enhance RL efficiency. CISPO clips importance sampling weights rather than token updates, outperforming other competitive RL variants. Combining hybrid-attention and CISPO enables MiniMax-M1's full RL training on 512 H800 GPUs to complete in only three weeks, with a rental cost of just $534,700. We release two versions of MiniMax-M1 models with 40K and 80K thinking budgets respectively, where the 40K model represents an intermediate phase of the 80K training. Experiments on standard benchmarks show that our models are comparable or superior to strong open-weight models such as the original DeepSeek-R1 and Qwen3-235B, with particular strengths in complex software engineering, tool utilization, and long-context tasks. We publicly release MiniMax-M1 at https://github.com/MiniMax-AI/MiniMax-M1.

cs.CL↗

arXiv2025

Optimization of bi-directional gated loop cell based on multi-head attention mechanism for SSD health state classification model

Aiming at the critical role of SSD health state prediction in data reliability assurance, this study proposes a hybrid BiGRU-MHA model that incorporates a multi-head attention mechanism to enhance the accuracy and stability of storage device health classification. The model innovatively integrates temporal feature extraction and key information focusing capabilities. Specifically, it leverages the bidirectional timing modeling advantages of the BiGRU network to capture both forward and backward dependencies of SSD degradation features. Simultaneously, the multi-head attention mechanism dynamically assigns feature weights, improving the model's sensitivity to critical health indicators. Experimental results show that the proposed model achieves classification accuracies of 92.70% on the training set and 92.44% on the test set, with a minimal performance gap of only 0.26%, demonstrating excellent generalization ability. Further analysis using the receiver operating characteristic (ROC) curve shows an area under the curve (AUC) of 0.94 on the test set, confirming the model's robust binary classification performance. This work not only presents a new technical approach for SSD health prediction but also addresses the generalization bottleneck of traditional models, offering a verifiable method with practical value for preventive maintenance of industrial-grade storage systems. The results show the model can significantly reduce data loss risks by providing early failure warnings and help optimize maintenance costs, supporting intelligent decision-making in building reliable storage systems for cloud computing data centers and edge storage environments.

cs.LG↗

arXiv2025

Spatial and temporal evolutions of blue-core helicon discharge driven by planar antenna with concentric rings

The spatial and temporal evolutions of blue-core helicon discharge driven by a planar antenna with four concentric rings are explored on the Linear Experimental Advanced Device (LEAD). The discharge experiences distinct density jumps from E mode to H mode, W mode, and blue-core mode, when RF input power increases. This is similar to previous observations using other typical helicon antennas; however, this special antenna could drive modes of even higher levels for which the blue-core plasma column is actually hollow in radius, i.e. peaking off-axis, which was not presented before. The column shows counterclockwise rotation for blue-core mode and clockwise rotation for non-blue-core mode. The reason could be attributed to the radial electric field differenceses for both modes which reverses the rotation direction via ExB drive. Moreover, the centrifugal instability of blue-core helicon plasma is computed using a two-fluid flowing plasma model. It shows that the instability is strong for small axial wave number but becomes weak for large axial wave number. Perturbed density peaks at radius of 0.045 m, while the equilibrium density gradient peaks at radius of 0.055 m. The coincidence of their radial locations suggests that it is a resistive drift mode driven by density gradient. The blue-core mode weakens once the magnetic field or flow rate exceeds the threshold value. Increasing power further leads to a smoother plasma density gradient. The electron temperature profiles decrease with increased power, and the radial gradient of the electron temperature inside the core is smaller as the magnetic field changes. To our best knowledge, it is the first detailed characterization of blue-core helicon plasma driven by planar antenna, especially in terms of azimuthal rotation and centrifugal instability.

physics.plasm-ph↗

arXiv2025

CoQMoE: Co-Designed Quantization and Computation Orchestration for Mixture-of-Experts Vision Transformer on FPGA

Vision Transformers (ViTs) exhibit superior performance in computer vision tasks but face deployment challenges on resource-constrained devices due to high computational/memory demands. While Mixture-of-Experts Vision Transformers (MoE-ViTs) mitigate this through a scalable architecture with sub-linear computational growth, their hardware implementation on FPGAs remains constrained by resource limitations. This paper proposes a novel accelerator for efficiently implementing quantized MoE models on FPGAs through two key innovations: (1) A dual-stage quantization scheme combining precision-preserving complex quantizers with hardware-friendly simplified quantizers via scale reparameterization, with only 0.28 $\%$ accuracy loss compared to full precision; (2) A resource-aware accelerator architecture featuring latency-optimized streaming attention kernels and reusable linear operators, effectively balancing performance and resource consumption. Experimental results demonstrate that our accelerator achieves nearly 155 frames per second, a 5.35$\times$ improvement in throughput, and over $80\%$ energy reduction compared to state-of-the-art (SOTA) FPGA MoE accelerators, while maintaining $<1\%$ accuracy loss across vision benchmarks. Our implementation is available at https://github.com/DJ000011/CoQMoE.

cs.AR↗

arXiv2025

LLIA -- Enabling Low-Latency Interactive Avatars: Real-Time Audio-Driven Portrait Video Generation with Diffusion Models

Diffusion-based models have gained wide adoption in the virtual human generation due to their outstanding expressiveness. However, their substantial computational requirements have constrained their deployment in real-time interactive avatar applications, where stringent speed, latency, and duration requirements are paramount. We present a novel audio-driven portrait video generation framework based on the diffusion model to address these challenges. Firstly, we propose robust variable-length video generation to reduce the minimum time required to generate the initial video clip or state transitions, which significantly enhances the user experience. Secondly, we propose a consistency model training strategy for Audio-Image-to-Video to ensure real-time performance, enabling a fast few-step generation. Model quantization and pipeline parallelism are further employed to accelerate the inference speed. To mitigate the stability loss incurred by the diffusion process and model quantization, we introduce a new inference strategy tailored for long-duration video generation. These methods ensure real-time performance and low latency while maintaining high-fidelity output. Thirdly, we incorporate class labels as a conditional input to seamlessly switch between speaking, listening, and idle states. Lastly, we design a novel mechanism for fine-grained facial expression control to exploit our model's inherent capacity. Extensive experiments demonstrate that our approach achieves low-latency, fluid, and authentic two-way communication. On an NVIDIA RTX 4090D, our model achieves a maximum of 78 FPS at a resolution of 384x384 and 45 FPS at a resolution of 512x512, with an initial video generation latency of 140 ms and 215 ms, respectively.

cs.CV↗

arXiv2025

Single-pulse-based interstellar scintillation studies of RRATs

The nature of irregularly spaced pulses of rotating radio transients (RRATs) complicates interstellar scintillation studies. In this letter, we report the primary scintillation parameters of a sample of RRATs using pairwise correlations of pulse spectra. Moreover, from the measured scintillation velocities, we constrain their transverse velocities. We also find a reduced modulation index, $\rm{m=0.13\pm0.01}$, for RRAT~J1538+2345. Several possible explanations are discussed. Furthermore, the single-pulse-based interstellar scintillation technique is applicable to other pulsar populations, including nulling pulsars and those with short scintillation timescales, and fast radio bursts.

astro-ph.HE↗