arXiv subjects

Wentao Ma

Publications and source records attributed to Wentao Ma.

At least 37 records · Page 2Linked to original sources

arXiv2025

M2C: An Open-Source Software for Multiphysics Simulation of Compressible Multi-Material Flows and Fluid-Structure Interactions

M2C (Multiphysics Modeling and Computation) is an open-source software for simulating multi-material fluid flows and fluid-structure interactions under extreme conditions, such as high pressures, high temperatures, shock waves, and large interface deformations. It employs a finite volume method to solve the compressible Navier-Stokes equations and supports a wide range of thermodynamic equations of state. M2C incorporates models of laser radiation and absorption, phase transition, and ionization, coupled with continuum dynamics. Multi-material interfaces are evolved using a level set method, while fluid-structure interfaces are tracked using an embedded boundary method. Advective fluxes across interfaces are computed using FIVER (FInite Volume method based on Exact multi-material Riemann problems). For two-way fluid-structure interaction, M2C is coupled with the open-source structural dynamics solver Aero-S using a partitioned procedure. The M2C code is written in C++ and parallelized with MPI for high-performance computing. The source package includes a set of example problems for demonstration and user training. Accuracy is verified through benchmark cases such as Riemann problems, interface evolution, single-bubble dynamics, and ionization response. Several multiphysics applications are also presented, including laser-induced thermal cavitation, explosion and blast mitigation, and hypervelocity impact.

physics.flu-dyn↗

arXiv2025

Vamba: Understanding Hour-Long Videos with Hybrid Mamba-Transformers

State-of-the-art transformer-based large multimodal models (LMMs) struggle to handle hour-long video inputs due to the quadratic complexity of the causal self-attention operations, leading to high computational costs during training and inference. Existing token compression-based methods reduce the number of video tokens but often incur information loss and remain inefficient for extremely long sequences. In this paper, we explore an orthogonal direction to build a hybrid Mamba-Transformer model (VAMBA) that employs Mamba-2 blocks to encode video tokens with linear complexity. Without any token reduction, VAMBA can encode more than 1024 frames (640$\times$360) on a single GPU, while transformer-based models can only encode 256 frames. On long video input, VAMBA achieves at least 50% reduction in GPU memory usage during training and inference, and nearly doubles the speed per training step compared to transformer-based LMMs. Our experimental results demonstrate that VAMBA improves accuracy by 4.3% on the challenging hour-long video understanding benchmark LVBench over prior efficient video LMMs, and maintains strong performance on a broad spectrum of long and short video understanding tasks.

cs.CV↗

arXiv2025

SpokenWOZ: A Large-Scale Speech-Text Benchmark for Spoken Task-Oriented Dialogue Agents

Task-oriented dialogue (TOD) models have made significant progress in recent years. However, previous studies primarily focus on datasets written by annotators, which has resulted in a gap between academic research and real-world spoken conversation scenarios. While several small-scale spoken TOD datasets are proposed to address robustness issues such as ASR errors, they ignore the unique challenges in spoken conversation. To tackle the limitations, we introduce SpokenWOZ, a large-scale speech-text dataset for spoken TOD, containing 8 domains, 203k turns, 5.7k dialogues and 249 hours of audios from human-to-human spoken conversations. SpokenWOZ further incorporates common spoken characteristics such as word-by-word processing and reasoning in spoken language. Based on these characteristics, we present cross-turn slot and reasoning slot detection as new challenges. We conduct experiments on various baselines, including text-modal models, newly proposed dual-modal models, and LLMs, e.g., ChatGPT. The results show that the current models still have substantial room for improvement in spoken conversation, where the most advanced dialogue state tracker only achieves 25.65% in joint goal accuracy and the SOTA end-to-end model only correctly completes the user request in 52.1% of dialogues. The dataset, code, and leaderboard are available: https://spokenwoz.github.io/.

cs.CL↗

arXiv2025

EPO: Explicit Policy Optimization for Strategic Reasoning in LLMs via Reinforcement Learning

Large Language Models (LLMs) have shown impressive reasoning capabilities in well-defined problems with clear solutions, such as mathematics and coding. However, they still struggle with complex real-world scenarios like business negotiations, which require strategic reasoning-an ability to navigate dynamic environments and align long-term goals amidst uncertainty. Existing methods for strategic reasoning face challenges in adaptability, scalability, and transferring strategies to new contexts. To address these issues, we propose explicit policy optimization (EPO) for strategic reasoning, featuring an LLM that provides strategies in open-ended action space and can be plugged into arbitrary LLM agents to motivate goal-directed behavior. To improve adaptability and policy transferability, we train the strategic reasoning model via multi-turn reinforcement learning (RL),utilizing process rewards and iterative self-play. Experiments across social and physical domains demonstrate EPO's ability of long-term goal alignment through enhanced strategic reasoning, achieving state-of-the-art performance on social dialogue and web navigation tasks. Our findings reveal various collaborative reasoning mechanisms emergent in EPO and its effectiveness in generating novel strategies, underscoring its potential for strategic reasoning in real-world applications. Code and data are available at https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/EPO.

cs.CL↗

arXiv2025

VideoEval-Pro: Robust and Realistic Long Video Understanding Evaluation

Large multimodal models (LMMs) have recently emerged as a powerful tool for long video understanding (LVU), prompting the development of standardized LVU benchmarks to evaluate their performance. However, our investigation reveals a rather sober lesson for existing LVU benchmarks. First, most existing benchmarks rely heavily on multiple-choice questions (MCQs), whose evaluation results are inflated due to the possibility of guessing the correct answer; Second, a significant portion of questions in these benchmarks have strong priors to allow models to answer directly without even reading the input video. For example, Gemini-1.5-Pro can achieve over 50\% accuracy given a random frame from a long video on Video-MME. We also observe that increasing the number of frames does not necessarily lead to improvement on existing benchmarks, which is counterintuitive. As a result, the validity and robustness of current LVU benchmarks are undermined, impeding a faithful assessment of LMMs' long-video understanding capability. To tackle this problem, we propose VideoEval-Pro, a realistic LVU benchmark containing questions with open-ended short-answer, which truly require understanding the entire video. VideoEval-Pro assesses both segment-level and full-video understanding through perception and reasoning tasks. By evaluating 21 proprietary and open-source video LMMs, we conclude the following findings: (1) video LMMs show drastic performance ($>$25\%) drops on open-ended questions compared with MCQs; (2) surprisingly, higher MCQ scores do not lead to higher open-ended scores on VideoEval-Pro; (3) compared to other MCQ benchmarks, VideoEval-Pro benefits more from increasing the number of input frames. Our results show that VideoEval-Pro offers a more realistic and reliable measure of long video understanding, providing a clearer view of progress in this domain.

cs.CV↗

arXiv2025

ProT-GFDM: A Generative Fractional Diffusion Model for Protein Generation

This work introduces the generative fractional diffusion model for protein generation (ProT-GFDM), a novel generative framework that employs fractional stochastic dynamics for protein backbone structure modeling. This approach builds on the continuous-time score-based generative diffusion modeling paradigm, where data are progressively transformed into noise via a stochastic differential equation and reversed to generate structured samples. Unlike classical methods that rely on standard Brownian motion, ProT-GFDM employs a fractional stochastic process with superdiffusive properties to improve the capture of long-range dependencies in protein structures. Trained on protein fragments from the Protein Data Bank, ProT-GFDM outperforms conventional score-based models, achieving a 7.19% increase in density, a 5.66% improvement in coverage, and a 1.01% reduction in the Frechet inception distance. By integrating fractional dynamics with computationally efficient sampling, the proposed framework advances generative modeling for structured biological data, with implications for protein design and computational drug discovery.

q-bio.QM↗

arXiv2025

SDPO: Segment-Level Direct Preference Optimization for Social Agents

Social agents powered by large language models (LLMs) can simulate human social behaviors but fall short in handling complex social dialogues. Direct Preference Optimization (DPO) has proven effective in aligning LLM behavior with human preferences across various agent tasks. However, standard DPO focuses solely on individual turns, which limits its effectiveness in multi-turn social interactions. Several DPO-based multi-turn alignment methods with session-level data have shown potential in addressing this problem.While these methods consider multiple turns across entire sessions, they are often overly coarse-grained, introducing training noise, and lack robust theoretical support. To resolve these limitations, we propose Segment-Level Direct Preference Optimization (SDPO), which dynamically select key segments within interactions to optimize multi-turn agent behavior. SDPO minimizes training noise and is grounded in a rigorous theoretical framework. Evaluations on the SOTOPIA benchmark demonstrate that SDPO-tuned agents consistently outperform both existing DPO-based methods and proprietary LLMs like GPT-4o, underscoring SDPO's potential to advance the social intelligence of LLM-based agents. We release our code and data at https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/SDPO.

cs.AI↗

arXiv2024

FlowBench: Revisiting and Benchmarking Workflow-Guided Planning for LLM-based Agents

LLM-based agents have emerged as promising tools, which are crafted to fulfill complex tasks by iterative planning and action. However, these agents are susceptible to undesired planning hallucinations when lacking specific knowledge for expertise-intensive tasks. To address this, preliminary attempts are made to enhance planning reliability by incorporating external workflow-related knowledge. Despite the promise, such infused knowledge is mostly disorganized and diverse in formats, lacking rigorous formalization and comprehensive comparisons. Motivated by this, we formalize different formats of workflow knowledge and present FlowBench, the first benchmark for workflow-guided planning. FlowBench covers 51 different scenarios from 6 domains, with knowledge presented in diverse formats. To assess different LLMs on FlowBench, we design a multi-tiered evaluation framework. We evaluate the efficacy of workflow knowledge across multiple formats, and the results indicate that current LLM agents need considerable improvements for satisfactory planning. We hope that our challenging benchmark can pave the way for future agent planning research.

cs.CL↗

arXiv2024

PTA: Enhancing Multimodal Sentiment Analysis through Pipelined Prediction and Translation-based Alignment

Multimodal aspect-based sentiment analysis (MABSA) aims to understand opinions in a granular manner, advancing human-computer interaction and other fields. Traditionally, MABSA methods use a joint prediction approach to identify aspects and sentiments simultaneously. However, we argue that joint models are not always superior. Our analysis shows that joint models struggle to align relevant text tokens with image patches, leading to misalignment and ineffective image utilization. In contrast, a pipeline framework first identifies aspects through MATE (Multimodal Aspect Term Extraction) and then aligns these aspects with image patches for sentiment classification (MASC: Multimodal Aspect-Oriented Sentiment Classification). This method is better suited for multimodal scenarios where effective image use is crucial. We present three key observations: (a) MATE and MASC have different feature requirements, with MATE focusing on token-level features and MASC on sequence-level features; (b) the aspect identified by MATE is crucial for effective image utilization; and (c) images play a trivial role in previous MABSA methods due to high noise. Based on these observations, we propose a pipeline framework that first predicts the aspect and then uses translation-based alignment (TBA) to enhance multimodal semantic consistency for better image utilization. Our method achieves state-of-the-art (SOTA) performance on widely used MABSA datasets Twitter-15 and Twitter-17. This demonstrates the effectiveness of the pipeline approach and its potential to provide valuable insights for future MABSA research. For reproducibility, the code and checkpoint will be released.

cs.CL↗

arXiv2023

Self-Explanation Prompting Improves Dialogue Understanding in Large Language Models

Task-oriented dialogue (TOD) systems facilitate users in executing various activities via multi-turn dialogues, but Large Language Models (LLMs) often struggle to comprehend these intricate contexts. In this study, we propose a novel "Self-Explanation" prompting strategy to enhance the comprehension abilities of LLMs in multi-turn dialogues. This task-agnostic approach requires the model to analyze each dialogue utterance before task execution, thereby improving performance across various dialogue-centric tasks. Experimental results from six benchmark datasets confirm that our method consistently outperforms other zero-shot prompts and matches or exceeds the efficacy of few-shot prompts, demonstrating its potential as a powerful tool in enhancing LLMs' comprehension in complex dialogue tasks.

cs.CL↗

arXiv2023

UniPCM: Universal Pre-trained Conversation Model with Task-aware Automatic Prompt

Recent research has shown that multi-task pre-training greatly improves the model's robustness and transfer ability, which is crucial for building a high-quality dialog system. However, most previous works on multi-task pre-training rely heavily on human-defined input format or prompt, which is not optimal in quality and quantity. In this work, we propose to use Task-based Automatic Prompt generation (TAP) to automatically generate high-quality prompts. Using the high-quality prompts generated, we scale the corpus of the pre-trained conversation model to 122 datasets from 15 dialog-related tasks, resulting in Universal Pre-trained Conversation Model (UniPCM), a powerful foundation model for various conversational tasks and different dialog systems. Extensive experiments have shown that UniPCM is robust to input prompts and capable of various dialog-related tasks. Moreover, UniPCM has strong transfer ability and excels at low resource scenarios, achieving SOTA results on 9 different datasets ranging from task-oriented dialog to open-domain conversation. Furthermore, we are amazed to find that TAP can generate prompts on par with those collected with crowdsourcing. The code is released with the paper.

cs.CL↗

arXiv2023

Long-Pulse Laser-Induced Cavitation: A Race Between Advection and Phase Transition

Vapor bubbles generated by long-pulsed laser often have complex non-spherical shapes that reflect some characteristics (e.g., direction, width) of the laser beam. The transition between two commonly observed shapes - namely, a rounded pear-like shape and an elongated conical shape - is studied using a new computational model that combines compressible multiphase fluid dynamics with laser radiation and phase transition. Two laboratory experiments are simulated, in which Holmium:YAG and Thulium fiber lasers are used separately to generate bubbles of different shapes. In both cases, the bubble morphology predicted by the simulation agrees reasonably well with the experimental measurement. The simulated laser radiance, temperature, velocity, and pressure fields are analyzed to explain bubble dynamics and energy transmission. It is found that due to the lasting energy input (i.e. long-pulsed laser), the vapor bubble's dynamics is driven not only by advection, but also by the continuation of vaporization. Notably, vaporization lasts less than 1 microsecond in the case of the pear-shaped bubble, versus more than 50 microseconds for the elongated bubble. It is hypothesized that the bubble's shape is the result of a competition. When the speed of advection is higher than that of vaporization, the bubble tends to grow spherically. Otherwise, it elongates along the laser beam direction. To clarify and test this hypothesis, the two speeds are defined analytically using a simplified model, then estimated for the experiments using simulation results. The results support the hypothesis. They also suggest that a higher laser absorption coefficient and a narrower beam facilitate bubble elongation.

physics.flu-dyn↗

arXiv2023

Efficient Solution of Bimaterial Riemann Problems for Compressible Multi-Material Flow Simulations

When solving compressible multi-material flow problems, an unresolved challenge is the computation of advective fluxes across material interfaces that separate drastically different thermodynamic states and relations. A popular idea in this regard is to locally construct bimaterial Riemann problems, and to apply their exact solutions in flux computation. For general equations of state, however, finding the exact solution of a Riemann problem is expensive as it requires nested loops. Multiplied by the large number of Riemann problems constructed during a simulation, the computational cost often becomes prohibitive. The work presented in this paper aims to accelerate the solution of bimaterial Riemann problems without introducing approximations or offline precomputation tasks. The basic idea is to exploit some special properties of the Riemann problem equations, and to recycle previous solutions as much as possible. Following this idea, four acceleration methods are developed, including (1) a change of integration variable through rarefaction fans, (2) storing and reusing integration trajectory data, (3) step size adaptation, and (4) constructing an R-tree on the fly to generate initial guesses. The performance of these acceleration methods are assessed using four example problems in underwater explosion, laser-induced cavitation, and hypervelocity impact. These problems exhibit strong shock waves, large interface deformation, contact of multiple (>2) interfaces, and interaction between gases and condensed matters. In these challenging cases, the solution of bimaterial Riemann problems is accelerated by 37 to 87 times. As a result, the total cost of advective flux computation, which includes the exact Riemann problem solution at material interfaces and the numerical flux calculation over the entire computational domain, is accelerated by 18 to 81 times.

physics.comp-ph↗

arXiv2023

Data-driven Approximation of Distributionally Robust Chance Constraints using Bayesian Credible Intervals

The non-convexity and intractability of distributionally robust chance constraints make them challenging to cope with. From a data-driven perspective, we propose formulating it as a robust optimization problem to ensure that the distributionally robust chance constraint is satisfied with high probability. To incorporate available data and prior distribution knowledge, we construct ambiguity sets for the distributionally robust chance constraint using Bayesian credible intervals. We establish the congruent relationship between the ambiguity set in Bayesian distributionally robust chance constraints and the uncertainty set in a specific robust optimization. In contrast to most existent uncertainty set construction methods which are only applicable for particular settings, our approach provides a unified framework for constructing uncertainty sets under different marginal distribution assumptions, thus making it more flexible and widely applicable. Additionally, under the concavity assumption, our method provides strong finite sample probability guarantees for optimal solutions. The practicality and effectiveness of our approach are illustrated with numerical experiments on portfolio management and queuing system problems. Overall, our approach offers a promising solution to distributionally robust chance constrained problems and has potential applications in other fields.

math.OC↗

arXiv2023

Speech-Text Dialog Pre-training for Spoken Dialog Understanding with Explicit Cross-Modal Alignment

Recently, speech-text pre-training methods have shown remarkable success in many speech and natural language processing tasks. However, most previous pre-trained models are usually tailored for one or two specific tasks, but fail to conquer a wide range of speech-text tasks. In addition, existing speech-text pre-training methods fail to explore the contextual information within a dialogue to enrich utterance representations. In this paper, we propose Speech-text dialog Pre-training for spoken dialog understanding with ExpliCiT cRoss-Modal Alignment (SPECTRA), which is the first-ever speech-text dialog pre-training model. Concretely, to consider the temporality of speech modality, we design a novel temporal position prediction task to capture the speech-text alignment. This pre-training task aims to predict the start and end time of each textual word in the corresponding speech waveform. In addition, to learn the characteristics of spoken dialogs, we generalize a response selection task from textual dialog pre-training to speech-text dialog pre-training scenarios. Experimental results on four different downstream speech-text tasks demonstrate the superiority of SPECTRA in learning speech-text alignment and multi-turn dialog context.

cs.CL↗

arXiv2023

Fluid-Solid Coupled Simulation of Hypervelocity Impact and Plasma Formation

The generation of plasma from hypervelocity impacts is an active research topic due to its important science and engineering ramifications in various applications. Previous studies have mainly focused on the ionization of the solid materials that constitute the projectile and the target. In this letter, we consider impact events that occur in a fluid (e.g.,~gas) medium, and present a multiphysics computational modeling approach and associated analysis to predict the behavior of the dynamic fluid-solid interaction that causes the surrounding fluid to ionize. The proposed computational framework is applied to a specific case involving a system of three interacting domains: a copper rod projectile impacting onto a soda lime glass target in a neon gas environment. The impact velocity is varied between 3 km/s and 6 km/s in different simulations. The computational model couples the compressible inviscid Navier-Stokes equations with the Saha ionization equations. The three material interfaces formed among the projectile, the target, and the ambient gas are tracked implicitly by solving two level set equations that share the same velocity field. The mass, momentum, and energy fluxes across the interfaces are computed using the FInite Volume method with Exact two-material Riemann problems (FIVER). The simulation result reveals a region of neon gas with high velocity, temperature, pressure, and mass density, formed in the early stage of the impact mainly due to the hypersonic compression of the fluid between the projectile and the target. For impact velocities higher than 4 km/s, ionization is predicted in this region.

physics.plasm-ph↗

arXiv2023

Gate Recurrent Unit Network based on Hilbert-Schmidt Independence Criterion for State-of-Health Estimation

State-of-health (SOH) estimation is a key step in ensuring the safe and reliable operation of batteries. Due to issues such as varying data distribution and sequence length in different cycles, most existing methods require health feature extraction technique, which can be time-consuming and labor-intensive. GRU can well solve this problem due to the simple structure and superior performance, receiving widespread attentions. However, redundant information still exists within the network and impacts the accuracy of SOH estimation. To address this issue, a new GRU network based on Hilbert-Schmidt Independence Criterion (GRU-HSIC) is proposed. First, a zero masking network is used to transform all battery data measured with varying lengths every cycle into sequences of the same length, while still retaining information about the original data size in each cycle. Second, the Hilbert-Schmidt Independence Criterion (HSIC) bottleneck, which evolved from Information Bottleneck (IB) theory, is extended to GRU to compress the information from hidden layers. To evaluate the proposed method, we conducted experiments on datasets from the Center for Advanced Life Cycle Engineering (CALCE) of the University of Maryland and NASA Ames Prognostics Center of Excellence. Experimental results demonstrate that our model achieves higher accuracy than other recurrent models.

cs.LG↗

arXiv2022

Computational Analysis of Bubble-Structure Interactions in Near-Field Underwater Explosion

The response of underwater structures to a near-field explosion is coupled with the dynamics of the explosion bubble and the surrounding water. This multiphase fluid-structure interaction process is investigated using a model problem that features the yielding and collapse of a thin-walled aluminum cylinder. A recently developed computational framework that couples a compressible fluid dynamics solver with a structural dynamics solver is employed. The fluid-structure and liquid-gas interfaces are tracked using embedded boundary and level set methods. The conservation law across the interfaces is enforced by solving one-dimensional bimaterial Riemann problems. The initial pressure inside the explosion bubble is varied by two orders of magnitude in different test cases. Three different modes of collapse are discovered, including an horizontal collapse (i.e. with one lobe extending towards the explosive charge) that appears counterintuitive, yet has been observed in previous laboratory experiments. Because of the transition of modes, the time it takes for the structure to reach self-contact does not decrease monotonically as the explosion magnitude increases. The flow fields, the bubble dynamics, and the transient structural deformation are visualized to elucidate the cause of each collapse mode and the mode transitions. The result suggests that the pressure pulse resulting from the contraction of the explosion bubble has significant effect on the structure's collapse. The phase difference between the structural vibration and bubble oscillation influences the structure's mode of collapse. Furthermore, the transient structural deformation has clear effect on the bubble dynamics, leading to a two-way interaction. A liquid jet that points away from the structure is observed. Compared to the liquid jets produced by bubbles collapsing near a rigid wall, this jet is in the opposite direction.

physics.flu-dyn↗