SearcharxivSearch

arXiv subjects

Guoqiang Hu

Publications and source records attributed to Guoqiang Hu.

At least 19 recordsLinked to original sources

StepAudio 2.5 Technical Report

Unified audio-language modeling has emerged as a prominent trend in modern speech systems, promising to bring the reasoning capabilities of large language models to auditory tasks. However, existing unified foundations often struggle to match the depth of specialized systems across automatic speech recognition (ASR), text-to-speech synthesis (TTS), and realtime spoken interaction. Bridging this gap remains an open challenge. This report presents StepAudio 2.5, a unified audio-language foundation model that matches or exceeds specialized systems across all three capabilities. Rather than treating these tasks as architecturally distinct, we operate on the premise that once text and audio share a multimodal representational space, task specialization becomes a matter of operational regimes: data construction, optimization targets, and decoding constraints. Guided by this insight, we advance the post-training paradigm from standard supervised learning to task-tailored Reinforcement Learning from Human Feedback (RLHF), using it as the primary mechanism to define complex optimization targets. We leverage this RLHF-centric alignment, alongside specialized decoding, to shape a shared backbone into three distinct operational modes. Concretely, the ASR branch advances transcription efficiency via verifiable multi-token decoding; the TTS branch achieves controllable, expressive synthesis through preference-based RLHF and context-rich supervision; and the Realtime branch realizes low-latency, persona-consistent dialogue via generative reward modeling within an RLHF framework. On standard benchmarks, StepAudio 2.5 achieves state-of-the-art results across ASR, TTS, and Realtime, demonstrating that a singular audio-language foundation can successfully internalize the distinct deployment objectives of speech understanding, generation, and live interaction.

eess.AS

Step-Audio-EditX Technical Report

We present Step-Audio-EditX, the first open-source LLM-based audio model excelling at expressive and iterative audio editing encompassing emotion, speaking style, and paralinguistics alongside robust zero-shot text-to-speech (TTS) capabilities. Our core innovation lies in leveraging only large-margin synthetic data, which circumvents the need for embedding-based priors or auxiliary modules. This large-margin learning approach enables both iterative control and high expressivity across voices, and represents a fundamental pivot from the conventional focus on representation-level disentanglement. Evaluation results demonstrate that Step-Audio-EditX surpasses both MiniMax-2.6-hd and Doubao-Seed-TTS-2.0 in emotion editing and other fine-grained control tasks.

cs.CL

MULTI-Bench: A Multi-Turn Interactive Benchmark for Assessing Emotional Intelligence ability of Spoken Dialogue Models

Spoken Dialogue Models (SDMs) have advanced rapidly, yet their ability to sustain genuinely interactive multi-turn conversations remains underexplored, as most benchmarks focus on single-turn exchanges. We introduce Multi-Bench, the first benchmark explicitly designed to evaluate SDMs in multi-turn interactive dialogue with an emphasis on emotional intelligence. Multi-Bench employs a hierarchical structure with a basic track for emotion understanding and reasoning and an advanced track for emotion support and application. It comprises five carefully designed tasks and about 3.2K samples, ranging from emotion recognition to complex reasoning and interactive dialogue, supported by a reproducible evaluation framework. We evaluate six representative SDMs on eight subsets of Multi-Bench. Results show that while current SDMs achieve good performance on basic understanding tasks, they still have room for improvement in advanced multi-turn interactive dialogue and reasoning-related tasks, particularly in emotion awareness and application.

eess.AS

Step-Audio 2 Technical Report

This paper presents Step-Audio 2, an end-to-end multi-modal large language model designed for industry-strength audio understanding and speech conversation. By integrating a latent audio encoder and reasoning-centric reinforcement learning (RL), Step-Audio 2 achieves promising performance in automatic speech recognition (ASR) and audio understanding. To facilitate genuine end-to-end speech conversation, Step-Audio 2 incorporates the generation of discrete audio tokens into language modeling, significantly enhancing its responsiveness to paralinguistic information such as speaking styles and emotions. To effectively leverage the rich textual and acoustic knowledge in real-world data, Step-Audio 2 integrates retrieval-augmented generation (RAG) and is able to call external tools such as web search to mitigate hallucination and audio search to switch timbres. Trained on millions of hours of speech and audio data, Step-Audio 2 delivers intelligence and expressiveness across diverse conversational scenarios. Evaluation results demonstrate that Step-Audio 2 achieves state-of-the-art performance on various audio understanding and conversational benchmarks compared to other open-source and commercial solutions. Please visit https://github.com/stepfun-ai/Step-Audio2 for more information.

cs.CL

ManiGaussian++: General Robotic Bimanual Manipulation with Hierarchical Gaussian World Model

Multi-task robotic bimanual manipulation is becoming increasingly popular as it enables sophisticated tasks that require diverse dual-arm collaboration patterns. Compared to unimanual manipulation, bimanual tasks pose challenges to understanding the multi-body spatiotemporal dynamics. An existing method ManiGaussian pioneers encoding the spatiotemporal dynamics into the visual representation via Gaussian world model for single-arm settings, which ignores the interaction of multiple embodiments for dual-arm systems with significant performance drop. In this paper, we propose ManiGaussian++, an extension of ManiGaussian framework that improves multi-task bimanual manipulation by digesting multi-body scene dynamics through a hierarchical Gaussian world model. To be specific, we first generate task-oriented Gaussian Splatting from intermediate visual features, which aims to differentiate acting and stabilizing arms for multi-body spatiotemporal dynamics modeling. We then build a hierarchical Gaussian world model with the leader-follower architecture, where the multi-body spatiotemporal dynamics is mined for intermediate visual representation via future scene prediction. The leader predicts Gaussian Splatting deformation caused by motions of the stabilizing arm, through which the follower generates the physical consequences resulted from the movement of the acting arm. As a result, our method significantly outperforms the current state-of-the-art bimanual manipulation techniques by an improvement of 20.2% in 10 simulated tasks, and achieves 60% success rate on average in 9 challenging real-world tasks. Our code is available at https://github.com/April-Yz/ManiGaussian_Bimanual.

cs.RO

Extending Cox Proportional Hazards Model with Symbolic Non-Linear Log-Risk Functions for Survival Analysis

The Cox proportional hazards (CPH) model has been widely applied in survival analysis to estimate relative risks across different subjects given multiple covariates. Traditional CPH models rely on a linear combination of covariates weighted with coefficients as the log-risk function, which imposes a strong and restrictive assumption, limiting generalization. Recent deep learning methods enable non-linear log-risk functions. However, they often lack interpretability due to the end-to-end training mechanisms. The implementation of Kolmogorov-Arnold Networks (KAN) offers new possibilities for extending the CPH model with fully transparent and symbolic non-linear log-risk functions. In this paper, we introduce Generalized Cox Proportional Hazards (GCPH) model, a novel method for survival analysis that leverages KAN to enable a non-linear mapping from covariates to survival outcomes in a fully symbolic manner. GCPH maintains the interpretability of traditional CPH models while allowing for the estimation of non-linear log-risk functions. Experiments conducted on both synthetic data and various public benchmarks demonstrate that GCPH achieves competitive performance in terms of prediction accuracy and exhibits superior interpretability compared to current state-of-the-art methods.

cs.LG

Rethinking Remaining Useful Life Prediction with Scarce Time Series Data: Regression under Indirect Supervision

Supervised time series prediction relies on directly measured target variables, but real-world use cases such as predicting remaining useful life (RUL) involve indirect supervision, where the target variable is labeled as a function of another dependent variable. Trending temporal regression techniques rely on sequential time series inputs to capture temporal patterns, requiring interpolation when dealing with sparsely and irregularly sampled covariates along the timeline. However, interpolation can introduce significant biases, particularly with highly scarce data. In this paper, we address the RUL prediction problem with data scarcity as time series regression under indirect supervision. We introduce a unified framework called parameterized static regression, which takes single data points as inputs for regression of target values, inherently handling data scarcity without requiring interpolation. The time dependency under indirect supervision is captured via a parametrical rectification (PR) process, approximating a parametric function during inference with historical posteriori estimates, following the same underlying distribution used for labeling during training. Additionally, we propose a novel batch training technique for tasks in indirect supervision to prevent overfitting and enhance efficiency. We evaluate our model on public benchmarks for RUL prediction with simulated data scarcity. Our method demonstrates competitive performance in prediction accuracy when dealing with highly scarce time series data.

cs.LG

Proactive Depot Discovery: A Generative Framework for Flexible Location-Routing

The Location-Routing Problem (LRP), which combines the challenges of facility (depot) locating and vehicle route planning, is critically constrained by the reliance on predefined depot candidates, limiting the solution space and potentially leading to suboptimal outcomes. Previous research on LRP without predefined depots is scant and predominantly relies on heuristic algorithms that iteratively attempt depot placements across a planar area. Such approaches lack the ability to proactively generate depot locations that meet specific geographic requirements, revealing a notable gap in current research landscape. To bridge this gap, we propose a data-driven generative DRL framework, designed to proactively generate depots for LRP without predefined depot candidates, solely based on customer requests data which include geographic and demand information. It can operate in two distinct modes: direct generation of exact depot locations, and the creation of a multivariate Gaussian distribution for flexible depots sampling. By extracting depots' geographic pattern from customer requests data, our approach can dynamically respond to logistical needs, identifying high-quality depot locations that further reduce total routing costs compared to traditional methods. Extensive experiments demonstrate that, for a same group of customer requests, compared with those depots identified through random attempts, our framework can proactively generate depots that lead to superior solution routes with lower routing cost. The implications of our framework potentially extend into real-world applications, particularly in emergency medical rescue and disaster relief logistics, where rapid establishment and adjustment of depot locations are paramount, showcasing its potential in addressing LRP for dynamic and unpredictable environments.

cs.LG

Recent Advances in End-to-End Simultaneous Speech Translation

Simultaneous speech translation (SimulST) is a demanding task that involves generating translations in real-time while continuously processing speech input. This paper offers a comprehensive overview of the recent developments in SimulST research, focusing on four major challenges. Firstly, the complexities associated with processing lengthy and continuous speech streams pose significant hurdles. Secondly, satisfying real-time requirements presents inherent difficulties due to the need for immediate translation output. Thirdly, striking a balance between translation quality and latency constraints remains a critical challenge. Finally, the scarcity of annotated data adds another layer of complexity to the task. Through our exploration of these challenges and the proposed solutions, we aim to provide valuable insights into the current landscape of SimulST research and suggest promising directions for future exploration.

cs.SD

A Mel Spectrogram Enhancement Paradigm Based on CWT in Speech Synthesis

Acoustic features play an important role in improving the quality of the synthesised speech. Currently, the Mel spectrogram is a widely employed acoustic feature in most acoustic models. However, due to the fine-grained loss caused by its Fourier transform process, the clarity of speech synthesised by Mel spectrogram is compromised in mutant signals. In order to obtain a more detailed Mel spectrogram, we propose a Mel spectrogram enhancement paradigm based on the continuous wavelet transform (CWT). This paradigm introduces an additional task: a more detailed wavelet spectrogram, which like the post-processing network takes as input the Mel spectrogram output by the decoder. We choose Tacotron2 and Fastspeech2 for experimental validation in order to test autoregressive (AR) and non-autoregressive (NAR) speech systems, respectively. The experimental results demonstrate that the speech synthesised using the model with the Mel spectrogram enhancement paradigm exhibits higher MOS, with an improvement of 0.14 and 0.09 compared to the baseline model, respectively. These findings provide some validation for the universality of the enhancement paradigm, as they demonstrate the success of the paradigm in different architectures.

cs.SD

Simul-Whisper: Attention-Guided Streaming Whisper with Truncation Detection

As a robust and large-scale multilingual speech recognition model, Whisper has demonstrated impressive results in many low-resource and out-of-distribution scenarios. However, its encoder-decoder structure hinders its application to streaming speech recognition. In this paper, we introduce Simul-Whisper, which uses the time alignment embedded in Whisper's cross-attention to guide auto-regressive decoding and achieve chunk-based streaming ASR without any fine-tuning of the pre-trained model. Furthermore, we observe the negative effect of the truncated words at the chunk boundaries on the decoding results and propose an integrate-and-fire-based truncation detection model to address this issue. Experiments on multiple languages and Whisper architectures show that Simul-Whisper achieves an average absolute word error rate degradation of only 1.46% at a chunk size of 1 second, which significantly outperforms the current state-of-the-art baseline.

cs.SD

Enhancing Unsupervised Anomaly Detection with Score-Guided Network

Anomaly detection plays a crucial role in various real-world applications, including healthcare and finance systems. Owing to the limited number of anomaly labels in these complex systems, unsupervised anomaly detection methods have attracted great attention in recent years. Two major challenges faced by the existing unsupervised methods are: (i) distinguishing between normal and abnormal data in the transition field, where normal and abnormal data are highly mixed together; (ii) defining an effective metric to maximize the gap between normal and abnormal data in a hypothesis space, which is built by a representation learner. To that end, this work proposes a novel scoring network with a score-guided regularization to learn and enlarge the anomaly score disparities between normal and abnormal data. With such score-guided strategy, the representation learner can gradually learn more informative representation during the model training stage, especially for the samples in the transition field. We next propose a score-guided autoencoder (SG-AE), incorporating the scoring network into an autoencoder framework for anomaly detection, as well as other three state-of-the-art models, to further demonstrate the effectiveness and transferability of the design. Extensive experiments on both synthetic and real-world datasets demonstrate the state-of-the-art performance of these score-guided models (SGMs).

cs.LG

Duality-Based Distributed Optimization With Communication Delays in Multi-Cluster Networks

In this work, we consider solving a distributed optimization problem (DOP) in a multi-agent network with multiple agent clusters. In each cluster, the agents manage separable cost functions composed of possibly non-smooth components and aim to achieve an agreement on a common decision of the cluster. The global cost function is considered as the sum of the individual cost functions associated with affine coupling constraints on the clusters' decisions. To solve this problem, the dual problem is formulated by the concept of Fenchel conjugate. Then an asynchronous distributed dual proximal gradient (Asyn-DDPG) algorithm is proposed based on a cluster-based partial and mixed consensus protocol, by which the agents are only required to communicate with their neighbors with communication delays. An ergodic convergence result is provided, and the feasibility of the proposed algorithm is verified by solving a social welfare optimization problem in the simulation.

math.OC

Distributed Nash Equilibrium Seeking in N-Cluster Games with Fully Uncoordinated Constant Step-Sizes

Distributed optimization and Nash equilibrium (NE) seeking problems have drawn much attention in the control community recently. This paper studies a class of non-cooperative games, known as N-cluster game, which subsumes both cooperative and non-cooperative nature among multiple agents in the two problems: solving distributed optimization problem within the cluster, while playing a non-cooperative game across the clusters. Moreover, we consider a partial-decision information game setup, i.e., the agents do not have direct access to other agents' decisions, and hence need to communicate with each other through a directed graph. To solve the N-cluster game problem, we propose a distributed NE seeking algorithm by a synthesis of consensus and gradient tracking. Unlike other existing discrete-time methods for N-cluster games where either a common step-size is publicly known by all agents or only known by agents from the same cluster, the proposed algorithm can work with fully uncoordinated constant step-sizes, which allows the agents (both within and across the clusters) to choose their own preferred step-sizes. We prove that all agents' decisions converge linearly to their corresponding NE so long as the largest step-size and the heterogeneity of the step-sizes are small. We verify the derived results through a numerical example in a Cournot competition game.

math.OC

A Proximal Linearization-based Decentralized Method for Nonconvex Problems with Nonlinear Constraints

Decentralized optimization for non-convex problems are now demanding by many emerging applications (e.g., smart grids, smart building, etc.). Though dramatic progress has been achieved in convex problems, the results for non-convex cases, especially with non-linear constraints, are still largely unexplored. This is mainly due to the challenges imposed by the non-linearity and non-convexity, which makes establishing the convergence conditions bewildered. This paper investigates decentralized optimization for a class of structured non-convex problems characterized by: (i) nonconvex global objective function (possibly nonsmooth) and (ii) coupled nonlinear constraints and local bounded convex constraints w.r.t. the agents. For such problems, a decentralized approach called Proximal Linearizationbased Decentralized Method (PLDM) is proposed. Different from the traditional (augmented) Lagrangian-based methods which usually require the exact (local) optima at each iteration, the proposed method leverages a proximal linearization-based technique to update the decision variables iteratively, which makes it computationally efficient and viable for the non-linear cases. Under some standard conditions, the PLDM global convergence and local convergence rate to the epsilon-critical points are studied based on the Kurdyka-Lojasiewicz property which holds for most analytical functions. Finally, the performance and efficacy of the method are illustrated through a numerical example and an application to multi-zone heating, ventilation and air-conditioning (HVAC) control.

math.OC

Optimal Network Charge for Peer-to-Peer Energy Trading: A Grid Perspective

Peer-to-peer (P2P) energy trading is a promising market scheme to accommodate the increasing distributed energy resources (DERs). However, how P2P to be integrated into the existing power systems remains to be investigated. In this paper, we apply network charge as a means for the grid operator to attribute transmission loss and ensure network constraints for empowering P2P transaction. The interaction between the grid operator and the prosumers is modeled as a Stackelberg game, which yields a bi-level optimization problem. We prove that the Stackelberg game admits an equilibrium network charge price. Besides, we propose a method to obtain the network charge price by converting the bi-level optimization into a single-level mixed-integer quadratic programming (MIQP), which can handle a reasonable scale of prosumers efficiently. Simulations on the IEEE bus systems show that the proposed optimal network charge is favorable as it can benefit both the grid operator and the prosumers for empowering the P2P market, and achieves near-optimal social welfare. Moreover, the results show that the presence of energy storage will make the prosumers more sensitive to the network charge price changes.

cs.CE

Social Profit Optimization with Demand Response Management in Electricity Market: A Multi-timescale Leader-following Approach

In the electricity market, it is quite common that the market participants make "selfish" strategies to harvest the maximum profits for themselves, which may cause the social benefit loss and impair the sustainability of the society in the long term. Regarding this issue, in this work, we will discuss how the social profit can be improved through strategic demand response (DR) management. Specifically, we explore two interaction mechanisms in the market: Nash equilibrium (NE) and Stackelberg equilibrium (SE) among utility companies (UCs) and user-UC interactions, respectively. At the user side, each user determines the optimal energy-purchasing strategy to maximize its own profit. At the UC side, a governmental UC (g-UC) is considered, who aims to optimize the social profit of the market. Meanwhile, normal UCs play games to maximize their own profits. As a result, a basic leader-following problem among the UCs is formulated under the coordination of the independent system operator (ISO). Moreover, by using our proposed demand function amelioration (DFA) strategy, a multi-timescale leader-following problem is formulated. In this case, the maximal market efficiency can be achieved without changing the "selfish instinct" of normal UCs. In addition, by considering the local constraints for the UCs, two projection-based pricing algorithms are proposed for UCs, which can provide approximate optimal solutions for the resulting non-convex social profit optimization problems. The feasibility of the proposed algorithms is verified by using the concept of price of anarchy (PoA) in a multi-UC multi-user market model in the simulation.

eess.SY

Distributed Optimization with Coupling Constraints Based on Dual Proximal Gradient Method in Multi-Agent Networks

In this paper, we aim to solve a distributed optimization problem with affine coupling constraints in a multi-agent network, where the cost function of the agents is composed of smooth and possibly non-smooth parts. To solve this problem, we resort to the dual problem by deriving the Fenchel conjugate, resulting in a consensus-based constrained optimization problem. Then, we propose a distributed dual proximal gradient algorithm, where the agents make decisions based on the information of immediate neighbors. Provided that the non-smooth parts in the primal cost functions are with some simple structures, we only need to update dual variables by some simple operations, by which the overall computational complexity can be reduced. An ergodic convergence rate of the proposed algorithm is derived and the feasibility is numerically verified by solving a social welfare optimization problem in the electricity market.

math.OC