arXiv subjects

Hongyuan Wang

Publications and source records attributed to Hongyuan Wang.

At least 19 recordsLinked to original sources

arXiv2026

StepAudio 3 Realtime Technical Report

Realtime spoken interaction demands deep reasoning, prompt responses, and fluid turn-taking. We present StepAudio 3 Realtime, an audio-language foundation model organized around a continuous listen-converse-think-act loop. Deep Perception captures rich acoustic cues to interpret user intent, while Seamless Duplex models synchronized audio streams to handle pauses, backchannels, and interruptions naturally. Crucially, we resolve the tension between deep deliberation and latency via Think-While-Speaking, executing private reasoning in parallel with spoken delivery. In reasoning mode, StepAudio 3 reaches a 73.0 macro average on StepAudioChat. With Think-While-Speaking, it achieves dialogue and reasoning performance comparable to dedicated reasoning models while speaking in real time. Furthermore, an integrated Voice Agent handles asynchronous tool execution without disrupting the dialogue flow. StepAudio 3 Realtime achieves top-tier performance across key dimensions: an exceptional 90.6 on the MMSU benchmark, 98.9 Overall on the Artificial Analysis Full-Duplex Bench, and a 56.0% macro task-success rate on $τ$-Voice.

cs.SD↗

arXiv2026

Towards Universal Physical Adversarial Attacks via a Joint Multi-Objective and Multi-Model Optimization Framework

Physical adversarial attacks often overfit single surrogate models and optimization objectives. While ensemble attacks can mitigate this, existing methods struggle with severe gradient conflicts within restricted physical texture spaces, significantly degrading cross-model transferability. To bridge this gap, this paper proposes a Joint Multi-Objective and Multi-Model Optimization Framework (JMOF) that leverages quantitative similarity analysis to select the optimal surrogate model ensemble. Within JMOF, a dual-level mechanism jointly suppresses prediction outputs and flattens intermediate feature distributions, balancing attack efficiency with deep generalization. Additionally, an Orthogonal Gradient Alignment (OGA) strategy resolves cross-model gradient conflicts, transforming mutually repulsive gradients into synergistic optimization directions. Extensive simulated and real-world experiments demonstrate that JMOF outperforms state-of-the-art baselines against diverse black-box detectors. Crucially, JMOF exhibits substantial cross-vision-task generalization, generating attacks capable of simultaneously deceiving object detection and semantic segmentation or monocular depth estimation models. This research advances the generalization limits of physical adversarial attacks, providing a robust framework for evaluating visual AI vulnerabilities in real-world deployments.

cs.CV↗

arXiv2026

ViStoryBench: Comprehensive Benchmark Suite for Story Visualization

Story visualization aims to generate coherent image sequences that faithfully represent a narrative and match given character references. Despite progress in generative models, existing benchmarks remain narrow in scope, often limited to short prompts, lacking character references, or single-image cases, failing to reflect real-world narrative complexity and obscuring true model performance.We introduce ViStoryBench, a comprehensive benchmark designed to evaluate story visualization models across varied narrative structures, visual styles, and character settings. It features richly annotated multi-shot scripts derived from curated stories spanning literature, film, and folklore. Large language models assist in story summarization and script generation, with all outputs verified by humans for coherence and fidelity. Character references are carefully curated to maintain consistency across different artistic styles. ViStoryBench proposes a suite of multi-dimensional automated metrics to evaluate character consistency, style similarity, prompt alignment, aesthetic quality, and artifacts like copy-paste behavior. These metrics are validated through human studies and used to assess a broad range of open-source and commercial models, enabling systematic analysis and encouraging advances in visual storytelling.

cs.CV↗

arXiv2026

Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters

We introduce Step 3.5 Flash, a sparse Mixture-of-Experts (MoE) model that bridges frontier-level agentic intelligence and computational efficiency. We focus on what matters most when building agents: sharp reasoning and fast, reliable execution. Step 3.5 Flash pairs a 196B-parameter foundation with 11B active parameters for efficient inference. It is optimized with interleaved 3:1 sliding-window/full attention and Multi-Token Prediction (MTP-3) to reduce the latency and cost of multi-round agentic interactions. To reach frontier-level intelligence, we design a scalable reinforcement learning framework that combines verifiable signals with preference feedback, while remaining stable under large-scale off-policy training, enabling consistent self-improvement across mathematics, code, and tool use. Step 3.5 Flash demonstrates strong performance across agent, coding, and math tasks, achieving 85.4% on IMO-AnswerBench, 86.4% on LiveCodeBench-v6 (2024.08-2025.05), 88.2% on tau2-Bench, 69.0% on BrowseComp (with context management), and 51.0% on Terminal-Bench 2.0, comparable to frontier models such as GPT-5.2 xHigh and Gemini 3.0 Pro. By redefining the efficiency frontier, Step 3.5 Flash provides a high-density foundation for deploying sophisticated agents in real-world industrial environments.

cs.CL↗

arXiv2025

Step-DeepResearch Technical Report

As LLMs shift toward autonomous agents, Deep Research has emerged as a pivotal metric. However, existing academic benchmarks like BrowseComp often fail to meet real-world demands for open-ended research, which requires robust skills in intent recognition, long-horizon decision-making, and cross-source verification. To address this, we introduce Step-DeepResearch, a cost-effective, end-to-end agent. We propose a Data Synthesis Strategy Based on Atomic Capabilities to reinforce planning and report writing, combined with a progressive training path from agentic mid-training to SFT and RL. Enhanced by a Checklist-style Judger, this approach significantly improves robustness. Furthermore, to bridge the evaluation gap in the Chinese domain, we establish ADR-Bench for realistic deep research scenarios. Experimental results show that Step-DeepResearch (32B) scores 61.4% on Scale AI Research Rubrics. On ADR-Bench, it significantly outperforms comparable models and rivals SOTA closed-source models like OpenAI and Gemini DeepResearch. These findings prove that refined training enables medium-sized models to achieve expert-level capabilities at industry-leading cost-efficiency.

cs.CL↗

arXiv2025

Step-3 is Large yet Affordable: Model-system Co-design for Cost-effective Decoding

Large language models (LLMs) face low hardware efficiency during decoding, especially for long-context reasoning tasks. This paper introduces Step-3, a 321B-parameter VLM with hardware-aware model-system co-design optimized for minimizing decoding costs. Step-3 innovates in two key dimensions: (1) A novel Multi-Matrix Factorization Attention (MFA) mechanism that significantly reduces both KV cache size and computation while maintaining high attention expressiveness, and (2) Attention-FFN Disaggregation (AFD), a distributed inference system that decouples attention and Feed-Forward Network (FFN) layers into specialized subsystems. This co-design achieves unprecedented cost efficiency: Step-3 significantly reduces theoretical decoding costs compared with models like DeepSeek-V3 and Qwen3 MoE 235B, with the gains widening at longer context. Step-3 achieves low cost while activating 38B parameters per token (more than DeepSeek-V3 and Qwen3 MoE 235B), demonstrating that hardware-aligned attention arithmetic intensity, MoE sparsity, and AFD are critical to cost-effectiveness. We perform a head-to-head comparison with DeepSeek-V3 in its favorable scenarios. Our implementation on Hopper GPUs achieves a decoding throughput of up to 4,039 tokens per second per GPU under 50ms TPOT SLA (4K context, FP8, no MTP). It is higher than DeepSeek-V3's 2,324 in the same setup and sets a new Pareto frontier for LLM decoding.

cs.LG↗

arXiv2025

Step-Audio-AQAA: a Fully End-to-End Expressive Large Audio Language Model

Large Audio-Language Models (LALMs) have significantly advanced intelligent human-computer interaction, yet their reliance on text-based outputs limits their ability to generate natural speech responses directly, hindering seamless audio interactions. To address this, we introduce Step-Audio-AQAA, a fully end-to-end LALM designed for Audio Query-Audio Answer (AQAA) tasks. The model integrates a dual-codebook audio tokenizer for linguistic and semantic feature extraction, a 130-billion-parameter backbone LLM and a neural vocoder for high-fidelity speech synthesis. Our post-training approach employs interleaved token-output of text and audio to enhance semantic coherence and combines Direct Preference Optimization (DPO) with model merge to improve performance. Evaluations on the StepEval-Audio-360 benchmark demonstrate that Step-Audio-AQAA excels especially in speech control, outperforming the state-of-art LALMs in key areas. This work contributes a promising solution for end-to-end LALMs and highlights the critical role of token-based vocoder in enhancing overall performance for AQAA tasks.

cs.SD↗

arXiv2025

Complementary Advantages: Exploiting Cross-Field Frequency Correlation for NIR-Assisted Image Denoising

Existing single-image denoising algorithms often struggle to restore details when dealing with complex noisy images. The introduction of near-infrared (NIR) images offers new possibilities for RGB image denoising. However, due to the inconsistency between NIR and RGB images, the existing works still struggle to balance the contributions of two fields in the process of image fusion. In response to this, in this paper, we develop a cross-field Frequency Correlation Exploiting Network (FCENet) for NIR-assisted image denoising. We first propose the frequency correlation prior based on an in-depth statistical frequency analysis of NIR-RGB image pairs. The prior reveals the complementary correlation of NIR and RGB images in the frequency domain. Leveraging frequency correlation prior, we then establish a frequency learning framework composed of Frequency Dynamic Selection Mechanism (FDSM) and Frequency Exhaustive Fusion Mechanism (FEFM). FDSM dynamically selects complementary information from NIR and RGB images in the frequency domain, and FEFM strengthens the control of common and differential features during the fusion process of NIR and RGB features. Extensive experiments on simulated and real data validate that the proposed method outperforms other state-of-the-art methods. The code will be released at https://github.com/yuchenwang815/FCENet.

cs.CV↗

arXiv2025

Step-Audio: Unified Understanding and Generation in Intelligent Speech Interaction

Real-time speech interaction, serving as a fundamental interface for human-machine collaboration, holds immense potential. However, current open-source models face limitations such as high costs in voice data collection, weakness in dynamic control, and limited intelligence. To address these challenges, this paper introduces Step-Audio, the first production-ready open-source solution. Key contributions include: 1) a 130B-parameter unified speech-text multi-modal model that achieves unified understanding and generation, with the Step-Audio-Chat version open-sourced; 2) a generative speech data engine that establishes an affordable voice cloning framework and produces the open-sourced lightweight Step-Audio-TTS-3B model through distillation; 3) an instruction-driven fine control system enabling dynamic adjustments across dialects, emotions, singing, and RAP; 4) an enhanced cognitive architecture augmented with tool calling and role-playing abilities to manage complex tasks effectively. Based on our new StepEval-Audio-360 evaluation benchmark, Step-Audio achieves state-of-the-art performance in human evaluations, especially in terms of instruction following. On open-source benchmarks like LLaMA Question, shows 9.3% average performance improvement, demonstrating our commitment to advancing the development of open-source multi-modal language technologies. Our code and models are available at https://github.com/stepfun-ai/Step-Audio.

cs.CL↗

arXiv2024

Learning Exhaustive Correlation for Spectral Super-Resolution: Where Spatial-Spectral Attention Meets Linear Dependence

Spectral super-resolution that aims to recover hyperspectral image (HSI) from easily obtainable RGB image has drawn increasing interest in the field of computational photography. The crucial aspect of spectral super-resolution lies in exploiting the correlation within HSIs. However, two types of bottlenecks in existing Transformers limit performance improvement and practical applications. First, existing Transformers often separately emphasize either spatial-wise or spectral-wise correlation, disrupting the 3D features of HSI and hindering the exploitation of unified spatial-spectral correlation. Second, existing self-attention mechanism always establishes full-rank correlation matrix by learning the correlation between pairs of tokens, leading to its inability to describe linear dependence widely existing in HSI among multiple tokens. To address these issues, we propose a novel Exhaustive Correlation Transformer (ECT) for spectral super-resolution. First, we propose a Spectral-wise Discontinuous 3D (SD3D) splitting strategy, which models unified spatial-spectral correlation by integrating spatial-wise continuous splitting strategy and spectral-wise discontinuous splitting strategy. Second, we propose a Dynamic Low-Rank Mapping (DLRM) model, which captures linear dependence among multiple tokens through a dynamically calculated low-rank dependence map. By integrating unified spatial-spectral attention and linear dependence, our ECT can model exhaustive correlation within HSI. The experimental results on both simulated and real data indicate that our method achieves state-of-the-art performance. Codes and pretrained models will be available later.

eess.IV↗

arXiv2022

Spacecraft depth completion based on the gray image and the sparse depth map

Perceiving the three-dimensional (3D) structure of the spacecraft is a prerequisite for successfully executing many on-orbit space missions, and it can provide critical input for many downstream vision algorithms. In this paper, we propose to sense the 3D structure of spacecraft using light detection and ranging sensor (LIDAR) and a monocular camera. To this end, Spacecraft Depth Completion Network (SDCNet) is proposed to recover the dense depth map based on gray image and sparse depth map. Specifically, SDCNet decomposes the object-level spacecraft depth completion task into foreground segmentation subtask and foreground depth completion subtask, which segments the spacecraft region first and then performs depth completion on the segmented foreground area. In this way, the background interference to foreground spacecraft depth completion is effectively avoided. Moreover, an attention-based feature fusion module is also proposed to aggregate the complementary information between different inputs, which deduces the correlation between different features along the channel and the spatial dimension sequentially. Besides, four metrics are also proposed to evaluate object-level depth completion performance, which can more intuitively reflect the quality of spacecraft depth completion results. Finally, a large-scale satellite depth completion dataset is constructed for training and testing spacecraft depth completion algorithms. Empirical experiments on the dataset demonstrate the effectiveness of the proposed SDCNet, which achieves 0.25m mean absolute error of interest and 0.759m mean absolute truncation error, surpassing state-of-the-art methods by a large margin. The spacecraft pose estimation experiment is also conducted based on the depth completion results, and the experimental results indicate that the predicted dense depth map could meet the needs of downstream vision tasks.

cs.CV↗

arXiv2022

SHREC'22 Track: Sketch-Based 3D Shape Retrieval in the Wild

Sketch-based 3D shape retrieval (SBSR) is an important yet challenging task, which has drawn more and more attention in recent years. Existing approaches address the problem in a restricted setting, without appropriately simulating real application scenarios. To mimic the realistic setting, in this track, we adopt large-scale sketches drawn by amateurs of different levels of drawing skills, as well as a variety of 3D shapes including not only CAD models but also models scanned from real objects. We define two SBSR tasks and construct two benchmarks consisting of more than 46,000 CAD models, 1,700 realistic models, and 145,000 sketches in total. Four teams participated in this track and submitted 15 runs for the two tasks, evaluated by 7 commonly-adopted metrics. We hope that, the benchmarks, the comparative results, and the open-sourced evaluation code will foster future research in this direction among the 3D object retrieval community.

cs.CV↗

arXiv2021

Multi-Features Guidance Network for partial-to-partial point cloud registration

To eliminate the problems of large dimensional differences, big semantic gap, and mutual interference caused by hybrid features, in this paper, we propose a novel Multi-Features Guidance Network for partial-to-partial point cloud registration(MFG). The proposed network mainly includes four parts: keypoints' feature extraction, correspondences searching, correspondences credibility computation, and SVD, among which correspondences searching and correspondence credibility computation are the cores of the network. Unlike the previous work, we utilize the shape features and the spatial coordinates to guide correspondences search independently and fusing the matching results to obtain the final matching matrix. In the correspondences credibility computation module, based on the conflicted relationship between the features matching matrix and the coordinates matching matrix, we score the reliability for each correspondence, which can reduce the impact of mismatched or non-matched points. Experimental results show that our network outperforms the current state-of-the-art while maintaining computational efficiency.

cs.CV↗

arXiv2021

Multiple magnetic topological phases in bulk van der Waals crystal MnSb4Te7

The magnetic van der Waals crystals MnBi2Te4/(Bi2Te3)n have drawn significant attention due to their rich topological properties and the tunability by external magnetic field. Although the MnBi2Te4/(Bi2Te3)n family have been intensively studied in the past few years, their close relatives, the MnSb2Te4/(Sb2Te3)n family, remain much less explored. In this work, combining magnetotransport measurements, angle-resolved photoemission spectroscopy, and first principles calculations, we find that MnSb4Te7, the n = 1 member of the MnSb2Te4/(Sb2Te3)n family, is a magnetic topological system with versatile topological phases which can be manipulated by both carrier doping and magnetic field. Our calculations unveil that its A-type antiferromagnetic (AFM) ground state stays in a Z_2 AFM topological insulator phase, which can be converted to an inversion-symmetry-protected axion insulator phase when in the ferromagnetic (FM) state. Moreover, when this system in the FM phase is slightly carrier doped on either the electron or hole side, it becomes a Weyl semimetal with multiple Weyl nodes in the highest valence bands and lowest conduction bands, which are manifested by the measured notable anomalous Hall effect. Our work thus introduces a new magnetic topological material with different topological phases which are highly tunable by carrier doping or magnetic field.

cond-mat.str-el↗

arXiv2020

High-Pressure Crystal Growth, Superconducting Properties, and Electronic Band Structure of Nb2P5

Orthorhombic (space group: Pnma) Nb2P5 is a high-pressure phase that is quenchable to ambient pressure, which could viewed as the zigzag infinite P chain-inserted NbP2. We report herein the high-pressure crystal growth of Nb2P5 and the discovery of its superconducting transition at Tc ~ 2.6 K. The electrical resistivity, magnetization, and specific heat capacity measurements on the high-quality crystal unveiled a conventional type-II weakly coupled s-wave nature of the superconductivity, with the upper critical field Hc2(0) ~ 0.5 T, the electron-phonon coupling strength λep ~ 0.5 - 0.8, and the Ginzburg-Landau parameter \k{appa} ~ 100. The ab initio calculations on the electronic band structure unveiled nodal-line structures protected by different symmetries. The one caused by band inversion, for example, on the Γ-X and U-R paths of the Brillouin zone, likely could bring nontrivial topology and hence possible nontrivial surface state on the surface. The surface states on the (100), (010) and (110) surfaces were also calculated and discussed. The discovery of the phosphorus-rich Nb2P5 superconductor would be instructive for the design of more metal phosphides superconductors which might host unconventional superconductivity or potential technical applications.

cond-mat.supr-con↗

arXiv2020

An Improved Distributed Nonlinear Observer for Leader-Following Consensus Via Differential Geometry Approach

This paper is concerned with the leader-following output consensus problem in the framework of distributed nonlinear observers. In stead of certain hypotheses on the leader system, a group of geometric conditions is put forward to develop a novel distributed observer strategy with less conservatism, thereby definitely improving the applicability of the existing results. To be more specific, the improved distributed observer can precisely handle consensus problems for some nonlinear leader systems which are invalid for the traditional strategies with the certain assumption, such as Elastic Shaft Single Linkage Manipulator (ESSLM) systems and most of first-order nonlinear systems. We prove the sufficient conditions for the exponential stability of our distributed observer's error dynamic by proposing two pioneered lemmas to show the relationship between the maximum eigenvalues of two matrices appearing in Lyapunov type matrices. Then, a partial feedback linearization method with zero dynamic proposed in differential geometry is employed to design a purely decentralized control law for the affine nonlinear multi-agent system. With this advancement, the existing results can be regarded as a specific case owing to that the followers can be chosen as an arbitrary minimum phase affine smooth nonlinear system. At last, the novel distributed observer and the improved purely decentralized control law are applied in the distributed control framework to construct a closed-loop system. We also prove the stability of closed-loop system to achieve leader-following consensus, i.e., the distributed control framework is proved to satisfy certainty equivalence principle. Our method is illustrated by ESSLM system and Van der Pol system as leader.

math.OC↗

arXiv2020

Non-Coulomb strong electron-hole binding in $Ta_2NiSe_5$ revealed by time- and angle-resolved photoemission spectroscopy

We reveal an ultrafast purely electronic phase transition in \TNS, which is a plausible excitonic insulator, after excited by an ultrafast infrared laser pulse. Specifically, the order parameter of the strong electron-hole binding shrinks with enhancing the pump pulse, and above a critical pump fluence, a photo-excited semimetallic state is experimentally identified with the absence of ultrafast structural transition. In addition, the bare valence and conduction bands and also the effective exciton binding energy in \TNS~are determined. These findings and detailed analysis suggest a bare nonequilibrium semimetallic phase in $Ta_2NiSe_5$ and the strong electron-hole binding cannot be exclusively driven by Coulomb interaction.

cond-mat.str-el↗

arXiv2019

Magnetotransport properties of the layered CaAl2Si2 semimetal hosting multiple nontrivial topological states

Combination of different nontrivial topological states in a single material is capable of realizing multiple functionalities and exotic physics, but such materials are still very sparse. We report herein the results of magnetotransport measurements and ab initio calculations on single crystalline CaAl2Si2 semimetal. The transport properties could be well understood in connection with the two-band model, agreeing well with the theoretical calculations indicating four main sheets of Fermi surface consisting of three hole pockets centered at the Γ point and one electron pocket centered at the M point in the Brillouin zone. The single fundamental frequency imposed in the quantum oscillations of magnetoresistance corresponds to the electron Fermi pocket. Without spin-orbit coupling (SOC), the ab initio calculations suggest CaAl2Si2 as a system hosting a topological nodal-line setting around the Γ point in the Brillouin zone close to the Fermi level. Once including the SOC, the fragile nodal-line will be gapped and a pair of Dirac points emerge along the high symmetric Γ-A direction, which is about 1.22 eV below the Fermi level. The SOC can also induce a topological insulator state along the Γ-A direction with a gap of about 3 meV. The results demonstrate CaAl2Si2 as an excellent platform for the study of novel topological physics with multiple topological states.

cond-mat.str-el↗