arXiv subjects

Hongxing Wang

Publications and source records attributed to Hongxing Wang.

15 recordsLinked to original sources

arXiv2026

How Much Dense Attention is Necessary? Oracle-Guided Sparse Prefill for Full/GQA Layers in Hybrid Long-Context Models

Long-context prefill remains expensive because full/GQA layers still score the historical sequence, even in hybrid models with local, sparse, linear, or recurrent components. We study how much dense attention is needed to preserve task-level behavior under explicit support granularity and top-k budgets. We introduce an attention-mass top-k oracle for existing GQA checkpoints: for each layer and query position, it computes dense attention, selects head-averaged token support, and recomputes attention only on that support. The oracle is a diagnostic reference, not a deployable accelerator, and separates sparse-budget feasibility from indexer error and runtime realization effects. On Qwen-family retrieval-heavy evaluations, the longest per-query oracle rows stay within 1 point of dense, and a Qwen3.5-9B RULER-style sweep from 4K to 100K stays within 0.48 points. Guided by the oracle, we derive a head-collapsed auxiliary indexer trained by KL distillation from dense attention-mass distributions while keeping the backbone frozen. With separately distilled Qwen3.5-0.8B and Qwen3.5-9B indexers, the reported 16K/32K validation macro gaps are +2.04 and +1.13 points, treated as quality preservation rather than improvement; fused selection-block-shared support can introduce a larger realization gap. Preliminary single-card TTFT measurements show distilled-indexer sparse serving speedups of 1.71x for Qwen3.5-0.8B on NPU and 1.93x for Qwen3.5-9B on GPU against its dense FlashAttention-2 baseline. Additional random-init stress rows reach 3.44x, indicating sparse-runtime headroom but not validated output quality. This first release separates oracle feasibility, distilled-indexer quality, and runtime headroom, leaving a fully matched quality-latency frontier to future work.

cs.LG↗

arXiv2025

Aligning Instance-Semantic Sparse Representation towards Unsupervised Object Segmentation and Shape Abstraction with Repeatable Primitives

Understanding 3D object shapes necessitates shape representation by object parts abstracted from results of instance and semantic segmentation. Promising shape representations enable computers to interpret a shape with meaningful parts and identify their repeatability. However, supervised shape representations depend on costly annotation efforts, while current unsupervised methods work under strong semantic priors and involve multi-stage training, thereby limiting their generalization and deployment in shape reasoning and understanding. Driven by the tendency of high-dimensional semantically similar features to lie in or near low-dimensional subspaces, we introduce a one-stage, fully unsupervised framework towards semantic-aware shape representation. This framework produces joint instance segmentation, semantic segmentation, and shape abstraction through sparse representation and feature alignment of object parts in a high-dimensional space. For sparse representation, we devise a sparse latent membership pursuit method that models each object part feature as a sparse convex combination of point features at either the semantic or instance level, promoting part features in the same subspace to exhibit similar semantics. For feature alignment, we customize an attention-based strategy in the feature space to align instance- and semantic-level object part features and reconstruct the input shape using both of them, ensuring geometric reusability and semantic consistency of object parts. To firm up semantic disambiguation, we construct cascade unfrozen learning on geometric parameters of object parts.

cs.CV↗

arXiv2024

Modality-Aware Shot Relating and Comparing for Video Scene Detection

Video scene detection involves assessing whether each shot and its surroundings belong to the same scene. Achieving this requires meticulously correlating multi-modal cues, $\it{e.g.}$ visual entity and place modalities, among shots and comparing semantic changes around each shot. However, most methods treat multi-modal semantics equally and do not examine contextual differences between the two sides of a shot, leading to sub-optimal detection performance. In this paper, we propose the $\bf{M}$odality-$\bf{A}$ware $\bf{S}$hot $\bf{R}$elating and $\bf{C}$omparing approach (MASRC), which enables relating shots per their own characteristics of visual entity and place modalities, as well as comparing multi-shots similarities to have scene changes explicitly encoded. Specifically, to fully harness the potential of visual entity and place modalities in modeling shot relations, we mine long-term shot correlations from entity semantics while simultaneously revealing short-term shot correlations from place semantics. In this way, we can learn distinctive shot features that consolidate coherence within scenes and amplify distinguishability across scenes. Once equipped with distinctive shot features, we further encode the relations between preceding and succeeding shots of each target shot by similarity convolution, aiding in the identification of scene ending shots. We validate the broad applicability of the proposed components in MASRC. Extensive experimental results on public benchmark datasets demonstrate that the proposed MASRC significantly advances video scene detection.

cs.CV↗

arXiv2024

STAMPsy: Towards SpatioTemporal-Aware Mixed-Type Dialogues for Psychological Counseling

Online psychological counseling dialogue systems are trending, offering a convenient and accessible alternative to traditional in-person therapy. However, existing psychological counseling dialogue systems mainly focus on basic empathetic dialogue or QA with minimal professional knowledge and without goal guidance. In many real-world counseling scenarios, clients often seek multi-type help, such as diagnosis, consultation, therapy, console, and common questions, but existing dialogue systems struggle to combine different dialogue types naturally. In this paper, we identify this challenge as how to construct mixed-type dialogue systems for psychological counseling that enable clients to clarify their goals before proceeding with counseling. To mitigate the challenge, we collect a mixed-type counseling dialogues corpus termed STAMPsy, covering five dialogue types, task-oriented dialogue for diagnosis, knowledge-grounded dialogue, conversational recommendation, empathetic dialogue, and question answering, over 5,000 conversations. Moreover, spatiotemporal-aware knowledge enables systems to have world awareness and has been proven to affect one's mental health. Therefore, we link dialogues in STAMPsy to spatiotemporal state and propose a spatiotemporal-aware mixed-type psychological counseling dataset. Additionally, we build baselines on STAMPsy and develop an iterative self-feedback psychological dialogue generation framework, named Self-STAMPsy. Results indicate that clarifying dialogue goals in advance and utilizing spatiotemporal states are effective.

cs.AI↗

arXiv2024

Weak Dual Drazin Inverse and its Characterizations and Properties

The dual Drazin inverse is an important dual generalized inverse. In this paper, to extend it we introduce the weak dual Drazin inverse which is unique and exists for any square dual matrix. When the dual Drazin inverse exists, it coincides with the weak dual Drazin inverse. In addition, we introduce the weak dual group inverse and apply it to studying one type restricted dual matrix equation.

math.RA↗

arXiv2024

Beam dynamics induced by the quantum metric of exceptional rings

Topological physics has broadened its scope from the study of topological insulating phases to include nodal phases containing band structure singularities. The geometry of the corresponding quantum states is described by the quantum metric which provides a theoretical framework for explaining phenomena that conventional approaches fail to address. The field has become even broader by encompassing non-Hermitian singularities: in addition to Dirac, Weyl nodes, or nodal lines, it is now common to encounter exceptional points, exceptional or Weyl rings, and even Weyl spheres. They give access to fascinating effects that cannot be reached within the Hermitian picture. However, the quantum geometry of non-Hermitian singularities is not a straightforward extension of the Hermitian one, remaining far less understood. Here, we study experimentally and theoretically the dynamics of wave packets at exceptional rings stemming from Dirac points in a photonic honeycomb lattice. First, we demonstrate a transition between conical diffraction and non-Hermitian broadening in real space. Next, we predict and demonstrate a new non-Hermitian effect in the reciprocal space, induced by the non-orthogonality of the eigenstates. We call it transverse non-Hermitian drift, and its description requires biorthogonal quantum metric. The non-Hermitian drift can be used for applications in beam steering.

cond-mat.mes-hall↗

arXiv2024

DualStreamFoveaNet: A Dual Stream Fusion Architecture with Anatomical Awareness for Robust Fovea Localization

Accurate fovea localization is essential for analyzing retinal diseases to prevent irreversible vision loss. While current deep learning-based methods outperform traditional ones, they still face challenges such as the lack of local anatomical landmarks around the fovea, the inability to robustly handle diseased retinal images, and the variations in image conditions. In this paper, we propose a novel transformer-based architecture called DualStreamFoveaNet (DSFN) for multi-cue fusion. This architecture explicitly incorporates long-range connections and global features using retina and vessel distributions for robust fovea localization. We introduce a spatial attention mechanism in the dual-stream encoder to extract and fuse self-learned anatomical information, focusing more on features distributed along blood vessels and significantly reducing computational costs by decreasing token numbers. Our extensive experiments show that the proposed architecture achieves state-of-the-art performance on two public datasets and one large-scale private dataset. Furthermore, we demonstrate that the DSFN is more robust on both normal and diseased retina images and has better generalization capacity in cross-dataset experiments.

cs.CV↗

arXiv2024

Dual minus partial order

In this paper, we introduce the Dual-minus partial order, get some characterizations of the partial order, and prove that both the dual star partial order and the dual sharp partial order are Dual-minus-type partial orders. Based on the Dual-minus partial order, we introduce the Dual-minus sharp partial order and the Dual-minus star partial order, which are also Dual-minus-type partial orders. In addition, we discuss relationships among the Dual-minus sharp partial order, the D-sharp partial order and the G-sharp partial order(the Dual-minus star partial order, the D-star partial order and the P-star partial order).

math.RA↗

arXiv2022

Characterizations and properties of dual matrix star orders

In this paper, we introduce D-star order, T-star order and P-star order on the class of dual matrices. By applying matrix decomposition and dual generalized inverses, we discuss properties, characterizations and relations among these orders, and illustrate their relations with examples.

math.RA↗

arXiv2022

The Dual Index and Dual Core Generalized Inverse

In this paper, we introduce the dual index and dual core generalized inverse (DCGI). By applying rank equation, generalized inverse and matrix decomposition, we give several characterizations of the dual index when it is equal to one. And we get that if DCGI exists, then it is unique. We derive a compact formula for DCGI and a series of equivalent characterizations of the existence of the inverse. It is worth nothing that the dual index of $\hat{M}$ is equal to one if and only if its DCGI exists. When the dual index of $\hat{M}$ is equal to one, we study dual Moore-Penrose generalized inverse (DMPGI) and dual group generalized inverse (DGGI), and consider the relationships among DCGI, DMPGI, DGGI, Moore-Penrose dual generalized inverse (MPDGI) and other dual generalized inverses. In addition, we consider symmetric dual matrix and its dual generalized inverses. At last, two examples are given to illustrate the application of DCGI in linear dual equations.

math.RA↗

arXiv2022

Dual $r$-Rank Decomposition and Its Applications

In this paper, we introduce the dual $r$-rank decomposition of dual matrix, get its existence condition and equivalent form of the decomposition, as well as derive some characterizations of dual Moore-Penrose generalized inverse(DMPGI). Based on DMPGI, we introduce one special dual matrix(dual EP matrix). By applying the dual $r$-rank decomposition we derive several characterizations of dual EP matrix, dual idempotent matrix, dual generalized inverses, and the relationships among dual Penrose equations.

math.RA↗

arXiv2022

The m-core-EP inverse in Minkowski space

In this paper, we introduce the m-core-EP inverse in Minkowski space, consider its properties, and get several sufficient and necessary conditions for the existence of the m-core-EP inverse. We give the m-core-EP decomposition in Minkowski space, and note that not every square matrix has the decomposition. Furthermore, by applying the m-core-EP inverse and the m-core-EP decomposition, we introduce the m-core-EP order and give some characterizations of it.

math.FA↗

arXiv2019

The core inverse and constrained matrix approximation problem

In this paper,we study the constrained matrix approximation problem in the Frobenius norm by using the core inverse:\begin{align}\nonumber \left\|{Mx - b} \right\|_F=\min\ \ {\rm subject\ to} \ \ {x\in\mathcal{R}(M)} ,\end{align} where $M\in\mathbb{C}^{\texttt{CM}}_n$. We get the unique solution to the problem, provide two Cramer's rules for the unique solution, and establish two new expressions for the core inverse.

math.RA↗

arXiv2017

Weak group inverse

In this paper, we introduce a weak group inverse (called the WG inverse in the present paper) for square matrices of an arbitrary index, and give some of its characterizations and properties. Furthermore, we introduce two orders: one is a pre-order and the other is a partial order, and derive several characterizations of the two orders. At last, one characterization of the core-EP order is derived by using the WG inverses.

math.RA↗

arXiv2016

Core-EP Decomposition and its Applications

In this paper, we introduce the notion of the core-EP decomposition and some of its properties. By using the decomposition, we derive several characterizations of the core-EP inverse, introduce a pre-order(i.e. the core-EP order) and a partial order(i.e. the core-minus partial order), and characterize the properties of both orders.

math.RA↗