arXiv subjects

Haibin Chen

Publications and source records attributed to Haibin Chen.

At least 19 recordsLinked to original sources

arXiv2026

A Homogeneous Tensor Framework for High-Order Trust-Region and Spherical Polynomial Optimization

High-order methods can improve worst-case evaluation complexity, but for orders $p\geq3$ their Taylor subproblems are nonconvex polynomial optimization problems and are generally difficult to solve. We develop a radius-controlled boundary approach based on homogeneous tensor representations. By augmenting the step with a constant coordinate, any $p$th-order Taylor polynomial can be represented exactly as an order-$p$ homogeneous tensor form; at a prescribed radius, the boundary model is a spherical polynomial optimization problem. The representation applies to arbitrary $p$, while the algorithmic development focuses on the cubic case $p=3$. For an inhomogeneous cubic on the sphere, we introduce a quadratic shift and prove, under an explicit shift bound, equivalence with a three-block multilinear formulation at global optimality. This motivates a proximal alternating minimization (PAM) method with closed-form block updates; its objective values decrease and every accumulation point is stationary. We embed the boundary-step mechanism in an Adaptive Homogeneous Tensor Method (Ada--HTM). Under explicit smoothness, safeguarded-decrease, weak-curvature nondegeneracy, and local-refinement conditions, Ada--HTM attains the adaptive-regularization-type (AR$p$-type) evaluation complexity $\mathcal{O}(\epsilon^{-(p+1)/p})$ for first-order stationarity. Numerically, PAM matches order-$2$ moment--sum-of-squares (SOS) certificates on the structured cubic instances for which certification is tractable, scales particularly well for low-rank tensors, and makes Ada--HTM competitive with trust-region and cubic-regularization methods, with its largest gains on ill-conditioned and badly-scaled problems.

math.OC↗

arXiv2026

ShopX: A Foundation Model for Intent-to-Item Fulfillment in Agentic Shopping

The wave of AI-native applications is moving shopping beyond page- and feed-based browsing toward intent-driven experiences orchestrated by LLM agents. A common design wraps an LLM around existing search and recommendation pipelines, forcing complex intents through low-bandwidth retrieval or ranking interfaces and leaving a gap between language understanding and item-space fulfillment. Generative recommendation gives LLMs a direct item-space interface through semantic IDs (SIDs), but existing models mainly generate candidates for retrieval rather than translate flexible intents into item-space outcomes. We propose ShopX to address this bottleneck by unifying intent understanding, execution planning, and flexible SID-native item-space operations into a single foundation model. We deploy ShopX in agentic shopping workflows through a model-native item-fulfillment framework with a serving harness that defines a model-facing action protocol and exposes support surfaces for context access, catalog grounding, and state management. Within this framework, ShopX plans and composes SID-based item-space operations such as SID beam-search retrieval, listwise ranking, or product bundling. This model-centric design reduces lossy hand-offs between agent orchestration and item-space execution. To build ShopX, we design semantically recoverable, LLM-operable SIDs and a training recipe that equips a general LLM for flexible multi-turn item-space fulfillment while retaining the knowledge and instruction-following abilities needed by a shopping agent. We evaluate the ShopX framework against tool-mediated agentic systems on single- and multi-turn fulfillment tasks derived from anonymized Taobao production logs, showing that model-native fulfillment improves overall framework behavior, especially on complex or ambiguous requests.

cs.IR↗

arXiv2026

PoC: Performance-oriented Context Compression for Large Language Models via Performance Prediction

While context compression can mitigate the growing inference costs of Large Language Models (LLMs) by shortening contexts, existing methods that specify a target compression ratio or length suffer from unpredictable performance degradation, hindering their reliable deployment. We introduce a paradigm shift to Performance-oriented Context Compression (PoC), where developers specify an acceptable performance floor instead of a compression ratio. PoC employs a lightweight performance predictor to automatically find the most aggressive compression ratio that satisfies this constraint before steering an off-the-shelf compressor. We design and compare two predictor variants: a simple context-agnostic predictor and a more sophisticated context-aware one that considers the input's inherent compressibility. On both question-answering and summarization benchmarks, the context-aware predictor consistently achieves lower performance prediction error than the context-agnostic predictor, while the resulting context-aware PoC attains a superior overall performance. Our work paves the way for a more reliable, efficient, and performance-aware deployment of context compression for LLMs.

cs.CL↗

arXiv2026

Expert Divergence Learning for MoE-based Language Models

The Mixture-of-Experts (MoE) architecture is a powerful technique for scaling language models, yet it often suffers from expert homogenization, where experts learn redundant functionalities, thereby limiting MoE's full potential. To address this, we introduce Expert Divergence Learning, a novel pre-training strategy that explicitly encourages functional specialization among experts. Our method incorporates a label-driven auxiliary loss that leverages domain labels inherent in pre-training corpora to maximize the Jensen-Shannon Divergence between the expert routing distributions of different data domains. This optimization objective guides the model to develop diverged routing policies for varied domains and closer routing policies for the same domain, which leads to emergent and organized expert specialization. We validate our approach by pre-training MoE models of up to 15 billion parameters from scratch. Experimental results demonstrate that models trained with Expert Divergence Learning not only achieve a lower language modeling loss but also exhibit significant performance improvements across a diverse range of downstream benchmarks. Further analysis confirms that our method effectively mitigates expert homogenization and brings greater functional specialization, all with negligible computational overhead during training.

cs.LG↗

arXiv2026

CoMeT: Collaborative Memory Transformer for Efficient Long Context Modeling

The quadratic complexity and indefinitely growing key-value (KV) cache of standard Transformers pose a major barrier to long-context processing. To overcome this, we introduce the Collaborative Memory Transformer (CoMeT), a novel architecture that enables LLMs to handle arbitrarily long sequences with constant memory usage and linear time complexity. Designed as an efficient, plug-in module, CoMeT can be integrated into pre-trained models with only minimal fine-tuning. It operates on sequential data chunks, using a dual-memory system to manage context: a temporary memory on a FIFO queue for recent events, and a global memory with a gated update rule for long-range dependencies. These memories then act as a dynamic soft prompt for the next chunk. To enable efficient fine-tuning on extremely long contexts, we introduce a novel layer-level pipeline parallelism strategy. The effectiveness of our approach is remarkable: a model equipped with CoMeT and fine-tuned on 32k contexts can accurately retrieve a passkey from any position within a 1M token sequence. On the SCROLLS benchmark, CoMeT surpasses other efficient methods and achieves performance comparable to a full-attention baseline on summarization tasks. Its practical effectiveness is further validated on real-world agent and user behavior QA tasks. The code is available at: https://github.com/LivingFutureLab/Comet

cs.LG↗

arXiv2026

Data Distribution Matters: A Data-Centric Perspective on Context Compression for Large Language Model

The deployment of Large Language Models (LLMs) in long-context scenarios is hindered by computational inefficiency and significant information redundancy. Although recent advancements have widely adopted context compression to address these challenges, existing research only focus on model-side improvements, the impact of the data distribution itself on context compression remains largely unexplored. To bridge this gap, we are the first to adopt a data-centric perspective to systematically investigate how data distribution impacts compression quality, including two dimensions: input data and intrinsic data (i.e., the model's internal pretrained knowledge). We evaluate the semantic integrity of compressed representations using an autoencoder-based framework to systematically investigate it. Our experimental results reveal that: (1) encoder-measured input entropy negatively correlates with compression quality, while decoder-measured entropy shows no significant relationship under a frozen-decoder setting; and (2) the gap between intrinsic data of the encoder and decoder significantly diminishes compression gains, which is hard to mitigate. Based on these findings, we further present practical guidelines to optimize compression gains.

cs.CL↗

arXiv2026

Read As Human: Compressing Context via Parallelizable Close Reading and Skimming

Large Language Models (LLMs) demonstrate exceptional capability across diverse tasks. However, their deployment in long-context scenarios is hindered by two challenges: computational inefficiency and redundant information. We propose RAM (Read As HuMan), a context compression framework that adopts an adaptive hybrid reading strategy, to address these challenges. Inspired by human reading behavior (i.e., close reading important content while skimming less relevant content), RAM partitions the context into segments and encodes them with the input query in parallel. High-relevance segments are fully retained (close reading), while low-relevance ones are query-guided compressed into compact summary vectors (skimming). Both explicit textual segments and implicit summary vectors are concatenated and fed into decoder to achieve both superior performance and natural language format interpretability. To refine the decision boundary between close reading and skimming, we further introduce a contrastive learning objective based on positive and negative query-segment pairs. Experiments demonstrate that RAM outperforms existing baselines on multiple question answering and summarization benchmarks across two backbones, while delivering up to a 12x end-to-end speedup on long inputs (average length 16K; maximum length 32K).

cs.CL↗

arXiv2026

PretrainRL: Alleviating Factuality Hallucination of Large Language Models at the Beginning

Large language models (LLMs), despite their powerful capabilities, suffer from factual hallucinations where they generate verifiable falsehoods. We identify a root of this issue: the imbalanced data distribution in the pretraining corpus, which leads to a state of "low-probability truth" and "high-probability falsehood". Recent approaches, such as teaching models to say "I don't know" or post-hoc knowledge editing, either evade the problem or face catastrophic forgetting. To address this issue from its root, we propose \textbf{PretrainRL}, a novel framework that integrates reinforcement learning into the pretraining phase to consolidate factual knowledge. The core principle of PretrainRL is "\textbf{debiasing then learning}." It actively reshapes the model's probability distribution by down-weighting high-probability falsehoods, thereby making "room" for low-probability truths to be learned effectively. To enable this, we design an efficient negative sampling strategy to discover these high-probability falsehoods and introduce novel metrics to evaluate the model's probabilistic state concerning factual knowledge. Extensive experiments on three public benchmarks demonstrate that PretrainRL significantly alleviates factual hallucinations and outperforms state-of-the-art methods.

cs.CL↗

arXiv2026

Biquadratic Cauchy Tensors and Spherical Biquadratic Polynomial Programming

This paper addresses biquadratic polynomial programming (BPP), an NP-hard optimization problem closely related to biquadratic tensors. We first establish several necessary and sufficient conditions for the positive semi-definiteness and positive definiteness of biquadratic Cauchy tensors. Leveraging the structured properties of these tensors, we then prove that the BPP and its equivalent multilinear formulation share the same set of optimal solutions. This result allows us to establish the global sequence convergence of the proximal alternating minimization (PAM) algorithm via the Kurdyka- Lojasiewicz (KL) property, extending the analysis in [8]. Furthermore, by reformulating the equivalent multilinear problem as an unconstrained optimization model, we enable the analysis of its KL exponent and derive an explicit expression for the convergence rate of PAM. Finally, numerical experiments are conducted on both biquadratic Cauchy tensors and general biquadratic tensor instances to evaluate the efficiency, stability, and practical performance of the proposed algorithm.

math.OC↗

arXiv2026

Completely Positive Reformulations of Polynomial Optimization Problems with Linear Inequality Constraints

Polynomial optimization encompasses a broad class of problems in which both the objective function and constraints are polynomial functions of the decision variables. In recent years, a substantial body of research has focused on reformulating polynomial optimization problems (POPs) as conic programs over the cone of completely positive tensors (CPTs). In this article, we propose several new completely positive reformulations for a class of POPs with linear inequality constraints. Our approach begins by lifting these problems into a novel convex optimization framework, wherein the variables are represented as combinations of symmetric rank-one tensors. Based on this lifted formulation, we present a general characterization of POPs with linear inequality constraints that can be reformulated as conic programs over the CPT cone. Additionally, we construct the dual formulations of the resulting completely positive programs. Under mild assumptions, we prove that these dual problems are strictly feasible and strong duality holds.

math.OC↗

arXiv2025

Tensor Based Proximal Alternating Minimization Method for A Kind of Inhomogeneous Quartic Optimization Problem

In this paper, we propose an efficient numerical approach for solving a specific type of quartic inhomogeneous polynomial optimization problem inspired by practical applications. The primary contribution of this work lies in establishing an inherent equivalence between the quartic inhomogeneous polynomial optimization problem and a multilinear optimization problem (MOP). This result extends the equivalence between fourth-order homogeneous polynomial optimization and multilinear optimization in the existing literature to the equivalence between fourth-order inhomogeneous polynomial optimization and multilinear optimization. By leveraging the multi-block structure embedded within the MOP, a tensor-based proximal alternating minimization algorithm is proposed to approximate the optimal value of the quartic problem. Under mild assumptions, the convergence of the algorithm is rigorously proven. Finally, the effectiveness of the proposed algorithm is demonstrated through preliminary computational results obtained using synthetic datasets.

math.OC↗

arXiv2025

Completely Positive Biquadratic Tensors

In this paper, we systemically introduce completely positive biquadratic (CPB) tensors and copositive biquadratic tensors. We show that all weakly CPB tensors are sum of squares tensors, the CPB tensor cone and the copositive biquadratic tensor cone are dual cone to each other. We also show that the outer product of two completely positive matrices is a CPB tensor, and the outer product of two copositive matrices is a copositive biquadratic tensor. We then study two easily checkable subclasses of CPB tensors, namely positive biquadratic Cauchy tensors and biquadratic Pascal tensors. We show that a biquadratic Pascal tensor is both strongly CPB and positive definite.

math.RA↗

arXiv2025

How to inject knowledge efficiently? Knowledge Infusion Scaling Law for Pre-training Large Language Models

Large language models (LLMs) have attracted significant attention due to their impressive general capabilities across diverse downstream tasks. However, without domain-specific optimization, they often underperform on specialized knowledge benchmarks and even produce hallucination. Recent studies show that strategically infusing domain knowledge during pretraining can substantially improve downstream performance. A critical challenge lies in balancing this infusion trade-off: injecting too little domain-specific data yields insufficient specialization, whereas excessive infusion triggers catastrophic forgetting of previously acquired knowledge. In this work, we focus on the phenomenon of memory collapse induced by over-infusion. Through systematic experiments, we make two key observations, i.e. 1) Critical collapse point: each model exhibits a threshold beyond which its knowledge retention capabilities sharply degrade. 2) Scale correlation: these collapse points scale consistently with the model's size. Building on these insights, we propose a knowledge infusion scaling law that predicts the optimal amount of domain knowledge to inject into large LLMs by analyzing their smaller counterparts. Extensive experiments across different model sizes and pertaining token budgets validate both the effectiveness and generalizability of our scaling law.

cs.CL↗

arXiv2025

RecGPT Technical Report

Recommender systems are among the most impactful applications of artificial intelligence, serving as critical infrastructure connecting users, merchants, and platforms. However, most current industrial systems remain heavily reliant on historical co-occurrence patterns and log-fitting objectives, i.e., optimizing for past user interactions without explicitly modeling user intent. This log-fitting approach often leads to overfitting to narrow historical preferences, failing to capture users' evolving and latent interests. As a result, it reinforces filter bubbles and long-tail phenomena, ultimately harming user experience and threatening the sustainability of the whole recommendation ecosystem. To address these challenges, we rethink the overall design paradigm of recommender systems and propose RecGPT, a next-generation framework that places user intent at the center of the recommendation pipeline. By integrating large language models (LLMs) into key stages of user interest mining, item retrieval, and explanation generation, RecGPT transforms log-fitting recommendation into an intent-centric process. To effectively align general-purpose LLMs to the above domain-specific recommendation tasks at scale, RecGPT incorporates a multi-stage training paradigm, which integrates reasoning-enhanced pre-alignment and self-training evolution, guided by a Human-LLM cooperative judge system. Currently, RecGPT has been fully deployed on the Taobao App. Online experiments demonstrate that RecGPT achieves consistent performance gains across stakeholders: users benefit from increased content diversity and satisfaction, merchants and the platform gain greater exposure and conversions. These comprehensive improvement results across all stakeholders validates that LLM-driven, intent-centric design can foster a more sustainable and mutually beneficial recommendation ecosystem.

cs.IR↗

arXiv2025

ECKGBench: Benchmarking Large Language Models in E-commerce Leveraging Knowledge Graph

Large language models (LLMs) have demonstrated their capabilities across various NLP tasks. Their potential in e-commerce is also substantial, evidenced by practical implementations such as platform search, personalized recommendations, and customer service. One primary concern associated with LLMs is their factuality (e.g., hallucination), which is urgent in e-commerce due to its significant impact on user experience and revenue. Despite some methods proposed to evaluate LLMs' factuality, issues such as lack of reliability, high consumption, and lack of domain expertise leave a gap between effective assessment in e-commerce. To bridge the evaluation gap, we propose ECKGBench, a dataset specifically designed to evaluate the capacities of LLMs in e-commerce knowledge. Specifically, we adopt a standardized workflow to automatically generate questions based on a large-scale knowledge graph, guaranteeing sufficient reliability. We employ the simple question-answering paradigm, substantially improving the evaluation efficiency by the least input and output tokens. Furthermore, we inject abundant e-commerce expertise in each evaluation stage, including human annotation, prompt design, negative sampling, and verification. Besides, we explore the LLMs' knowledge boundaries in e-commerce from a novel perspective. Through comprehensive evaluations of several advanced LLMs on ECKGBench, we provide meticulous analysis and insights into leveraging LLMs for e-commerce.

cs.CL↗

arXiv2025

A Comprehensive Framework for Electroweak Phase Transitions: Thermal History and Dynamics from Bubble Nucleation to Percolation

The electroweak phase transition (EWPT) is crucial for cosmology and particle physics, with a profound impact on electroweak baryogenesis, symmetry breaking, and gravitational wave (GW) signals. However, many studies overlook key aspects of EWPT dynamics, leading to misidentified patterns and overestimated GW signals. To address these gaps, we present a comprehensive framework for analyzing EWPTs, focusing on the vacuum's thermal history and dynamics from bubble nucleation to percolation. Using the $\mathbb{Z}_2$-odd real scalar singlet model, we demonstrate the occurrence of spontaneous $\mathbb{Z}_2$ symmetry breaking in the high-temperature vacuum, leading to diverse EWPT processes, including multi-step transitions and inverse symmetry breaking. We identify four distinct EWPT patterns, each characterized by unique symmetry-breaking mechanisms and associated with bubbles exhibiting distinct field configurations, which can be analyzed using a formalism based on energy density distributions developed here. A key finding is that bubble nucleation fails in extremely strong phase transitions (PTs) with low nucleation rates, or in ultra-fast PTs involving inverse $s$-bubbles that collapse instantly upon formation, both of which lead to false vacuum trapping and the absence of observable GW signals. In first-order PTs where nucleation succeeds, stronger transitions occur later in the universe's evolution, while weaker transitions proceed more rapidly. Multi-step transitions involving (inverse) $\mathbb{Z}_2$ symmetry breaking give rise to complex transition sequences and exotic bubble dynamics, such as sequential nucleation or the coexistence of bubbles from different vacua -- phenomena with significant implications for GW spectra, dark matter, and baryogenesis. This work advances our understanding of EWPT dynamics and lays the groundwork for future studies of EWPTs in BSM physics.

hep-ph↗

arXiv2025

ChineseEcomQA: A Scalable E-commerce Concept Evaluation Benchmark for Large Language Models

With the increasing use of Large Language Models (LLMs) in fields such as e-commerce, domain-specific concept evaluation benchmarks are crucial for assessing their domain capabilities. Existing LLMs may generate factually incorrect information within the complex e-commerce applications. Therefore, it is necessary to build an e-commerce concept benchmark. Existing benchmarks encounter two primary challenges: (1) handle the heterogeneous and diverse nature of tasks, (2) distinguish between generality and specificity within the e-commerce field. To address these problems, we propose \textbf{ChineseEcomQA}, a scalable question-answering benchmark focused on fundamental e-commerce concepts. ChineseEcomQA is built on three core characteristics: \textbf{Focus on Fundamental Concept}, \textbf{E-commerce Generality} and \textbf{E-commerce Expertise}. Fundamental concepts are designed to be applicable across a diverse array of e-commerce tasks, thus addressing the challenge of heterogeneity and diversity. Additionally, by carefully balancing generality and specificity, ChineseEcomQA effectively differentiates between broad e-commerce concepts, allowing for precise validation of domain capabilities. We achieve this through a scalable benchmark construction process that combines LLM validation, Retrieval-Augmented Generation (RAG) validation, and rigorous manual annotation. Based on ChineseEcomQA, we conduct extensive evaluations on mainstream LLMs and provide some valuable insights. We hope that ChineseEcomQA could guide future domain-specific evaluations, and facilitate broader LLM adoption in e-commerce applications.

cs.CL↗

arXiv2025

Tensor-based Dinkelbach method for computing generalized tensor eigenvalues and its applications

In this paper, we propose a novel tensor-based Dinkelbach--Type method for computing extremal tensor generalized eigenvalues. We show that the extremal tensor generalized eigenvalue can be reformulated as a critical subproblem of the classical Dinkelbach--Type method, which can subsequently be expressed as a multilinear optimization problem (MOP). The MOP is solved under a spherical constraint using an efficient proximal alternative minimization method, in which we rigorously establish the global convergence. Additionally, the equivalent MOP is reformulated as an unconstrained optimization problem, allowing for the analysis of the Kurdyka-Lojasiewicz (KL) exponent and providing an explicit expression for the convergence rate of the proposed algorithm. Preliminary numerical experiments on solving extremal tensor generalized eigenvalues and minimizing high-order trust-region subproblems are provided, validating the efficacy and practical utility of the proposed method.

math.NA↗