Searcharxiv⌕ Search

arXiv subjects

Defeng Sun

Publications and source records attributed to Defeng Sun.

At least 19 recordsLinked to original sources
arXiv2026

DARE: Aligning LLM Agents with the R Statistical Ecosystem via Distribution-Aware Retrieval

Large Language Model (LLM) agents excel at automating data science workflows, yet often underutilize rigorous statistical methods in R. Existing Retrieval-Augmented Generation (RAG) approaches rely on semantic similarity, ignoring the data-distribution and structural characteristics crucial for determining a statistical method's applicability. To address this, we propose DARE (Distribution-Aware Retrieval Embedding), a representation learning framework incorporating these data characteristics into R function retrieval. Our contributions are fourfold: (i) RPKB, a curated knowledge base of 8,191 high-quality R functions; (ii) DARE, the distribution-aware retrieval framework; (iii) RCodingAgent, an R-oriented agent integrating DARE for end-to-end analysis; and (iv) RStatBench, a comprehensive benchmark of 100 statistical tasks. Experiments demonstrate that DARE improves NDCG@10 by 13.43 to 32.65 percentage points across evaluated embedding models, with a MiniLM-based DARE achieving 93.92% NDCG@10 alongside high efficiency. Furthermore, integrating DARE into RCodingAgent nearly doubles mean downstream accuracy on RStatBench, increasing from 31.8% to 62.2% across ten LLMs. This work helps bridge modern LLM automation with R's established statistical computing ecosystem.

cs.IR↗
arXiv2026

Learning a Flow to Self-Supervised Representations

We introduce Flow-Based Distribution Matching (FBDM), which learns self-supervised representations through non-adversarial velocity regression. The encoder and auxiliary field are trained jointly by regressing analytic velocities along spherical paths from learned representations to reference targets. Both augmented views share a reference target, with a capacity limit on each reference center and an explicit view-alignment loss. A structured reference supports more components than the auxiliary flow dimension while retaining geometric separation. The auxiliary field is discarded after pretraining; only the backbone is retained downstream. On three ResNet-18 benchmarks with matched auxiliary parameter counts, FBDM provides 2.17 to 2.36 times higher per-epoch training throughput than critic-based Distribution Matching (DM), with about 0.21 GiB additional peak allocated GPU memory. Experiments show that FBDM is competitive with existing SSL methods from CIFAR to ImageNet and performs comparably to DM on their shared benchmarks. Under stated conditions, our population analysis bounds nearest-centroid classification error on the initial normalized pretraining representation by the FBDM loss.

cs.CV↗
arXiv2026

Recent Developments in HPR Methods for Large-Scale Linear Programming

We present recent developments in Halpern Peaceman-Rachford (HPR) methods for solving large-scale linear programming problems. We first introduce the semi-proximal HPR framework and summarize its convergence guarantees, together with its connections to Halpern PDHG and ergodic Peaceman-Rachford iterations. To improve practical performance, we develop adaptive restart and penalty parameter update strategies. We then describe several computational enhancements, including GPU-based preprocessing, diagonal scaling, reusable execution plans for sparse matrix-vector multiplication, and sieving techniques. We further discuss multi-GPU parallelization and lossless model storage and loading for handling large-scale instances. These developments form the foundation of HPR-LP, a GPU-accelerated solver for large-scale linear programming.

math.OC↗
arXiv2026

Recent Developments in HPR Methods for Large-Scale Convex Composite Quadratic Programming

We present recent developments in Halpern Peaceman--Rachford (HPR) methods for solving large-scale convex composite quadratic programming problems. Within a unified HPR framework applied to the restricted Wolfe dual, we develop several algorithmic variants based on different choices of the semi-proximal operator $\mathcal{T}_1$ for the joint dual subproblem. Specifically, an sGS-based construction leads to a sequential $w$--$y$--$w$ update scheme, while a block-linearized construction eliminates cross-block coupling and enables parallel dual updates. To improve the latter, we further introduce a coupling-aware construction that exploits a second spectral moment to obtain a tighter row majorizer. We extend the framework to convex quadratically constrained quadratic programming through factor-based second-order cone reformulations. Additional computational enhancements include equality Gram scaling and asynchronous CPU polishing. We establish convergence results under fixed metrics and discuss practical adaptive strategies.

math.OC↗
arXiv2026

On the Nonoccurrence of Gradient Restart in Nesterov Accelerated Gradient Method

Gradient restarting often improves the practical performance of accelerated gradient methods for unconstrained convex optimization problems, but a general theoretical guarantee of such improvement remains unclear. In this paper, we construct smooth, strongly convex objectives with Lipschitz-continuous gradients and corresponding initial points for which gradient restart never occurs, both for Nesterov's accelerated gradient method and for its ordinary differential equation model. From these examples, we conclude that strong convexity and smoothness alone do not guarantee that gradient restart improves the convergence behavior of the accelerated method or its continuous-time model. In particular, for the continuous-time example, the objective gap is asymptotic to $1/(πt^3)$. We therefore obtain a negative answer to the open problem raised by Su, Boyd, and Cand{è}s [\textit{J. Mach. Learn. Res.}, 2016, 17(153), 1--43] of whether gradient restart guarantees linear convergence of the continuous dynamics for smooth strongly convex objectives.

math.OC↗
arXiv2026

A Level Set Method with Secant Iterations for the Least-Squares Constrained Nuclear Norm Minimization

We present an efficient algorithm for least-squares constrained nuclear norm minimization, a computationally challenging problem with broad applications. Our approach combines a level set method with secant iterations and a proximal generation method. As a key theoretical contribution, we establish the nonsingularity of the Clarke generalized Jacobian for a general class of projection norm functions over closed convex sets. This property and the (strong) semismoothness of our value function yield fast local convergence of the secant method. For the resulting nuclear norm regularized subproblems, we develop a proximal generation method that exploits low-rank structures without compromising convergence. Extensive numerical experiments demonstrate the superior performance of our approach compared to state-of-the-art methods.

math.OC↗
arXiv2026

GPU-Accelerated Preprocessing for Large-Scale Linear Programming

GPU-accelerated first-order methods are increasingly effective for solving large-scale linear programs (LPs), yet LP preprocessing remains predominantly CPU-based and can become costly. A key challenge is that dependencies among presolve reductions and frequent modifications to sparse LP data make efficient GPU parallelization difficult. To address this challenge, we develop GPU-Preprocessor, a GPU-native LP preprocessing framework that integrates folding and presolve. To efficiently implement folding on GPUs, we adapt a randomized partition refinement algorithm to LP folding, using sparse matrix-vector products and parallel sorting to construct stable row and column partitions. For presolve, we introduce a new piecewise-linear reformulation rule that exploits underlying convex piecewise-linear structure to obtain sparser LP representations. We also develop a framework that combines phase scheduling with a Scan-Plan-Apply design to coordinate presolve rules and reduce sparse-matrix reconstruction overhead through batched updates. Numerical experiments on three LP benchmark sets show that GPU-Preprocessor achieves greater average problem-size reductions than PSLP and Gurobi Presolve, with SGM10 preprocessing-time speedups of up to $43.9\times$ and $677.7\times$, respectively. Here, SGM10 denotes the shifted geometric mean with a 10-second shift. On the largest tested quadratic assignment problem LP relaxation, these speedups reach $126.3\times$ and $2{,}207.9\times$. When integrated with the GPU-accelerated LP solver HPR-LP, GPU-Preprocessor achieves SGM10 speedups of $2.5\times$ to $5.0\times$ in combined preprocessing and solving time over HPR-LP without preprocessing. The code is available at https://github.com/PolyU-IOR/GPU-Preprocessor.

math.OC↗
arXiv2026

Solution Set Dimensions in Convex Optimization and Primal-Polar Facial Signatures

We study the possible dimensions of solution sets in linearly perturbed convex optimization problems and their behavior under convex duality. The dimensions of solution sets and inverse solution sets capture two complementary forms of nonuniqueness: that of optimal solutions for a given perturbation, and that of perturbations for which a given point is optimal. Although these two families are linked by Fenchel conjugacy, we show that their collections of dimensions can be prescribed independently for finite supercoercive convex functions. As a geometric consequence, we prove that the facial dimension signatures of a convex body and its polar can be prescribed independently. The connection between these two realization results is obtained by homogenizing the epigraphs of a function and its conjugate. In contrast, under simultaneous facial exposedness, we establish a dimensional obstruction relating the smallest positive proper face dimensions of the body and its polar. These results resolve two open questions posed by Roshchina and Tunçel.

math.OC↗
arXiv2026

Bringing Generative Learning to Representation Learning: Self-Supervised Transfer Learning as Distribution Matching

Most self-supervised learning objectives defend against collapse but leave the target representation law unspecified. We formulate representation learning as Distribution Matching (DM), learning an augmentation-invariant encoder whose push-forward law matches an explicit reference law. The reference law specifies what the learned representation distribution should look like, while the matching mechanism determines how this target is matched. Within DM, generative learning and representation learning can be viewed as directionally inverse distributional processes: generative learning maps a tractable reference to data, whereas representation learning maps data to a designed reference law. For the Mallows-distance instantiation studied here, we connect the population objective to class-centre separation and classification error and derive a finite-sample bound on downstream classification error. Simulations and image benchmarks show manifold rectification, fine-grained structure and transferability.

stat.ML↗
arXiv2026

olBlend: Language-Guided Molecular Blend Retrieval with Mixture Odor Prediction

Converting open-ended natural-language odor descriptions into molecular mixtures that can be physically prepared from an available inventory remains an important challenge in computational olfaction. We propose olBlend, a language-guided framework for molecular mixture retrieval. olBlend first uses MixPOM to predict odor probabilities for candidate molecular pairs and stores these predictions for retrieval. It then uses a language model to translate each odor description into three types of semantic requirements: lead attributes that define the main odor, ambient attributes that describe its background, and absent attributes that should be avoided. A structured scoring algorithm uses the precomputed odor probabilities to score how well each candidate mixture meets these requirements, then retrieves and ranks mixtures from the available inventory. Evaluation on established odor datasets shows that MixPOM improves mixture odor prediction over averaged single-molecule predictions. Retrieval comparisons across natural-language queries demonstrate how background and exclusion requirements guide mixture selection. We further prepare retrieved formulations as physical odor samples, demonstrating the workflow from natural-language descriptions to realizable molecular formulations.

cs.HC↗
arXiv2026

SLICE-Mem: Salience-Gated Lightweight Information Curation for Efficient LLM Agent Memory

Long-term memory enables large language model (LLM) agents to maintain personalized and temporally coherent interactions, but repeated LLM inference for memory admission, extraction, and organization introduces substantial cost and latency. We present SLICE-Mem, a lightweight memory framework that combines salience-guided admission and predicate-centered information extraction. A compact classifier identifies user turns worth retaining, and a semantic-dependency-based extractor converts admitted turns into concise memory entries. This online pipeline requires no generative LLM calls, while cross-entry reconciliation is deferred to an offline consolidation stage. We evaluate SLICE-Mem on the weekly subset of Memora and LongMemEval-S using 8B and 30B Qwen backbones for memory construction and consolidation on both benchmarks. Compared with the generative-memory baseline MemOS under the Qwen3-30B configuration, SLICE-Mem's online pipeline achieves 46.60 versus 46.68 FAMA (Forgetting-Aware Memory Accuracy) on Memora Weekly and 56.8% versus 59.3% accuracy on LongMemEval-S, while reducing total LLM token consumption by 92.4% and 99.5%, respectively. These results demonstrate that SLICE-Mem can preserve useful evidence for long-term memory with substantially less reliance on LLM-based memory construction.

cs.CL↗
arXiv2026

Learning Universal Costs via Multi-Observation Inverse Optimal Transport

Classical Inverse Optimal Transport (IOT) typically infers a cost function from a single observed transport plan, which may be sensitive to noise and observational biases in practice. To address this limitation, we propose the Multi-Observation Inverse Optimal Transport (M-IOT) framework, which learns a universal cost function from multiple transport plans. We reformulate the resulting bilevel optimization problem into a tractable single-level convex program and characterize the existence and geometric structure of the optimal solution set. Moreover, we propose a Block Coordinate Descent with Anderson-type Extrapolation (BCDwAE) algorithm and establish its Q-linear convergence. Extensive experiments on synthetic data, urban mobility flows, and e-commerce recommendations demonstrate that M-IOT reduces recovery error compared to single-observation baselines and extracts robust structural patterns from noisy multi-view data.

math.OC↗
arXiv2026

Semidefinite Implementations of the Proximal Point Algorithm and Consequences for Progressive Decoupling in Optimization

The proximal point algorithm for finding a zero of a maximal monotone mapping is the engine that drives augmented Lagrangian methods and various splitting schemes in convex optimization, where proximal terms that are merely positive semidefinite can be helpful in preconditioning. Here a semidefinite extension of that algorithm is developed with new features which guarantee convergence, typically even linear convergence, in broader territory than previously foreseen. Moreover, inexact minimization is allowed in the subproblems in a variable-metric implementation which might open up quasi-Newton-like approaches. The key to the extension is applying an advanced form of the positive definite proximal point algorithm to a projected monotone mapping derived from the given mapping, and combining that with a completion step. For this to work, the maximality of the projected mapping is essential, but maximality is not automatic. The circumstances that guarantee maximality are definitively analyzed and convenient criteria for maximality are thereby identified. Consequences of the extended algorithm are worked out for application to convex inf-projection, augmented Lagrangian steps, and bracketed progressive decoupling based on Spingarn's partial inverse as a scheme for problem decomposition.

math.OC↗
arXiv2026

DSAEval: Evaluating Data Science Agents on a Wide Range of Real-World Data Science Problems

Recent LLM-based data agents aim to automate data science tasks ranging from data analysis to deep learning. However, the open-ended nature of real-world data science problems, which often span multiple taxonomies and lack standard answers, poses a significant challenge for evaluation. To address this, we introduce DSAEval, a benchmark comprising 641 real-world data science problems grounded in 285 diverse datasets, covering both structured and unstructured data (e.g., image and text). DSAEval incorporates three distinctive features: (1) Multimodal Environment Perception, which enables agents to interpret observations from multiple modalities, including text and vision; (2) Multi-Query Interactions, which mirror the iterative and cumulative nature of real-world data science projects; and (3) Multi-Dimensional Evaluation, which provides a holistic assessment across reasoning, code, and results. We systematically evaluate 13 recent advanced agentic LLMs using DSAEval. Our results show that Claude-Sonnet-4.5 achieves the strongest overall performance, MiMo-V2-Pro and GPT-5.2 lead in duration and step efficiency, respectively, and MiMo-V2-Flash is the most cost-effective. We further demonstrate that multimodal perception consistently improves performance on vision-related tasks, with gains ranging from 2.04% to 11.30%. Overall, while current data science agents perform well on structured data and routine data analysis workflows, substantial challenges remain in unstructured domains. Finally, we offer critical insights and outline future research directions.

cs.AI↗
arXiv2026

Diffusion Based Unpaired Data Learning for Inverse Problems

Data is important in many deep learning-based inverse problem solvers. However, obtaining sufficient paired data in many scenarios remains highly challenging, while unpaired data is cheap. To maximize data utilization, this paper proposes LUD-DIF, a diffusion-based approach for solving inverse problems with unpaired data. Starting from the evidence lower bound (ELBO) of the joint distribution, we decouple it into two independent diffusion processes under the weak-coupling assumption. The method provides theoretical support from a variational inference perspective, derives the loss function, quantitatively analyzes the error bound introduced by the assumption, and offers a theorem-motivated heuristic for hyperparameter selection. Experimental results demonstrate that LUD-DIF achieves outstanding performance on multiple image inverse problems, validating its effectiveness and generalization capability in unpaired inverse problem settings.

cs.CV↗
arXiv2026

Maximal monotonicity of piecewise polyhedral mappings

Maximal monotone mappings that are piecewise polyhedral arise from subdifferentials of convex functions and saddle functions that are piecewise linear-quadratic and enter into algorithmic constructions of importance in linear-quadratic optimization and associated splitting methods. The question of whether those constructions preserve maximal monotonicity is then crucial. The usual answers to that invoke constraint qualifications involving the nonemptiness of intersections of certain relative interiors, but it is shown here that the piecewise polyhedral structure allows the relative interiors to be bypassed.

math.OC↗
arXiv2026

ADMM Fails to Achieve an $O(K^{-1})$ Ergodic KKT Residual Bound

The Karush--Kuhn--Tucker (KKT) residual is a fundamental measure of first-order optimality and, under an error bound condition, is comparable to the distance to the KKT solution set up to constant factors. Despite the $O(K^{-1})$ ergodic rates known for objective error and feasibility violations, we show that the KKT residual of classical ADMM cannot, in general, satisfy a uniform $O(K^{-1})$ bound. Specifically, we construct a fixed-dimensional, horizon-dependent family of two-block convex optimization problems for which the KKT residual is $Ω(K^{-1/2})$ at both the last iterate and the equal-weight ergodic average at the prescribed horizon $K$. Consequently, a uniform $O(K^{-1})$ KKT residual bound is impossible for either output.

math.OC↗
arXiv2026

The Aubin Property for Generalized Equations over $C^2$-cone Reducible Sets

This paper establishes the equivalence between the Aubin property and strong regularity for generalized equations over $C^2$-cone reducible sets. This result resolves a long-standing question in variational analysis and extends the classical equivalence theorem for polyhedral sets to a broad class of non-polyhedral sets. Our proof strategy departs from traditional variational techniques, integrating insights from convex geometry with powerful tools from algebraic topology. At the heart of our analysis is a novel index theorem for a class of functions involving metric projections onto arbitrary closed convex sets. Its proof exploits the geometry of normal cones and topological degree theory. We then use a lift of the diffeomorphism provided by the $C^2$-cone reduction to reduce the original generalized equations to the setting of the index theorem. The homological inverse mapping theorem then yields the desired strong regularity. We show that the $C^2$-cone reducibility assumption cannot be removed in general by constructing a three-dimensional semialgebraic counterexample. This result unifies and extends existing stability results for conventional nonlinear programming, nonlinear second-order cone programming, and nonlinear semidefinite programming under a single general framework, and removes the local optimality assumption imposed in the existing non-polyhedral results.

math.OC↗