arXiv subjects

Zhening Li

Publications and source records attributed to Zhening Li.

17 recordsLinked to original sources

arXiv2025

EnCompass: Enhancing Agent Programming with Search Over Program Execution Paths

We introduce a new approach to agent programming, the development of LLM-based agents. Current approaches to agent programming often entangle two aspects of agent design: the core workflow logic and the inference-time strategy (e.g., tree search). We introduce "probabilistic angelic nondeterminism" ("PAN"), a programming model that disentangles these two concerns, allowing the programmer to describe the agent workflow and independently experiment with different inference-time strategies by simply changing a few inputs. We provide an implementation of PAN in Python as the EnCompass framework, which uses a Python decorator to compile agent workflow programs into a search space. We present three case studies that demonstrate how the framework lets the programmer quickly improve the reliability of an agent and easily switch between different inference-time strategies, all with little additional coding.

cs.AI↗

arXiv2025

On decomposability and subdifferential of the tensor nuclear norm

We study the decomposability and the subdifferential of the tensor nuclear norm. Both concepts are well understood and widely applied in matrices but remain unclear for higher-order tensors. We show that the tensor nuclear norm admits a full decomposability over specific subspaces and determine the largest possible subspaces that allow the full decomposability. We derive novel inclusions of the subdifferential of the tensor nuclear norm and study its subgradients in a variety of subspaces of interest. All the results hold for tensors of an arbitrary order. As an immediate application, we establish the statistical performance of the tensor robust principal component analysis, the first such result for tensors of an arbitrary order.

math.OC↗

arXiv2025

Beyond I-Con: Exploring New Dimension of Distance Measures in Representation Learning

The Information Contrastive (I-Con) framework revealed that over 23 representation learning methods implicitly minimize KL divergence between data and learned distributions that encode similarities between data points. However, a KL-based loss may be misaligned with the true objective, and properties of KL divergence such as asymmetry and unboundedness may create optimization challenges. We present Beyond I-Con, a framework that enables systematic discovery of novel loss functions by exploring alternative statistical divergences. Key findings: (1) on unsupervised clustering of DINO-ViT embeddings, we achieve state-of-the-art results by modifying the PMI algorithm to use total variation (TV) distance; (2) supervised contrastive learning with Euclidean distance as the feature space metric is improved by replacing the standard loss function with Jenson-Shannon divergence (JSD); (3) on dimensionality reduction, we achieve superior qualitative results and better performance on downstream tasks than SNE by replacing KL with a bounded $f$-divergence. Our results highlight the importance of considering divergence choices in representation learning optimization.

cs.LG↗

arXiv2025

The First Impression Problem: Internal Bias Triggers Overthinking in Reasoning Models

Reasoning models often exhibit overthinking, characterized by redundant reasoning steps. We identify \emph{internal bias} elicited by the input question as a key trigger of such behavior. Upon encountering a problem, the model immediately forms a preliminary guess about the answer, which we term an internal bias since it may not be explicitly generated, and it arises without systematic reasoning. When this guess conflicts with its subsequent reasoning, the model tends to engage in excessive reflection, resulting in wasted computation. We validate the association between internal bias and overthinking across multiple models and diverse reasoning tasks. To demonstrate the causal relationship more rigorously, we conduct two counterfactual interventions, showing that removing the input question after the model reduces the redundant reasoning across various complex reasoning tasks, and manually injecting bias affects overthinking accordingly. Further interpretability experiments suggest that excessive attention to the input question serves as a key mechanism through which internal bias influences subsequent reasoning trajectories. Finally, we evaluated several methods aimed at mitigating overthinking, yet the influence of internal bias persisted under all conditions.

cs.AI↗

arXiv2024

When Do Skills Help Reinforcement Learning? A Theoretical Analysis of Temporal Abstractions

Skills are temporal abstractions that are intended to improve reinforcement learning (RL) performance through hierarchical RL. Despite our intuition about the properties of an environment that make skills useful, a precise characterization has been absent. We provide the first such characterization, focusing on the utility of deterministic skills in deterministic sparse-reward environments with finite action spaces. We show theoretically and empirically that RL performance gain from skills is worse in environments where solutions to states are less compressible. Additional theoretical results suggest that skills benefit exploration more than they benefit learning from existing experience, and that using unexpressive skills such as macroactions may worsen RL performance. We hope our findings can guide research on automatic skill discovery and help RL practitioners better decide when and how to use skills.

cs.LG↗

arXiv2024

Nestedness Promotes Stability in Maximum-Entropy Bipartite Food Webs

Food web topology and energy flow rates across food web linkages can influence ecosystem properties such as stability. Stability predictions from current models of energy flow are often sensitive to details in their formulation, and their complexity makes it difficult to elucidate underlying mechanisms of general phenomena. Here, within the maximum information entropy inference framework (MaxEnt), we derive a simple formula for the energy flow carried by each linkage between two adjacent trophic layers. Inputs to the model are the topological structure of the food web and aggregate energy fluxes entering or exiting each species node. For ecosystems with interactions dominated by consumer-resource interactions between two trophic layers, we construct a model of species dynamics based on the energy flow predictions from the MaxEnt model. Mathematical analyses and simulations of the model show that a food web topology with a higher matrix dipole moment promotes stability against small perturbations in population sizes, where the \textit{matrix dipole moment} is a simple nestedness metric that we introduce. Since nested bipartite subnetworks arise naturally in food webs, our result provides an explanation for the stability of natural communities.

q-bio.PE↗

arXiv2023

Predictive Chemistry Augmented with Text Retrieval

This paper focuses on using natural language descriptions to enhance predictive models in the chemistry field. Conventionally, chemoinformatics models are trained with extensive structured data manually extracted from the literature. In this paper, we introduce TextReact, a novel method that directly augments predictive chemistry with texts retrieved from the literature. TextReact retrieves text descriptions relevant for a given chemical reaction, and then aligns them with the molecular representation of the reaction. This alignment is enhanced via an auxiliary masked LM objective incorporated in the predictor training. We empirically validate the framework on two chemistry tasks: reaction condition recommendation and one-step retrosynthesis. By leveraging text retrieval, TextReact significantly outperforms state-of-the-art chemoinformatics models trained solely on molecular data.

cs.CL↗

arXiv2023

$\ell_p$-sphere covering and approximating nuclear $p$-norm

The spectral $p$-norm and nuclear $p$-norm of matrices and tensors appear in various applications albeit both are NP-hard to compute. The former sets a foundation of $\ell_p$-sphere constrained polynomial optimization problems and the latter has been found in many rank minimization problems in machine learning. We study approximation algorithms of the tensor nuclear $p$-norm with an aim to establish the approximation bound matching the best one of its dual norm, the tensor spectral $p$-norm. Driven by the application of sphere covering to approximate both tensor spectral and nuclear norms ($p=2$), we propose several types of hitting sets that approximately represent $\ell_p$-sphere with adjustable parameters for different levels of approximations and cardinalities, providing an independent toolbox for decision making on $\ell_p$-spheres. Using the idea in robust optimization and second-order cone programming, we obtain the first polynomial-time algorithm with an $\Omega(1)$-approximation bound for the computation of the matrix nuclear $p$-norm when $p\in(2,\infty)$ is a rational, paving a way for applications in modeling with the matrix nuclear $p$-norm. These two new results enable us to propose various polynomial-time approximation algorithms for the computation of the tensor nuclear $p$-norm using tensor partitions, convex optimization and duality theory, attaining the same approximation bound to the best one of the tensor spectral $p$-norm. We believe the ideas of $\ell_p$-sphere covering with its applications in approximating nuclear $p$-norm would be useful to tackle optimization problems on other sets such as the binary hypercube with its applications in graph theory and neural networks, the nonnegative sphere with its applications in copositive programming and nonnegative matrix factorization.

math.OC↗

arXiv2023

Approximating Tensor Norms via Sphere Covering: Bridging the Gap Between Primal and Dual

The matrix spectral and nuclear norms appear in enormous applications. The generalizations of these norms to higher-order tensors is becoming increasingly important but unfortunately they are NP-hard to compute or even approximate. Although the two norms are dual to each other, the best known approximation bound achieved by polynomial-time algorithms for the tensor nuclear norm is worse than that for the tensor spectral norm. In this paper, we bridge this gap by proposing deterministic algorithms with the best bound for both tensor norms. Our methods not only improve the approximation bound for the nuclear norm, but are also data independent and easily implementable comparing to existing approximation methods for the tensor spectral norm. The main idea is to construct a selection of unit vectors that can approximately represent the unit sphere, in other words, a collection of spherical caps to cover the sphere. For this purpose, we explicitly construct several collections of spherical caps for sphere covering with adjustable parameters for different levels of approximations and cardinalities. These readily available constructions are of independent interest as they provide a powerful tool for various decision making problems on spheres and related problems. We believe the ideas of constructions and the applications to approximate tensor norms can be useful to tackle optimization problems over other sets such as the binary hypercube.

math.OC↗

arXiv2022

Complexity and computation for the spectral norm and nuclear norm of order three tensors with one fixed dimension

The recent decade has witnessed a surge of research in modelling and computing from two-way data (matrices) to multiway data (tensors). However, there is a drastic phase transition for most tensor optimization problems when the order of a tensor increases from two (a matrix) to three: Most tensor problems are NP-hard while that for matrices are easy. It triggers a question on where exactly the transition occurs. The paper aims to study this kind of question for the spectral norm and the nuclear norm. Although computing the spectral norm for a general $\ell\times m\times n$ tensor is NP-hard, we show that it can be computed in polynomial time if $\ell$ is fixed. This is the same for the nuclear norm. While these polynomial-time methods are not implementable in practice, we propose fully polynomial-time approximation schemes (FPTAS) for the spectral norm based on spherical grids and for the nuclear norm with further help of duality theory and semidefinite optimization. Numerical experiments on simulated data show that our FPTAS can compute these tensor norms for small $\ell \le 6$ but large $m, n\ge50$. To the best of our knowledge, this is the first method that can compute the nuclear norm of general asymmetric tensors. Both our polynomial-time algorithms and FPTAS can be extended to higher-order tensors as well.

math.OC↗

arXiv2022

LEMMA: Bootstrapping High-Level Mathematical Reasoning with Learned Symbolic Abstractions

Humans tame the complexity of mathematical reasoning by developing hierarchies of abstractions. With proper abstractions, solutions to hard problems can be expressed concisely, thus making them more likely to be found. In this paper, we propose Learning Mathematical Abstractions (LEMMA): an algorithm that implements this idea for reinforcement learning agents in mathematical domains. LEMMA augments Expert Iteration with an abstraction step, where solutions found so far are revisited and rewritten in terms of new higher-level actions, which then become available to solve new problems. We evaluate LEMMA on two mathematical reasoning tasks--equation solving and fraction simplification--in a step-by-step fashion. In these two domains, LEMMA improves the ability of an existing agent, both solving more problems and generalizing more effectively to harder problems than those seen during training.

cs.AI↗

arXiv2022

Extreme ratio between spectral and Frobenius norms of nonnegative tensors

One of the fundamental problems in multilinear algebra, the minimum ratio between the spectral and Frobenius norms of tensors, has received considerable attention in recent years. While most values are unknown for real and complex tensors, the asymptotic order of magnitude and tight lower bounds have been established. However, little is known about nonnegative tensors. In this paper, we present an almost complete picture of the ratio for nonnegative tensors. In particular, we provide a tight lower bound that can be achieved by a wide class of nonnegative tensors under a simple necessary and sufficient condition, which helps to characterize the extreme tensors and obtain results such as the asymptotic order of magnitude. We show that the ratio for symmetric tensors is no more than that for general tensors multiplied by a constant depending only on the order of tensors, hence determining the asymptotic order of magnitude for real, complex, and nonnegative symmetric tensors. We also find that the ratio is in general different to the minimum ratio between the Frobenius and nuclear norms for nonnegative tensors, a sharp contrast to the case for real tensors and complex tensors.

math.NA↗

arXiv2022

MolScribe: Robust Molecular Structure Recognition with Image-To-Graph Generation

Molecular structure recognition is the task of translating a molecular image into its graph structure. Significant variation in drawing styles and conventions exhibited in chemical literature poses a significant challenge for automating this task. In this paper, we propose MolScribe, a novel image-to-graph generation model that explicitly predicts atoms and bonds, along with their geometric layouts, to construct the molecular structure. Our model flexibly incorporates symbolic chemistry constraints to recognize chirality and expand abbreviated structures. We further develop data augmentation strategies to enhance the model robustness against domain shifts. In experiments on both synthetic and realistic molecular images, MolScribe significantly outperforms previous models, achieving 76-93% accuracy on public benchmarks. Chemists can also easily verify MolScribe's prediction, informed by its confidence estimation and atom-level alignment with the input image. MolScribe is publicly available through Python and web interfaces: https://github.com/thomas0809/MolScribe.

cs.CV↗

arXiv2018

On orthogonal tensors and best rank-one approximation ratio

As is well known, the smallest possible ratio between the spectral norm and the Frobenius norm of an $m \times n$ matrix with $m \le n$ is $1/\sqrt{m}$ and is (up to scalar scaling) attained only by matrices having pairwise orthonormal rows. In the present paper, the smallest possible ratio between spectral and Frobenius norms of $n_1 \times \dots \times n_d$ tensors of order $d$, also called the best rank-one approximation ratio in the literature, is investigated. The exact value is not known for most configurations of $n_1 \le \dots \le n_d$. Using a natural definition of orthogonal tensors over the real field (resp., unitary tensors over the complex field), it is shown that the obvious lower bound $1/\sqrt{n_1 \cdots n_{d-1}}$ is attained if and only if a tensor is orthogonal (resp., unitary) up to scaling. Whether or not orthogonal or unitary tensors exist depends on the dimensions $n_1,\dots,n_d$ and the field. A connection between the (non)existence of real orthogonal tensors of order three and the classical Hurwitz problem on composition algebras can be established: existence of orthogonal tensors of size $\ell \times m \times n$ is equivalent to the admissibility of the triple $[\ell,m,n]$ to the Hurwitz problem. Some implications for higher-order tensors are then given. For instance, real orthogonal $n \times \dots \times n$ tensors of order $d \ge 3$ do exist, but only when $n = 1,2,4,8$. In the complex case, the situation is more drastic: unitary tensors of size $\ell \times m \times n$ with $\ell \le m \le n$ exist only when $\ell m \le n$. Finally, some numerical illustrations for spectral norm computation are presented.

math.NA↗

arXiv2018

On decompositions and approximations of conjugate partial-symmetric complex tensors

Conjugate partial-symmetric (CPS) tensors are the high-order generalization of Hermitian matrices. As the role played by Hermitian matrices in matrix theory and quadratic optimization, CPS tensors have shown growing interest recently in tensor theory and optimization, particularly in many application-driven complex polynomial optimization problems. In this paper, we study CPS tensors with a focus on ranks, rank-one decompositions and approximations, as well as their applications. The analysis is conducted along side with a more general class of complex tensors called partial-symmetric tensors. We prove constructively that any CPS tensor can be decomposed into a sum of rank-one CPS tensors, which provides an alternative definition of CPS tensors via linear combinations of rank-one CPS tensors. Three types of ranks for CPS tensors are defined and shown to be different in general. This leads to the invalidity of the conjugate version of Comon's conjecture. We then study rank-one approximations and matricizations of CPS tensors. By carefully unfolding CPS tensors to Hermitian matrices, rank-one equivalence can be preserved. This enables us to develop new convex optimization models and algorithms to compute best rank-one approximation of CPS tensors. Numerical experiments from various data are performed to justify the capability of our methods.

math.OC↗

arXiv2016

Approximation Algorithms for Optimization of Real-Valued General Conjugate Complex Forms

Complex polynomial optimization has recently gained more and more attention in both theory and practice. In this paper, we study the optimization of a real-valued general conjugate complex form over various popular constraint sets including the m-th roots of complex unity, the complex unit circle, and the complex unit sphere. A real-valued general conjugate complex form is a homogenous polynomial function of complex variables as well as their conjugates, and always takes real values. General conjugate form optimization is a wide class of complex polynomial optimization models, which include many homogenous polynomial optimization in the real domain with either discrete or continuous variables, and Hermitian quadratic form optimization as well as its higher degree extensions. All the problems under consideration are NP-hard in general and we focus on polynomial-time approximation algorithms with worst-case performance ratios. These approximation ratios improve previous results when restricting our problems to some special classes of complex polynomial optimization, and improve or equate previous results when restricting our problems to some special classes of polynomial optimization in the real domain. These algorithms are based on tensor relaxation and random sampling. Our novel technical contributions are to establish the first set of probability lower bounds for random sampling over the m-th root of unity, the complex unit circle, and the complex unit sphere, and propose the first polarization formula linking general conjugate forms and complex multilinear forms.

math.OC↗

arXiv2016

Characterizing Real-Valued Multivariate Complex Polynomials and Their Symmetric Tensor Representations

In this paper we study multivariate polynomial functions in complex variables and the corresponding associated symmetric tensor representations. The focus is on finding conditions under which such complex polynomials/tensors always take real values. We introduce the notion of symmetric conjugate forms and general conjugate forms, and present characteristic conditions for such complex polynomials to be real-valued. As applications of our results, we discuss the relation between nonnegative polynomials and sums of squares in the context of complex polynomials. Moreover, new notions of eigenvalues/eigenvectors for complex tensors are introduced, extending properties from the Hermitian matrices. Finally, we discuss an important property for symmetric tensors, which states that the largest absolute value of eigenvalue of a symmetric real tensor is equal to its largest singular value; the result is known as Banach's theorem. We show that a similar result holds in the complex case as well.

math.OC↗