arXiv subjects

Jinze Zhao

Publications and source records attributed to Jinze Zhao.

9 recordsLinked to original sources

arXiv2026

Constant Steps Are s-Composable: An Exact Interpolation Certificate for Gradient Descent

Grimmer, Shu, and Wang asked whether a balanced constant schedule is $s$-composable at every horizon. More precisely, for an integer $n\geq 1$, let $\bar{h}=1+r$, where $r\in(0,1)$ is the unique solution of $$r^n\bigl(1+n(1+r)\bigr)=1,$$ and run gradient descent for $n$ steps with normalized stepsize $\bar{h}$. The cases $n=1,2$ were known, while the general case $n\geq3$ was left open. We prove the conjecture for every $n$. Our proof gives an explicit, dimension-free nonnegative linear combination of the smooth convex interpolation inequalities. The certificate is assembled from matrices supported on contiguous index intervals. Its off-diagonal entries are automatically positive, its quadratic part is diagonal, and its remaining multipliers reduce to two scalar families. We derive closed forms for those families and prove positivity using strict concavity and elementary rational inequalities. Consequently, for every $L$-smooth convex function, the constant schedule satisfies the sharp mixed terminal Lyapunov inequality conjectured in the original paper, together with the associated simultaneous objective-gap and gradient-norm bounds. All exceptional horizons and boundary indices are treated explicitly.

math.OC↗

arXiv2026

Small-Ball Marginals Do Not Control Restricted Eigenvalues by Euclidean Gaussian Width

Banerjee, Chen, and Sivakumar asked at COLT 2015 whether a uniform small-ball condition on the rows of a random design matrix forces a restricted-eigenvalue lower bound whose sample complexity is governed by the ordinary Euclidean Gaussian width of an arbitrary spherical subset. We give a negative answer to the natural distribution-free formulation of that question. For every sample size $n$, we construct a centered, genuinely heavy-tailed row distribution in dimension $p=4^n+1$ and a set $A=C\cap\mathbb{S}^{p-1}$, where $C$ is a closed polyhedral convex cone, such that \[ \inf_{v\ne 0}\mathbb{P}\!\left( \left|\left\langle Z,v\right\rangle\right| \ge \frac{\left\lVert v\right\rVert_2}{\sqrt{2}} \right)\ge \frac{1}{12} \quad\text{and}\quad w(A)<2. \] Nevertheless, for the matrix $X$ with $n$ independent copies of $Z$ as rows, \[ \mathbb{P}\!\left( \inf_{u\in A}\left\lVert Xu\right\rVert_2^2=0 \right) \ge 1-\exp(-2^n). \] Thus no positive constants depending only on the fixed small-ball parameters can yield a lower bound of the proposed form $c_1n-c_2w(A)^2$ with high probability. The construction isolates the obstruction: a marginal small-ball lower bound controls every fixed direction, but does not control the distribution-dependent complexity of searching over many directions. We state the quantifiers explicitly and discuss why isotropic or upper-tail assumptions lead to a different, still meaningful problem.

math.PR↗

arXiv2026

Predicting Diagonalizability of a Mean Matrix

Wu and Santhanam asked whether one can determine, from an increasing i.i.d. sample of binary random matrices, whether the unknown mean matrix is diagonalizable while making only finitely many errors almost surely. We answer this question affirmatively, for diagonalizability over either $\mathbb{R}$ or $\mathbb{C}$. The main observation is a general principle: every semialgebraic property of a fixed-dimensional bounded mean parameter is eventually almost surely predictable. We give a self-contained shrinking-confidence-set proof and an explicit predictor obtained from polynomial sign tests. Tarski--Seidenberg quantifier elimination shows that both the real- and complex-diagonalizable loci are semialgebraic, despite being neither closed nor open. We further extend the positive result to unbounded observations with any fixed finite moment of order $r>1$, using the Marcinkiewicz--Zygmund strong law. Combined with the Dembo--Peres topological criterion, this yields a sharp contrast: over the class of all merely integrable matrix laws, diagonalizability is not eventually almost surely predictable when the dimension is at least two. The construction is effective for fixed dimension, although no practical complexity bound is claimed.

math.ST↗

arXiv2026

From Approachability Residuals to Anytime-Valid Evidence: The Online Convex Geometry of Testing by Betting

Betting-based sequential tests and Blackwell approachability are linked by a rate-explicit reduction through support-function residuals. For a compact convex target $S$ and vector observations $r_t$, an OCO learner selects a predictable normal $w_t$ and produces $q_t=\langle w_t,r_t\rangle-h_S(w_t)$. We prove the exact pathwise identity $$ \dist(\bar r_T,S) =\frac1T\sum_{t=1}^Tq_t+\frac{\Reg_T}{T}. $$ When $|q_t|\leq B$, composing this identity with one-sided betting yields a finite-time transfer: if the OCO and log-wealth regrets are at most $a_T$ and $\ell_T$, respectively, then a target gap exceeding \[ \frac{a_T}{T} +2B\sqrt{\frac{\log(1/α)+\ell_T}{T}} \] forces rejection by time $T$, while non-rejection certifies the converse radius. We then formulate a controlled stochastic experiment in which an action selected after $w_t$ satisfies Blackwell's supporting-halfspace condition for every null mean payoff. The resulting wealth is an e-process under adaptive nulls; sublinear OCO regret gives stochastic approachability, whereas persistent mean separation under an alternative gives exponential wealth at rate at least $δ^2/(4B^2)$. Deterministic Blackwell games and passive tests are, respectively, the noise-free and singleton-action cases of this protocol. Bounded two-sample means, kernel MMD, and active heterogeneous data sources instantiate the reduction. The resulting connection is exact algebraically, quantitative at finite time, and operational when experiments are controlled.

cs.LG↗

arXiv2026

Pyramidal Width Can Increase Under Vertex Insertion

Lacoste-Julien and Jaggi conjectured in 2015 that the pyramidal width of a polytope cannot increase when a vertex is added, provided that every old point remains a vertex. We give an exact counterexample with six integer points in $\R^3$. For \[ P=\conv\{v_0,\ldots,v_4\},\qquad Q=\conv\{v_0,\ldots,v_5\}, \] where \[ \begin{aligned} v_0&=(-1,-3,-1), & v_1&=(3,2,-2), & v_2&=(0,2,1),\\ v_3&=(-1,-3,3), & v_4&=(-2,0,1), & v_5&=(-1,0,-2), \end{aligned} \] all five vertices of $P$ remain vertices of $Q$, but \[ \PWidth(P)^2=\frac{48}{353} \quad\text{and}\quad \PWidth(Q)^2=\frac{36}{133}. \] Thus vertex insertion increases pyramidal width by the factor $\sqrt{1059/532}\approx 1.410886779$. The proof uses the equivalence between pyramidal width and facial distance, certifies both face lattices by integer supporting hyperplanes, and evaluates every facial distance by a finite rational calculation. A dependency-free exact verifier accompanies the paper.

cs.LG↗

arXiv2025

Sparse Mixture-of-Experts for Compositional Generalization: Empirical Evidence and Theoretical Foundations of Optimal Sparsity

Sparse Mixture-of-Experts (SMoE) architectures have gained prominence for their ability to scale neural networks, particularly transformers, without a proportional increase in computational cost. Despite their success, their role in compositional generalization, i.e., adapting to novel combinations of known components, remains under-explored. This study challenges the assumption that minimal expert activation suffices for task generalization and investigates the relationship between task complexity and optimal sparsity in SMoE models. Through empirical evaluations on the SRAVEN symbolic reasoning task and the SKILL-MIX benchmark, we demonstrate that (i) the number of activated experts consistently increases with the perceived task difficulty to maintain performance; and (ii) the optimal number of activated experts scales proportionally with task complexity. Our theoretical analysis derives a scaling law for optimal sparsity by balancing approximation and estimation errors, revealing alignment with empirical observations. We formally show that the optimal sparsity lies between minimal activation (1-2 experts) and full activation, with the exact number scaling proportionally to task complexity and further influenced by the size of the training data and the complexity of the model. These findings offer practical insights for designing SMoE models that achieve computational efficiency while enabling robust compositional generalization.

cs.LG↗

arXiv2025

Meta ControlNet: Enhancing Task Adaptation via Meta Learning

Diffusion-based image synthesis has attracted extensive attention recently. In particular, ControlNet that uses image-based prompts exhibits powerful capability in image tasks such as canny edge detection and generates images well aligned with these prompts. However, vanilla ControlNet generally requires extensive training of around 5000 steps to achieve a desirable control for a single task. Recent context-learning approaches have improved its adaptability, but mainly for edge-based tasks, and rely on paired examples. Thus, two important open issues are yet to be addressed to reach the full potential of ControlNet: (i) zero-shot control for certain tasks and (ii) faster adaptation for non-edge-based tasks. In this paper, we introduce a novel Meta ControlNet method, which adopts the task-agnostic meta learning technique and features a new layer freezing design. Meta ControlNet significantly reduces learning steps to attain control ability from 5000 to 1000. Further, Meta ControlNet exhibits direct zero-shot adaptability in edge-based tasks without any finetuning, and achieves control within only 100 finetuning steps in more complex non-edge tasks such as Human Pose, outperforming all existing methods. The codes is available in https://github.com/JunjieYang97/Meta-ControlNet.

cs.CV↗

arXiv2024

Generalization Error Analysis for Sparse Mixture-of-Experts: A Preliminary Study

Mixture-of-Experts (MoE) represents an ensemble methodology that amalgamates predictions from several specialized sub-models (referred to as experts). This fusion is accomplished through a router mechanism, dynamically assigning weights to each expert's contribution based on the input data. Conventional MoE mechanisms select all available experts, incurring substantial computational costs. In contrast, Sparse Mixture-of-Experts (Sparse MoE) selectively engages only a limited number, or even just one expert, significantly reducing computation overhead while empirically preserving, and sometimes even enhancing, performance. Despite its wide-ranging applications and these advantageous characteristics, MoE's theoretical underpinnings have remained elusive. In this paper, we embark on an exploration of Sparse MoE's generalization error concerning various critical factors. Specifically, we investigate the impact of the number of data samples, the total number of experts, the sparsity in expert selection, the complexity of the routing mechanism, and the complexity of individual experts. Our analysis sheds light on \textit{how \textbf{sparsity} contributes to the MoE's generalization}, offering insights from the perspective of classical learning theory.

cs.LG↗

arXiv2023

CrossEAI: Using Explainable AI to generate better bounding boxes for Chest X-ray images

Explainability is critical for deep learning applications in healthcare which are mandated to provide interpretations to both patients and doctors according to legal regulations and responsibilities. Explainable AI methods, such as feature importance using integrated gradients, model approximation using LIME, or neuron activation and layer conductance to provide interpretations for certain health risk predictions. In medical imaging diagnosis, disease classification usually achieves high accuracy, but generated bounding boxes have much lower Intersection over Union (IoU). Different methods with self-supervised or semi-supervised learning strategies have been proposed, but few improvements have been identified for bounding box generation. Previous work shows that bounding boxes generated by these methods are usually larger than ground truth and contain major non-disease area. This paper utilizes the advantages of post-hoc AI explainable methods to generate bounding boxes for chest x-ray image diagnosis. In this work, we propose CrossEAI which combines heatmap and gradient map to generate more targeted bounding boxes. By using weighted average of Guided Backpropagation and Grad-CAM++, we are able to generate bounding boxes which are closer to the ground truth. We evaluate our model on a chest x-ray dataset. The performance has significant improvement over the state of the art model with the same setting, with $9\%$ improvement in average of all diseases over all IoU. Moreover, as a model that does not use any ground truth bounding box information for training, we achieve same performance in general as the model that uses $80\%$ of the ground truth bounding box information for training

eess.IV↗