arXiv subjects

Mihailo Stojnic

Publications and source records attributed to Mihailo Stojnic.

At least 19 recordsLinked to original sources

arXiv2026

Proving Lehner's formulas via RDT -- indefinite and asymmetric scenarios

The strong asymptotic freeness established in [28,53] allows for the study of norms of Gaussian matrix polynomials via corresponding free operator counterparts. For spectral edges of semicircular free counterparts to symmetric Kronecker-Gaussian matrices, Lehner in [39] determined closed-form analytical characterizations. As an alternative to classical spectral methods, in [62], we created a Random Duality Theory (RDT) based framework for studying these problems and reproved Lehner's formula for definite matrix coefficients. In this work, we develop the RDT machinery further and achieve strong progress in several key directions: (i) Symmetric Square Case: We consider indefinite matrix coefficients and provide spectral edges lower bounds that match Lehner's formula. (ii) Asymmetric Non-Square Variants: We establish RDT asymmetric analogues to Lehner formulas and prove that they lower-bound the spectral edges. (iii) Interlaced Decoupling Property: We uncover that the obtained analogues exhibit a remarkable interlaced decoupling property. Additionally, we consider semi-definite programming (SDP) formulations that allow for the practical solution of the obtained analytical characterizations. Theoretical predictions obtained through solving SDPs are compared to numerical simulations, showing a striking agreement even for problem dimensions on the order of a few hundreds.

math.PR↗

arXiv2026

An RDT based confirmation of Lehner's formula for Kronecker-Gaussian matrices

Remarkable breakthroughs [25,43] established the so-called strong asymptotic freeness between classical Gaussian random ensembles and their semicircular free counterparts. Along the same lines, the Lehner formula [31], associated with the free counterpart, precisely determines the spectral edges of Kronecker-Gaussian matrices. We here revisit and study this formula without the utilization of random matrix theory and spectral methods. In particular, relying on concepts utilized within \emph{Random Duality Theory} (RDT) [45,46,51], we reconfirm Lehner's formula and effectively reprove the key asymptotic freeness results obtained via spectral methods in [25,43].

math.PR↗

arXiv2026

Precise sample covariance spectral norm error -- an RDT view

We study the sample covariance error of centered Gaussians. A remarkable breakthrough [66] established the correct error scaling order and explicitly revealed the critical role of both the effective rank and the true covariance spectrum. In this work, we move beyond scaling characterizations and determine the precise limiting value of the error's spectral norm. To do so, we develop a generic framework based on Random Duality Theory (RDT). Within this framework, we first determine closed-form, explicit RDT-based upper bounds. We then establish complementary lower bounds by introducing a novel bilinear-quadratic RDT lower-bounding mechanism. By combining this mechanism with a two-replica systems bounding strategy, we show that our lower and upper bounds match in large-dimensional contexts. Our theoretical results are supplemented with numerical evaluations and simulations, demonstrating an excellent agreement already for problem sizes on the order of thousands.

math.ST↗

arXiv2026

An RDT based approach to large deviations of Wishart and Wigner matrices spectral edges

We present a novel methodology for studying \emph{large deviations principles} (LDPs) of random matrices. By utilizing a partially lifted variant of \emph{random duality theory} (RDT), we develop a generic LDP framework that completely circumvents traditional random matrix theory (RMT) methods. To demonstrate the framework's simplicity and accuracy, we apply it to the Wishart and Wigner GOE classical statistical ensembles. In both cases, we obtain elegant LDP characterizations of the upper and lower spectral edges that fully match the results achieved through traditional \emph{Coulomb gas} methodologies in [85,95].

math.PR↗

arXiv2026

Ultrametric OGP - parametric RDT \emph{symmetric} binary perceptron connection

In [97,99,100], an fl-RDT framework is introduced to characterize \emph{statistical computational gaps} (SCGs). Studying \emph{symmetric binary perceptrons} (SBPs), [100] obtained an \emph{algorithmic} threshold estimate $α_a\approx α_c^{(7)}\approx 1.6093$ at the 7th lifting level (for $κ=1$ margin), closely approaching $1.58$ local entropy (LE) prediction [18]. In this paper, we further connect parametric RDT to overlap gap properties (OGPs), another key geometric feature of the solution space. Specifically, for any positive integer $s$, we consider $s$-level ultrametric OGPs ($ult_s$-OGPs) and rigorously upper-bound the associated constraint densities $α_{ult_s}$. To achieve this, we develop an analytical union-bounding program consisting of combinatorial and probabilistic components. By casting the combinatorial part as a convex problem and the probabilistic part as a nested integration, we conduct numerical evaluations and obtain that the tightest bounds at the first two levels, $\barα_{ult_1} \approx 1.6578$ and $\barα_{ult_2} \approx 1.6219$, closely approach the 3rd and 4th lifting level parametric RDT estimates, $α_c^{(3)} \approx 1.6576$ and $α_c^{(4)} \approx 1.6218$. We also observe excellent agreement across other key parameters, including overlap values and the relative sizes of ultrametric clusters. Based on these observations, we propose several conjectures linking $ult$-OGP and parametric RDT. Specifically, we conjecture that algorithmic threshold $α_a=\lim_{s\rightarrow\infty} α_{ult_s} = \lim_{s\rightarrow\infty} \barα{ult_s} = \lim_{r\rightarrow\infty} α_{c}^{(r)}$, and $α_{ult_s} \leq α_{c}^{(s+2)}$ (with possible equality for some (maybe even all) $s$). Finally, we discuss the potential existence of a full isomorphism connecting all key parameters of $ult$-OGP and parametric RDT.

cs.LG↗

arXiv2026

Parametric RDT approach to computational gap of symmetric binary perceptron

We study potential presence of statistical-computational gaps (SCG) in symmetric binary perceptrons (SBP) via a parametric utilization of \emph{fully lifted random duality theory} (fl-RDT) [96]. A structural change from decreasingly to arbitrarily ordered $c$-sequence (a key fl-RDT parametric component) is observed on the second lifting level and associated with \emph{satisfiability} ($α_c$) -- \emph{algorithmic} ($α_a$) constraints density threshold change thereby suggesting a potential existence of a nonzero computational gap $SCG=α_c-α_a$. The second level estimate is shown to match the theoretical $α_c$ whereas the $r\rightarrow \infty$ level one is proposed to correspond to $α_a$. For example, for the canonical SBP ($κ=1$ margin) we obtain $α_c\approx 1.8159$ on the second and $α_a\approx 1.6021$ (with converging tendency towards $\sim 1.59$ range) on the seventh level. Our propositions remarkably well concur with recent literature: (i) in [20] local entropy replica approach predicts $α_{LE}\approx 1.58$ as the onset of clustering defragmentation (presumed driving force behind locally improving algorithms failures); (ii) in $α\rightarrow 0$ regime we obtain on the third lifting level $κ\approx 1.2385\sqrt{\frac{α_a}{-\log\left ( α_a \right ) }}$ which qualitatively matches overlap gap property (OGP) based predictions of [43] and identically matches local entropy based predictions of [24]; (iii) $c$-sequence ordering change phenomenology mirrors the one observed in asymmetric binary perceptron (ABP) in [98] and the negative Hopfield model in [100]; and (iv) as in [98,100], we here design a CLuP based algorithm whose practical performance closely matches proposed theoretical predictions.

stat.ML↗

arXiv2025

Binary perceptron computational gap -- a parametric fl RDT view

Recent studies suggest that asymmetric binary perceptron (ABP) likely exhibits the so-called statistical-computational gap characterized with the appearance of two phase transitioning constraint density thresholds: \textbf{\emph{(i)}} the \emph{satisfiability threshold} $α_c$, below/above which ABP succeeds/fails to operate as a storage memory; and \textbf{\emph{(ii)}} \emph{algorithmic threshold} $α_a$, below/above which one can/cannot efficiently determine ABP's weight so that it operates as a storage memory. We consider a particular parametric utilization of \emph{fully lifted random duality theory} (fl RDT) [85] and study its potential ABP's algorithmic implications. A remarkable structural parametric change is uncovered as one progresses through fl RDT lifting levels. On the first two levels, the so-called $\c$ sequence -- a key parametric fl RDT component -- is of the (natural) decreasing type. A change of such phenomenology on higher levels is then connected to the $α_c$ -- $α_a$ threshold change. Namely, on the second level concrete numerical values give for the critical constraint density $α=α_c\approx 0.8331$. While progressing through higher levels decreases this estimate, already on the fifth level we observe a satisfactory level of convergence and obtain $α\approx 0.7764$. This allows to draw two striking parallels: \textbf{\emph{(i)}} the obtained constraint density estimate is in a remarkable agrement with range $α\in (0.77,0.78)$ of clustering defragmentation (believed to be responsible for failure of locally improving algorithms) [17,88]; and \textbf{\emph{(ii)}} the observed change of $\c$ sequence phenomenology closely matches the one of the negative Hopfield model for which the existence of efficient algorithms that closely approach similar type of threshold has been demonstrated recently [87].

stat.ML↗

arXiv2025

Ground state energies of multipartite $p$-spin models -- partially lifted RDT view

We consider ground state energies (GSE) of multipartite $p$-spin models. Relying on partially lifted random duality theory (pl RDT) concepts we introduce an analytical mechanism that produces easy to compute lower and upper GSE bounds for \emph{any} spin sets. We uncover that these bounds actually match in case of fully spherical sets thereby providing optimal GSE values for spherical multipartite pure $p$-spin models. Numerical evidence further suggests that our upper and lower bounds may match even in the Ising scenarios. As such developments are rather intriguing, we formulate several questions regarding the connection between our bounds matching generality on the one side and the spin sets structures on the other.

math.PR↗

arXiv2025

CLuP practically achieves $\sim 1.77$ positive and $\sim 0.33$ negative Hopfield model ground state free energy

We study algorithmic aspects of finding $n$-dimensional \emph{positive} and \emph{negative} Hopfield ($\pm$Hop) model ground state free energies. This corresponds to classical maximization of random positive/negative semi-definite quadratic forms over binary $\left \{\pm \frac{1}{\sqrt{n}} \right \}^n$ vectors. The key algorithmic question is whether these problems can be computationally efficiently approximated within a factor $\approx 1$. Following the introduction and success of \emph{Controlled Loosening-up} (CLuP-SK) algorithms in finding near ground state energies of closely related Sherrington-Kirkpatrick (SK) models [82], we here propose a CLuP$\pm$Hop counterparts for $\pm$Hop models. Fully lifted random duality theory (fl RDT) [78] is utilized to characterize CLuP$\pm$Hop \emph{typical} dynamics. An excellent agreement between practical performance and theoretical predictions is observed. In particular, for $n$ as small as few thousands CLuP$\pm$Hop achieve $\sim 1.77$ and $\sim 0.33$ as the ground state free energies of the positive and negative Hopfield models. At the same time we obtain on the 6th level of lifting (6-spl RDT) corresponding theoretical thermodynamic ($n\rightarrow\infty$) limits $\approx 1.7784$ and $\approx 0.3281$. This positions determining Hopfield models near ground state energies as \emph{typically} easy problems. Moreover, the very same 6th lifting level evaluations allow to uncover a fundamental intrinsic difference between two models: $+$Hop's near optimal configurations are \emph{typically close} to each other whereas the $-$Hop's are \emph{typically far away}.

cond-mat.dis-nn↗

arXiv2025

A CLuP algorithm to practically achieve $\sim 0.76$ SK--model ground state free energy

We consider algorithmic determination of the $n$-dimensional Sherrington-Kirkpatrick (SK) spin glass model ground state free energy. It corresponds to a binary maximization of an indefinite quadratic form and under the \emph{worst case} principles of the classical NP complexity theory it is hard to approximate within a $\log(n)^{const.}$ factor. On the other hand, the SK's random nature allows (polynomial) spectral methods to \emph{typically} approach the optimum within a constant factor. Naturally one is left with the fundamental question: can the residual (constant) \emph{computational gap} be erased? Following the success of \emph{Controlled Loosening-up} (CLuP) algorithms in planted models, we here devise a simple practical CLuP-SK algorithmic procedure for (non-planted) SK models. To analyze the \emph{typical} success of the algorithm we associate to it (random) CLuP-SK models. Further connecting to recent random processes studies [94,97], we characterize the models and CLuP-SK algorithm via fully lifted random duality theory (fl RDT) [98]. Moreover, running the algorithm we demonstrate that its performance is in an excellent agrement with theoretical predictions. In particular, already for $n$ on the order of a few thousands CLuP-SK achieves $\sim 0.76$ ground state free energy and remarkably closely approaches theoretical $n\rightarrow\infty$ limit $\approx 0.763$. For all practical purposes, this renders computing SK model's near ground state free energy as a \emph{typically} easy problem.

cond-mat.dis-nn↗

arXiv2025

Fully lifted \emph{blirp} interpolation -- a large deviation view

[104] introduced a powerful \emph{fully lifted} (fl) statistical interpolating mechanism. It established a nested connection between blirps (bilinearly indexed random processes) and their decoupled (linearly indexed) comparative counterparts. We here revisit the comparison from [104] and introduce its a \emph{large deviation} upgrade. The new machinery allows to substantially widen the [104]'s range of applicability. In addition to \emph{typical}, studying analytically much harder \emph{atypical} random structures features is now possible as well. To give a bit of a practical flavor, we show how the obtained results connect to the so-called \emph{local entropies} (LE) and their predicated role in understanding solutions clustering and associated \emph{computational gaps} in hard random optimization problems. As was the case in [104], even though the technical considerations often appear as fairly involved, the final interpolating forms admit elegant expressions thereby providing a relatively easy to use tool readily available for further studies. Moreover, as the considered models encompass all well known random structures discussed in [104], the obtained results automatically apply to them as well.

math.PR↗

arXiv2025

A large deviation view of \emph{stationarized} fully lifted blirp interpolation

We consider \emph{bilinearly indexed random processes} (blirp) and study their interpolating comparative mechanisms. Generic introduction of the \emph{fully lifted} (fl) blirp interpolation in [105] was followed by a corresponding stationarization counterpart in [103]. A \emph{large deviation} upgrade of [105] introduced in companion paper [106] is complemented here with the corresponding one of [103]. Similarly to [106], the mechanism that we introduce extends the range of [103]'s applicability so that it encompasses random structures \emph{atypical} features. Among others these include the \emph{local entropies} (LE) which explain atypical solutions clusterings in hard random optimization problems believed to be directly responsible for the presumable existence of the so-called \emph{computational gaps}. Moreover (and similar to [105]), despite on occasion somewhat involved technical considerations, the final forms of the uncovered fundamental interpolating parameters relations are rather elegant and as such provide a valuable tool readily available for further use.

math.PR↗

arXiv2025

Rare dense solutions clusters in asymmetric binary perceptrons -- local entropy via fully lifted RDT

We study classical asymmetric binary perceptron (ABP) and associated \emph{local entropy} (LE) as potential source of its algorithmic hardness. Isolation of \emph{typical} ABP solutions in SAT phase seemingly suggests a universal algorithmic hardness. Paradoxically, efficient algorithms do exist even for constraint densities $α$ fairly close but at a finite distance (\emph{computational gap}) from the capacity. In recent years, existence of rare large dense clusters and magical ability of fast algorithms to find them have been posited as the conceptual resolution of this paradox. Monotonicity or breakdown of the LEs associated with such \emph{atypical} clusters are predicated to play a key role in their thinning-out or even complete defragmentation. Invention of fully lifted random duality theory (fl RDT) [90,93,94] allows studying random structures \emph{typical} features. A large deviation upgrade, sfl LD RDT [96,97], moves things further and enables \emph{atypical} features characterizations as well. Utilizing the machinery of [96,97] we here develop a generic framework to study LE as an ABP's atypical feature. Already on the second level of lifting we discover that the LE results are closely matching those obtained through replica methods. For classical zero threshold ABP, we obtain that LE breaks down for $α$ in $(0.77,0.78)$ interval which basically matches $α\sim 0.75-0.77$ range that currently best ABP solvers can handle and effectively indicates that LE's behavior might indeed be among key reflections of the ABP's computational gaps presumable existence.

stat.ML↗

arXiv2025

Phase transition of \emph{descending} phase retrieval algorithms

We study theoretical limits of \emph{descending} phase retrieval algorithms. Utilizing \emph{Random duality theory} (RDT) we develop a generic program that allows statistical characterization of various algorithmic performance metrics. Through these we identify the concepts of \emph{parametric manifold} and its \emph{funneling points} as key mathematical objects that govern the underlying algorithms' behavior. An isomorphism between single funneling point manifolds and global convergence of descending algorithms is established. The structure and shape of the parametric manifold as well as its dependence on the sample complexity are studied through both plain and lifted RDT. Emergence of a phase transition is observed. Namely, as sample complexity increases, parametric manifold transitions from a multi to a single funneling point structure. This in return corresponds to a transition from the scenarios where descending algorithms generically fail to the scenarios where they succeed in solving phase retrieval. We also develop and implement a practical algorithmic variant that in a hybrid alternating fashion combines a barrier and a plain gradient descent. Even though the theoretical results are obtained for infinite dimensional scenarios (and consequently non-jittery parametric manifolds), we observe a strong agrement between theoretical and simulated phase transitions predictions for fairly small dimensions on the order of a few hundreds.

stat.ML↗

arXiv2025

Optimal spectral initializers impact on phase retrieval phase transitions -- an RDT view

We analyze the relation between spectral initializers and theoretical limits of \emph{descending} phase retrieval algorithms (dPR). In companion paper [104], for any sample complexity ratio, $α$, \emph{parametric manifold}, ${\mathcal {PM}}(α)$, is recognized as a critically important structure that generically determines dPRs abilities to solve phase retrieval (PR). Moreover, overlap between the algorithmic solution and the true signal is positioned as a key ${\mathcal {PM}}$'s component. We here consider the so-called \emph{overlap optimal} spectral initializers (OptSpins) as dPR's starting points and develop a generic \emph{Random duality theory} (RDT) based program to statistically characterize them. In particular, we determine the functional structure of OptSpins and evaluate the starting overlaps that they provide for the dPRs. Since ${\mathcal {PM}}$'s so-called \emph{flat regions} are highly susceptible to \emph{local jitteriness} and as such are key obstacles on dPR's path towards PR's global optimum, a precise characterization of the starting overlap allows to determine if such regions can be successfully circumvented. Through the presented theoretical analysis we observe two key points in that regard: \textbf{\emph{(i)}} dPR's theoretical phase transition (critical $α$ above which they solve PR) might be difficult to practically achieve as the ${\mathcal {PM}}$'s flat regions are large causing the associated OptSpins to fall exactly within them; and \textbf{\emph{(ii)}} Opting for so-called ``\emph{safer compression}'' and slightly increasing $α$ (by say $15\%$) shrinks flat regions and allows OptSpins to fall outside them and dPRs to ultimately solve PR. Numerical simulations are conducted as well and shown to be in an excellent agreement with theoretical predictions.

stat.ML↗

arXiv2025

Phase retrieval with rank $d$ measurements -- \emph{descending} algorithms phase transitions

Companion paper [118] developed a powerful \emph{Random duality theory} (RDT) based analytical program to statistically characterize performance of \emph{descending} phase retrieval algorithms (dPR) (these include all variants of gradient descents and among them widely popular Wirtinger flows). We here generalize the program and show how it can be utilized to handle rank $d$ positive definite phase retrieval (PR) measurements (with special cases $d=1$ and $d=2$ serving as emulations of the real and complex phase retrievals, respectively). In particular, we observe that the minimal sample complexity ratio (number of measurements scaled by the dimension of the unknown signal) which ensures dPR's success exhibits a phase transition (PT) phenomenon. For both plain and lifted RDT we determine phase transitions locations. To complement theoretical results we implement a log barrier gradient descent variant and observe that, even in small dimensional scenarios (with problem sizes on the order of 100), the simulated phase transitions are in an excellent agreement with the theoretical predictions.

stat.ML↗

arXiv2024

Deep ReLU networks -- injectivity capacity upper bounds

We study deep ReLU feed forward neural networks (NN) and their injectivity abilities. The main focus is on \emph{precisely} determining the so-called injectivity capacity. For any given hidden layers architecture, it is defined as the minimal ratio between number of network's outputs and inputs which ensures unique recoverability of the input from a realizable output. A strong recent progress in precisely studying single ReLU layer injectivity properties is here moved to a deep network level. In particular, we develop a program that connects deep $l$-layer net injectivity to an $l$-extension of the $\ell_0$ spherical perceptrons, thereby massively generalizing an isomorphism between studying single layer injectivity and the capacity of the so-called (1-extension) $\ell_0$ spherical perceptrons discussed in [82]. \emph{Random duality theory} (RDT) based machinery is then created and utilized to statistically handle properties of the extended $\ell_0$ spherical perceptrons and implicitly of the deep ReLU NNs. A sizeable set of numerical evaluations is conducted as well to put the entire RDT machinery in practical use. From these we observe a rapidly decreasing tendency in needed layers' expansions, i.e., we observe a rapid \emph{expansion saturation effect}. Only $4$ layers of depth are sufficient to closely approach level of no needed expansion -- a result that fairly closely resembles observations made in practical experiments and that has so far remained completely untouchable by any of the existing mathematical methodologies.

stat.ML↗

arXiv2024

Injectivity capacity of ReLU gates

We consider the injectivity property of the ReLU networks layers. Determining the ReLU injectivity capacity (ratio of the number of layer's inputs and outputs) is established as isomorphic to determining the capacity of the so-called $\ell_0$ spherical perceptron. Employing \emph{fully lifted random duality theory} (fl RDT) a powerful program is developed and utilized to handle the $\ell_0$ spherical perceptron and implicitly the ReLU layers injectivity. To put the entire fl RDT machinery in practical use, a sizeable set of numerical evaluations is conducted as well. The lifting mechanism is observed to converge remarkably fast with relative corrections in the estimated quantities not exceeding $\sim 0.1\%$ already on the third level of lifting. Closed form explicit analytical relations among key lifting parameters are uncovered as well. In addition to being of incredible importance in handling all the required numerical work, these relations also shed a new light on beautiful parametric interconnections within the lifting structure. Finally, the obtained results are also shown to fairly closely match the replica predictions from [40].

stat.ML↗