Searcharxiv⌕ Search

arXiv subjects

Keisuke Yano

Publications and source records attributed to Keisuke Yano.

At least 19 recordsLinked to original sources
arXiv2026

From DPPs to $k$-DPPs: identifiability analysis via spectral decomposition

We study the geometry of determinantal point processes (DPPs) through the spectral decomposition $L=U\Lambda U^{\top}$. The spectrum $\Lambda$ governs the cardinality distribution via elementary symmetric polynomials, while the eigenspace orientation $U$ governs the conditional law within each fixed-cardinality stratum. Conditioning on cardinality $k$ yields the $k$-DPP, for which the identifiability structure changes fundamentally: the spectral parameter becomes identifiable only up to a common scale, and the eigenspace rotation parameter is identifiable only through squared minors of the eigenvector matrix. We characterize the identifiability gap precisely, via three explicit invariances (scale, sign similarity, and eigenspace rotation) and a dimension-counting theorem showing the existence of additional continuous non-identifiability whenever $\binom{N}{k}<N(N+1)/2$. In contrast, for the full DPP the non-identifiability comes only from the discrete sign similarity.

stat.ML↗
arXiv2026

Feature Starvation as Geometric Instability in Sparse Autoencoders

Sparse autoencoders (SAEs) are used to disentangle the dense, polysemantic internal representations of large language models (LLMs) into interpretable, monosemantic concepts. However, standard $\ell_1$-regularized SAEs suffer from feature starvation (dead neurons) and shrinkage bias, often requiring computationally expensive heuristic resampling and nondifferentiable hard-masking methods to bypass these challenges. We argue that feature starvation is not merely an empirical artifact of poor data diversity, but a fundamental optimization-geometric pathology of overcomplete dictionaries: the $\ell_1$-induced sparse coding map is unstable and fundamentally misaligned with shallow, amortized encoders. To address this structural instability, we introduce adaptive elastic net SAEs (AEN-SAEs), a fully differentiable architecture grounded in classical sparse regression. AEN-SAEs combine an $\ell_2$ structural term that enforces strong convexity and Lipschitz stability with adaptive $\ell_1$ reweighting that eliminates shrinkage bias and suppresses spurious features, thereby jointly controlling the curvature and interaction structure of the induced polyhedral geometry. Theoretically, we show that AEN-SAEs yield a Lipschitz-continuous sparse coding map and recover the global feature support under mild assumptions. Empirically, across synthetic settings and LLMs (Pythia 70M, Llama 3.1 8B), AEN-SAEs mitigate feature starvation without auxiliary heuristics while maintaining competitive reconstruction abilities.

cs.LG↗
arXiv2026

Riemannian Optimization over Symmetric Positive Definite Matrices with the Alpha-Procrustes Geometry

In Riemannian optimization, it is well known that the condition number of the Riemannian Hessian at an optimum strongly influences the asymptotic convergence behavior of optimization algorithms. On the manifold of symmetric positive definite (SPD) matrices, several commonly used metrics for optimization, such as the Affine-Invariant (AI) and Bures--Wasserstein (BW) metrics, tend to become ill-conditioned as the underlying SPD matrix becomes ill-conditioned. As a result, even when the Euclidean Hessian remains uniformly well-conditioned on the SPD manifold, optimization may still become difficult near an optimum associated with an ill-conditioned SPD matrix. In this paper, we address this issue through the Alpha-Procrustes (AP) geometry on the SPD manifold. This geometry generalizes several well-known metrics, including the Log-Euclidean (LE) metric for \(\alpha=0\) and the BW metric for \(\alpha=1/2\). We first show that, when \(\alpha=1\), all eigenvalues of the Riemannian metric operator induced by the AP geometry are uniformly bounded independently of the underlying SPD matrix. Therefore, under the assumption that the Euclidean Hessian satisfies the uniform spectral bounds, all the eigenvalues of the corresponding Riemannian Hessian are uniformly bounded independently of the underlying SPD matrix. Consequently, the case \(\alpha=1\) provides a robust geometric framework for several Riemannian optimization problems involving ill-conditioned SPD matrices. Finally, we validate our theoretical findings through extensive numerical experiments across a range of applications.

math.OC↗
arXiv2026

Trajectory-Restricted Optimization Conditions and Geometry-Aware Linear Convergence

Linear convergence of first-order methods is typically characterized by global optimization conditions whose constants reflect worst-case geometry of the ambient space. In high-dimensional or structured problems, these global constants can be arbitrarily conservative and fail to capture the geometry actually encountered by optimization trajectories. In this paper, we develop a trajectory-restricted framework for linear convergence based on localized geometric regularity. We introduce restricted variants of the Polyak--{\L}ojasiewicz inequality, error bound, and quadratic growth conditions that are required to hold only on subsets of the domain. We show that classical convergence guarantees extend under these localized conditions, and in key cases, we develop new arguments that yield explicit relationships between the corresponding constants. The resulting rates are governed by geometric quantities associated with the regions traversed by the algorithm. For polyhedral composite problems, we prove that convergence is controlled by restricted Hoffman constants corresponding to the active polyhedral faces visited along the trajectory. Once the iterates enter a well-conditioned face, the effective condition number improves accordingly. Our work provides a geometric quantification for fast local convergence after active-set or manifold identification and more broadly suggests that linear convergence is fundamentally governed by the geometry of the subsets explored by the algorithm, rather than by worst-case global conditioning.

math.OC↗
arXiv2026

Empirical Bayes Predictive Density Estimation under Covariate Shift in Large Imbalanced Linear Mixed Models

We study empirical Bayes (EB) predictive density estimation in linear mixed models (LMMs) with large number of units, which induce a high dimensional random effects space. Focusing on Kullback Leibler (KL) risk minimization, we develop a calibration framework to optimally tune predictive densities derived from on a broad class of flexible priors. Our proposed method addresses two key challenges in predictive inference: (a) severe data scarcity leading to highly imbalanced designs, in which replicates are available for only a small subset of units; and (b) distributional shifts in future covariates. To estimate predictive KL risk in LMMs, we use a data-fission approach that leverages exchangeability in the covariate distribution. We establish convergence rates for our proposed risk estimators and show how their efficiency deteriorates as data scarcity increases. Our results imply the decision-theoretic optimality of the proposed EB predictive density estimator. The theoretical development relies on a novel probabilistic analysis of the interaction between data fission, sample reuse, and the predictive heat-equation representation of George et al. (2006), which expresses predictive KL risk through expected log-marginals. Extensive simulation studies demonstrate strong predictive performance and robustness of the proposed approach across diverse regimes with varying degrees of data scarcity and covariate shift.

stat.ME↗
arXiv2026

Rigorous Formulation of Finite-Sample and Finite-Window Effects in Galaxy Clustering

Galaxy surveys provide finite catalogs of objects observed within bounded volumes, yet clustering statistics are often interpreted using theoretical frameworks developed for infinite point processes. In this work, we formulate key statistical quantities directly for finite point processes and examine the structural consequences of finite-number and finite-window constraints. We show that several well-known features of galaxy survey analysis arise naturally from finiteness alone. In particular, non-vanishing higher-order connected correlations can occur even in statistically independent samples when the total number of points is fixed, and the integral constraint in two-point statistics appears as an exact identity implied by the finite-number condition rather than as an estimator artifact. We further demonstrate that counts-in-cells and point-centered environmental measures correspond to distinct statistical ensembles. Using Palm conditioning, we derive an exact relation between random-cell and point-centered statistics, showing that the latter probe a tilted version of the underlying distribution. These results provide a probabilistic framework for separating structural effects imposed by finite sampling from correlations reflecting genuine astrophysical processes. The formulation presented here remains valid for realistic survey geometries and finite data sets and clarifies the interpretation of commonly used clustering statistics in galaxy surveys.

astro-ph.GA↗
arXiv2026

The geometry of Stein's method of moments: A canonical decomposition via score matching

In this paper, we elucidate the geometry of Stein's method of moments (SMoM). SMoM is a parameter estimation method based on the Stein operator, and yields a wide class of estimators that do not depend on the normalizing constant. We present a canonical decomposition of an SMoM estimator after centering the score matching estimator, which sheds light on the central role of the score matching within the SMoM framework. Using this decomposition, we construct an SMoM estimator that improves upon the score matching estimator in the asymptotic variance. We also discuss the connection between SMoM and the Wasserstein geometry. Specifically, using the Wasserstein score function, we provide a geometrical interpretation of the gap in the asymptotic variance between the score matching estimator and the maximum likelihood estimator. Furthermore, it is shown that the score matching estimator is asymptotically efficient if and only if the Fisher score functions span the same space as the Wasserstein score functions.

math.ST↗
arXiv2025

Dual Riemannian Newton Method on Statistical Manifolds

In probabilistic modeling, parameter estimation is commonly formulated as a minimization problem on a parameter manifold. Optimization in such spaces requires geometry-aware methods that respect the underlying information structure. While the natural gradient leverages the Fisher information metric as a form of Riemannian gradient descent, it remains a first-order method and often exhibits slow convergence near optimal solutions. Existing second-order manifold algorithms typically rely on the Levi-Civita connection, thus overlooking the dual-connection structure that is central to information geometry. We propose the dual Riemannian Newton method, a Newton-type optimization algorithm on manifolds endowed with a metric and a pair of dual affine connections. The dual Riemannian Newton method explicates how duality shapes second-order updates: when the retraction (a local surrogate of the exponential map) is defined by one connection, the associated Newton equation is posed with its dual. We establish local quadratic convergence and validate the theory with experiments on representative statistical models. Thus, the dual Riemannian Newton method thus delivers second-order efficiency while remaining compatible with the dual structures that underlie modern information-geometric learning and inference.

stat.CO↗
arXiv2024

Duality induced by an embedding structure of determinantal point process

This paper investigates the information geometrical structure of a determinantal point process (DPP). It demonstrates that a DPP is embedded in the exponential family of log-linear models. The extent of deviation from an exponential family is analyzed using the $\mathrm{e}$-embedding curvature tensor, which identifies partially flat parameters of a DPP. On the basis of this embedding structure, the duality related to a marginal kernel and an $L$-ensemble kernel is discovered.

math.ST↗
arXiv2023

Matching prior pairs connecting Maximum A Posteriori estimation and posterior expectation

Bayesian statistics has two common measures of central tendency of a posterior distribution: posterior means and Maximum A Posteriori (MAP) estimates. In this paper, we discuss a connection between MAP estimates and posterior means. We derive an asymptotic condition for a pair of prior densities under which the posterior mean based on one prior coincides with the MAP estimate based on the other prior. A sufficient condition for the existence of this prior pair relates to $\alpha$-flatness of the statistical model in information geometry. We also construct a matching prior pair using $\alpha$-parallel priors. Our result elucidates an interesting connection between regularization in generalized linear regression models and posterior expectation.

math.ST↗
arXiv2023

On robustness of Spectral R\'{e}nyi divergence

This paper studies a specific class of statistical divergences for spectral densities of time series: the spectral $\alpha$-R\'{e}nyi divergences, which include the Itakura-Saito divergence as a limiting case. The aim of this paper is to highlight both information-theoretic and statistical properties of spectral $\alpha$-R\'{e}nyi divergences. We reveal the connection between the spectral $\alpha$-R\'{e}nyi divergence and the $\gamma$-divergence in robust statistics, and a variational representation of the spectral $\alpha$-R\'{e}nyi divergence. Inspired by these results suggesting "robustness" of spectral $\alpha$-R\'{e}nyi divergence, we show that the minimum spectral R\'{e}nyi divergence estimate has a stable optimization path with respect to outliers in the frequency domain, unlike the minimum Itakura-Saito divergence estimator, and thus it delivers more stable estimates, reducing the need for intricate pre-processing.

math.ST↗
arXiv2023

Minimum information dependence modeling

We propose a method to construct a joint statistical model for mixed-domain data to analyze their dependence. Multivariate Gaussian and log-linear models are particular examples of the proposed model. It is shown that the functional equation defining the model has a unique solution under fairly weak conditions. The model is characterized by two orthogonal parameters: the dependence parameter and the marginal parameter. To estimate the dependence parameter, a conditional inference together with a sampling procedure is proposed and is shown to provide a consistent estimator. Illustrative examples of data analyses involving penguins and earthquakes are presented.

stat.ME↗
arXiv2023

A generalization gap estimation for overparameterized models via the Langevin functional variance

This paper discusses the estimation of the generalization gap, the difference between generalization performance and training performance, for overparameterized models including neural networks. We first show that a functional variance, a key concept in defining a widely-applicable information criterion, characterizes the generalization gap even in overparameterized settings where a conventional theory cannot be applied. As the computational cost of the functional variance is expensive for the overparameterized models, we propose an efficient approximation of the function variance, the Langevin approximation of the functional variance (Langevin FV). This method leverages only the $1$st-order gradient of the squared loss function, without referencing the $2$nd-order gradient; this ensures that the computation is efficient and the implementation is consistent with gradient-based optimization algorithms. We demonstrate the Langevin FV numerically by estimating the generalization gaps of overparameterized linear regression and non-linear neural network models, containing more than a thousand of parameters therein.

stat.ML↗
arXiv2023

Posterior Covariance Information Criterion for Weighted Inference

For predictive evaluation based on quasi-posterior distributions, we develop a new information criterion, the posterior covariance information criterion (PCIC. PCIC generalises the widely applicable information criterion WAIC so as to effectively handle predictive scenarios where likelihoods for the estimation and the evaluation of the model may be different. A typical example of such scenarios is the weighted likelihood inference, including prediction under covariate shift and counterfactual prediction. The proposed criterion utilises a posterior covariance form and is computed by using only one Markov chain Monte Carlo run. Through numerical examples, we demonstrate how PCIC can apply in practice. Further, we show that PCIC is asymptotically unbiased to the quasi-Bayesian generalization error under mild conditions in weighted inference with both regular and singular statistical models.

stat.ME↗
arXiv2022

Posterior covariance information criterion for general loss functions

We propose a novel computationally low-cost method for estimating a general predictive measure of generalised Bayesian inference. The proposed method utilises posterior covariance and provides estimators of the Gibbs and the plugin generalisation errors. We present theoretical guarantees of the proposed method, clarifying the connection to the Bayesian sensitivity analysis and the infinitesimal jackknife approximation of Bayesian leave-one-out cross validation. We illustrate several applications of our methods, including applications to differential privacy-preserving learning, the Bayesian hierarchical modeling, the Bayesian regression in the presence of influential observations, and the bias reduction of the widely-applicable information criterion. The applicability in high dimensions is also discussed.

stat.ME↗
arXiv2022

Structured regularization based velocity structure estimation in local earthquake tomography for the adaptation to velocity discontinuities

We propose a local earthquake tomography method that applies a structured regularization technique to determine sharp changes in Earth's seismic velocity structure using arrival time data of direct waves. Our approach focuses on the ability to better image two common features that are observed in Earth's seismic velocity structure: sharp changes in velocities that correspond to material boundaries, such as the Conrad and Moho discontinuities; and gradual changes in velocity that are associated with pressure and temperature distributions in the crust and mantle. We employ different penalty terms in the vertical and horizontal directions to refine the earthquake tomography. We utilize a vertical-direction (depth) penalty that takes the form of the l1-sum of the l2-norms of the second-order differences of the horizontal units in the vertical direction. This penalty is intended to represent sharp velocity changes caused by discontinuities by creating a piecewise linear depth profile of seismic velocity. We set a horizontal-direction penalty term on the basis of the l2-norm to express gradual velocity tendencies in the horizontal direction, which has been often used in conventional tomography methods. We use a synthetic data set to demonstrate that our method provides significant improvements over velocity structures estimated using conventional methods by obtaining stable estimates of both steep and gradual changes in velocity. Furthermore, we apply our proposed method to real seismic data in central Japan and present the potential of our method for detecting velocity discontinuities using the observed arrival times from a small number of local earthquakes.

stat.AP↗
arXiv2020

Minimax Predictive Density for Sparse Count Data

This paper discusses predictive densities under the Kullback--Leibler loss for high-dimensional Poisson sequence models under sparsity constraints. Sparsity in count data implies zero-inflation. We present a class of Bayes predictive densities that attain asymptotic minimaxity in sparse Poisson sequence models. We also show that our class with an estimator of unknown sparsity level plugged-in is adaptive in the asymptotically minimax sense. For application, we extend our results to settings with quasi-sparsity and with missing-completely-at-random observations. The simulation studies as well as application to real data illustrate the efficiency of the proposed Bayes predictive densities.

math.ST↗