arXiv subjects

Misha Kilmer

Publications and source records attributed to Misha Kilmer.

16 recordsLinked to original sources

arXiv2026

Structure-Informed Bounds on the Kronecker Rank of Block-Structured Matrices

We derive theoretical bounds on the Kronecker rank of block-structured matrices that possess both inner and outer structure. Building on the matrix-to-tensor and tensor-to-matrix framework of Kilmer and Saibaba ($\textit{SIAM J. Matrix Anal. Appl.}$, 2022), we show that the Kronecker rank of a matrix $\mathbf{A}$ equals the dimension of the span of its distinct blocks (the $\textit{inner blockspan}$), and equals as well the dimension of the corresponding span for a permuted matrix $\mathbf{B}$ whose inner and outer structures are interchanged. We give two proofs of this equality: one via a direct dimension argument, and one via an explicit isomorphism between the outer blockspan of $\mathbf{A}$ and the column space of the second-mode unfolding of the associated tensor. These results yield nested containments that translate structural information, such as Toeplitz, Hankel, banded, or sparse patterns, into computable upper bounds on the Kronecker rank. We also establish a duality between element-level sparsity in $\mathbf{A}$ and block-level sparsity in $\mathbf{B}$. Numerical experiments confirm the theory across several classes of structured matrices and, for two sparse matrices drawn from the SuiteSparse collection, provide a structural explanation for observed singular value decay.

math.NA↗

arXiv2026

Exact Symmetry as Algebra: A Machine-Verified Tensor Calculus that Enforces Physical Selection Rules

Symmetry is central to the physical sciences, yet machine learning usually captures it only approximately, leaving a residual per-step equivariance error $\varepsilon$ that compounds with depth $M$ as $M\varepsilon$, whereas exact equivariance holds at unbounded depth; we demonstrate this divergence at fourteen orders of magnitude. We show that a symmetry can be made exact by construction, as the multiplication rule of a tensor algebra. In the resulting $\starG$ algebra, defined by any finite group $G$, the group-Fourier transform block-diagonalizes every tensor into irreducible-representation blocks, making equivariance intrinsic; requiring equivariance conversely \emph{forces} this suitably normalized transform, so the algebra is determined by $G$ rather than chosen. The standard matrix toolbox, including a Frobenius-optimal low-rank factorization, transfers blockwise, machine-checked in Lean~4 under an explicit axiom budget, and extends unchanged to band-limited compact groups and, under periodic boundary conditions, to all 230 crystallographic space groups and the compact little-group fibers of Euclidean and Poincar\'e symmetry. This exactness is an applied capability: on inorganic-crystal elastic tensors the algebra enforces point-group selection rules exactly on the output of \emph{any} predictor, driving a trained graph network's forbidden-channel leakage from $10^{-2}$ to machine zero, eliminating mechanically unstable predictions, and recovering viable materials that an unconstrained screen discards; on molecular data, with no quantum-mechanical input, it exposes octahedral selection-rule signatures consistent with the Wigner--Eckart theorem. Matched networks lead on pooled molecular accuracy, which we report plainly: the contribution is a complementary algebraic calculus, structural and diagnostic, exact at any depth.

cs.LG↗

arXiv2025

Alternating steepest descent methods for tensor completion with applications to spectromicroscopy

In this paper we develop two new Tensor Alternating Steepest Descent algorithms for tensor completion in the low-rank $\star_{M}$-product format, whereby we aim to reconstruct an entire low-rank tensor from a small number of measurements thereof. Both algorithms are rooted in the Alternating Steepest Descent (ASD) method for matrix completion, first proposed in [J. Tanner and K. Wei, Appl. Comput. Harmon. Anal., 40 (2016), pp. 417-429]. In deriving the new methods we target the X-ray spectromicroscopy undersampling problem, whereby data are collected by scanning a specimen on a rectangular viewpoint with X-ray beams of different energies. The recorded absorptions coefficients of the mixed specimen materials are naturally stored in a third-order tensor, with spatial horizontal and vertical axes, and an energy axis. To speed the X-ray spectromicroscopy measurement process up, only a fraction of tubes from (a reshaped version of) this tensor are fully scanned, leading to a tensor completion problem. In this framework we can apply any transform (such as the Fourier transform) to the tensor tube by tube, providing a natural way to work with the $\star_{M}$-tensor algebra, and propose: (1) a tensor completion algorithm that is essentially ASD reformulated in the $\star_{M}$-induced metric space and (2) a tensor completion algorithm that solves a set of (readily parallelizable) independent matrix completion problems for the frontal slices of the transformed tensor. The two new methods are tested on real X-ray spectromicroscopy data, demonstrating that they achieve the same reconstruction error with fewer samples from the tensor compared to the matrix completion algorithms applied to a flattened tensor.

math.NA↗

arXiv2025

Efficient Dynamic Image Reconstruction with motion estimation

Dynamic inverse problems are challenging to solve due to the need to identify and incorporate appropriate regularization in both space and time. Moreover, the very large scale nature of such problems in practice presents an enormous computational challenge. In this work, in addition to the use of edge-enhancing regularization of spatial features, we propose a new regularization method that incorporates a temporal model that estimates the motion of objects in time. In particular, we consider the optical flow model that simultaneously estimates the motion and provides an approximation for the desired image, and we incorporate this information into the cost functional as an additional form of temporal regularization. We propose a computationally efficient algorithm to solve the jointly regularized problem that leverages a generalized Krylov subspace method. We illustrate the effectiveness of the prescribed approach on a wide range of numerical experiments, including limited angle and single-shot computerized tomography.

math.NA↗

arXiv2022

Parametric Level-sets Enhanced To Improve Reconstruction (PaLEnTIR)

We introduce PaLEnTIR, a significantly enhanced parametric level-set (PaLS) method addressing the restoration and reconstruction of piecewise constant objects. Our key contribution involves a unique PaLS formulation utilizing a single level-set function to restore scenes containing multi-contrast piecewise-constant objects without requiring knowledge of the number of objects or their contrasts. Unlike standard PaLS methods employing radial basis functions (RBFs), our model integrates anisotropic basis functions (ABFs), thereby expanding its capacity to represent a wider class of shapes. Furthermore, PaLEnTIR improves the conditioning of the Jacobian matrix, required as part of the parameter identification process, and consequently accelerates optimization methods. We validate PaLEnTIR's efficacy through diverse experiments encompassing sparse and limited angle of view X-ray computed tomography (2D and 3D), nonlinear diffuse optical tomography (DOT), denoising, and deconvolution tasks using both real and simulated data sets.

math.NA↗

arXiv2020

A survey of subspace recycling iterative methods

This survey concerns subspace recycling methods, a popular class of iterative methods that enable effective reuse of subspace information in order to speed up convergence and find good initial guesses over a sequence of linear systems with slowly changing coefficient matrices, multiple right-hand sides, or both. The subspace information that is recycled is usually generated during the run of an iterative method (usually a Krylov subspace method) on one or more of the systems. Following introduction of definitions and notation, we examine the history of early augmentation schemes along with deflation preconditioning schemes and their influence on the development of recycling methods. We then discuss a general residual constraint framework through which many augmented Krylov and recycling methods can both be viewed. We review several augmented and recycling methods within this framework. We then discuss some known effective strategies for choosing subspaces to recycle before taking the reader through more recent developments that have generalized recycling for (sequences of) shifted linear systems, some of them with multiple right-hand sides in mind. We round out our survey with a brief review of application areas that have seen benefit from subspace recycling methods.

math.NA↗

arXiv2019

Tensor-Tensor Products for Optimal Representation and Compression

In this era of big data, data analytics and machine learning, it is imperative to find ways to compress large data sets such that intrinsic features necessary for subsequent analysis are not lost. The traditional workhorse for data dimensionality reduction and feature extraction has been the matrix SVD, which presupposes that the data has been arranged in matrix format. Our main goal in this study is to show that high-dimensional data sets are more compressible when treated as tensors (aka multiway arrays) and compressed via tensor-SVDs under the tensor-tensor product structures in (Kilmer and Martin, 2011; Kernfeld et al., 2015). We begin by proving Eckart Young optimality results for families of tensor-SVDs under two different truncation strategies. As such optimality properties can be proven in both matrix and tensor-based algebras, a fundamental question arises: does the tensor construct subsume the matrix construct in terms of representation efficiency? The answer is yes, as shown when we prove that a tensor-tensor representation of an equal dimensional spanning space can be superior to its matrix counterpart. We then investigate how the compressed representation provided by the truncated tensor-SVD is related both theoretically and in compression performance to its closest tensor-based analogue, truncated HOSVD (De Lathauwer et al., 2000; De Lathauwer and Vandewalle, 2004), thereby showing the potential advantages of our tensor-based algorithms. Finally, we propose new tensor truncated SVD variants, namely multi-way tensor SVDs, provide further approximated representation efficiency and discuss under which conditions they are considered optimal. We conclude with a numerical study demonstrating the utility of the theory.

math.NA↗

arXiv2018

Stable Tensor Neural Networks for Rapid Deep Learning

We propose a tensor neural network ($t$-NN) framework that offers an exciting new paradigm for designing neural networks with multidimensional (tensor) data. Our network architecture is based on the $t$-product (Kilmer and Martin, 2011), an algebraic formulation to multiply tensors via circulant convolution. In this $t$-product algebra, we interpret tensors as $t$-linear operators analogous to matrices as linear operators, and hence our framework inherits mimetic matrix properties. To exemplify the elegant, matrix-mimetic algebraic structure of our $t$-NNs, we expand on recent work (Haber and Ruthotto, 2017) which interprets deep neural networks as discretizations of non-linear differential equations and introduces stable neural networks which promote superior generalization. Motivated by this dynamic framework, we introduce a stable $t$-NN which facilitates more rapid learning because of its reduced, more powerful parameterization. Through our high-dimensional design, we create a more compact parameter space and extract multidimensional correlations otherwise latent in traditional algorithms. We further generalize our $t$-NN framework to a family of tensor-tensor products (Kernfeld, Kilmer, and Aeron, 2015) which still induce a matrix-mimetic algebraic structure. Through numerical experiments on the MNIST and CIFAR-10 datasets, we demonstrate the more powerful parameterizations and improved generalizability of stable $t$-NNs.

cs.LG↗

arXiv2017

Image classification using local tensor singular value decompositions

From linear classifiers to neural networks, image classification has been a widely explored topic in mathematics, and many algorithms have proven to be effective classifiers. However, the most accurate classifiers typically have significantly high storage costs, or require complicated procedures that may be computationally expensive. We present a novel (nonlinear) classification approach using truncation of local tensor singular value decompositions (tSVD) that robustly offers accurate results, while maintaining manageable storage costs. Our approach takes advantage of the optimality of the representation under the tensor algebra described to determine to which class an image belongs. We extend our approach to a method that can determine specific pairwise match scores, which could be useful in, for example, object recognition problems where pose/position are different. We demonstrate the promise of our new techniques on the MNIST data set.

stat.ML↗

arXiv2016

A Randomized Tensor Singular Value Decomposition based on the t-product

The tensor Singular Value Decomposition (t-SVD) for third order tensors that was proposed by Kilmer and Martin~\cite{2011kilmer} has been applied successfully in many fields, such as computed tomography, facial recognition, and video completion. In this paper, we propose a method that extends a well-known randomized matrix method to the t-SVD. This method can produce a factorization with similar properties to the t-SVD, but is more computationally efficient on very large datasets. We present details of the algorithm, theoretical results, and provide numerical results that show the promise of our approach for compressing and analyzing datasets. We also present an improved analysis of the randomized subspace iteration for matrices, which may be of independent interest to the scientific community.

math.NA↗

arXiv2015

Multilinear Subspace Clustering

In this paper we present a new model and an algorithm for unsupervised clustering of 2-D data such as images. We assume that the data comes from a union of multilinear subspaces (UOMS) model, which is a specific structured case of the much studied union of subspaces (UOS) model. For segmentation under this model, we develop Multilinear Subspace Clustering (MSC) algorithm and evaluate its performance on the YaleB and Olivietti image data sets. We show that MSC is highly competitive with existing algorithms employing the UOS model in terms of clustering performance while enjoying improvement in computational complexity.

cs.IT↗

arXiv2015

Clustering multi-way data: a novel algebraic approach

In this paper, we develop a method for unsupervised clustering of two-way (matrix) data by combining two recent innovations from different fields: the Sparse Subspace Clustering (SSC) algorithm [10], which groups points coming from a union of subspaces into their respective subspaces, and the t-product [18], which was introduced to provide a matrix-like multiplication for third order tensors. Our algorithm is analogous to SSC in that an "affinity" between different data points is built using a sparse self-representation of the data. Unlike SSC, we employ the t-product in the self-representation. This allows us more flexibility in modeling; infact, SSC is a special case of our method. When using the t-product, three-way arrays are treated as matrices whose elements (scalars) are n-tuples or tubes. Convolutions take the place of scalar multiplication. This framework allows us to embed the 2-D data into a vector-space-like structure called a free module over a commutative ring. These free modules retain many properties of complex inner-product spaces, and we leverage that to provide theoretical guarantees on our algorithm. We show that compared to vector-space counterparts, SSmC achieves higher accuracy and better able to cluster data with less preprocessing in some image clustering problems. In particular we show the performance of the proposed method on Weizmann face database, the Extended Yale B Face database and the MNIST handwritten digits database.

cs.LG↗

arXiv2014

Novel methods for multilinear data completion and de-noising based on tensor-SVD

In this paper we propose novel methods for completion (from limited samples) and de-noising of multilinear (tensor) data and as an application consider 3-D and 4- D (color) video data completion and de-noising. We exploit the recently proposed tensor-Singular Value Decomposition (t-SVD)[11]. Based on t-SVD, the notion of multilinear rank and a related tensor nuclear norm was proposed in [11] to characterize informational and structural complexity of multilinear data. We first show that videos with linear camera motion can be represented more efficiently using t-SVD compared to the approaches based on vectorizing or flattening of the tensors. Since efficiency in representation implies efficiency in recovery, we outline a tensor nuclear norm penalized algorithm for video completion from missing entries. Application of the proposed algorithm for video recovery from missing entries is shown to yield a superior performance over existing methods. We also consider the problem of tensor robust Principal Component Analysis (PCA) for de-noising 3-D video data from sparse random corruptions. We show superior performance of our method compared to the matrix robust PCA adapted to this setting as proposed in [4].

cs.CV↗

arXiv2014

Fast algorithms for hyperspectral Diffuse Optical Tomography

The image reconstruction of chromophore concentrations using Diffuse Optical Tomography (DOT) data can be described mathematically as an ill-posed inverse problem. Recent work has shown that the use of hyperspectral DOT data, as opposed to data sets comprising of a single or, at most, a dozen wavelengths, has the potential for improving the quality of the reconstructions. The use of hyperspectral diffuse optical data in the formulation and solution of the inverse problem poses a significant computational burden. The forward operator is, in actuality, nonlinear. However, under certain assumptions, a linear approximation, called the Born approximation, provides a suitable surrogate for the forward operator, and we assume this to be true in the present work. Computation of the Born matrix requires the solution of thousands of large scale discrete PDEs and the reconstruction problem, requires matrix-vector products with the (dense) Born matrix. In this paper, we address both of these difficulties, thus making the Born approach a computational viable approach for hyDOT reconstruction. In this paper, we assume that the images we wish to reconstruct are anomalies of unknown shape and constant value, described using a parametric level set approach, (PaLS) on a constant background. Specifically, to address the issue of the PDE solves, we develop a novel recycling-based Krylov subspace approach that leverages certain system similarities across wavelengths. To address expense of using the Born operator in the inversion, we present a fast algorithm for compressing the Born operator that locally compresses across wavelengths for a given source-detector set and then recursively combines the low-rank factors to provide a global low-rank approximation. This low-rank approximation can be used implicitly to speed up the recovery of the shape parameters and the chromophore concentrations.

math.NA↗

arXiv2013

Novel Factorization Strategies for Higher Order Tensors: Implications for Compression and Recovery of Multi-linear Data

In this paper we propose novel methods for compression and recovery of multilinear data under limited sampling. We exploit the recently proposed tensor- Singular Value Decomposition (t-SVD)[1], which is a group theoretic framework for tensor decomposition. In contrast to popular existing tensor decomposition techniques such as higher-order SVD (HOSVD), t-SVD has optimality properties similar to the truncated SVD for matrices. Based on t-SVD, we first construct novel tensor-rank like measures to characterize informational and structural complexity of multilinear data. Following that we outline a complexity penalized algorithm for tensor completion from missing entries. As an application, 3-D and 4-D (color) video data compression and recovery are considered. We show that videos with linear camera motion can be represented more efficiently using t-SVD compared to traditional approaches based on vectorizing or flattening of the tensors. Application of the proposed tensor completion algorithm for video recovery from missing entries is shown to yield a superior performance over existing methods. In conclusion we point out several research directions and implications to online prediction of multilinear data.

cs.IT↗

arXiv2011

Parametric Level Set Methods for Inverse Problems

In this paper, a parametric level set method for reconstruction of obstacles in general inverse problems is considered. General evolution equations for the reconstruction of unknown obstacles are derived in terms of the underlying level set parameters. We show that using the appropriate form of parameterizing the level set function results a significantly lower dimensional problem, which bypasses many difficulties with traditional level set methods, such as regularization, re-initialization and use of signed distance function. Moreover, we show that from a computational point of view, low order representation of the problem paves the path for easier use of Newton and quasi-Newton methods. Specifically for the purposes of this paper, we parameterize the level set function in terms of adaptive compactly supported radial basis functions, which used in the proposed manner provides flexibility in presenting a larger class of shapes with fewer terms. Also they provide a "narrow-banding" advantage which can further reduce the number of active unknowns at each step of the evolution. The performance of the proposed approach is examined in three examples of inverse problems, i.e., electrical resistance tomography, X-ray computed tomography and diffuse optical tomography.

math.AP↗