arXiv subjects

Michael P. Friedlander

Publications and source records attributed to Michael P. Friedlander.

At least 19 recordsLinked to original sources

arXiv2026

A Scale-Shape Dual Newton Method for Entropic Least Squares

We give a damped inexact Newton method for entropy-regularized least-squares on the nonnegative orthant that converges globally at a linear rate with $O(\log\epsilon^{-1})$ iteration complexity, locally at a superlinear-to-quadratic rate, and is immune to the finite-precision overflow that limits classical dual solvers. A scale-shape decomposition of the primal -- separating its scale from its direction -- produces a dual with a nonsingular Jacobian. Objectives and Jacobians are evaluated through stable log-sum-exp and softmax primitives. Lambert W bounds on the scale uniformly control the Jacobian's spectrum, from which both rates follow. The solution map is jointly Lipschitz in the data, regularization parameter, and reference measure, and extends continuously to the vanishing-regularization limit. Experiments on a problem from analytic continuation of quantum Monte Carlo data confirm the predicted overflow resilience and convergence behavior.

math.OC↗

arXiv2026

Convergence Rate of the Last Iterate of Stochastic Proximal Algorithms

We analyze two classical algorithms for solving additively composite convex optimization problems where the objective is the sum of a smooth term and a nonsmooth regularizer: proximal stochastic gradient method for a single regularizer; and the randomized incremental proximal method, which uses the proximal operator of a randomly selected function when the regularizer is given as the sum of many nonsmooth functions. We focus on relaxing the bounded variance assumption that is common, yet stringent, for getting last iterate convergence rates. We prove the $\widetilde{O}(1/\sqrt{T})$ rate of convergence for the last iterate of both algorithms under componentwise convexity and smoothness, which is optimal up to log terms. Our results apply directly to graph-guided regularizers that arise in multi-task and federated learning, where the regularizer decomposes as a sum over edges of a collaboration graph.

math.OC↗

arXiv2025

Multiscale Methods for Discretized Continuous Optimization: Convergence and Cost Analysis

Discretized versions of optimization problems over continuous arguments are routinely solved at a single fine resolution, incurring a per-iteration cost that grows, often superlinearly, with the number of grid points. This paper analyzes a multiscale method that instead solves a hierarchy of increasingly fine dyadic discretizations. Linear interpolation of each coarse solution warm starts the next finer scale using any q-linearly convergent update rule as the inner solver. Each coarse problem is a consistent discretization of the continuous problem. Structural properties such as convexity and smoothness are preserved. For problems with Lipschitz-continuous solutions, two variants of the method converge to the fine-scale solution with explicit error bounds. The fine-scale solution in turn approximates the continuous solution once the grid is sufficiently fine, with quantified constants. The total cost to reach a fixed accuracy is provably lower than that of single-scale optimization whenever the cost of one update grows at least linearly in the problem size. Numerical experiments on probability density demixing problems, including geological survey data, show four- to sevenfold speedups while using a fraction of the memory.

math.NA↗

arXiv2025

STARK denoises spatial transcriptomics images via adaptive regularization

We present an approach to denoising spatial transcriptomics images that is particularly effective for uncovering cell identities in the regime of ultra-low sequencing depths, and also allows for interpolation of gene expression. The method -- Spatial Transcriptomics via Adaptive Regularization and Kernels (STARK) -- augments kernel ridge regression with an incrementally adaptive graph Laplacian regularizer. In each iteration, we (1) perform kernel ridge regression with a fixed graph to update the image, and (2) update the graph based on the new image. The kernel ridge regression step involves reducing the infinite dimensional problem on a space of images to finite dimensions via a modified representer theorem. Starting with a purely spatial graph, and updating it as we improve our image makes the graph more robust to noise in low sequencing depth regimes. We show that the aforementioned approach optimizes a block-convex objective through an alternating minimization scheme wherein the sub-problems have closed form expressions that are easily computed. This perspective allows us to prove convergence of the iterates to a stationary point of this non-convex objective. Statistically, such stationary points converge to the ground truth with rate $\mathcal{O}(R^{-1/2})$ where $R$ is the number of reads. In numerical experiments on real spatial transcriptomics data, the denoising performance of STARK, evaluated in terms of label transfer accuracy, shows consistent improvement over the competing methods tested.

stat.ML↗

arXiv2025

The noiseless limit and improved-prior limit of the maximum entropy method and their implications for the analytic continuation problem

Quantum Monte Carlo (QMC) methods are uniquely capable of providing exact simulations of quantum many-body systems. Unfortunately, the applications of a QMC simulation are limited because extracting dynamic properties requires solving the analytic continuation (AC) problem. Across the many fields that use QMC methods, there is no universally accepted analytic continuation algorithm for extracting dynamic properties, but many publications compare to the maximum entropy method. We investigate when entropy maximization is an acceptable approach. We show that stochastic sampling algorithms reduce to entropy maximization when the Bayesian prior is near to the true solution. We investigate when is Bryan's controversial optimization algorithm [Bryan, Eur. Biophys. J. 18, 165-174 (1990)] for entropy maximization (sometimes known as the maximum entropy method) appropriate to use. We show that Bryan's algorithm is appropriate when the noise is near zero or when the Bayesian prior is near to the true solution. We also investigate the mean squared error, finding a better scaling when the Bayesian prior is near the true solution than when the noise is near zero. We point to examples of improved data-driven Bayesian priors that have already leveraged this advantage. We support these results by solving the double Gaussian problem using both Bryan's algorithm and the newly formulated dual approach to entropy maximization [Chuna et al., J. Phys. A: Math. Theor. 58, 335203 (2025)].

physics.comp-ph↗

arXiv2025

Average-case thresholds for exact regularization of linear programs

Small regularizers can preserve linear programming solutions exactly. This paper provides the first average-case analysis of exact regularization: with a standard Gaussian cost vector and fixed constraint set, bounds are established for the probability that exact regularization succeeds as a function of regularization strength. Failure is characterized via the Gaussian measure of inner cones, controlled by novel two-sided bounds on the measure of shifted cones. Results reveal dimension-dependent scaling laws and connect exact regularization of linear programs to their polyhedral geometry via the normal fan and the Gaussian (solid-angle) measure of its cones. Computable bounds are obtained in several canonical settings, including regularized optimal transport. Numerical experiments corroborate the predicted scalings and thresholds.

math.OC↗

arXiv2025

Decentralized Optimization with Topology-Independent Communication

Distributed optimization requires nodes to coordinate, yet full synchronization scales poorly. When $n$ nodes collaborate through $m$ pairwise regularizers, standard methods demand $\mathcal{O}(m)$ communications per iteration. This paper proposes randomized local coordination: each node independently samples one regularizer uniformly and coordinates only with nodes sharing that term. This exploits partial separability, where each regularizer $G_j$ depends on a subset $S_j \subseteq \{1,\ldots,n\}$ of nodes. For graph-guided regularizers where $|S_j|=2$, expected communication drops to exactly 2 messages per iteration. This method achieves $\tilde{\mathcal{O}}(\varepsilon^{-2})$ iterations for convex objectives and under strong convexity, $\mathcal{O}(\varepsilon^{-1})$ to an $\varepsilon$-solution and $\mathcal{O}(\log(1/\varepsilon))$ to a neighborhood. Replacing the proximal map of the sum $\sum_j G_j$ with the proximal map of a single randomly selected regularizer $G_j$ preserves convergence while eliminating global coordination. Experiments validate both convergence rates and communication efficiency across synthetic and real-world datasets.

cs.LG↗

arXiv2025

Scalable Data-Driven Basis Selection for Linear Machine Learning Interatomic Potentials

Machine learning interatomic potentials (MLIPs) provide an effective approach for accurately and efficiently modeling atomic interactions, expanding the capabilities of atomistic simulations to complex systems. However, a priori feature selection leads to high complexity, which can be detrimental to both computational cost and generalization, resulting in a need for hyperparameter tuning. We demonstrate the benefits of active set algorithms for automated data-driven feature selection. The proposed methods are implemented within the Atomic Cluster Expansion (ACE) framework. Computational tests conducted on a variety of benchmark datasets indicate that sparse ACE models consistently enhance computational efficiency, generalization accuracy and interpretability over dense ACE models. An added benefit of the proposed algorithms is that they produce entire paths of models with varying cost/accuracy ratio.

physics.comp-ph↗

arXiv2025

Estimates of the dynamic structure factor for the finite temperature electron liquid via analytic continuation of path integral Monte Carlo data

Understanding the dynamic properties of the uniform electron gas (UEG) is important for numerous applications ranging from semiconductor physics to exotic warm dense matter. In this work, we apply the maximum entropy method (MEM), as implemented in Chuna \emph{et al.}~[arXiv:2501.01869], to \emph{ab initio} path integral Monte Carlo (PIMC) results for the imaginary-time correlation function $F(q,\tau)$ to estimate the dynamic structure factor $S(q,\omega)$ over an unprecedented range of densities at the electronic Fermi temperature. To conduct the MEM, we propose to construct the Bayesian prior $\mu$ from the PIMC data. Constructing the static approximation leads to a drastic improvement in $S(q,\omega)$ estimate over using the more simple random phase approximation (RPA) as the Bayesian prior. We find good agreement with existing results by Dornheim \emph{et al.}~[\textit{Phys.~Rev.~Lett.}~\textbf{121}, 255001 (2018)], where they are available. In addition, we present new results for the strongly coupled electron liquid regime with $r_s=50,...,200$, which reveal a pronounced roton-type feature and an incipient double peak structure in $S(q,\omega)$ at intermediate wavenumbers. We also find that our dynamic structure factors satisfy known sum rules, even though these sum rules are not enforced explicitly. An advantage of our set-up is that it is not specific to the UEG, thereby opening up new avenues to study the dynamics of real warm dense matter systems based on cutting-edge PIMC simulations in future works.

cond-mat.str-el↗

arXiv2025

Dual formulation of the maximum entropy method applied to analytic continuation of quantum Monte Carlo data

Many fields of physics use quantum Monte Carlo techniques, but struggle to estimate dynamic spectra via the analytic continuation of imaginary-time quantum Monte Carlo data. One of the most ubiquitous approaches to analytic continuation is the maximum entropy method (MEM). We supply a dual Newton optimization algorithm to be used within the MEM and provide analytic bounds for the algorithm's error. The MEM is typically used with Bryan's controversial algorithm [Rothkopf, "Bryan's Maximum Entropy Method" Data 5.3 (2020)]. We present new theoretical issues that are not yet in the literature. Our algorithm has all the theoretical benefits of Bryan's algorithm without these theoretical issues. We compare the MEM with Bryan's optimization to the MEM with our dual Newton optimization on test problems from lattice quantum chromodynamics and plasma physics. These comparisons show that in the presence of noise the dual Newton algorithm produces better estimates and error bars; this indicates the limits of Bryan's algorithm's applicability. We use the MEM to investigate authentic quantum Monte Carlo data for the uniform electron gas at warm dense matter conditions and further substantiate the roton-type feature in the dispersion relation.

physics.comp-ph↗

arXiv2022

Knowledge-Injected Federated Learning

Federated learning is an emerging technique for training models from decentralized data sets. In many applications, data owners participating in the federated learning system hold not only the data but also a set of domain knowledge. Such knowledge includes human know-how and craftsmanship that can be extremely helpful to the federated learning task. In this work, we propose a federated learning framework that allows the injection of participants' domain knowledge, where the key idea is to refine the global model with knowledge locally. The scenario we consider is motivated by a real industry-level application, and we demonstrate the effectiveness of our approach to this application.

cs.LG↗

arXiv2022

Cardinality-constrained structured data-fitting problems

A memory-efficient framework is described for the cardinality-constrained structured data-fitting problem. Dual-based atom-identification rules are proposed that reveal the structure of the optimal primal solution from near-optimal dual solutions. These rules allow for a simple and computationally cheap algorithm for translating any feasible dual solution to a primal solution that satisfies the cardinality constraint. Rigorous guarantees are provided for obtaining a near-optimal primal solution given any dual-based method that generates dual iterates converging to an optimal dual solution. Numerical experiments on real-world datasets support confirm the analysis and demonstrate the efficiency of the proposed approach.

math.OC↗

arXiv2022

Polar Deconvolution of Mixed Signals

The signal demixing problem seeks to separate a superposition of multiple signals into its constituent components. This paper studies a two-stage approach that first decompresses and subsequently deconvolves the noisy and undersampled observations of the superposition using two convex programs. Probabilistic error bounds are given on the accuracy with which this process approximates the individual signals. The theory of polar convolution of convex sets and gauge functions plays a central role in the analysis and solution process. If the measurements are random and the noise is bounded, this approach stably recovers low-complexity and mutually incoherent signals, with high probability and with near-optimal sample complexity. We develop an efficient algorithm, based on level-set and conditional-gradient methods, that solves the convex optimization problems with sublinear iteration complexity and linear space requirements. Numerical experiments on both real and synthetic data confirm the theory and the efficiency of the approach.

cs.IR↗

arXiv2022

Improving Fairness for Data Valuation in Horizontal Federated Learning

Federated learning is an emerging decentralized machine learning scheme that allows multiple data owners to work collaboratively while ensuring data privacy. The success of federated learning depends largely on the participation of data owners. To sustain and encourage data owners' participation, it is crucial to fairly evaluate the quality of the data provided by the data owners and reward them correspondingly. Federated Shapley value, recently proposed by Wang et al. [Federated Learning, 2020], is a measure for data value under the framework of federated learning that satisfies many desired properties for data valuation. However, there are still factors of potential unfairness in the design of federated Shapley value because two data owners with the same local data may not receive the same evaluation. We propose a new measure called completed federated Shapley value to improve the fairness of federated Shapley value. The design depends on completing a matrix consisting of all the possible contributions by different subsets of the data owners. It is shown under mild conditions that this matrix is approximately low-rank by leveraging concepts and tools from optimization. Both theoretical analysis and empirical evaluation verify that the proposed measure does improve fairness in many circumstances.

cs.LG↗

arXiv2022

A dual approach for federated learning

We study the federated optimization problem from a dual perspective and propose a new algorithm termed federated dual coordinate descent (FedDCD), which is based on a type of coordinate descent method developed by Necora et al.[Journal of Optimization Theory and Applications, 2017]. Additionally, we enhance the FedDCD method with inexact gradient oracles and Nesterov's acceleration. We demonstrate theoretically that our proposed approach achieves better convergence rates than the state-of-the-art primal federated optimization algorithms under certain situations. Numerical experiments on real-world datasets support our analysis.

cs.LG↗

arXiv2022

Fair and efficient contribution valuation for vertical federated learning

Federated learning is an emerging technology for training machine learning models across decentralized data sources without sharing data. Vertical federated learning, also known as feature-based federated learning, applies to scenarios where data sources have the same sample IDs but different feature sets. To ensure fairness among data owners, it is critical to objectively assess the contributions from different data sources and compensate the corresponding data owners accordingly. The Shapley value is a provably fair contribution valuation metric originating from cooperative game theory. However, its straight-forward computation requires extensively retraining a model on each potential combination of data sources, leading to prohibitively high communication and computation overheads due to multiple rounds of federated learning. To tackle this challenge, we propose a contribution valuation metric called vertical federated Shapley value (VerFedSV) based on the classic Shapley value. We show that VerFedSV not only satisfies many desirable properties of fairness but is also efficient to compute. Moreover, VerFedSV can be adapted to both synchronous and asynchronous vertical federated learning algorithms. Both theoretical analysis and extensive experimental results demonstrate the fairness, efficiency, adaptability, and effectiveness of VerFedSV.

cs.LG↗

arXiv2021

Online mirror descent and dual averaging: keeping pace in the dynamic case

Online mirror descent (OMD) and dual averaging (DA) -- two fundamental algorithms for online convex optimization -- are known to have very similar (and sometimes identical) performance guarantees when used with a fixed learning rate. Under dynamic learning rates, however, OMD is provably inferior to DA and suffers a linear regret, even in common settings such as prediction with expert advice. We modify the OMD algorithm through a simple technique that we call stabilization. We give essentially the same abstract regret bound for OMD with stabilization and for DA by modifying the classical OMD convergence analysis in a careful and modular way that allows for straightforward and flexible proofs. Simple corollaries of these bounds show that OMD with stabilization and DA enjoy the same performance guarantees in many applications -- even under dynamic learning rates. We also shed light on the similarities between OMD and DA and show simple conditions under which stabilized-OMD and DA generate the same iterates.

cs.LG↗

arXiv2021

Quantum Algorithms for Structured Prediction

We introduce two quantum algorithms for solving structured prediction problems. We first show that a stochastic gradient descent that uses the quantum minimum finding algorithm and takes its probabilistic failure into account solves the structured prediction problem with a runtime that scales with the square root of the size of the label space, and in $\widetilde O\left(1/ε\right)$ with respect to the precision, $ε$, of the solution. Motivated by robust inference techniques in machine learning, we then introduce another quantum algorithm that solves a smooth approximation of the structured prediction problem with a similar quantum speedup in the size of the label space and a similar scaling in the precision parameter. In doing so, we analyze a variant of stochastic gradient descent for convex optimization in the presence of an additive error in the calculation of the gradients, and show that its convergence rate does not deteriorate if the additive errors are of the order $O(\sqrtε)$. This algorithm uses quantum Gibbs sampling at temperature $Ω(ε)$ as a subroutine. Based on these theoretical observations, we propose a method for using quantum Gibbs samplers to combine feedforward neural networks with probabilistic graphical models for quantum machine learning. Our numerical results using Monte Carlo simulations on an image tagging task demonstrate the benefit of the approach.

cs.LG↗