arXiv subjects

Cash Cherry

Publications and source records attributed to Cash Cherry.

4 recordsLinked to original sources

arXiv2026

Gradient-Free Optimization for Matrix functions

We consider the task of optimizing smooth, possibly non-convex functions of a matrix variable given access only to directional derivatives rather than full gradients. This setting arises when fine-tuning large neural networks on consumer-grade hardware: the network's weights are matrices, memory constraints rule out backward-mode automatic differentiation, but directional derivatives remain available through forward mode. We frame gradient estimation in this setting as a structured recovery problem, in the spirit of signal processing. From this perspective we provide three contributions. First, we introduce an alternative to the standard random gradient estimator; the difference corresponds to replacing the adjoint of the sampling operator with its pseudoinverse. Second, when the gradient satisfies an approximate low-rank condition, techniques from matrix sensing yield a family of highly accurate gradient estimators that drop into any first-order method. Third, we note that while the computational cost of such estimators is high, this can be amortized by combining them with a matrix-aware optimizer such as spectral descent. Specifically, the gradient estimator computes a factorization of the gradient, allowing for the projection step of spectral descent to be done at no extra cost. We demonstrate our findings with two careful numerical experiments on synthetic functions with approximately low-rank gradients. We show that by exploiting this low-rank property one obtains much faster convergence to good approximate solutions.

math.OC↗

arXiv2026

A Gauss-Newton Method with No Additional PDE Solves Beyond Gradient Evaluation for Large-Scale PDE-Constrained Inverse Problems

Partial Differential Equation (PDE)-constrained optimization problems often take the form of an optimization of an objective function given as a sum of loss terms. Each function or gradient evaluation requires one or more PDE solves, which render these problems computationally demanding. While Gauss-Newton methods are well-suited for large-scale PDE-constrained optimization, their application to settings such as Full-Waveform Inversion (FWI) is hindered by the need for additional PDE solves to compute Jacobian-vector products. This paper proposes a Gauss-Newton approach that eliminates the need for extra PDE solves beyond those required for gradient computation. Our numerical experiments on FWI demonstrate that the proposed method achieves the efficiency of gradient-based schemes while retaining the fast convergence of Gauss-Newton methods.

math.OC↗

arXiv2024

Applications of Automatic Differentiation in Image Registration

We demonstrate that automatic differentiation (AD), which has become commonly available in machine learning frameworks, is an efficient way to explore ideas that lead to algorithmic improvement in multi-scale affine image registration and affine super-resolution problems. In our first experiment on multi-scale registration, we implement an ODE predictor-corrector method involving a derivative with respect to the scale parameter and the Hessian of an image registration objective function, both of which would be difficult to compute without AD. Our findings indicate that exact Hessians are necessary for the method to provide any benefits over a traditional multi-scale method; a Gauss-Newton Hessian approximation fails to provide such benefits. In our second experiment, we implement a variable projected Gauss-Newton method for super-resolution and use AD to differentiate through the iteratively computed projection, a method previously unaddressed in the literature. We show that Jacobians obtained without differentiating through the projection are poor approximations to the true Jacobians of the variable projected forward map and explore the performance of other approximations in the problem of super-resolution. By addressing these problems, this work contributes to the application of AD in image registration and sets a precedent for further use of machine learning tools in this field.

math.OC↗

arXiv2024

Rigid-Recurrent Sequences for Actions of Finite Exponent Groups

The focus of this paper is to better understand the coexistence of rigidity, weak mixing, and recurrence by constructing thin sets in the product of countably many copies of the finite cyclic group of order q. A Kronecker-type set K is a subset of this group on which every continuous function into the complex unit circle equals the restriction, to K, of a character in the group's Pontryagin dual. Ackelsberg proves that if, for all q > 1, there exists a perfect Kronecker-type set generating a dense subgroup, then there exist large rigidity sequences for weak mixing systems of actions by countable discrete abelian groups. Ackelsberg shows the existence of such sets for prime values of q, while we construct them for all q > 1.

math.DS↗