arXiv subjects

Michael F. Zimmer

Publications and source records attributed to Michael F. Zimmer.

8 recordsLinked to original sources

arXiv2025

Constants of Motion for Conserved and Non-conserved Dynamics

This paper begins with a dynamical model that was obtained by applying a machine learning technique (FJet) to time-series data; this dynamical model is then analyzed with Lie symmetry techniques to obtain constants of motion. This analysis is performed on both the conserved and non-conserved cases of the 1D and 2D harmonic oscillators. For the 1D oscillator, constants are found in the cases where the system is underdamped, overdamped, and critically damped. The novel existence of such a constant for a non-conserved model is interpreted as a manifestation of the conservation of energy of the {\em total} system (i.e., oscillator plus dissipative environment). For the 2D oscillator, constants are found for the isotropic and anisotropic cases, including when the frequencies are incommensurate; it is also generalized to arbitrary dimensions. In addition, a constant is identified which generalizes angular momentum for all ratios of the frequencies. The approach presented here can produce {\em multiple} constants of motion from a {\em single}, generic data set.

cs.LG↗

arXiv2025

Comment on "Machine learning conservation laws from differential equations"

The paper [1] by Liu, Madhavan, and Tegmark sought to use machine learning methods to elicit known conservation laws for several systems. However, in their example of a damped 1D harmonic oscillator they made seven serious errors, causing both their method and result to be incorrect. In this Comment, those errors are reviewed.

cs.LG↗

arXiv2024

Extracting Dynamical Models from Data

The problem of determining the underlying dynamics of a system when only given data of its state over time has challenged scientists for decades. In this paper, the approach of using machine learning to model the updates of the phase space variables is introduced; this is done as a function of the phase space variables. (More generally, the modeling is done over functions of the jet space.) This approach (named FJet) allows one to accurately replicate the dynamics, and is demonstrated on the examples of the damped harmonic oscillator, the damped pendulum, and the Duffing oscillator; the underlying differential equation is also accurately recovered for each example. In addition, the results in no way depend on how the data is sampled over time (i.e., regularly or irregularly). It is demonstrated that a regression implementation of FJet is similar to the model resulting from a Taylor series expansion of the Runge-Kutta (RK) numerical integration scheme. This identification confers the advantage of explicitly revealing the function space to use in the modeling, as well as the associated uncertainty quantification for the updates. Finally, it is shown in the undamped harmonic oscillator example that the stability of the updates is stable $10^9$ times longer than with $4$th-order RK (with time step $0.1$).

cs.LG↗

arXiv2021

Neograd: Near-Ideal Gradient Descent

The purpose of this paper is to improve upon existing variants of gradient descent by solving two problems: (1) removing (or reducing) the plateau that occurs while minimizing the cost function, (2) continually adjusting the learning rate to an "ideal" value. The approach taken is to approximately solve for the learning rate as a function of a trust metric. When this technique is hybridized with momentum, it creates an especially effective gradient descent variant, called NeogradM. It is shown to outperform Adam on several test problems, and can easily reach cost function values that are smaller by a factor of $10^8$, for example.

cs.LG↗

arXiv2021

2nd-order Updates with 1st-order Complexity

It has long been a goal to efficiently compute and use second order information on a function ($f$) to assist in numerical approximations. Here it is shown how, using only basic physics and a numerical approximation, such information can be accurately obtained at a cost of ${\cal O}(N)$ complexity, where $N$ is the dimensionality of the parameter space of $f$. In this paper, an algorithm ({\em VA-Flow}) is developed to exploit this second order information, and pseudocode is presented. It is applied to two classes of problems, that of inverse kinematics (IK) and gradient descent (GD). In the IK application, the algorithm is fast and robust, and is shown to lead to smooth behavior even near singularities. For GD the algorithm also works very well, provided the cost function is locally well-described by a polynomial.

cs.LG↗

arXiv2020

Two direct solvers for a system of linear equations

A system of linear equations is normally understood as a linear mapping between two vector spaces. However, most direct solutions (e.g., QR, LU, ...) rely on the inelegant approach of back-substitution: a significant departure from such a characterization. In this paper, two new methods are introduced which respect the underlying vector space throughout, whether it be the row or column space of the coefficient matrix. Both solutions produce a generalized inverse as well as a projection operator for the null space. They also have the unique feature of admitting an online solution, which allows a solution to be computed as the data becomes available.

math.RA↗

arXiv2017

Speedup from a different parametrization within the Neural Network algorithm

A different parametrization of the hyperplanes is used in the neural network algorithm. As demonstrated on several autoencoder examples it significantly outperforms the usual parametrization, reaching lower training error values with only a fraction of the number of epochs. It's argued that it makes it easier to understand and initialize the parameters.

cs.LG↗

arXiv2013

A row space method for solving a system of linear equations

A new algorithm is presented for computing a direct solution to a system of consistent linear equations. It produces a minimum norm particular solution, a generalized inverse (of type {124}), and a null space projection operator. In addition, the algorithm permits an online formulation so that computations may proceed as the data become available. The algorithm does not require the solution of a triangular system of equations, nor does it rely on block partitioned matrices.

math.RA↗