arXiv subjects

Oluwasegun A. Somefun

Publications and source records attributed to Oluwasegun A. Somefun.

3 recordsLinked to original sources

arXiv2026

A Trust-region Framework for Moment Estimation

In this paper, we develop a trust-region framework for understanding the behavior of adaptive moment estimation mechanisms, such as \textsc{Adam}, in stochastic gradient optimization. Specifically, the magnitude of the update step associated with each individual parameter is constrained by a finite-order $p$-moment trust-region, with $p\ge1$. The resulting derivation leads to a family of learning-rate mechanisms based on second-moment estimation and normalized $p$-th-moment estimation. For $p=4$, this involves kurtosis estimation. Subsequent derivations provide a unified interpretation of moment-estimation-based normalization, learning-rate scheduling, momentum as a spectral first-order lowpass regularization, and operator-level spectral-norm normalization within a common trust-region framework. Preliminary experiments on GPT2-124M trained on FineWeb-Edu and TinyStories suggest that the fourth-moment realization provides its greatest benefit when trust-region constraints are weak. As progressively stronger trust-region controls are introduced, the second-moment realization becomes increasingly competitive, often achieving slightly lower validation loss than its corresponding fourth-moment realization.

cs.LG↗

arXiv2021

Uniformly-Damped Binomial Filters: Five-percent Maximum Overshoot Optimal Response Design

In this paper, the five-percent maximum overshoot design of uniformly-damped binomial filters (transfer-functions) is introduced. First, the butterworth filter response is represented as a damped-binomial filter response. To extend the maximum-overshoot response of the second-order butterworth to higher orders, the binomial theorem is extended to the uniformly-damped binomial theorem. It is shown that the five-percent uniformly-damped binomial filter is a compromise between the butterworth filter and the standard binomial filter, with respect to the filter-approximation problem in the time and frequency domain. Finally, this paper concludes that in applications of interest, such as step-tracking, where both strong filtering and a fast, smooth transient-response, with negligible overshoot are desired, the response of the normalized five-percent uniformly-damped binomial form is a candidate replacement for both the butterworth and standard binomial filter forms.

eess.SY↗

arXiv2020

From the logistic-sigmoid to nlogistic-sigmoid: modelling the COVID-19 pandemic growth

Real-world growth processes, such as epidemic growth, are inherently noisy, uncertain and often involve multiple growth phases. The logistic-sigmoid function has been suggested and applied in the domain of modelling such growth processes. However, existing definitions are limiting, as they do not consider growth as restricted in two-dimension. Additionally, as the number of growth phases increase, the modelling and estimation of logistic parameters becomes more cumbersome, requiring more complex tools and analysis. To remedy this, we introduce the nlogistic-sigmoid function as a compact, unified modern definition of logistic growth for modelling such real-world growth phenomena. Also, we introduce two characteristic metrics of the logistic-sigmoid curve that can give more robust projections on the state of the growth process in each dimension. Specifically, we apply this function to modelling the daily World Health Organization published COVID-19 time-series data of infection and death cases of the world and countries of the world to date. Our results demonstrate statistically significant goodness of fit greater than or equal to 99% for affected countries of the world exhibiting patterns of either single or multiple stages of the ongoing COVID-19 outbreak, such as the USA. Consequently, this modern logistic definition and its metrics, as a machine learning tool, can help to provide clearer and more robust monitoring and quantification of the ongoing pandemic growth process.

cs.NE↗