arXiv subjects

Michael Levine

Publications and source records attributed to Michael Levine.

16 recordsLinked to original sources

arXiv2025

Rates of Convergence of Maximum Smoothed Log-Likelihood Estimators for Semi-Parametric Multivariate Mixtures

Theoretical guarantees are established for a standard estimator in a semi-parametric finite mixture model, where each component density is modeled as a product of univariate densities under a conditional independence assumption. The focus is on the estimator that maximizes a smoothed log-likelihood function, which can be efficiently computed using a majorization-minimization algorithm. This smoothed likelihood applies a nonlinear regularization operator defined as the exponential of a kernel convolution on the logarithm of each component density. Consistency of the estimators is demonstrated by leveraging classical M-estimation frameworks under mild regularity conditions. Subsequently, convergence rates for both finite- and infinite-dimensional parameters are derived by exploiting structural properties of the smoothed likelihood, the behavior of the iterative optimization algorithm, and a thorough study of the profile smoothed likelihood. This work provides the first rigorous theoretical guarantees for this estimation approach, bridging the gap between practical algorithms and statistical theory in semi-parametric mixture modeling.

math.ST↗

arXiv2025

A monotonic MM-type algorithm for estimation of nonparametric finite mixture models with dependent marginals

In this manuscript, we consider a finite nonparametric mixture model with non-independent marginal density functions. Dependence between the marginal densities is modeled using a copula device. Until recently, no deterministic algorithms capable of estimating components of such a model have been available. A deterministic algorithm that is capable of this has been proposed in \citet*{levine2024smoothed}. That algorithm seeks to maximize a smoothed nonparametric penalized log-likelihood; it seems to perform well in practice but does not possess the monotonicity property. In this manuscript, we introduce a deterministic MM (Minorization-Maximization) algorithm for estimation of components of this model that is also maximizing a smoothed penalized nonparametric log-likelihood but that is monotonic with respect to this objective functional. Besides the convergence of the objective functional, the convergence of a subsequence of arguments of this functional, generated by this algorithm, is also established. The behavior of this algorithm is illustrated using both simulated datasets as well as a real dataset. The results illustrate performance that is at least comparable to the earlier algorithm of \citet*{levine2024smoothed}. A discussion of the results and possible future research directions make up the last part of the manuscript.

stat.ME↗

arXiv2023

Operator Guidance Informed by AI-Augmented Simulations

This paper will present a multi-fidelity, data-adaptive approach with a Long Short-Term Memory (LSTM) neural network to estimate ship response statistics in bimodal, bidirectional seas. The study will employ a fast low-fidelity, volume-based tool SimpleCode and a higher-fidelity tool known as the Large Amplitude Motion Program (LAMP). SimpleCode and LAMP data were generated by common bi-modal, bi-directional sea conditions in the North Atlantic as training data. After training an LSTM network with LAMP ship motion response data, a sample route was traversed and randomly sampled historical weather was input into SimpleCode and the LSTM network, and compared against the higher fidelity results.

cs.AI↗

arXiv2023

Non-Steepness and Maximum Likelihood Estimation Properties of the Truncated Multivariate Normal Distributions

This article considers exponential families of truncated multivariate normal distributions with one-sided truncation for some or all coordinates. We observe that if all components are one-sided truncated then this family is not full. The family of truncated multivariate normal distributions is extended to a full family, and the extended family is investigated in detail. We identify the canonical parameter space of the extended family and establish that the family is not regular and not even steep. We also consider maximum likelihood estimation for the location vector parameter and the positive definite (symmetric) matrix dispersion parameter of a truncated non-singular multivariate normal distribution. It is shown that if the sample size is sufficiently large then, almost surely, the maximizer of the likelihood function is unique, provided that it exists. It is also shown that each solution to the score equations for the location and dispersion parameters satisfies the method-of-moments equations. Finally, it is observed that similar results arise in the case of an arbitrary number of truncated components.

math.ST↗

arXiv2022

A smoothed semiparametric likelihood for estimation of nonparametric finite mixture models with a copula-based dependence structure

In this manuscript, we consider a finite multivariate nonparametric mixture model where the dependence between the marginal densities is modeled using the copula device. Pseudo EM stochastic algorithms were recently proposed to estimate all of the components of this model under a location-scale constraint on the marginals. Here, we introduce a deterministic algorithm that seeks to maximize a smoothed semiparametric likelihood. No location-scale assumption is made about the marginals. The algorithm is monotonic in one special case, and, in another, leads to ``approximate monotonicity'' -- whereby the difference between successive values of the objective function becomes non-negative up to an additive term that becomes negligible after a sufficiently large number of iterations. The behavior of this algorithm is illustrated on several simulated datasets. The results suggest that, under suitable conditions, the proposed algorithm may indeed be monotonic in general. A discussion of the results and some possible future research directions round out our presentation.

stat.CO↗

arXiv2022

Nonparametric clustering of RNA-sequencing data

Identification of clusters of co-expressed genes in transcriptomic data is a difficult task. Most algorithms used for this purpose can be classified into two broad categories: distance-based or model-based approaches. Distance-based approaches typically utilize a distance function between pairs of data objects and group similar objects together into clusters. Model-based approaches are based on using the mixture-modeling framework. Compared to distance-based approaches, model-based approaches offer better interpretability because each cluster can be explicitly characterized in terms of the proposed model. However, these models present a particular difficulty in identifying a correct multivariate distribution that a mixture can be based upon. In this manuscript, we review some of the approaches used to select a distribution for the needed mixture model first. Then, we propose avoiding this problem altogether by using a nonparametric MSL (Maximum Smoothed Likelihood) algorithm. This algorithm was proposed earlier in statistical literature but has not been, to the best of our knowledge, applied to transcriptomics data. The salient feature of this approach is that it avoids explicit specification of distributions of individual biological samples altogether, thus making the task of a practitioner easier. When used on a real dataset, the algorithm produces a large number of biologically meaningful clusters and compares favorably to the two other mixture-based algorithms commonly used for RNA-seq data clustering. Our code is publicly available in Github at https://github.com/Matematikoi/non_parametric_clustering.

stat.AP↗

arXiv2020

Estimation and Inference of Time-Varying Auto-Covariance under Complex Trend: A Difference-based Approach

We propose a difference-based nonparametric methodology for the estimation and inference of the time-varying auto-covariance functions of a locally stationary time series when it is contaminated by a complex trend with both abrupt and smooth changes. Simultaneous confidence bands (SCB) with asymptotically correct coverage probabilities are constructed for the auto-covariance functions under complex trend. A simulation-assisted bootstrapping method is proposed for the practical construction of the SCB. Detailed simulation and a real data example round out our presentation.

math.ST↗

arXiv2019

Estimation of group means in generalized linear mixed models

In this manuscript, we investigate the concept of the mean response for a treatment group mean as well as its estimation and prediction for generalized linear models with a subject-wise random effect. Generalized linear models are commonly used to analyze categorical data. The model-based mean for a treatment group usually estimates the response at the mean covariate. However, the mean response for the treatment group for studied population is at least equally important in the context of clinical trials. New methods were proposed to estimate such a mean response in generalized linear models; however, this has only been done when there are no random effects in the model. We suggest that, in a generalized linear mixed model (GLMM), there are at least two possible definitions of a treatment group mean response that can serve as estimation/prediction targets. The estimation of these treatment group means is important for healthcare professionals to be able to understand the absolute benefit versus risk. For both of these treatment group means, we propose a new set of methods that suggests how to estimate/predict both of them in a GLMM models with a univariate subject-wise random effect. Our methods also suggest an easy way of constructing corresponding confidence and prediction intervals for both possible treatment group means. Simulations show that proposed confidence and prediction intervals provide correct empirical coverage probability under most circumstances. Proposed methods have also been applied to analyze hypoglycemia data from diabetes clinical trials.

stat.AP↗

arXiv2019

Independence Properties of the Truncated Multivariate Elliptical Distributions

Truncated multivariate distributions arise extensively in econometric modelling when non-negative random variables are intrinsic to the data-generation process. More broadly, truncated multivariate distributions have appeared in censored and truncated regression models, simultaneous equations modelling, multivariate regression, and applications going back to the now-classic papers of Amemiya (1974) and Heckman (1976). In some applications of truncated multivariate distributions, there arises the problem of characterizing the distribution through correlation and independence properties of sub-vectors. In this paper, we characterize the truncated multivariate normal random vectors for which two complementary sub-vectors are mutually independent. Further, we characterize the multivariate truncated elliptical distributions, proving that if two complementary sub-vectors are mutually independent then the distribution of the joint vector is truncated multivariate normal, as is the distribution of each sub-vector. As an application, we apply the independence criterion to test the hypothesis of independence of the entrance examination scores and subsequent course averages achieved by a sample of university students; to do so, we verify the regularity conditions underpinning a classical theorem of Wilks on the asymptotic null distribution of the likelihood ratio test statistic.

math.ST↗

arXiv2019

ACF estimation via difference schemes for a semiparametric model with m-dependent errors

In this manuscript, we discuss a class of difference-based estimators of the autocovariance structure in a semiparametric regression model where the signal is discontinuous and the errors are serially correlated. The signal in this model consists of a sum of the function with jumps and an identifiable smooth function. A simpler form of this model has been considered earlier under the name of Nonparametric Jump Regression (NJRM). The estimators proposed allow us to bypass a complicated problem of prior estimation of the mean signal in such a model. We provide finite-sample expressions for biases and variance of the proposed estimators when the errors are Gaussian. Gaussianity in the above is only needed to provide explicit closed form expressions for biases and variances of our estimators. Moreover, we observe that the mean squared error of the proposed variance estimator does not depend on either the unknown smooth function that is a part of the mean signal nor on the values of difference sequence coefficients. Our approach also suggests sufficient conditions for $\sqrt{n}-$ consistency of the proposed estimators.

math.ST↗

arXiv2018

Minimax Optimal Estimation in Partially Linear Additive Models under High Dimension

In this paper, we derive minimax rates for estimating both parametric and nonparametric components in partially linear additive models with high dimensional sparse vectors and smooth functional components. The minimax lower bound for Euclidean components is the typical sparse estimation rate that is independent of nonparametric smoothness indices. However, the minimax lower bound for each component function exhibits an interplay between the dimensionality and sparsity of the parametric component and the smoothness of the relevant nonparametric component. Indeed, the minimax risk for smooth nonparametric estimation can be slowed down to the sparse estimation rate whenever the smoothness of the nonparametric component or dimensionality of the parametric component is suffciently large. In the above setting, we demonstrate that penalized least square estimators can nearly achieve minimax lower bounds.

math.ST↗

arXiv2017

Identifiability of two-component skew normal mixtures with one known component

We give sufficient identifiability conditions for estimating mixing proportions in two-component mixtures of skew normal distributions with one known component. We consider the univariate case as well as two multivariate extensions: a multivariate skew normal distribution (MSN) by Azzalini and Dalla Valle (1996) and the canonical fundamental skew normal distribution (CFUSN) by Arellano-Valle and Genton (2005). The characteristic function of the CFUSN distribution is additionally derived.

math.ST↗

arXiv2017

Robust functional estimation in the multivariate partial linear model

We consider the problem of adaptive estimation of the functional component in a multivariate partial linear model where the argument of the function is defined on a $q$-dimensional grid. Obtaining an adaptive estimator of this functional component is an important practical problem in econometrics where exact distributions of random errors and the parametric component are mostly unknown and cannot safely assumed to be normal. An estimator of the functional component that is adaptive in the mean squared sense over the wide range of multivariate Besov classes and robust to a wide choice of distributions of the linear component and random errors is constructed. It is also shown that the same estimator is locally adaptive over the same range of Besov classes and robust over large collections of distributions of the linear component and random errors as well. At any fixed point, this estimator also attains a local adaptive minimax rate. The procedure needed to obtain such an estimator turns out to depend on the choice of the right shrinkage approach in the wavelet domain. We show that one possible approach is to use the multivariate version of the classical BlockJS method. The multivariate version of BlockJS is developed in the manuscript and is shown to represent an independent interest. Finally, the Besov space scale over which the proposed estimator is locally adaptive is shown to depend on the dimensionality of the domain of the functional component; the higher the dimension, the larger the smoothness indicator of Besov spaces must be.

math.ST↗

arXiv2017

A maximum smoothed likelihood based estimation for two component semiparametric density mixtures with a known component

We consider a semiparametric mixture of two univariate density functions where one of them is known while the weight and the other function are unknown. Such mixtures have a history of application to the problem of detecting differentially expressed genes under two or more conditions in microarray data. Until now, some additional knowledge about the unknown component (e.g. the fact that it belongs to a location family) has been assumed. As opposed to this approach, we do not assume any additional structure on the unknown density function. For this mixture model, we derive a new sufficient identifiability condition and pinpoint a specific class of distributions describing the unknown component for which this condition is mostly satisfied. Our approach to estimation of this model is based on an idea of applying a maximum smoothed likelihood to what would otherwise have been an ill-posed problem. We introduce an iterative MM (Majorization-Minimization) algorithm that estimates all of the model parameters. We establish that the algorithm possesses a descent property with respect to a log-likelihood objective functional and prove that the algorithm converges to a minimizer of such an objective functional. Finally, we also illustrate the performance of our algorithm in a simulation study and using a real dataset.

math.ST↗

arXiv2016

Asymptotic properties of functionals of increments of a continuous semimartingale with stochastic sampling times

This paper is concerned with asymptotic behavior of a variety of functionals of increments of continuous semimartingales. Sampling times are assumed to follow a rather general discretization scheme. If an underlying semimartingale is thought of as a financial asset price process, a general sampling scheme like the one employed in this paper is capable of reflecting what happens whenever the financial trading data are recorded in a tick-by-tick fashion. A law of large numbers and a central limit theorem are proved after an appropriate normalization. One application of our result is an extension of the realized kernel estimator of integrated volatility to the case of random sampling times.

math.PR↗

arXiv2008

Effect of mean on variance function estimation in nonparametric regression

Variance function estimation in nonparametric regression is considered and the minimax rate of convergence is derived. We are particularly interested in the effect of the unknown mean on the estimation of the variance function. Our results indicate that, contrary to the common practice, it is not desirable to base the estimator of the variance function on the residuals from an optimal estimator of the mean when the mean function is not smooth. Instead it is more desirable to use estimators of the mean with minimal bias. On the other hand, when the mean function is very smooth, our numerical results show that the residual-based method performs better, but not substantial better than the first-order-difference-based estimator. In addition our asymptotic results also correct the optimal rate claimed in Hall and Carroll [J. Roy. Statist. Soc. Ser. B 51 (1989) 3--14].

math.ST↗