Searcharxiv⌕ Search

arXiv subjects

George V. Moustakides

Publications and source records attributed to George V. Moustakides.

At least 19 recordsLinked to original sources

Data-Driven Estimation of Conditional Expectations, Application to Optimal Stopping and Reinforcement Learning

When the underlying conditional density is known, conditional expectations can be computed analytically or numerically. When, however, such knowledge is not available and instead we are given a collection of training data, the goal of this work is to propose simple and purely data-driven means for estimating directly the desired conditional expectation. Because conditional expectations appear in the description of a number of stochastic optimization problems with the corresponding optimal solution satisfying a system of nonlinear equations, we extend our data-driven method to cover such cases as well. We test our methodology by applying it to Optimal Stopping and Optimal Action Policy in Reinforcement Learning.

stat.ML↗

Quickest Change Detection with Controlled Sensing

In the problem of quickest change detection, a change occurs at some unknown time in the distribution of a sequence of random vectors that are monitored in real time, and the goal is to detect this change as quickly as possible subject to a certain false alarm constraint. In this work we consider this problem in the presence of parametric uncertainty in the post-change regime and controlled sensing. That is, the post-change distribution contains an unknown parameter, and the distribution of each observation, before and after the change, is affected by a control action. In this context, in addition to a stopping rule that determines the time at which it is declared that the change has occurred, one also needs to determine a sequential control policy, which chooses the control action at each time based on the already collected observations. We formulate this problem mathematically using Lorden's minimax criterion, and assuming that there are finitely many possible actions and post-change parameter values. We then propose a specific procedure for this problem that employs an adaptive CuSum statistic in which (i) the estimate of the parameter is based on a fixed number of the more recent observations, and (ii) each action is selected to maximize the Kullback-Leibler divergence of the next observation based on the current parameter estimate, apart from a small number of exploration times. We show that this procedure, which we call the Windowed Chernoff-CuSum (WCC), is first-order asymptotically optimal under Lorden's minimax criterion, for every possible possible value of the unknown post-change parameter, as the mean time to false alarm goes to infinity. We also provide simulation results to illustrate the performance of the WCC procedure.

cs.IT↗

Optimal Stopping Methodology for the Secretary Problem with Random Queries

Candidates arrive sequentially for an interview process which results in them being ranked relative to their predecessors. Based on the ranks available at each time, one must develop a decision mechanism that selects or dismisses the current candidate in an effort to maximize the chance to select the best. This classical version of the ``Secretary problem'' has been studied in depth using mostly combinatorial approaches, along with numerous other variants. In this work we consider a particular new version where during reviewing one is allowed to query an external expert to improve the probability of making the correct decision. Unlike existing formulations, we consider experts that are not necessarily infallible and may provide suggestions that can be faulty. For the solution of our problem we adopt a probabilistic methodology and view the querying times as consecutive stopping times which we optimize with the help of optimal stopping theory. For each querying time we must also design a mechanism to decide whether we should terminate the search at the querying time or not. This decision is straightforward under the usual assumption of infallible experts but, when experts are faulty, it has a far more intricate structure.

cs.DS↗

Window-Limited CUSUM for Sequential Change Detection

We study the parametric online changepoint detection problem, where the underlying distribution of the streaming data changes from a known distribution to an alternative that is of a known parametric form but with unknown parameters. We propose a joint detection/estimation scheme, which we call Window-Limited CUSUM, that combines the cumulative sum (CUSUM) test with a sliding window-based consistent estimate of the post-change parameters. We characterize the optimal choice of the window size and show that the Window-Limited CUSUM enjoys first-order asymptotic optimality as average run length approaches infinity under the optimal choice of window length. Compared to existing schemes with similar asymptotic optimality properties, our test can be much faster computed because it can recursively update the CUSUM statistic by employing the estimate of the post-change parameters. A parallel variant is also proposed that facilitates the practical implementation of the test. Numerical simulations corroborate our theoretical findings.

math.ST↗

A combinatorial proof for the secretary problem with multiple choices

The Secretary problem is a classical sequential decision-making question that can be succinctly described as follows: a set of rank-ordered applicants are interviewed sequentially for a single position. Once an applicant is interviewed, an immediate and irrevocable decision is made if the person is to be offered the job or not and only applicants observed so far can be used in the decision process. The problem of interest is to identify the stopping rule that maximizes the probability of hiring the highest-ranked applicant. A multiple-choice version of the Secretary problem, known as the Dowry problem, assumes that one is given a fixed integer budget for the total number of selections allowed to choose the best applicant. It has been solved using tools from dynamic programming and optimal stopping theory. We provide the first combinatorial proof for a related new \emph{query-based model} for which we are allowed to solicit the response of an expert to determine if an applicant is optimal. Since the selection criteria differ from those of the Dowry problem we obtain nonidentical expected stopping times. Our result indicates that an optimal strategy is the $(a_s, a_{s-1}, \ldots, a_1)$-strategy, i.e., for the $i^{th}$ selection, where $1 \le i \le s$ and $1 \le j = s+1-i \le s$, we reject the first $a_j$ applicants, wait until the decision of the $(i-1)^{th}$ selection (if $i \ge 2$), and then accept the next applicant whose qualification is better than all previously appeared applicants. Furthermore, our optimal strategy is right-hand based, i.e., the optimal strategies for two models with $s_1$ and $s_2$ selections in total ($s_1 < s_2$) share the same sequence $a_1, a_2, \ldots, a_{s_1}$ when it is viewed from the right. When the total number of applicants tends to infinity, our result agrees with the thresholds obtained by Gilbert and Mosteller.

math.CO↗

Query-Based Selection of Optimal Candidates under the Mallows Model

We study the secretary problem in which rank-ordered lists are generated by the Mallows model and the goal is to identify the highest-ranked candidate through a sequential interview process which does not allow rejected candidates to be revisited. The main difference between our formulation and existing models is that, during the selection process, we are given a fixed number of opportunities to query an infallible expert whether the current candidate is the highest-ranked or not. If the response is positive, the selection process terminates, otherwise, the search continues until a new potentially optimal candidate is identified. Our optimal interview strategy, as well as the expected number of candidates interviewed and the expected number of queries used, can be determined through the evaluation of well-defined recurrence relations. Specifically, if we are allowed to query $s-1$ times and to make a final selection without querying (thus, making $s$ selections in total) then the optimum scheme is characterized by $s$ thresholds that depend on the parameter $θ$ of the Mallows distribution but are independent on the maximum number of queries.

stat.ME↗

Data-Driven Parameter Estimation

Optimum parameter estimation methods require knowledge of a parametric probability density that statistically describes the available observations. In this work we examine Bayesian and non-Bayesian parameter estimation problems under a data-driven formulation where the necessary parametric probability density is replaced by available data. We present various data-driven versions that either result in neural network approximations of the optimum estimators or in well defined optimization problems that can be solved numerically. In particular, for the data-driven equivalent of non-Bayesian estimation we end up with optimization problems similar to the ones encountered for the design of generative networks.

stat.AP↗

Designing GANs: A Likelihood Ratio Approach

We are interested in the design of generative networks. The training of these mathematical structures is mostly performed with the help of adversarial (min-max) optimization problems. We propose a simple methodology for constructing such problems assuring, at the same time, consistency of the corresponding solution. We give characteristic examples developed by our method, some of which can be recognized from other applications, and some are introduced here for the first time. We present a new metric, the likelihood ratio, that can be employed online to examine the convergence and stability during the training of different Generative Adversarial Networks (GANs). Finally, we compare various possibilities by applying them to well-known datasets using neural networks of different configurations and sizes.

cs.LG↗

Sequential Subspace Change-Point Detection

We consider the online monitoring of multivariate streaming data for changes that are characterized by an unknown subspace structure manifested in the covariance matrix. In particular, we consider the covariance structure changes from an identity matrix to an unknown spiked covariance model. We assume the post-change distribution is unknown, and propose two detection procedures: the Largest-Eigenvalue Shewhart chart and the Subspace-CUSUM detection procedure. We present theoretical approximations to the average run length (ARL) and the expected detection delay (EDD) for the Largest-Eigenvalue Shewhart chart and also provide analysis for tuning parameters of the Subspace-CUSUM procedure. The performance of the proposed methods is illustrated using simulation and real data for human gesture detection and seismic event detection.

math.ST↗

Quickest Detection of Moving Anomalies in Sensor Networks

The problem of sequentially detecting a moving anomaly which affects different parts of a sensor network with time is studied. Each network sensor is characterized by a non-anomalous and anomalous distribution, governing the generation of sensor data. Initially, the observations of each sensor are generated according to the corresponding non-anomalous distribution. After some unknown but deterministic time instant, a moving anomaly emerges, affecting different sets of sensors as time progresses. As a result, the observations of the affected sensors are generated according to the corresponding anomalous distribution. Our goal is to design a stopping procedure to detect the emergence of the anomaly as quickly as possible, subject to constraints on the frequency of false alarms. The problem is studied in a quickest change detection framework where it is assumed that the evolution of the anomaly is unknown but deterministic. To this end, we propose a modification of Lorden's worst average detection delay metric to account for the trajectory of the anomaly that maximizes the detection delay of a candidate detection procedure. We establish that a Cumulative Sum-type test solves the resulting sequential detection problem exactly when the sensors are homogeneous. For the case of heterogeneous sensors, the proposed detection scheme can be modified to provide a first-order asymptotically optimal algorithm. We conclude by presenting numerical simulations to validate our theoretical analysis.

math.ST↗

Image De-Quantization Using Generative Models as Priors

Image quantization is used in several applications aiming in reducing the number of available colors in an image and therefore its size. De-quantization is the task of reversing the quantization effect and recovering the original multi-chromatic level image. Existing techniques achieve de-quantization by imposing suitable constraints on the ideal image in order to make the recovery problem feasible since it is otherwise ill-posed. Our goal in this work is to develop a de-quantization mechanism through a rigorous mathematical analysis which is based on the classical statistical estimation theory. In this effort we incorporate generative modeling of the ideal image as a suitable prior information. The resulting technique is simple and capable of de-quantizing successfully images that have experienced severe quantization effects. Interestingly, our method can recover images even if the quantization process is not exactly known and contains unknown parameters.

cs.CV↗

Image Restoration from Parametric Transformations using Generative Models

When images are statistically described by a generative model we can use this information to develop optimum techniques for various image restoration problems as inpainting, super-resolution, image coloring, generative model inversion, etc. With the help of the generative model it is possible to formulate, in a natural way, these restoration problems as Statistical estimation problems. Our approach, by combining maximum a-posteriori probability with maximum likelihood estimation, is capable of restoring images that are distorted by transformations even when the latter contain unknown parameters. The resulting optimization is completely defined with no parameters requiring tuning. This must be compared with the current state of the art which requires exact knowledge of the transformations and contains regularizer terms with weights that must be properly defined. Finally, we must mention that we extend our method to accommodate mixtures of multiple images where each image is described by its own generative model and we are able of successfully separating each participating image from a single mixture.

eess.IV↗

Training Neural Networks for Likelihood/Density Ratio Estimation

Various problems in Engineering and Statistics require the computation of the likelihood ratio function of two probability densities. In classical approaches the two densities are assumed known or to belong to some known parametric family. In a data-driven version we replace this requirement with the availability of data sampled from the densities of interest. For most well known problems in Detection and Hypothesis testing we develop solutions by providing neural network based estimates of the likelihood ratio or its transformations. This task necessitates the definition of proper optimizations which can be used for the training of the network. The main purpose of this work is to offer a simple and unified methodology for defining such optimization problems with guarantees that the solution is indeed the desired function. Our results are extended to cover estimates for likelihood ratios of conditional densities and estimates for statistics encountered in local approaches.

eess.SP↗

Adaptive Blind Separation of Two Dependent Sources

We consider the problem of adaptive blind separation of two sources from their instantaneous mixtures. We focus on the case where the two sources are not necessarily independent. By analyzing a general form of adaptive algorithms we show that separation is possible not only for independent sources but also for sources that are dependent provided their joint pdf satisfies certain symmetry conditions. A very interesting problem consists in identifying the class of dependent sources that are non-separable, namely, the counterpart of Gaussian sources of the independent case. We corroborate our theoretical analysis with a number of simulations and give examples of dependent sources that can be easily separated.

eess.SP↗

Optimizing Shallow Networks for Binary Classification

Data driven classification that relies on neural networks is based on optimization criteria that involve some form of distance between the output of the network and the desired label. Using the same mathematical analysis, for a multitude of such measures, we can show that their optimum solution matches the ideal likelihood ratio test classifier. In this work we introduce a different family of optimization problems which is not covered by the existing approaches and, therefore, opens possibilities for new training algorithms for neural network based classification. We give examples that lead to algorithms that are simple in implementation, exhibit stable convergence characteristics and are antagonistic to the most popular existing techniques.

cs.LG↗

Detecting Changes in Hidden Markov Models

We consider the problem of sequential detection of a change in the statistical behavior of a hidden Markov model. By adopting a worst-case analysis with respect to the time of change and by taking into account the data that can be accessed by the change-imposing mechanism we offer alternative formulations of the problem. For each formulation we derive the optimum Shewhart test that maximizes the worst-case detection probability while guaranteeing infrequent false alarms.

math.ST↗

Asynchronous Multi-Sensor Change-Point Detection for Seismic Tremors

We consider the sequential change-point detection for asynchronous multi-sensors, where each sensor observe a signal (due to change-point) at different times. We propose an asynchronous Subspace-CUSUM procedure based on jointly estimating the unknown signal waveform and the unknown relative delays between the sensors. Using the estimated delays, we can align signals and use the subspace to combine the multiple sensor observations. We derive the optimal drift parameter for the proposed procedure, and characterize the relationship between the expected detection delay, average run length (of false alarms), and the energy of the time-varying signal. We demonstrate the good performance of the proposed procedure using simulation and real data. We also demonstrate that the proposed procedure outperforms the well-known `one-shot procedure' in detecting weak and asynchronous signals.

stat.AP↗

Kernel-Based Training of Generative Networks

Generative adversarial networks (GANs) are designed with the help of min-max optimization problems that are solved with stochastic gradient-type algorithms which are known to be non-robust. In this work we revisit a non-adversarial method based on kernels which relies on a pure minimization problem and propose a simple stochastic gradient algorithm for the computation of its solution. Using simplified tools from Stochastic Approximation theory we demonstrate that batch versions of the algorithm or smoothing of the gradient do not improve convergence. These observations allow for the development of a training algorithm that enjoys reduced computational complexity and increased robustness while exhibiting similar synthesis characteristics as classical GANs.

cs.LG↗