SearcharxivSearch

arXiv subjects

Youngjo Lee

Publications and source records attributed to Youngjo Lee.

At least 19 recordsLinked to original sources

Uncertainty-Aware Ideal Point Estimation via Variational EM

Roll-call data analysis aims to estimate legislators' ideal points and quantify the associated uncertainty. Existing approaches either rely on Bayesian methods implemented via Markov chain Monte Carlo sampling or focus primarily on point estimation, with uncertainty typically assessed through resampling procedures such as the bootstrap. Consequently, the computational burden of these approaches can become substantial when applied to large roll-call datasets. To address this challenge, we propose a computationally efficient likelihood method for estimating ideal points and their standard errors. Leveraging the P\'{o}lya--Gamma identity, we develop a variational expectation--maximization algorithm for estimating ideal points and introduce a variational Louis' method to approximate the observed Fisher information for standard error estimation. Numerical studies and applications to U.S. congressional roll-call data demonstrate that the proposed method produces accurate ideal point estimates and reliable standard errors while being substantially more computationally efficient than existing approaches.

stat.ME

Epistemic Confidence Statement via Extended Likelihood

Fisher's fiducial probability has recently attracted renewed attention under the notion of epistemic confidence. Epistemic confidence statements can be formulated through extended likelihoods, thereby clarifying several long-standing controversies regarding its fiducial probability properties. It establishes a direct connection between Fisher's epistemic notion of confidence for observed data and Neyman's frequentist aleatory coverage probability for future data, thereby enabling extension of epistemic confidence statements for multidimensional parameters. We demonstrate how higher-order asymptotic theory can be applied to refine the first-order asymptotic epistemic confidence statements of the observed region, as a direct consequence of extended likelihood property.

math.ST

Subject-specific Deep Neural Networks for Count Data with High-cardinality Categorical Features

There is a growing interest in subject-specific predictions using deep neural networks (DNNs) because real-world data often exhibit correlations, which has been typically overlooked in traditional DNN frameworks. In this paper, we propose a novel hierarchical likelihood learning framework for introducing gamma random effects into the Poisson DNN, so as to improve the prediction performance by capturing both nonlinear effects of input variables and subject-specific cluster effects. The proposed method simultaneously yields maximum likelihood estimators for fixed parameters and best unbiased predictors for random effects by optimizing a single objective function. This approach enables a fast end-to-end algorithm for handling clustered count data, which often involve high-cardinality categorical features. Furthermore, state-of-the-art network architectures can be easily implemented into the proposed h-likelihood framework. As an example, we introduce multi-head attention layer and a sparsemax function, which allows feature selection in high-dimensional settings. To enhance practical performance and learning efficiency, we present an adjustment procedure for prediction of random parameters and a method-of-moments estimator for pretraining of variance component. Various experiential studies and real data analyses confirm the advantages of our proposed methods.

cs.LG

Statistical Inference for Random Unknowns via Modifications of Extended Likelihood

Fisher's likelihood is widely used for statistical inference for fixed unknowns. This paper aims to extend two important likelihood-based methods, namely the maximum likelihood procedure for point estimation and the confidence procedure for interval estimation, to embrace a broader class of statistical models with additional random unknowns. We propose the new h-likelihood and the h-confidence by modifying extended likelihoods. Maximization of the h-likelihood yields both maximum likelihood estimators of fixed unknowns and asymptotically optimal predictors for random unknowns, achieving the generalized Cram\'er-Rao lower bound. The h-likelihood further offers advantages in scalability for large datasets and complex models. The h-confidence could yield a valid interval estimation and prediction by maintaining the coverage probability for both fixed and random unknowns in small samples. We study approximate methods for the h-likelihood and h-confidence, which can be applied to a general class of models with additional random unknowns.

math.ST

Point Mass in the Confidence Distribution: Is it a Drawback or an Advantage?

Stein's (1959) problem highlights the phenomenon called the probability dilution in high dimensional cases, which is known as a fundamental deficiency in probabilistic inference. The satellite conjunction problem also suffers from probability dilution that poor-quality data can lead to a dilution of collision probability. Though various methods have been proposed, such as generalized fiducial distribution and the reference posterior, they could not maintain the coverage probability of confidence intervals (CIs) in both problems. On the other hand, the confidence distribution (CD) has a point mass at zero, which has been interpreted paradoxical. However, we show that this point mass is an advantage rather than a drawback, because it gives a way to maintain the coverage probability of CIs. More recently, `false confidence theorem' was presented as another deficiency in probabilistic inferences, called the false confidence. It was further claimed that the use of consonant belief can mitigate this deficiency. However, we show that the false confidence theorem cannot be applied to the CD in both Stein's and satellite conjunction problems. It is crucial that a confidence feature, not a consonant one, is the key to overcome the deficiencies in probabilistic inferences. Our findings reveal that the CD outperforms the other existing methods, including the consonant belief, in the context of Stein's and satellite conjunction problems. Additionally, we demonstrate the ambiguity of coverage probability in an observed CI from the frequentist CI procedure, and show that the CD provides valuable information regarding this ambiguity.

stat.ME

Deep Neural Networks for Semiparametric Frailty Models via H-likelihood

For prediction of clustered time-to-event data, we propose a new deep neural network based gamma frailty model (DNN-FM). An advantage of the proposed model is that the joint maximization of the new h-likelihood provides maximum likelihood estimators for fixed parameters and best unbiased predictors for random frailties. Thus, the proposed DNN-FM is trained by using a negative profiled h-likelihood as a loss function, constructed by profiling out the non-parametric baseline hazard. Experimental studies show that the proposed method enhances the prediction performance of the existing methods. A real data analysis shows that the inclusion of subject-specific frailties helps to improve prediction of the DNN based Cox model (DNN-Cox).

stat.ML

Robust propensity score weighting estimation under missing at random

Missing data is frequently encountered in many areas of statistics. Propensity score weighting is a popular method for handling missing data. The propensity score method employs a response propensity model, but correct specification of the statistical model can be challenging in the presence of missing data. Doubly robust estimation is attractive, as the consistency of the estimator is guaranteed when either the outcome regression model or the propensity score model is correctly specified. In this paper, we first employ information projection to develop an efficient and doubly robust estimator under indirect model calibration constraints. The resulting propensity score estimator can be equivalently expressed as a doubly robust regression imputation estimator by imposing the internal bias calibration condition in estimating the regression parameters. In addition, we generalize the information projection to allow for outlier-robust estimation. Some asymptotic properties are presented. The simulation study confirms that the proposed method allows robust inference against not only the violation of various model assumptions, but also outliers. A real-life application is presented using data from the Conservation Effects Assessment Project.

stat.ME

Enhanced Laplace Approximation

The Laplace approximation (LA) has been proposed as a method for approximating the marginal likelihood of statistical models with latent variables. However, the approximate maximum likelihood estimators (MLEs) based on the LA are often biased for binary or spatial data, and the corresponding Hessian matrix underestimates the standard errors of these approximate MLEs. A higher-order approximation has been proposed; however, it cannot be applied to complicated models such as correlated random effects models and does not provide consistent variance estimators. In this paper, we propose an enhanced LA (ELA) that provides the true MLE and its consistent variance estimator. We study its relationship to the variational Bayes method. We also introduce a new restricted maximum likelihood estimator (REMLE) for estimating dispersion parameters. The results of numerical studies show that the ELA provides a satisfactory MLE and REMLE, as well as their variance estimators for fixed parameters. The MLE and REMLE can be viewed as posterior mode and marginal posterior mode under flat priors, respectively. Some comparisons are also made with Bayesian procedures under different priors.

stat.ME

Maximum Likelihood Imputation

Maximum likelihood (ML) estimation is widely used in statistics. The h-likelihood has been proposed as an extension of Fisher's likelihood to statistical models including unobserved latent variables of recent interest. Its advantage is that the joint maximization gives ML estimators (MLEs) of both fixed and random parameters with their standard error estimates. However, the current h-likelihood approach does not allow MLEs of variance components as Henderson's joint likelihood does not in linear mixed models. In this paper, we show how to form the h-likelihood in order to facilitate joint maximization for MLEs of whole parameters. We also show the role of the Jacobian term which allows MLEs in the presence of unobserved latent variables. To obtain MLEs for fixed parameters, intractable integration is not necessary. As an illustration, we show one-shot ML imputation for missing data by treating them as realized but unobserved random parameters. We show that the h-likelihood bypasses the expectation step in the expectation-maximization (EM) algorithm and allows single ML imputation instead of multiple imputations. We also discuss the difference in predictions in random effects and missing data.

stat.ME

Deep learning study on the Dirac eigenvalue spectrum of staggered quarks

We study the chirality of staggered quarks on the Dirac eigenvalue spectrum using deep learning (DL) techniques. The Kluberg-Stern method to construct staggered bilinear operators conserves continuum property such as recursion relations, uniqueness of chirality, and Ward identities, which leads to a unique and characteristic pattern (we call it "leakage pattern (LP)") in the matrix elements of the chirality operator sandwiched between two quark eigenstates of staggered Dirac operator. DL analysis gives $99.4(2)\%$ accuracy on normal gauge configurations and $0.998$ AUC (Area Under ROC Curve) for classifying non-zero mode octets in the Dirac eigenvalue spectrum. It confirms that the leakage pattern is universal on normal gauge configurations. The multi-layer perceptron (MLP) method turns out to be the best DL model for our study on the LP.

hep-lat

Iteratively Selecting an Easy Reference Frame Makes Unsupervised Video Object Segmentation Easier

Unsupervised video object segmentation (UVOS) is a per-pixel binary labeling problem which aims at separating the foreground object from the background in the video without using the ground truth (GT) mask of the foreground object. Most of the previous UVOS models use the first frame or the entire video as a reference frame to specify the mask of the foreground object. Our question is why the first frame should be selected as a reference frame or why the entire video should be used to specify the mask. We believe that we can select a better reference frame to achieve the better UVOS performance than using only the first frame or the entire video as a reference frame. In our paper, we propose Easy Frame Selector (EFS). The EFS enables us to select an 'easy' reference frame that makes the subsequent VOS become easy, thereby improving the VOS performance. Furthermore, we propose a new framework named as Iterative Mask Prediction (IMP). In the framework, we repeat applying EFS to the given video and selecting an 'easier' reference frame from the video than the previous iteration, increasing the VOS performance incrementally. The IMP consists of EFS, Bi-directional Mask Prediction (BMP), and Temporal Information Updating (TIU). From the proposed framework, we achieve state-of-the-art performance in three UVOS benchmark sets: DAVIS16, FBMS, and SegTrack-V2.

cs.CV

On rereading Savage

If we accept Savage's set of axioms, then all uncertainties must be treated like ordinary probability. Savage espoused subjective probability, allowing, for example, the probability of Donald Trump's re-election. But Savage's probability also covers the objective version, such as the probability of heads in a fair toss of a coin. In other words, there is no distinction between objective and subjective probability. Savage's system has great theoretical implications; for example, prior probabilities can be elicited from subjective preferences, and then get updated by objective evidence, a learning step that forms the basis of Bayesian computations. Non-Bayesians have generally refused to accept the subjective aspect of probability or to allow priors in formal statistical modelling. As demanded, for example, by the late Dennis Lindley, since Bayesian probability is axiomatic, it is the non-Bayesians' duty to point out which axioms are not acceptable to them. This is not a simple request, since the Bayesian axioms are not commonly covered in our professional training, even in the Bayesian statistics courses. So our aim is to provide a readable exposition the Bayesian axioms from a close rereading Savage's classic book.

stat.OT

Epistemic confidence, the Dutch Book and relevant subsets

We use a logical device called the Dutch Book to establish epistemic confidence, defined as the sense of confidence \emph{in an observed} confidence interval. This epistemic property is unavailable -- or even denied -- in orthodox frequentist inference. In financial markets, including the betting market, the Dutch Book is also known as arbitrage or risk-free profitable transaction. A numerical confidence is deemed epistemic if its use as a betting price is protected from the Dutch Book by an external agent. Theoretically, to construct the Dutch Book, the agent must exploit unused information available in any relevant subset. Pawitan and Lee (2021) showed that confidence is an extended likelihood, and the likelihood principle states that the likelihood contains all the information in the data, hence leaving no relevant subset. Intuitively, this implies that confidence associated with the full likelihood is protected from the Dutch Book, and hence is epistemic. Our aim is to provide the theoretical support for this intuitive notion.

math.ST

Chiral symmetry and taste symmetry from the eigenvalue spectrum of staggered Dirac operators

We investigate general properties of the eigenvalue spectrum for improved staggered quarks. We introduce a new chirality operator $[γ_5 \otimes 1]$ and a new shift operator $[1 \otimes ξ_5]$, which respect the same recursion relation as the $γ_5$ operator in the continuum. Then we show that matrix elements of the chirality operator sandwiched between two eigenstates of the staggered Dirac operator are related to those of the shift operator by the Ward identity of the conserved $U(1)_A$ symmetry of staggered fermion actions. We perform a numerical study in quenched QCD using HYP staggered quarks to demonstrate the Ward identity. We introduce a new concept of leakage patterns which collectively represent the matrix elements of the chirality operator and the shift operator sandwiched between two eigenstates of the staggered Dirac operator. The leakage pattern provides a new method to identify zero modes and non-zero modes in the Dirac eigenvalue spectrum. This method is as robust as the spectral flow method but requires much less computing power. Analysis using a machine learning technique confirms that the leakage pattern is universal, since the staggered Dirac eigenmodes on normal gauge configurations respect it. In addition, the leakage pattern can be used to determine a ratio of renormalization factors as a by-product. We conclude that it might be possible and realistic to measure the topological charge $Q$ using the Atiya-Singer index theorem and the leakage pattern of the chirality operator in the staggered fermion formalism.

hep-lat

Popper's falsification and corroboration from the statistical perspectives

The role of probability appears unchallenged as the key measure of uncertainty, used among other things for practical induction in the empirical sciences. Yet, Popper was emphatic in his rejection of inductive probability and of the logical probability of hypotheses; furthermore, for him, the degree of corroboration cannot be a probability. Instead he proposed a deductive method of testing. In many ways this dialectic tension has many parallels in statistics, with the Bayesians on logico-inductive side vs the non-Bayesians or the frequentists on the other side. Simplistically Popper seems to be on the frequentist side, but recent synthesis on the non-Bayesian side might direct the Popperian views to a more nuanced destination. Logical probability seems perfectly suited to measure partial evidence or support, so what can we use if we are to reject it? For the past 100 years, statisticians have also developed a related concept called likelihood, which has played a central role in statistical modelling and inference. Remarkably, this Fisherian concept of uncertainty is largely unknown or at least severely under-appreciated in non-statistical literature. As a measure of corroboration, the likelihood satisfies the Popperian requirement that it is not a probability. Our aim is to introduce the likelihood and its recent extension via a discussion of two well-known logical fallacies in order to highlight that its lack of recognition may have led to unnecessary confusion in our discourse about falsification and corroboration of hypotheses. We highlight the 100 years of development of likelihood concepts. The year 2021 will mark the 100-year anniversary of the likelihood, so with this paper we wish it a long life and increased appreciation in non-statistical literature.

stat.OT

Resolving the induction problem: Can we state with complete confidence via induction that the sun rises forever?

Induction is a form of reasoning that starts with a particular example and generalizes to a rule, namely, a hypothesis. However, establishing the truth of a hypothesis is problematic due to the potential occurrence of conflicting events, also known as the induction problem. The sunrise problem, first introduced by Laplace (1814), is a quintessential example of the probability-based induction. In his solution, a zero probability is always assigned to the hypothesis that the sun rises forever, regardless of the number of observations made. This is a symptom of fundamental deficiency of probability-based induction: A hypothesis can never be accepted via the Bayes-Laplace approach. Alternative priors have been proposed to address this issue, but they have failed to fully overcome the deficiency. We investigate why this occurs and demonstrate that the confidence does not exhibit such a deficiency, as it is not a probability and therefore does not adhere to Bayes' rule. The confidence is neither a likelihood to allow not only a reconciliation between epistemic and aleatory interpretations of probability but also a resolution in agreement with the evidence by enabling us to accept a hypothesis with complete confidence as a rational decision.

stat.OT

Volume entropy and information flow in a brain graph

Entropy is a classical measure to quantify the amount of information or complexity of a system. Various entropy-based measures such as functional and spectral entropies have been proposed in brain network analysis. However, they are less widely used than traditional graph theoretic measures such as global and local efficiencies because either they are not well-defined on a graph or difficult to interpret its biological meaning. In this paper, we propose a new entropy-based graph invariant, called volume entropy. It measures the exponential growth rate of the number of paths in a graph, which is a relevant measure if information flows through the graph forever. We model the information propagation on a graph by the generalized Markov system associated to the weighted edge-transition matrix. We estimate the volume entropy using the stationary equation of the generalized Markov system. A prominent advantage of using the stationary equation is that it assigns certain distribution of weights on the edges of the brain graph, which we call the stationary distribution. The stationary distribution shows the information capacity of edges and the direction of information flow on a brain graph. The simulation results show that the volume entropy distinguishes the underlying graph topology and geometry better than the existing graph measures. In brain imaging data application, the volume entropy of brain graphs was significantly related to healthy normal aging from 20s to 60s. In addition, the stationary distribution of information propagation gives a new insight into the information flow of functional brain graph.

q-bio.NC

Robust functional regression model for marginal mean and subject-specific inferences

We introduce flexible robust functional regression models, using various heavy-tailed processes, including a Student $t$-process. We propose efficient algorithms in estimating parameters for the marginal mean inferences and in predicting conditional means as well interpolation and extrapolation for the subject-specific inferences. We develop bootstrap prediction intervals for conditional mean curves. Numerical studies show that the proposed model provides robust analysis against data contamination or distribution misspecification, and the proposed prediction intervals maintain the nominal confidence levels. A real data application is presented as an illustrative example.

stat.ME