arXiv subjects

Zhiliang Ying

Publications and source records attributed to Zhiliang Ying.

At least 19 recordsLinked to original sources

arXiv2026

A Dynamic Factor Model for Multivariate Counting Process Data

We propose a dynamic multiplicative factor model for process data, which arise from complex problem-solving items, an emerging testing mode in large-scale educational assessment. The proposed model can be viewed as an extension of the classical frailty models developed in survival analysis for multivariate recurrent event times, but with two important distinctions: (i) the factor (frailty) is of primary interest; (ii) covariates are internal and embedded in the factor. It allows us to explore low dimensional structure with meaningful interpretation. We show that the proposed model is identifiable and that the maximum likelihood estimators are consistent and asymptotically normal. Furthermore, to obtain a parsimonious model and to improve interpretation of parameters therein, variable selection and estimation for both fixed and random effects are developed through suitable penalisation. The computation is carried out by a stochastic EM combined with the Metropolis algorithm and the coordinate descent algorithm. Simulation studies demonstrate that the proposed approach provides an effective recovery of the true structure. The proposed method is applied to analysing the log-file of an item from the Programme for the International Assessment of Adult Competencies (PIAAC), where meaningful relationships are discovered.

stat.ME↗

arXiv2026

Exploratory Hierarchical Factor Analysis with an Application to Psychological Measurement

Hierarchical factor models, which include the bifactor model as a special case, are useful in social and behavioural sciences for measuring hierarchically structured constructs. Specifying a hierarchical factor model involves imposing hierarchically structured zero constraints on a factor loading matrix, which is often challenging. Therefore, an exploratory analysis is needed to learn the hierarchical factor structure from data. Unfortunately, there does not exist an identifiability theory for the learnability of this hierarchical structure, nor a computationally efficient method with provable performance. The method of Schmid-Leiman transformation, which is often regarded as the default method for exploratory hierarchical factor analysis, is flawed and likely to fail. The contribution of this paper is three-fold. First, an identifiability result is established for general hierarchical factor models, which shows that the hierarchical factor structure is learnable under mild regularity conditions. Second, a computationally efficient divide-and-conquer approach is proposed for learning the hierarchical factor structure. Finally, asymptotic theory is established for the proposed method, showing that it can consistently recover the true hierarchical factor structure as the sample size grows to infinity. The power of the proposed method is shown via simulation studies and a real data application to a personality test. The computation code for the proposed method is publicly available at https://github.com/EmetSelch97/EHFA/.

stat.ME↗

arXiv2025

Robust and Adaptive Spectral Method for Representation Multi-Task Learning with Contamination

Representation-based multi-task learning (MTL) improves efficiency by learning a shared structure across tasks, but its practical application is often hindered by contamination, outliers, or adversarial tasks. Most existing methods and theories assume a clean or near-clean setting, failing when contamination is significant. This paper tackles representation MTL with an unknown and potentially large contamination proportion, while also allowing for heterogeneity among inlier tasks. We introduce a Robust and Adaptive Spectral method (RAS) that can distill the shared inlier representation effectively and efficiently, while requiring no prior knowledge of the contamination level or the true representation dimension. Theoretically, we provide non-asymptotic error bounds for both the learned representation and the per-task parameters. These bounds adapt to inlier task similarity and outlier structure, and guarantee that RAS performs at least as well as single-task learning, thus preventing negative transfer. We also extend our framework to transfer learning with corresponding theoretical guarantees for the target task. Extensive experiments confirm our theory, showcasing the robustness and adaptivity of RAS, and its superior performance in regimes with up to 80\% task contamination.

stat.ML↗

arXiv2025

The Anytime Convergence of Stochastic Gradient Descent with Momentum: From a Continuous-Time Perspective

We study the stochastic optimization problem from a continuous-time perspective, with a focus on the Stochastic Gradient Descent with Momentum (SGDM) method. We show that the trajectory of SGDM, despite its \emph{stochastic} nature, converges in $L_2$-norm to a \emph{deterministic} second-order Ordinary Differential Equation (ODE) as the stepsize goes to zero. The connection between the ODE and the algorithm results in a useful development for the discrete-time convergence analysis. More specifically, we develop, through the construction of a suitable Lyapunov function, convergence results for the ODE, which are then translated to the corresponding convergence results for the discrete-time case. This approach yields a novel \emph{anytime} convergence guarantee for stochastic gradient methods. In particular, we prove that the sequence $\{ x_k \}$, governed by running SGDM on a smooth convex function $f$, satisfies \begin{align*} \mathbb{P}\left(f (x_k) - f^* \le C\left(1+\log\frac{1}β\right)\frac{\log k}{\sqrt{k}},\;\text{for all $k$}\right)\ge 1-β\quad\text{ for any $β>0$,} \end{align*} where $f^*=\min_{x\in\mathbb{R}^n} f(x)$, and $C$ is a constant. Rather than at a single step, this result captures the convergence behavior across the entire trajectory of the algorithm.

math.OC↗

arXiv2025

Breaking a Logarithmic Barrier in the Stopping Time Convergence Rate of Stochastic First-order Methods

This work provides a novel convergence analysis for stochastic optimization in terms of stopping times, addressing the practical reality that algorithms are often terminated adaptively based on observed progress. Unlike prior approaches, our analysis: 1. Directly characterizes convergence in terms of stopping times adapted to the underlying stochastic process. 2. Breaks a logarithmic barrier in existing results. Key to our results is the development of a lemma to control the large deviation property of almost super-martingales. This lemma might be of broader interest.

math.OC↗

arXiv2025

Exact Exploratory Bi-factor Analysis: A Constraint-based Optimisation Approach

Bi-factor analysis is a form of confirmatory factor analysis widely used in psychological and educational measurement. The use of a bi-factor model requires the specification of an explicit bi-factor structure on the relationship between the observed variables and the group factors. In practice, the bi-factor structure is sometimes unknown, in which case an exploratory form of bi-factor analysis is needed to find the bi-factor structure. Unfortunately, there are few methods for exploratory bi-factor analysis, with the exception of a rotation-based method proposed in Jennrich and Bentler (2011, 2012). However, this method only finds approximate bi-factor structures, as it does not yield an exact bi-factor loading structure, even after applying hard thresholding. In this paper, we propose a constraint-based optimisation method that learns an exact bi-factor loading structure from data, overcoming the issue with the rotation-based method. The key to the proposed method is a mathematical characterisation of the bi-factor loading structure as a set of equality constraints, which allows us to formulate the exploratory bi-factor analysis problem as a constrained optimisation problem in a continuous domain and solve the optimisation problem with an augmented Lagrangian method. The power of the proposed method is shown via simulation studies and a real data example. Extending the proposed method to exploratory hierarchical factor analysis is also discussed. The codes are available on ``https://anonymous.4open.science/r/Bifactor-ALM-method-757D".

stat.ME↗

arXiv2025

Dynamic Factor Analysis of High-dimensional Recurrent Events

Recurrent event time data arise in many studies, including biomedicine, public health, marketing, and social media analysis. High-dimensional recurrent event data involving many event types and observations have become prevalent with advances in information technology. This paper proposes a semiparametric dynamic factor model for the dimension reduction of high-dimensional recurrent event data. The proposed model imposes a low-dimensional structure on the mean intensity functions of the event types while allowing for dependencies. A nearly rate-optimal smoothing-based estimator is proposed. An information criterion that consistently selects the number of factors is also developed. Simulation studies demonstrate the effectiveness of these inference tools. The proposed method is applied to grocery shopping data, for which an interpretable factor structure is obtained.

stat.ME↗

arXiv2025

Multilayer Network Regression with Eigenvector Centrality and Community Structure

In the analysis of complex networks, centrality measures and community structures play pivotal roles. For multilayer networks, a critical challenge lies in effectively integrating information across diverse layers while accounting for the dependence structures both within and between layers. We propose an innovative two-stage regression model for multilayer networks, combining eigenvector centrality and network community structure within fourth-order tensor-like multilayer networks. We develop new community-based centrality measures, integrated into a regression framework. To address the inherent noise in network data, we conduct separate analyses of centrality measures with and without measurement errors and establish consistency for the least squares estimates in the regression model. The proposed methodology is applied to the world input-output dataset, investigating how input-output network data among different countries and industries influence the gross output of each industry.

stat.ME↗

arXiv2025

Semiparametric Modeling and Analysis for Longitudinal Network Data

We introduce a semiparametric latent space model for analyzing longitudinal network data. The model consists of a static latent space component and a time-varying node-specific baseline component. We develop a semiparametric efficient score equation for the latent space parameter by adjusting for the baseline nuisance component. Estimation is accomplished through a one-step update estimator and an appropriately penalized maximum likelihood estimator. We derive oracle error bounds for the two estimators and address identifiability concerns from a quotient manifold perspective. Our approach is demonstrated using the New York Citi Bike Dataset.

math.ST↗

arXiv2024

Simultaneous Identification of Sparse Structures and Communities in Heterogeneous Graphical Models

Exploring and detecting community structures hold significant importance in genetics, social sciences, neuroscience, and finance. Especially in graphical models, community detection can encourage the exploration of sets of variables with group-like properties. In this paper, within the framework of Gaussian graphical models, we introduce a novel decomposition of the underlying graphical structure into a sparse part and low-rank diagonal blocks (non-overlapped communities). We illustrate the significance of this decomposition through two modeling perspectives and propose a three-stage estimation procedure with a fast and efficient algorithm for the identification of the sparse structure and communities. Also on the theoretical front, we establish conditions for local identifiability and extend the traditional irrepresentability condition to an adaptive form by constructing an effective norm, which ensures the consistency of model selection for the adaptive $\ell_1$ penalized estimator in the second stage. Moreover, we also provide the clustering error bound for the K-means procedure in the third stage. Extensive numerical experiments are conducted to demonstrate the superiority of the proposed method over existing approaches in estimating graph structures. Furthermore, we apply our method to the stock return data, revealing its capability to accurately identify non-overlapped community structures.

stat.ML↗

arXiv2023

Markov Network for Modeling Local Item Dependence in Cognitively Diagnostic Classification Models

The study presents an exploratory graphical modeling approach for evaluating local item dependency within cognitively diagnostic classification models (DCMs). Current approaches to modeling local dependence require known item structure and have limited utility when such information is not available. In this study, we propose an exploratory approach to modeling local dependence so that items' own interactions can be revealed without dependency specification. The new framework is developed by integrating a Markov network into a generalized DCM. The framework unveils item interactions while performing regular cognitive diagnosis within a unified scheme. The inference on the model parameters is made on the regularized pseudo-likelihood and is implemented by an EM algorithm. Numerical experimentation from Monte Carlo simulation suggests that the proposed framework adequately recovers generating parameters and yields reliable standard error estimates. Compared with the regular DCM, the new model produced more accurate item parameter estimates as items exhibit local dependence. The study demonstrates application of the model using two real assessment data and discusses practical benefits of modeling local dependence.

stat.ME↗

arXiv2023

Adjusting for non-confounding covariates in case-control association studies

There is a considerable literature in case-control logistic regression on whether or not non-confounding covariates should be adjusted for. However, only limited and ad hoc theoretical results are available on this important topic. A constrained maximum likelihood method was recently proposed, which appears to be generally more powerful than logistic regression methods with or without adjusting for non-confounding covariates. This note provides a theoretical clarification for the case-control logistic regression with and without covariate adjustment and the constrained maximum likelihood method on their relative performances in terms of asymptotic relative efficiencies. We show that the benefit of covariate adjustment in the case-control logistic regression depends on the disease prevalence. We also show that the constrained maximum likelihood estimator gives an asymptotically uniformly most powerful test.

math.ST↗

arXiv2021

Accurate Assessment via Process Data

Accurate assessment of students' ability is the key task of a test. Assessments based on final responses are the standard. As the infrastructure advances, substantially more information is observed. One of such instances is the process data that is collected by computer-based interactive items, which contain a student's detailed interactive processes. In this paper, we show both theoretically and empirically that appropriately including such information in the assessment will substantially improve relevant assessment precision. The precision is measured empirically by out-of-sample test reliability.

stat.AP↗

arXiv2021

Item Response Theory -- A Statistical Framework for Educational and Psychological Measurement

Item response theory (IRT) has become one of the most popular statistical models for psychometrics, a field of study concerned with the theory and techniques of psychological measurement. The IRT models are latent factor models tailored to the analysis, interpretation, and prediction of individuals' behaviors in answering a set of measurement items that typically involve categorical response data. Many important questions of measurement are directly or indirectly answered through the use of IRT models, including scoring individuals' test performances, validating a test scale, linking two tests, among others. This paper provides a review of item response theory, including its statistical framework and psychometric applications. We establish connections between item response theory and related topics in statistics, including empirical Bayes, nonparametric methods, matrix completion, regularized estimation, and sequential analysis. Possible future directions of IRT are discussed from the perspective of statistical learning.

stat.ME↗

arXiv2021

External Correlates of Adult Digital Problem-Solving Behavior: Log Data Analysis of a Large-Scale Assessment

Using the action sequence data (i.e., log data) from the problem-solving in technology-rich environments assessment on the 2012 Programme for the International Assessment of Adult Competencies survey, the current study examines the associations between adult digital problem-solving behavior and several demographic and cognitive variables. Action sequence features extracted using multidimensional scaling (Tang, Wang, He, Liu, & Ying, 2019) and sequence-to-sequence autoencoders (Tang, Wang, Liu, & Ying, 2019) were used to predict test-taker external characteristics. Features extracted from action sequences were consistently found to contain more information on demographic and cognitive characteristics than final scores. Partial least squares analyses further revealed systematic associations between behavioral patterns and demographic/cognitive characteristics.

stat.AP↗

arXiv2021

Scalable Estimation and Inference with Large-scale or Online Survival Data

With the rapid development of data collection and aggregation technologies in many scientific disciplines, it is becoming increasingly ubiquitous to conduct large-scale or online regression to analyze real-world data and unveil real-world evidence. In such applications, it is often numerically challenging or sometimes infeasible to store the entire dataset in memory. Consequently, classical batch-based estimation methods that involve the entire dataset are less attractive or no longer applicable. Instead, recursive estimation methods such as stochastic gradient descent that process data points sequentially are more appealing, exhibiting both numerical convenience and memory efficiency. In this paper, for scalable estimation of large or online survival data, we propose a stochastic gradient descent method which recursively updates the estimates in an online manner as data points arrive sequentially in streams. Theoretical results such as asymptotic normality and estimation efficiency are established to justify its validity. Furthermore, to quantify the uncertainty associated with the proposed stochastic gradient descent estimator and facilitate statistical inference, we develop a scalable resampling strategy that specifically caters to the large-scale or online setting. Simulation studies and a real data application are also provided to assess its performance and illustrate its practical utility.

stat.ME↗

arXiv2020

Identifiability of Bifactor Models

The bifactor model and its extensions are multidimensional latent variable models, under which each item measures up to one subdimension on top of the primary dimension(s). Despite their wide applications to educational and psychological assessments, this type of multidimensional latent variable models may suffer from non-identifiability, which can further lead to inconsistent parameter estimation and invalid inference. The current work provides a relatively complete characterization of identifiability for the linear and dichotomous bifactor models and the linear extended bifactor model with correlated subdimensions. In addition, similar results for the two-tier models are also developed. Illustrative examples are provided on checking model identifiability through inspecting the factor loading structure. Simulation studies are reported that examine estimation consistency when the identifiability conditions are/are not satisfied.

math.ST↗

arXiv2020

Unfolding-Model-Based Visualization: Theory, Method and Applications

Multidimensional unfolding methods are widely used for visualizing item response data. Such methods project respondents and items simultaneously onto a low-dimensional Euclidian space, in which respondents and items are represented by ideal points, with person-person, item-item, and person-item similarities being captured by the Euclidian distances between the points. In this paper, we study the visualization of multidimensional unfolding from a statistical perspective. We cast multidimensional unfolding into an estimation problem, where the respondent and item ideal points are treated as parameters to be estimated. An estimator is then proposed for the simultaneous estimation of these parameters. Asymptotic theory is provided for the recovery of the ideal points, shedding lights on the validity of model-based visualization. An alternating projected gradient descent algorithm is proposed for the parameter estimation. We provide two illustrative examples, one on users' movie rating and the other on senate roll call voting.

stat.ME↗