arXiv subjects

Deyuan Li

Publications and source records attributed to Deyuan Li.

At least 19 recordsLinked to original sources

arXiv2026

CoVaR under Asymptotic Independence

Conditional value-at-risk (CoVaR) is one of the most important measures of systemic risk. It is defined as the high quantile conditional on a related variable being extreme, widely used in the field of quantitative risk management. In this work, we develop a semi-parametric methodology to estimate CoVaR for asymptotically independent pairs within the framework of bivariate extreme value theory. We use parametric modelling of the bivariate extremal structure to address data sparsity in the joint tail regions and prove consistency and asymptotic normality of the proposed estimator. The robust performance of the estimator is illustrated via simulation studies. Its application to the US stock returns data produces insightful dynamic CoVaR forecasts.

stat.ME↗

arXiv2025

Perturbation-based Inference for Extreme Value Index

The extreme value index (EVI) characterizes the tail behavior of a distribution and is crucial for extreme value theory. Inference on the EVI is challenging due to data scarcity in the tail region. We propose a novel method for constructing confidence intervals for the EVI using synthetic exceedances generated via perturbation. Rather than perturbing the entire sample, we add noise to exceedances above a high threshold and apply the generalized Pareto distribution (GPD) approximation. Confidence intervals are derived by simulating the distribution of pivotal statistics from the perturbed data. We show that the pivotal statistic is consistent, ensuring the proposed method provides consistent intervals for the EVI. Additionally, we demonstrate that the perturbed data is differentially private. When the GPD approximation is inadequate, we introduce a refined perturbation method. Simulation results show that our approach outperforms existing methods, providing robust and reliable inference.

stat.ME↗

arXiv2025

Max-Linear Tail Regression

The relationship between a response variable and its covariates can vary significantly, especially in scenarios where covariates take on extremely high or low values. This paper introduces a max-linear tail regression model specifically designed to capture such extreme relationships. To estimate the regression coefficients within this framework, we propose a novel M-estimator based on extreme value theory. The consistency and asymptotic normality of our proposed estimator are rigorously established under mild conditions. Simulation results demonstrate that our estimation method outperforms the conditional least squares approach. We validate the practical applicability of our model through two case studies: one using financial data and the other using rainfall data.

stat.ME↗

arXiv2024

High-Dimensional Extreme Quantile Regression

The estimation of conditional quantiles at extreme tails is of great interest in numerous applications. Various methods that integrate regression analysis with an extrapolation strategy derived from extreme value theory have been proposed to estimate extreme conditional quantiles in scenarios with a fixed number of covariates. However, these methods prove ineffective in high-dimensional settings, where the number of covariates increases with the sample size. In this article, we develop new estimation methods tailored for extreme conditional quantiles with high-dimensional covariates. We establish the asymptotic properties of the proposed estimators and demonstrate their superior performance through simulation studies, particularly in scenarios of growing dimension and high dimension where existing methods may fail. Furthermore, the analysis of auto insurance data validates the efficacy of our methods in estimating extreme conditional insurance claims and selecting important variables.

stat.ME↗

arXiv2024

M3SciQA: A Multi-Modal Multi-Document Scientific QA Benchmark for Evaluating Foundation Models

Existing benchmarks for evaluating foundation models mainly focus on single-document, text-only tasks. However, they often fail to fully capture the complexity of research workflows, which typically involve interpreting non-textual data and gathering information across multiple documents. To address this gap, we introduce M3SciQA, a multi-modal, multi-document scientific question answering benchmark designed for a more comprehensive evaluation of foundation models. M3SciQA consists of 1,452 expert-annotated questions spanning 70 natural language processing paper clusters, where each cluster represents a primary paper along with all its cited documents, mirroring the workflow of comprehending a single paper by requiring multi-modal and multi-document data. With M3SciQA, we conduct a comprehensive evaluation of 18 foundation models. Our results indicate that current foundation models still significantly underperform compared to human experts in multi-modal information retrieval and in reasoning across multiple scientific documents. Additionally, we explore the implications of these findings for the future advancement of applying foundation models in multi-modal scientific literature analysis.

cs.CL↗

arXiv2024

Aligning Multiclass Neural Network Classifier Criterion with Task Performance Metrics

Multiclass neural network classifiers are typically trained using cross-entropy loss but evaluated using metrics derived from the confusion matrix, such as Accuracy, $F_\beta$-Score, and Matthews Correlation Coefficient. This mismatch between the training objective and evaluation metric can lead to suboptimal performance, particularly when the user's priorities differ from what cross-entropy implicitly optimizes. For example, in the presence of class imbalance, $F_1$-Score may be preferred over Accuracy. Similarly, given a preference towards precision, the $F_{\beta=0.25}$-Score will better reflect this preference than $F_1$-Score. However, standard cross-entropy loss does not accommodate such a preference. Building on prior work leveraging soft-set confusion matrices and a continuous piecewise-linear Heaviside approximation, we propose Evaluation Aligned Surrogate Training (EAST), a novel approach to train multiclass classifiers using close surrogates of confusion-matrix based metrics, thereby aligning a neural network classifier's predictions more closely to a target evaluation metric than typical cross-entropy loss. EAST introduces three key innovations: First, we propose a novel dynamic thresholding approach during training. Second, we propose using a multiclass soft-set confusion matrix. Third, we introduce an annealing process that gradually aligns the surrogate loss with the target evaluation metric. Our theoretical analysis shows that EAST results in consistent estimators of the target evaluation metric. Furthermore, we show that the learned network parameters converge asymptotically to values that optimize for the target evaluation metric. Extensive experiments validate the effectiveness of our approach, demonstrating improved alignment between training objectives and evaluation metrics, while outperforming existing methods across many datasets.

cs.LG↗

arXiv2023

Distributed Inference for Tail Risk

For measuring tail risk with scarce extreme events, extreme value analysis is often invoked as the statistical tool to extrapolate to the tail of a distribution. The presence of large datasets benefits tail risk analysis by providing more observations for conducting extreme value analysis. However, large datasets can be stored distributedly preventing the possibility of directly analyzing them. In this paper, we introduce a comprehensive set of tools for examining the asymptotic behavior of tail empirical and quantile processes in the setting where data is distributed across multiple sources, for instance, when data are stored on multiple machines. Utilizing these tools, one can establish the oracle property for most distributed estimators in extreme value statistics in a straightforward way. The main theoretical challenge arises when the number of machines diverges to infinity. The number of machines resembles the role of dimensionality in high dimensional statistics. We provide various examples to demonstrate the practicality and value of our proposed toolkit.

stat.ME↗

arXiv2023

Tail Gini Functional under Asymptotic Independence

Tail Gini functional is a measure of tail risk variability for systemic risks, and has many applications in banking, finance and insurance. Meanwhile, there is growing attention on aymptotic independent pairs in quantitative risk management. This paper addresses the estimation of the tail Gini functional under asymptotic independence. We first estimate the tail Gini functional at an intermediate level and then extrapolate it to the extreme tails. The asymptotic normalities of both the intermediate and extreme estimators are established. The simulation study shows that our estimator performs comparatively well in view of both bias and variance. The application to measure the tail variability of weekly loss of individual stocks given the occurence of extreme events in the market index in Hong Kong Stock Exchange provides meaningful results, and leads to new insights in risk management.

stat.ME↗

arXiv2022

Estimating Extreme Value Index by Subsampling for Massive Datasets with Heavy-Tailed Distributions

Modern statistical analyses often encounter datasets with massive sizes and heavy-tailed distributions. For datasets with massive sizes, traditional estimation methods can hardly be used to estimate the extreme value index directly. To address the issue, we propose here a subsampling-based method. Specifically, multiple subsamples are drawn from the whole dataset by using the technique of simple random subsampling with replacement. Based on each subsample, an approximate maximum likelihood estimator can be computed. The resulting estimators are then averaged to form a more accurate one. Under appropriate regularity conditions, we show theoretically that the proposed estimator is consistent and asymptotically normal. With the help of the estimated extreme value index, we can estimate high-level quantiles and tail probabilities of a heavy-tailed random variable consistently. Extensive simulation experiments are provided to demonstrate the promising performance of our method. A real data analysis is also presented for illustration purpose.

stat.ME↗

arXiv2022

Bridging the Gap: Unifying the Training and Evaluation of Neural Network Binary Classifiers

While neural network binary classifiers are often evaluated on metrics such as Accuracy and $F_1$-Score, they are commonly trained with a cross-entropy objective. How can this training-evaluation gap be addressed? While specific techniques have been adopted to optimize certain confusion matrix based metrics, it is challenging or impossible in some cases to generalize the techniques to other metrics. Adversarial learning approaches have also been proposed to optimize networks via confusion matrix based metrics, but they tend to be much slower than common training methods. In this work, we propose a unifying approach to training neural network binary classifiers that combines a differentiable approximation of the Heaviside function with a probabilistic view of the typical confusion matrix values using soft sets. Our theoretical analysis shows the benefit of using our method to optimize for a given evaluation metric, such as $F_1$-Score, with soft sets, and our extensive experiments show the effectiveness of our approach in several domains.

cs.LG↗

arXiv2021

Adapting the Hill estimator to distributed inference: dealing with the bias

The distributed Hill estimator is a divide-and-conquer algorithm for estimating the extreme value index when data are stored in multiple machines. In applications, estimates based on the distributed Hill estimator can be sensitive to the choice of the number of the exceedance ratios used in each machine. Even when choosing the number at a low level, a high asymptotic bias may arise. We overcome this potential drawback by designing a bias correction procedure for the distributed Hill estimator, which adheres to the setup of distributed inference. The asymptotically unbiased distributed estimator we obtained, on the one hand, is applicable to distributed stored data, on the other hand, inherits all known advantages of bias correction methods in extreme value statistics.

stat.ME↗

arXiv2021

Modeling and Decoupling Systemic Risk

Identifying systemic risk patterns in geopolitical, economic, financial, environmental, transportation, epidemiological systems and their impacts is the key to risk management. This paper proposes a new nonlinear time series model: autoregressive conditional accelerated Fréchet (AcAF) model and introduces two new endopathic and exopathic competing risk measures for better learning risk patterns, decoupling systemic risk, and making better risk management. The paper establishes the probabilistic properties of stationarity and ergodicity of the AcAF model. Simulation demonstrates the efficiency of the proposed estimators and the AcAF model's flexibility in modeling heterogeneous data. Empirical studies on the stock returns in S&P 500 and the cryptocurrency trading show the superior performance of the proposed model in terms of the identified risk patterns, endopathic and exopathic competing risks, being informative with greater interpretability, enhancing the understanding of the systemic risks of a market and their causes, and making better risk management possible.

stat.AP↗

arXiv2021

Ehrhart-Equivalence, Equidecomposability, and Unimodular Equivalence of Integral Polytopes

Ehrhart polynomials are extensively-studied structures that interpolate the discrete volume of the dilations of integral $n$-polytopes. The coefficients of Ehrhart polynomials, however, are still not fully understood, and it is not known when two polytopes have equivalent Ehrhart polynomials. In this paper, we establish a relationship between Ehrhart-equivalence and other forms of equivalence: the $\operatorname{GL}_n(\mathbb{Z})$-equidecomposability and unimodular equivalence of two integral $n$-polytopes in $\mathbb{R}^n$. We conjecture that any two Ehrhart-equivalent integral $n$-polytopes $P,Q\subset\mathbb{R}^n$ are $\operatorname{GL}_n(\mathbb{Z})$-equidecomposable into $\frac{1}{(n-1)!}$-th unimodular simplices, thereby generalizing the known cases of $n=1, 2, 3$. We also create an algorithm to check for unimodular equivalence of any two integral $n$-simplices in $\mathbb{R}^n$. We then find and prove a new one-to-one correspondence between unimodular equivalence of integral $2$-simplices and the unimodular equivalence of their $n$-dimensional pyramids. Finally, we prove the existence of integral $n$-simplices in $\mathbb{R}^n$ that are not unimodularly equivalent for all $n \ge 2$.

math.CO↗

arXiv2021

Exponents of Jacobians of Graphs and Regular Matroids

Let $G$ be a finite undirected multigraph with no self-loops. The Jacobian $\operatorname{Jac}(G)$ is a finite abelian group associated with $G$ whose cardinality is equal to the number of spanning trees of $G$. There are only a finite number of biconnected graphs $G$ such that the exponent of $\operatorname{Jac}(G)$ equals $2$ or $3$. The definition of a Jacobian can also be extended to regular matroids as a generalization of graphs. We prove that there are finitely many connected regular matroids $M$ such that $\operatorname{Jac}(M)$ has exponent $2$ and characterize all such matroids.

math.CO↗

arXiv2020

Ulam Sets in New Settings

The classical Ulam sequence is defined recursively as follows: $a_1=1$, $a_2=2$, and $a_n$, for $n > 2$, is the smallest integer not already in the sequence that can be written uniquely as the sum of two distinct earlier terms. This sequence is known for its mysterious quasi-periodic behavior and its surprising rigidity when we let $a_2$ vary. This definition can be generalized to other sets of generators in different settings with a binary operation and a valid notion of size. Since there is not always a natural linear ordering of the elements, the resulting collections are called Ulam sets. In this paper, we study Ulam sets in new settings. First, we investigate the structure of canonical Ulam sets in free groups; this is the first investigation of Ulam sets in noncommutative groups. We prove several symmetry results and prove a periodicity result for eventually periodic words with fixed prefixes. Then, we study Ulam sets in $\mathbb{Z}\times (\mathbb{Z}/n\mathbb{Z})$ and prove regularity for an infinite class of initial sets. We also examine an intriguing phenomenon about decompositions of later elements into sums of the generators. Finally, we consider $\mathcal{V}$-sets, a variant where the summands are not required to be distinct; we focus on $\mathcal{V}$-sets in $\mathbb{Z}^2$.

math.CO↗

arXiv2018

Unexpectedly high salt accumulation inside carbon nanotubes soaked in very dilute salt solutions

We experimentally demonstrate the formation of salt aggregations with unexpectedly high concentration inside multi-walled carbon nanotubes (CNTs) soaked only in dilute salt solutions and even in solutions containing only traces of salts. This finding suggests the blocking of fluid across CNTs by the salt aggregations when CNTs are soaked in a dilute salt solution with the concentration of seawater or even lower, which may open new avenues for the development of novel CNT-based desalination techniques. The high salt accumulation of CNTs also provides a new CNT-based strategy for the collection/extraction of noble metal salts in solutions containing traces of noble metal salts. Theoretical analyses reveal that this high salt accumulation inside CNTs can be mainly attributed to the strong hydrated cation-pi interactions of hydrated cations and pi electrons in the aromatic rings of CNTs.

cond-mat.mtrl-sci↗

arXiv2017

NaCl crystal from salt solution with far below saturated concentration under ambient condition

Under ambient conditions, we directly observed NaCl crystals experimentally in the rGO membranes soaked in the salt solution with concentration below and far below the saturated concentration. Moreover, in most probability, the NaCl crystals show stoichiometries behavior. We attribute this unexpected crystallization to the cation-π interactions between the ions and the aromatic rings of the rGO.

physics.chem-ph↗

arXiv2016

Highly Efficient Ion Rejection by Graphene Oxide Membranes via Ion-controlling Interlayer Spacing

Because they may provide ultrathin, high-flux, and energy-efficient membranes for precise ionic and molecular sieving in aqueous solution, GO membranes (partially oxidized, stacked sheets of graphene) have shown great potential in water desalination and purification, gas and ion separation, biosensors, proton conductors, lithium-based batteries and super-capacitors. Unlike carbon nanotube (CNT) membranes, in which the nanotube pores have fixed sizes, the pores of GO membranes - the interlayer spacing between GO sheets - are of variable size. This presents a challenge for using GO membranes for filtration. Despite the great efforts to tune and fix the interlayer spacing, it remains difficult both to reduce the interlayer spacing sufficiently to exclude small ions while keeping this separation constant against the tendency of GO membranes to swell when immersed in aqueous solution, which greatly affects the applications of GO membranes. Here, we demonstrate experimentally that highly efficient and selective ion rejection by GO membranes can be readily achieved by controlling the interlayer spacing of GO membranes using cations (K+, Na+, Ca2+, Li+ and Mg2+) themselves. The interspacing can be controlled with precision as small as 1 A, and GO membranes controlled by one kind of cation can exclude other cations with a larger hydrated volume, which can only be accommodated with a larger interlayer spacing. First-principles calculations reveal that the strong noncovalent cation-pi interactions between hydrated cations in solution and aromatic ring structures in GO are the cause of this unexpected behavior. These findings open up new avenues for using GO membranes for water desalination and purification, lithium-based batteries and super-capacitors, molecular sieves for separating ions or molecules, and many other applications.

cond-mat.mtrl-sci↗