arXiv subjects

Ziheng Chen

Publications and source records attributed to Ziheng Chen.

At least 73 records · Page 4Linked to original sources

arXiv2024

Riemannian Multinomial Logistics Regression for SPD Neural Networks

Deep neural networks for learning Symmetric Positive Definite (SPD) matrices are gaining increasing attention in machine learning. Despite the significant progress, most existing SPD networks use traditional Euclidean classifiers on an approximated space rather than intrinsic classifiers that accurately capture the geometry of SPD manifolds. Inspired by Hyperbolic Neural Networks (HNNs), we propose Riemannian Multinomial Logistics Regression (RMLR) for the classification layers in SPD networks. We introduce a unified framework for building Riemannian classifiers under the metrics pulled back from the Euclidean space, and showcase our framework under the parameterized Log-Euclidean Metric (LEM) and Log-Cholesky Metric (LCM). Besides, our framework offers a novel intrinsic explanation for the most popular LogEig classifier in existing SPD networks. The effectiveness of our method is demonstrated in three applications: radar recognition, human action recognition, and electroencephalography (EEG) classification. The code is available at https://github.com/GitZH-Chen/SPDMLR.git.

cs.LG↗

arXiv2024

A Lie Group Approach to Riemannian Batch Normalization

Manifold-valued measurements exist in numerous applications within computer vision and machine learning. Recent studies have extended Deep Neural Networks (DNNs) to manifolds, and concomitantly, normalization techniques have also been adapted to several manifolds, referred to as Riemannian normalization. Nonetheless, most of the existing Riemannian normalization methods have been derived in an ad hoc manner and only apply to specific manifolds. This paper establishes a unified framework for Riemannian Batch Normalization (RBN) techniques on Lie groups. Our framework offers the theoretical guarantee of controlling both the Riemannian mean and variance. Empirically, we focus on Symmetric Positive Definite (SPD) manifolds, which possess three distinct types of Lie group structures. Using the deformation concept, we generalize the existing Lie groups on SPD manifolds into three families of parameterized Lie groups. Specific normalization layers induced by these Lie groups are then proposed for SPD neural networks. We demonstrate the effectiveness of our approach through three sets of experiments: radar recognition, human action recognition, and electroencephalography (EEG) classification. The code is available at https://github.com/GitZH-Chen/LieBN.git.

cs.LG↗

arXiv2024

Convergence rate and exponential stability of backward Euler method for neutral stochastic delay differential equations under generalized monotonicity conditions

This work focuses on the numerical approximations of neutral stochastic delay differential equations with their drift and diffusion coefficients growing super-linearly with respect to both delay variables and state variables. Under generalized monotonicity conditions, we prove that the backward Euler method not only converges strongly in the mean square sense with order $1/2$, but also inherit the mean square exponential stability of the original equations. As a byproduct, we obtain the same results on convergence rate and exponential stability of the backward Euler method for stochastic delay differential equations with generalized monotonicity conditions. These theoretical results are finally supported by several numerical experiments.

math.NA↗

arXiv2023

Riemannian Local Mechanism for SPD Neural Networks

The Symmetric Positive Definite (SPD) matrices have received wide attention for data representation in many scientific areas. Although there are many different attempts to develop effective deep architectures for data processing on the Riemannian manifold of SPD matrices, very few solutions explicitly mine the local geometrical information in deep SPD feature representations. Given the great success of local mechanisms in Euclidean methods, we argue that it is of utmost importance to ensure the preservation of local geometric information in the SPD networks. We first analyse the convolution operator commonly used for capturing local information in Euclidean deep networks from the perspective of a higher level of abstraction afforded by category theory. Based on this analysis, we define the local information in the SPD manifold and design a multi-scale submanifold block for mining local geometry. Experiments involving multiple visual tasks validate the effectiveness of our approach. The supplement and source code can be found in https://github.com/GitZH-Chen/MSNet.git.

cs.CV↗

arXiv2023

The Dark Side of Explanations: Poisoning Recommender Systems with Counterfactual Examples

Deep learning-based recommender systems have become an integral part of several online platforms. However, their black-box nature emphasizes the need for explainable artificial intelligence (XAI) approaches to provide human-understandable reasons why a specific item gets recommended to a given user. One such method is counterfactual explanation (CF). While CFs can be highly beneficial for users and system designers, malicious actors may also exploit these explanations to undermine the system's security. In this work, we propose H-CARS, a novel strategy to poison recommender systems via CFs. Specifically, we first train a logical-reasoning-based surrogate model on training data derived from counterfactual explanations. By reversing the learning process of the recommendation model, we thus develop a proficient greedy algorithm to generate fabricated user profiles and their associated interaction records for the aforementioned surrogate model. Our experiments, which employ a well-known CF generation method and are conducted on two distinct datasets, show that H-CARS yields significant and successful attack performance.

cs.IR↗

arXiv2022

ReLAX: Reinforcement Learning Agent eXplainer for Arbitrary Predictive Models

Counterfactual examples (CFs) are one of the most popular methods for attaching post-hoc explanations to machine learning (ML) models. However, existing CF generation methods either exploit the internals of specific models or depend on each sample's neighborhood, thus they are hard to generalize for complex models and inefficient for large datasets. This work aims to overcome these limitations and introduces ReLAX, a model-agnostic algorithm to generate optimal counterfactual explanations. Specifically, we formulate the problem of crafting CFs as a sequential decision-making task and then find the optimal CFs via deep reinforcement learning (DRL) with discrete-continuous hybrid action space. Extensive experiments conducted on several tabular datasets have shown that ReLAX outperforms existing CF generation baselines, as it produces sparser counterfactuals, is more scalable to complex target models to explain, and generalizes to both classification and regression tasks. Finally, to demonstrate the usefulness of our method in a real-world use case, we leverage CFs generated by ReLAX to suggest actions that a country should take to reduce the risk of mortality due to COVID-19. Interestingly enough, the actions recommended by our method correspond to the strategies that many countries have actually implemented to counter the COVID-19 pandemic.

cs.LG↗

arXiv2022

GREASE: Generate Factual and Counterfactual Explanations for GNN-based Recommendations

Recently, graph neural networks (GNNs) have been widely used to develop successful recommender systems. Although powerful, it is very difficult for a GNN-based recommender system to attach tangible explanations of why a specific item ends up in the list of suggestions for a given user. Indeed, explaining GNN-based recommendations is unique, and existing GNN explanation methods are inappropriate for two reasons. First, traditional GNN explanation methods are designed for node, edge, or graph classification tasks rather than ranking, as in recommender systems. Second, standard machine learning explanations are usually intended to support skilled decision-makers. Instead, recommendations are designed for any end-user, and thus their explanations should be provided in user-understandable ways. In this work, we propose GREASE, a novel method for explaining the suggestions provided by any black-box GNN-based recommender system. Specifically, GREASE first trains a surrogate model on a target user-item pair and its $l$-hop neighborhood. Then, it generates both factual and counterfactual explanations by finding optimal adjacency matrix perturbations to capture the sufficient and necessary conditions for an item to be recommended, respectively. Experimental results conducted on real-world datasets demonstrate that GREASE can generate concise and effective explanations for popular GNN-based recommender models.

cs.IR↗

arXiv2022

DreamNet: A Deep Riemannian Network based on SPD Manifold Learning for Visual Classification

Image set-based visual classification methods have achieved remarkable performance, via characterising the image set in terms of a non-singular covariance matrix on a symmetric positive definite (SPD) manifold. To adapt to complicated visual scenarios better, several Riemannian networks (RiemNets) for SPD matrix nonlinear processing have recently been studied. However, it is pertinent to ask, whether greater accuracy gains can be achieved by simply increasing the depth of RiemNets. The answer appears to be negative, as deeper RiemNets tend to lose generalization ability. To explore a possible solution to this issue, we propose a new architecture for SPD matrix learning. Specifically, to enrich the deep representations, we adopt SPDNet [1] as the backbone, with a stacked Riemannian autoencoder (SRAE) built on the tail. The associated reconstruction error term can make the embedding functions of both SRAE and of each RAE an approximate identity mapping, which helps to prevent the degradation of statistical information. We then insert several residual-like blocks with shortcut connections to augment the representational capacity of SRAE, and to simplify the training of a deeper network. The experimental evidence demonstrates that our DreamNet can achieve improved accuracy with increased depth of the network.

cs.CV↗

arXiv2022

RLOP: RL Methods in Option Pricing from a Mathematical Perspective

Abstract In this work, we build two environments, namely the modified QLBS and RLOP models, from a mathematics perspective which enables RL methods in option pricing through replicating by portfolio. We implement the environment specifications (the source code can be found at https://github.com/owen8877/RLOP), the learning algorithm, and agent parametrization by a neural network. The learned optimal hedging strategy is compared against the BS prediction. The effect of various factors is considered and studied based on how they affect the optimal price and position.

q-fin.PR↗

arXiv2022

AdaLoGN: Adaptive Logic Graph Network for Reasoning-Based Machine Reading Comprehension

Recent machine reading comprehension datasets such as ReClor and LogiQA require performing logical reasoning over text. Conventional neural models are insufficient for logical reasoning, while symbolic reasoners cannot directly apply to text. To meet the challenge, we present a neural-symbolic approach which, to predict an answer, passes messages over a graph representing logical relations between text units. It incorporates an adaptive logic graph network (AdaLoGN) which adaptively infers logical relations to extend the graph and, essentially, realizes mutual and iterative reinforcement between neural and symbolic reasoning. We also implement a novel subgraph-to-node message passing mechanism to enhance context-option interaction for answering multiple-choice questions. Our approach shows promising results on ReClor and LogiQA.

cs.CL↗

arXiv2022

Large deviations principle for stochastic delay differential equations with super-linearly growing coefficients

We utilize the weak convergence method to establish the Freidlin--Wentzell large deviations principle (LDP) for stochastic delay differential equations (SDDEs) with super-linearly growing coefficients, which covers a large class of cases with non-globally Lipschitz coefficients. The key ingredient in our proof is the uniform moment estimate of the controlled equation, where we handle the super-linear growth of the coefficients by an iterative argument. Our results allow both the drift and diffusion coefficients of the considered equations to super-linearly grow not only with respect to the delay variable but also to the state variable. This work extends the existing results which develop the LDPs for SDDEs with super-linearly growing coefficients only with respect to the delay variable.

math.PR↗

arXiv2021

Large deviations principles of sample paths and invariant measures of numerical methods for parabolic SPDEs

For parabolic stochastic partial differential equations (SPDEs), we show that the numerical methods, including the spatial spectral Galerkin method and further the full discretization via the temporal accelerated exponential Euler method, satisfy the uniform sample path large deviations. Combining the exponential tail estimate of invariant measures, we establish the large deviations principles (LDPs) of invariant measures of these numerical methods. Based on the error estimate between the rate function of the considered numerical methods and that of the original equation, we prove that these numerical methods can weakly asymptotically preserve the LDPs of sample paths and invariant measures of the original equation. This work provides an approach to proving the weakly asymptotical preservation for the above two LDPs for SPDEs with small noise via numerical methods, by means of the minimization sequences.

math.NA↗

arXiv2021

Costly Features Classification using Monte Carlo Tree Search

We consider the problem of costly feature classification, where we sequentially select the subset of features to make a balance between the classification error and the feature cost. In this paper, we first cast the task into a MDP problem and use Advantage Actor Critic algorithm to solve it. In order to further improve the agent's performance and make the policy explainable, we employ the Monte Carlo Tree Search to update the policy iteratively. During the procedure, we also consider its performance on the unbalanced dataset and its sensitivity to the missing value. We evaluate our model on multiple datasets and find it outperforms other methods.

cs.LG↗

arXiv2020

A full-discrete exponential Euler approximation of invariant measure for parabolic stochastic partial differential equations

We discrete the ergodic semilinear stochastic partial differential equations in space dimension $d \leq 3$ with additive noise, spatially by a spectral Galerkin method and temporally by an exponential Euler scheme. It is shown that both the spatial semi-discretization and the spatio-temporal full discretization are ergodic. Further, convergence orders of the numerical invariant measures, depending on the regularity of noise, are recovered based on an easy time-independent weak error analysis without relying on Malliavin calculus. To be precise, the convergence order is $1-ε$ in space and $\frac{1}{2}-ε$ in time for the space-time white noise case and $2-ε$ in space and $1-ε$ in time for the trace class noise case in space dimension $d = 1$, with arbitrarily small $ε>0$. Numerical results are finally reported to confirm these theoretical findings.

math.NA↗

arXiv2020

Reconstruction in an imaging calorimeter for HL-LHC

The CMS endcap calorimeter upgrade for the High Luminosity LHC in 2027 uses silicon sensors to achieve radiation tolerance, with the further benefit of a very high readout granularity. Small scintillator tiles with individual SiPM readout are used in regions permitted by the radiation levels. A reconstruction framework is being developed to fully exploit the granularity and other significant features of the detector like precision timing, especially in the high pileup environment of HL-LHC. An iterative clustering framework (TICL) has been put in place, and is being actively developed. The framework takes as input the clusters of energy deposited in individual calorimeter layers delivered by the CLUE algorithm, which has recently been revised and tuned. Mindful of the projected extreme pressure on computing capacity in the HL-LHC era, the algorithms are being designed with modern parallel architectures in mind. Important speedup has recently been obtained for the clustering algorithm by running it on GPUs. Machine learning techniques are being developed and integrated into the reconstruction framework. This paper will describe the approaches being considered and show first results.

physics.ins-det↗

arXiv2020

The Bayesian Inversion Problem for Thermal Average Sampling of Quantum Systems

In this article, we propose a novel method for sampling potential functions based on noisy observation data of a finite number of observables in quantum canonical ensembles, which leads to the accurate sampling of a wide class of test observables. The method is based on the Bayesian inversion framework, which provides a platform for analyzing the posterior distribution and naturally leads to an efficient numerical sampling algorithm. We highlight that, the stability estimate is obtained by treating the potential functions as intermediate variables in the following way: the discrepancy between two sets of observation data of training observables can bound the distance between corresponding posterior distributions of potential functions, while the latter naturally leads to a bound of the discrepancies between corresponding thermal averages of test observables. Besides, the training observables can be more flexible than finite samples of the local density function, which are mostly used in previous researches. The method also applies to the multi-level quantum systems in the non-adiabatic regime. In addition, we provide extensive numerical tests to verify the accuracy and efficiency of the proposed algorithm.

math.NA↗

arXiv2020

CLUE: A Fast Parallel Clustering Algorithm for High Granularity Calorimeters in High Energy Physics

One of the challenges of high granularity calorimeters, such as that to be built to cover the endcap region in the CMS Phase-2 Upgrade for HL-LHC, is that the large number of channels causes a surge in the computing load when clustering numerous digitised energy deposits (hits) in the reconstruction stage. In this article, we propose a fast and fully-parallelizable density-based clustering algorithm, optimized for high occupancy scenarios, where the number of clusters is much larger than the average number of hits in a cluster. The algorithm uses a grid spatial index for fast querying of neighbours and its timing scales linearly with the number of hits within the range considered. We also show a comparison of the performance on CPU and GPU implementations, demonstrating the power of algorithmic parallelization in the coming era of heterogeneous computing in high energy physics.

physics.ins-det↗

arXiv2019

Item Response Theory based Ensemble in Machine Learning

In this article, we propose a novel probabilistic framework to improve the accuracy of a weighted majority voting algorithm. In order to assign higher weights to the classifiers which can correctly classify hard-to-classify instances, we introduce the Item Response Theory (IRT) framework to evaluate the samples' difficulty and classifiers' ability simultaneously. Three models are created with different assumptions suitable for different cases. When making an inference, we keep a balance between the accuracy and complexity. In our experiment, all the base models are constructed by single trees via bootstrap. To explain the models, we illustrate how the IRT ensemble model constructs the classifying boundary. We also compare their performance with other widely used methods and show that our model performs well on 19 datasets.

stat.ML↗