Searcharxiv⌕ Search

arXiv subjects

Haotian Xu

Publications and source records attributed to Haotian Xu.

88 records · Page 5Linked to original sources

Change point inference in high-dimensional regression models under temporal dependence

This paper concerns about the limiting distributions of change point estimators, in a high-dimensional linear regression time series context, where a regression object $(y_t, X_t) \in \mathbb{R} \times \mathbb{R}^p$ is observed at every time point $t \in \{1, \ldots, n\}$. At unknown time points, called change points, the regression coefficients change, with the jump sizes measured in $\ell_2$-norm. We provide limiting distributions of the change point estimators in the regimes where the minimal jump size vanishes and where it remains a constant. We allow for both the covariate and noise sequences to be temporally dependent, in the functional dependence framework, which is the first time seen in the change point inference literature. We show that a block-type long-run variance estimator is consistent under the functional dependence, which facilitates the practical implementation of our derived limiting distributions. We also present a few important byproducts of our analysis, which are of their own interest. These include a novel variant of the dynamic programming algorithm to boost the computational efficiency, consistent change point localisation rates under temporal dependence and a new Bernstein inequality for data possessing functional dependence. Extensive numerical results are provided to support our theoretical results. The proposed methods are implemented in the R package \texttt{changepoints} \citep{changepoints_R}.

math.ST↗

No Train Still Gain. Unleash Mathematical Reasoning of Large Language Models with Monte Carlo Tree Search Guided by Energy Function

Large language models (LLMs) demonstrate impressive language understanding and contextual learning abilities, making them suitable for natural language processing (NLP) tasks and complex mathematical reasoning. However, when applied to mathematical reasoning tasks, LLMs often struggle to generate correct reasoning steps and answers despite having high probabilities for the solutions. To overcome this limitation and enhance the mathematical reasoning capabilities of fine-tuned LLMs without additional fine-tuning steps, we propose a method that incorporates Monte Carlo Tree Search (MCTS) and a lightweight energy function to rank decision steps and enable immediate reaction and precise reasoning. Specifically, we re-formulate the fine-tuned LLMs into a Residual-based Energy Model (Residual-EBM) and employ noise contrastive estimation to estimate the energy function's parameters. We then utilize MCTS with the energy function as a path verifier to search the output space and evaluate the reasoning path. Through extensive experiments on two mathematical reasoning benchmarks, GSM8k and AQUA-RAT, we demonstrate the exceptional capabilities of our method, which significantly improves the pass@1 metric of the fine-tuned model without requiring additional fine-tuning or reinforcement learning with human feedback alignment.

cs.AI↗

Efficient Exploration Using Extra Safety Budget in Constrained Policy Optimization

Reinforcement learning (RL) has achieved promising results on most robotic control tasks. Safety of learning-based controllers is an essential notion of ensuring the effectiveness of the controllers. Current methods adopt whole consistency constraints during the training, thus resulting in inefficient exploration in the early stage. In this paper, we propose an algorithm named Constrained Policy Optimization with Extra Safety Budget (ESB-CPO) to strike a balance between the exploration efficiency and the constraints satisfaction. In the early stage, our method loosens the practical constraints of unsafe transitions (adding extra safety budget) with the aid of a new metric we propose. With the training process, the constraints in our optimization problem become tighter. Meanwhile, theoretical analysis and practical experiments demonstrate that our method gradually meets the cost limit's demand in the final training stage. When evaluated on Safety-Gym and Bullet-Safety-Gym benchmarks, our method has shown its advantages over baseline algorithms in terms of safety and optimality. Remarkably, our method gains remarkable performance improvement under the same cost limit compared with baselines.

cs.RO↗

CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibility

With the rapid evolution of large language models (LLMs), there is a growing concern that they may pose risks or have negative social impacts. Therefore, evaluation of human values alignment is becoming increasingly important. Previous work mainly focuses on assessing the performance of LLMs on certain knowledge and reasoning abilities, while neglecting the alignment to human values, especially in a Chinese context. In this paper, we present CValues, the first Chinese human values evaluation benchmark to measure the alignment ability of LLMs in terms of both safety and responsibility criteria. As a result, we have manually collected adversarial safety prompts across 10 scenarios and induced responsibility prompts from 8 domains by professional experts. To provide a comprehensive values evaluation of Chinese LLMs, we not only conduct human evaluation for reliable comparison, but also construct multi-choice prompts for automatic evaluation. Our findings suggest that while most Chinese LLMs perform well in terms of safety, there is considerable room for improvement in terms of responsibility. Moreover, both the automatic and human evaluation are important for assessing the human values alignment in different aspects. The benchmark and code is available on ModelScope and Github.

cs.CL↗

Change point detection and inference in multivariable nonparametric models under mixing conditions

This paper studies multivariate nonparametric change point localization and inference problems. The data consists of a multivariate time series with potentially short range dependence. The distribution of this data is assumed to be piecewise constant with densities in a Hölder class. The change points, or times at which the distribution changes, are unknown. We derive the limiting distributions of the change point estimators when the minimal jump size vanishes or remains constant, a first in the literature on change point settings. We are introducing two new features: a consistent estimator that can detect when a change is happening in data with short-term dependence, and a consistent block-type long-run variance estimator. Numerical evidence is provided to back up our theoretical results.

math.ST↗

VizInspect Pro -- Automated Optical Inspection (AOI) solution

Traditional vision based Automated Optical Inspection (referred to as AOI in paper) systems present multiple challenges in factory settings including inability to scale across multiple product lines, requirement of vendor programming expertise, little tolerance to variations and lack of cloud connectivity for aggregated insights. The lack of flexibility in these systems presents a unique opportunity for a deep learning based AOI system specifically for factory automation. The proposed solution, VizInspect pro is a generic computer vision based AOI solution built on top of Leo - An edge AI platform. Innovative features that overcome challenges of traditional vision systems include deep learning based image analysis which combines the power of self-learning with high speed and accuracy, an intuitive user interface to configure inspection profiles in minutes without ML or vision expertise and the ability to solve complex inspection challenges while being tolerant to deviations and unpredictable defects. This solution has been validated by multiple external enterprise customers with confirmed value propositions. In this paper we show you how this solution and platform solved problems around model development, deployment, scaling multiple inferences and visualizations.

cs.AI↗

Generating Disentangled Arguments with Prompts: A Simple Event Extraction Framework that Works

Event Extraction bridges the gap between text and event signals. Based on the assumption of trigger-argument dependency, existing approaches have achieved state-of-the-art performance with expert-designed templates or complicated decoding constraints. In this paper, for the first time we introduce the prompt-based learning strategy to the domain of Event Extraction, which empowers the automatic exploitation of label semantics on both input and output sides. To validate the effectiveness of the proposed generative method, we conduct extensive experiments with 11 diverse baselines. Empirical results show that, in terms of F1 score on Argument Extraction, our simple architecture is stronger than any other generative counterpart and even competitive with algorithms that require template engineering. Regarding the measure of recall, it sets new overall records for both Argument and Trigger Extractions. We hereby recommend this framework to the community, with the code publicly available at https://git.io/GDAP.

cs.CL↗

Localising change points in piecewise polynomials of general degrees

In this paper we are concerned with a sequence of univariate random variables with piecewise polynomial means and independent sub-Gaussian noise. The underlying polynomials are allowed to be of arbitrary but fixed degrees. All the other model parameters are allowed to vary depending on the sample size. We propose a two-step estimation procedure based on the $\ell_0$-penalisation and provide upper bounds on the localisation error. We complement these results by deriving a global information-theoretic lower bounds, which show that our two-step estimators are nearly minimax rate-optimal. We also show that our estimator enjoys near optimally adaptive performance by attaining individual localisation errors depending on the level of smoothness at individual change points of the underlying signal. In addition, under a special smoothness constraint, we provide a minimax lower bound on the localisation errors. This lower bound is independent of the polynomial orders and is sharper than the global minimax lower bound.

math.ST↗

Stacked polarimeters with twisted black phosphorus

The real-time, in-line analysis of light polarization is critical in optical communication networks, which suffers from the complex systems with numerous bulky opto-electro-mechanical elements tandemly arranged along optical path. Here, we propose a fiber-integrated polarimeter with nano-thickness by vertically stacking three two-dimensional (2D) materials based photodetection units. We demonstrate a self-power-calibrated, ultrafast, unambiguous detection of linear (LP) and circular polarized (CP) light according to the symmetry broken induced linear photogalvanic effects (LPGE) and circular photogalvanic effects (CPGE) in black phosphorous (BP) units, which are twistedly stacked to substitute traditional mechanical rotation of polarizers. As a demonstration, we achieve Hadamard single-pixel polarimetric imaging by the polarimeter to recognize the polarization distributions, showing potential in high-speed polarization-division-multiplexed imaging and real-time polarized endoscopy. This work provides a new strategy for next-generation ultracompact optical and optoelectronic systems, and guides a way for developing high-resolution arrayed devices with multifunctional pixels.

physics.optics↗

An Improved Distributed Nonlinear Observer for Leader-Following Consensus Via Differential Geometry Approach

This paper is concerned with the leader-following output consensus problem in the framework of distributed nonlinear observers. In stead of certain hypotheses on the leader system, a group of geometric conditions is put forward to develop a novel distributed observer strategy with less conservatism, thereby definitely improving the applicability of the existing results. To be more specific, the improved distributed observer can precisely handle consensus problems for some nonlinear leader systems which are invalid for the traditional strategies with the certain assumption, such as Elastic Shaft Single Linkage Manipulator (ESSLM) systems and most of first-order nonlinear systems. We prove the sufficient conditions for the exponential stability of our distributed observer's error dynamic by proposing two pioneered lemmas to show the relationship between the maximum eigenvalues of two matrices appearing in Lyapunov type matrices. Then, a partial feedback linearization method with zero dynamic proposed in differential geometry is employed to design a purely decentralized control law for the affine nonlinear multi-agent system. With this advancement, the existing results can be regarded as a specific case owing to that the followers can be chosen as an arbitrary minimum phase affine smooth nonlinear system. At last, the novel distributed observer and the improved purely decentralized control law are applied in the distributed control framework to construct a closed-loop system. We also prove the stability of closed-loop system to achieve leader-following consensus, i.e., the distributed control framework is proved to satisfy certainty equivalence principle. Our method is illustrated by ESSLM system and Van der Pol system as leader.

math.OC↗

Robust Two-Step Wavelet-Based Inference for Time Series Models

Complex time series models such as (the sum of) ARMA$(p,q)$ models with additional noise, random walks, rounding errors and/or drifts are increasingly used for data analysis in fields such as biology, ecology, engineering and economics where the length of the observed signals can be extremely large. Performing inference on and/or prediction from these models can be highly challenging for several reasons: (i) the data may contain outliers that can adversely affect the estimation procedure; (ii) the computational complexity can become prohibitive when models include more than just a few parameters and/or the time series are large; (iii) model building and/or selection adds another layer of (computational) complexity to the previous task; and (iv) solutions that address (i), (ii) and (iii) simultaneously do not exist in practice. For this reason, this paper aims at jointly addressing these challenges by proposing a general framework for robust two-step estimation based on a bounded influence M-estimator of the wavelet variance. In this perspective, we first develop the conditions for the joint asymptotic normality of the latter estimator thereby providing the necessary tools to perform (direct) inference for scale-based analysis of signals. Taking advantage of the model-independent weights of this first-step estimator that are computed only once, we then develop the asymptotic properties of two-step robust estimators using the framework of the Generalized Method of Wavelet Moments (GMWM), hence defining the Robust GMWM (RGMWM) that we then use for robust model estimation and inference in a computationally efficient manner even for large time series. Simulation studies illustrate the good finite sample performance of the RGMWM estimator and applied examples highlight the practical relevance of the proposed approach.

stat.ME↗

Wavelet-Based Moment-Matching Techniques for Inertial Sensor Calibration

The task of inertial sensor calibration has required the development of various techniques to take into account the sources of measurement error coming from such devices. The calibration of the stochastic errors of these sensors has been the focus of increasing amount of research in which the method of reference has been the so-called "Allan variance slope method" which, in addition to not having appropriate statistical properties, requires a subjective input which makes it prone to mistakes. To overcome this, recent research has started proposing "automatic" approaches where the parameters of the probabilistic models underlying the error signals are estimated by matching functions of the Allan variance or Wavelet Variance with their model-implied counterparts. However, given the increased use of such techniques, there has been no study or clear direction for practitioners on which approach is optimal for the purpose of sensor calibration. This paper formally defines the class of estimators based on this technique and puts forward theoretical and applied results that, comparing with estimators in this class, suggest the use of the Generalized Method of Wavelet Moments as an optimal choice.

stat.ME↗

Multivariate Signal Modelling with Applications to Inertial Sensor Calibration

The common approach to inertial sensor calibration for navigation purposes has been to model the stochastic error signals of individual sensors independently, whether as components of a single inertial measurement unit (IMU) in different directions or arrayed in the same direction for redundancy. These signals usually have an extremely complex spectral structure that is often described using latent (or composite) models composed by a sum of underlying models. A large amount of research in this domain has been focused on the latter aspect through the proposal of various methods that have been able to improve the estimation of these models both from a computational and a statistical point of view. However, the separate calibration of the individual sensors is still unable to take into account the dependence between each of them which can have an important impact on the precision of the navigation systems. In this paper, we develop a new approach to simultaneously model both the individual signals and the dependence between them by studying the quantity called Wavelet Cross-Covariance and using it to extend the application of the Generalized Method of Wavelet Moments. This new method can be used in other settings for time series modeling, especially in cases where the dependence among signals may be hard to detect. Moreover, in the field of inertial sensor calibration, this approach can deliver important contributions among which the possibility to test dependence between sensors, integrate their dependence within the navigation filter and construct an optimal virtual sensor that can be used to simplify and improve navigation accuracy. The advantages of this method and its usefulness for inertial sensor calibration are highlighted through a simulation study and an applied example with a small array of XSens MTi-G IMUs.

stat.ME↗

Joint Stochastic Approximation learning of Helmholtz Machines

Though with progress, model learning and performing posterior inference still remains a common challenge for using deep generative models, especially for handling discrete hidden variables. This paper is mainly concerned with algorithms for learning Helmholz machines, which is characterized by pairing the generative model with an auxiliary inference model. A common drawback of previous learning algorithms is that they indirectly optimize some bounds of the targeted marginal log-likelihood. In contrast, we successfully develop a new class of algorithms, based on stochastic approximation (SA) theory of the Robbins-Monro type, to directly optimize the marginal log-likelihood and simultaneously minimize the inclusive KL-divergence. The resulting learning algorithm is thus called joint SA (JSA). Moreover, we construct an effective MCMC operator for JSA. Our results on the MNIST datasets demonstrate that the JSA's performance is consistently superior to that of competing algorithms like RWS, for learning a range of difficult models.

cs.LG↗

A Study of the Allan Variance for Constant-Mean Non-Stationary Processes

The Allan Variance (AV) is a widely used quantity in areas focusing on error measurement as well as in the general analysis of variance for autocorrelated processes in domains such as engineering and, more specifically, metrology. The form of this quantity is widely used to detect noise patterns and indications of stability within signals. However, the properties of this quantity are not known for commonly occurring processes whose covariance structure is non-stationary and, in these cases, an erroneous interpretation of the AV could lead to misleading conclusions. This paper generalizes the theoretical form of the AV to some non-stationary processes while at the same time being valid also for weakly stationary processes. Some simulation examples show how this new form can help to understand the processes for which the AV is able to distinguish these from the stationary cases and hence allow for a better interpretation of this quantity in applied cases.

math.ST↗

Joint Bayesian Gaussian discriminant analysis for speaker verification

State-of-the-art i-vector based speaker verification relies on variants of Probabilistic Linear Discriminant Analysis (PLDA) for discriminant analysis. We are mainly motivated by the recent work of the joint Bayesian (JB) method, which is originally proposed for discriminant analysis in face verification. We apply JB to speaker verification and make three contributions beyond the original JB. 1) In contrast to the EM iterations with approximated statistics in the original JB, the EM iterations with exact statistics are employed and give better performance. 2) We propose to do simultaneous diagonalization (SD) of the within-class and between-class covariance matrices to achieve efficient testing, which has broader application scope than the SVD-based efficient testing method in the original JB. 3) We scrutinize similarities and differences between various Gaussian PLDAs and JB, complementing the previous analysis of comparing JB only with Prince-Elder PLDA. Extensive experiments are conducted on NIST SRE10 core condition 5, empirically validating the superiority of JB with faster convergence rate and 9-13% EER reduction compared with state-of-the-art PLDA.

cs.SD↗