arXiv subjects

Xiaopeng Luo

Publications and source records attributed to Xiaopeng Luo.

15 recordsLinked to original sources

arXiv2026

Hierarchical Reference Sets for Robust Unsupervised Detection of Scattered and Clustered Outliers

Most real-world IoT data analysis tasks, such as clustering and anomaly event detection, are unsupervised and highly susceptible to the presence of outliers. In addition to sporadic scattered outliers caused by factors such as faulty sensor readings, IoT systems often exhibit clustered outliers. These occur when multiple devices or nodes produce similar anomalous measurements, for instance, owing to localized interference, emerging security threats, or regional false alarms, forming micro-clusters. These clustered outliers can be easily mistaken for normal behavior because of their relatively high local density, thereby obscuring the detection of both scattered and contextual anomalies. To address this, we propose a novel outlier detection paradigm that leverages the natural neighboring relationships using graph structures. This facilitates multi-perspective anomaly evaluation by incorporating reference sets at both local and global scales derived from the graph. Our approach enables the effective recognition of scattered outliers without interference from clustered anomalies, whereas the graph structure simultaneously helps reflect and isolate clustered outlier groups. Extensive experiments, including comparative performance analysis, ablation studies, validation on downstream clustering tasks, and evaluation of hyperparameter sensitivity, demonstrate the efficacy of the proposed method. The source code is available at https://github.com/gordonlok/DROD.

cs.LG↗

arXiv2026

Robust Categorical Data Clustering Guided by Multi-Granular Competitive Learning

Data set composed of categorical features is very common in big data analysis tasks. Since categorical features are usually with a limited number of qualitative possible values, the nested granular cluster effect is prevalent in the implicit discrete distance space of categorical data. That is, data objects frequently overlap in space or subspace to form small compact clusters, and similar small clusters often form larger clusters. However, the distance space cannot be well-defined like the Euclidean distance due to the qualitative categorical data values, which brings great challenges to the cluster analysis of categorical data. In view of this, we design a Multi-Granular Competitive Penalization Learning (MGCPL) algorithm to allow potential clusters to interactively tune themselves and converge in stages with different numbers of naturally compact clusters. To leverage MGCPL, we also propose a Cluster Aggregation strategy based on MGCPL Encoding (CAME) to first encode the data objects according to the learned multi-granular distributions, and then perform final clustering on the embeddings. It turns out that the proposed MGCPL-guided Categorical Data Clustering (MCDC) approach is competent in automatically exploring the nested distribution of multi-granular clusters and highly robust to categorical data sets from various domains. Benefiting from its linear time complexity, MCDC is scalable to large-scale data sets and promising in pre-partitioning data sets or compute nodes for boosting distributed computing. Extensive experiments with statistical evidence demonstrate its superiority compared to state-of-the-art counterparts on various real public data sets.

cs.LG↗

arXiv2026

HMVI: Unifying Heterogeneous Attributes with Natural Neighbors for Missing Value Inference

Missing value imputation is a fundamental challenge in machine intelligence, heavily dependent on data completeness. Current imputation methods often handle numerical and categorical attributes independently, overlooking critical interdependencies among heterogeneous features. To address these limitations, we propose a novel imputation approach that explicitly models cross-type feature dependencies within a unified framework. Our method leverages both complete and incomplete instances to ensure accurate and consistent imputation in tabular data. Extensive experimental results demonstrate that the proposed approach achieves superior performance over existing techniques and significantly enhances downstream machine learning tasks, providing a robust solution for real-world systems with missing data.

cs.LG↗

arXiv2025

Mask the Redundancy: Evolving Masking Representation Learning for Multivariate Time-Series Clustering

Multivariate Time-Series (MTS) clustering discovers intrinsic grouping patterns of temporal data samples. Although time-series provide rich discriminative information, they also contain substantial redundancy, such as steady-state machine operation records and zero-output periods of solar power generation. Such redundancy diminishes the attention given to discriminative timestamps in representation learning, thus leading to performance bottlenecks in MTS clustering. Masking has been widely adopted to enhance the MTS representation, where temporal reconstruction tasks are designed to capture critical information from MTS. However, most existing masking strategies appear to be standalone preprocessing steps, isolated from the learning process, which hinders dynamic adaptation to the importance of clustering-critical timestamps. Accordingly, this paper proposes the Evolving-masked MTS Clustering (EMTC) method, whose model architecture comprises Importance-aware Variate-wise Masking (IVM) and Multi-Endogenous Views (MEV) generation modules. IVM adaptively guides the model in learning more discriminative representations for clustering, while the reconstruction and cluster-guided contrastive learning pathways enhance and connect the representation learning to clustering tasks. Extensive experiments on 15 benchmark datasets demonstrate the superiority of EMTC over eight SOTA methods, where the EMTC achieves an average improvement of 4.85% in F1-Score over the strongest baselines.

cs.LG↗

arXiv2024

Learning Self-Growth Maps for Fast and Accurate Imbalanced Streaming Data Clustering

Streaming data clustering is a popular research topic in data mining and machine learning. Since streaming data is usually analyzed in data chunks, it is more susceptible to encounter the dynamic cluster imbalance issue. That is, the imbalance ratio of clusters changes over time, which can easily lead to fluctuations in either the accuracy or the efficiency of streaming data clustering. Therefore, we propose an accurate and efficient streaming data clustering approach to adapt the drifting and imbalanced cluster distributions. We first design a Self-Growth Map (SGM) that can automatically arrange neurons on demand according to local distribution, and thus achieve fast and incremental adaptation to the streaming distributions. Since SGM allocates an excess number of density-sensitive neurons to describe the global distribution, it can avoid missing small clusters among imbalanced distributions. We also propose a fast hierarchical merging strategy to combine the neurons that break up the relatively large clusters. It exploits the maintained SGM to quickly retrieve the intra-cluster distribution pairs for merging, which circumvents the most laborious global searching. It turns out that the proposed SGM can incrementally adapt to the distributions of new chunks, and the Self-grOwth map-guided Hierarchical merging for Imbalanced data clustering (SOHI) approach can quickly explore a true number of imbalanced clusters. Extensive experiments demonstrate that SOHI can efficiently and accurately explore cluster distributions for streaming data.

cs.LG↗

arXiv2021

Contraction methods for continuous optimization

Motivated by the grid search method and Bayesian optimization, we introduce the concept of contractibility and its applications in model-based optimization. First, a basic framework of contraction methods is established to construct a nonempty closed set sequence that contracts from the initial domain to the set of global minimizers. Then, from the perspective of whether the contraction can be carried out effectively, relevant conditions are introduced to divide all continuous optimization problems into three categories: (i) logarithmic time contractible, (ii) polynomial time contractible, or (iii) noncontractible. For every problem from the first two categories, there exists a contraction sequence that converges to the set of all global minimizers with linear convergence; for any problem from the last category, we discuss possible troubles caused by contraction. Finally, a practical algorithm is proposed with high probability bounds for convergence rate and complexity. It is shown that the contractibility contributes to practical applications and can also be seen as a complement to smoothness for distinguishing the optimization problems that are easy to solve.

math.OC↗

arXiv2020

Asymptotic proximal point methods: finding the global minima with linear convergence for a class of multiple minima problems

We propose and analyze asymptotic proximal point (APP) methods to find the global minimizer for a class of nonconvex, nonsmooth, or even discontinuous multiple minima functions. The method is based on an asymptotic representation of nonconvex proximal points so that it can find the global minimizer without being trapped in saddle points, local minima, or even discontinuities. Our main result shows that the method enjoys the global linear convergence for such a class of functions. Furthermore, the method is derivative-free and its per-iteration cost, i.e., the number of function evaluations, is also bounded, so it has a complexity bound $\mathcal{O}(\log\frac{1}ε)$ for finding a point such that the gap between this point and the global minimizer is less than $ε>0$. Numerical experiments and comparisons in various dimensions from $2$ to $500$ demonstrate the benefits of the method.

math.OC↗

arXiv2020

Derivative-free global minimization for a class of multiple minima problems

We prove that the finite-difference based derivative-free descent (FD-DFD) methods have a capability to find the global minima for a class of multiple minima problems. Our main result shows that, for a class of multiple minima objectives that is extended from strongly convex functions with Lipschitz-continuous gradients, the iterates of FD-DFD converge to the global minimizer $x_*$ with the linear convergence $\|x_{k+1}-x_*\|_2^2\leqslantρ^k \|x_1-x_*\|_2^2$ for a fixed $0<ρ<1$ and any initial iteration $x_1\in\mathbb{R}^d$ when the parameters are properly selected. Since the per-iteration cost, i.e., the number of function evaluations, is fixed and almost independent of the dimension $d$, the FD-DFD algorithm has a complexity bound $\mathcal{O}(\log\frac{1}ε)$ for finding a point $x$ such that the optimality gap $\|x-x_*\|_2^2$ is less than $ε>0$. Numerical experiments in various dimensions from $5$ to $500$ demonstrate the benefits of the FD-DFD method.

math.OC↗

arXiv2020

A New Accelerated Stochastic Gradient Method with Momentum

In this paper, we propose a novel accelerated stochastic gradient method with momentum, which momentum is the weighted average of previous gradients. The weights decays inverse proportionally with the iteration times. Stochastic gradient descent with momentum (Sgdm) use weights that decays exponentially with the iteration times to generate an momentum term. Using exponentially decaying weights, variants of Sgdm with well designed and complicated formats have been proposed to achieve better performance. The momentum update rules of our method is as simple as that of Sgdm. We provide theoretical convergence properties analyses for our method, which show both the exponentially decay weights and our inverse proportionally decay weights can limit the variance of the moving direction of parameters to be optimized to a region. Experimental results empirically show that our method works well with practical problems and outperforms Sgdm, and it outperforms Adam in convolutional neural networks.

cs.LG↗

arXiv2020

Can speed up the convergence rate of stochastic gradient methods to $\mathcal{O}(1/k^2)$ by a gradient averaging strategy?

In this paper we consider the question of whether it is possible to apply a gradient averaging strategy to improve on the sublinear convergence rates without any increase in storage. Our analysis reveals that a positive answer requires an appropriate averaging strategy and iterations that satisfy the variance dominant condition. As an interesting fact, we show that if the iterative variance we defined is always dominant even a little bit in the stochastic gradient iterations, the proposed gradient averaging strategy can increase the convergence rate $\mathcal{O}(1/k)$ to $\mathcal{O}(1/k^2)$ in probability for the strongly convex objectives with Lipschitz gradients. This conclusion suggests how we should control the stochastic gradient iterations to improve the rate of convergence.

math.OC↗

arXiv2020

Stochastic gradient-free descents

In this paper we propose stochastic gradient-free methods and accelerated methods with momentum for solving stochastic optimization problems. All these methods rely on stochastic directions rather than stochastic gradients. We analyze the convergence behavior of these methods under the mean-variance framework, and also provide a theoretical analysis about the inclusion of momentum in stochastic settings which reveals that the momentum term we used adds a deviation of order $\mathcal{O}(1/k)$ but controls the variance at the order $\mathcal{O}(1/k)$ for the $k$th iteration. So it is shown that, when employing a decaying stepsize $α_k=\mathcal{O}(1/k)$, the stochastic gradient-free methods can still maintain the sublinear convergence rate $\mathcal{O}(1/k)$ and the accelerated methods with momentum can achieve a convergence rate $\mathcal{O}(1/k^2)$ in probability for the strongly convex objectives with Lipschitz gradients; and all these methods converge to a solution with a zero expected gradient norm when the objective function is nonconvex, twice differentiable and bounded below.

math.OC↗

arXiv2019

Minima distribution for global optimization

This paper establishes a strict mathematical relationship between an arbitrary continuous function on a compact set and its global minima, like the well-known first order optimality condition for convex and differentiable functions. By introducing a class of nascent minima distribution functions that is only related to the target function and the given compact set, we construct a sequence that monotonically converges to the global minima on that given compact set. Then, we further consider some various sequences of sets where each sequence monotonically shrinks from the original compact set to the set of all global minimizers, and the shrink rate can be determined for continuously differentiable functions. Finally, we provide a different way of constructing the nascent minima distribution functions.

math.OC↗

arXiv2019

Numerical meshless solution of high-dimensional sine-Gordon equations via Fourier HDMR-HC approximation

In this paper, an implicit time stepping meshless scheme is proposed to find the numerical solution of high-dimensional sine-Gordon equations (SGEs) by combining the high dimensional model representation (HDMR) and the Fourier hyperbolic cross (HC) approximation. To ensure the sparseness of the relevant coefficient matrices of the implicit time stepping scheme, the whole domain is first divided into a set of subdomains, and the relevant derivatives in high-dimension can be separately approximated by the Fourier HDMR-HC approximation in each subdomain. The proposed method allows for stable large time-steps and a relatively small number of nodes with satisfactory accuracy. The numerical examples show that the proposed method is very attractive for simulating the high-dimensional SGEs.

math.NA↗

arXiv2019

Meshless Hermite-HDMR finite difference method for high-dimensional Dirichlet problems

In this paper, a meshless Hermite-HDMR finite difference method is proposed to solve high-dimensional Dirichlet problems. The approach is based on the local Hermite-HDMR expansion with an additional smoothing technique. First, we introduce the HDMR decomposition combined with the multiple Hermite series to construct a class of Hermite-HDMR approximations, and the relevant error estimate is theoretically built in a class of Hermite spaces. It can not only provide high order convergence but also retain good scaling with increasing dimensions. Then the Hermite-HDMR based finite difference method is particularly proposed for solving high-dimensional Dirichlet problems. By applying a smoothing process to the Hermite-HDMR approximations, numerical stability can be guaranteed even with a small number of nodes. Numerical experiments in dimensions up to $30$ show that resulting approximations are of very high quality.

math.NA↗

arXiv2019

Sparse residual tree and forest

Sparse residual tree (SRT) is an adaptive exploration method for multivariate scattered data approximation. It leads to sparse and stable approximations in areas where the data is sufficient or redundant, and points out the possible local regions where data refinement is needed. Sparse residual forest (SRF) is a combination of SRT predictors to further improve the approximation accuracy and stability according to the error characteristics of SRTs. The hierarchical parallel SRT algorithm is based on both tree decomposition and adaptive radial basis function (RBF) explorations, whereby for each child a sparse and proper RBF refinement is added to the approximation by minimizing the norm of the residual inherited from its parent. The convergence results are established for both SRTs and SRFs. The worst case time complexity of SRTs is $\mathcal{O}(N\log_2N)$ for the initial work and $\mathcal{O}(\log_2N)$ for each prediction, meanwhile, the worst case storage requirement is $\mathcal{O}(N\log_2N)$, where the $N$ data points can be arbitrary distributed. Numerical experiments are performed for several illustrative examples.

math.NA↗