arXiv subjects

Xin Luo

Publications and source records attributed to Xin Luo.

At least 91 records · Page 5Linked to original sources

arXiv2022

Multi-Metric AutoRec for High Dimensional and Sparse User Behavior Data Prediction

User behavior data produced during interaction with massive items in the significant data era are generally heterogeneous and sparse, leaving the recommender system (RS) a large diversity of underlying patterns to excavate. Deep neural network-based models have reached the state-of-the-art benchmark of the RS owing to their fitting capabilities. However, prior works mainly focus on designing an intricate architecture with fixed loss function and regulation. These single-metric models provide limited performance when facing heterogeneous and sparse user behavior data. Motivated by this finding, we propose a multi-metric AutoRec (MMA) based on the representative AutoRec. The idea of the proposed MMA is mainly two-fold: 1) apply different $L_p$-norm on loss function and regularization to form different variant models in different metric spaces, and 2) aggregate these variant models. Thus, the proposed MMA enjoys the multi-metric orientation from a set of dispersed metric spaces, achieving a comprehensive representation of user data. Theoretical studies proved that the proposed MMA could attain performance improvement. The extensive experiment on five real-world datasets proves that MMA can outperform seven other state-of-the-art models in predicting unobserved user behavior data.

cs.IR↗

arXiv2022

Surrogate-assisted level-based learning evolutionary search for heat extraction optimization of enhanced geothermal system

An enhanced geothermal system is essential to provide sustainable and long-term geothermal energy supplies and reduce carbon emissions. Optimal well-control scheme for effective heat extraction and improved heat sweep efficiency plays a significant role in geothermal development. However, the optimization performance of most existing optimization algorithms deteriorates as dimension increases. To solve this issue, a novel surrogate-assisted level-based learning evolutionary search algorithm (SLLES) is proposed for heat extraction optimization of enhanced geothermal system. SLLES consists of classifier-assisted level-based learning pre-screen part and local evolutionary search part. The cooperation of the two parts has realized the balance between the exploration and exploitation during the optimization process. After iteratively sampling from the design space, the robustness and effectiveness of the algorithm are proven to be improved significantly. To the best of our knowledge, the proposed algorithm holds state-of-the-art simulation-involved optimization framework. Comparative experiments have been conducted on benchmark functions, a two-dimensional fractured reservoir and a three-dimensional enhanced geothermal system. The proposed algorithm outperforms other five state-of-the-art surrogate-assisted algorithms on all selected benchmark functions. The results on the two heat extraction cases also demonstrate that SLLES can achieve superior optimization performance compared with traditional evolutionary algorithm and other surrogate-assisted algorithms. This work lays a solid basis for efficient geothermal extraction of enhanced geothermal system and sheds light on the model management strategies of data-driven optimization in the areas of energy exploitation.

cs.NE↗

arXiv2022

Highly-Accurate Electricity Load Estimation via Knowledge Aggregation

Mid-term and long-term electric energy demand prediction is essential for the planning and operations of the smart grid system. Mainly in countries where the power system operates in a deregulated environment. Traditional forecasting models fail to incorporate external knowledge while modern data-driven ignore the interpretation of the model, and the load series can be influenced by many complex factors making it difficult to cope with the highly unstable and nonlinear power load series. To address the forecasting problem, we propose a more accurate district level load prediction model Based on domain knowledge and the idea of decomposition and ensemble. Its main idea is three-fold: a) According to the non-stationary characteristics of load time series with obvious cyclicality and periodicity, decompose into series with actual economic meaning and then carry out load analysis and forecast. 2) Kernel Principal Component Analysis(KPCA) is applied to extract the principal components of the weather and calendar rule feature sets to realize data dimensionality reduction. 3) Give full play to the advantages of various models based on the domain knowledge and propose a hybrid model(XASXG) based on Autoregressive Integrated Moving Average model(ARIMA), support vector regression(SVR) and Extreme gradient boosting model(XGBoost). With such designs, it accurately forecasts the electricity demand in spite of their highly unstable characteristic. We compared our method with nine benchmark methods, including classical statistical models as well as state-of-the-art models based on machine learning, on the real time series of monthly electricity demand in four Chinese cities. The empirical study shows that the proposed hybrid model is superior to all competitors in terms of accuracy and prediction bias.

eess.SP↗

arXiv2022

A Node-collaboration-informed Graph Convolutional Network for Precise Representation to Undirected Weighted Graphs

An undirected weighted graph (UWG) is frequently adopted to describe the interactions among a solo set of nodes from real applications, such as the user contact frequency from a social network services system. A graph convolutional network (GCN) is widely adopted to perform representation learning to a UWG for subsequent pattern analysis tasks such as clustering or missing data estimation. However, existing GCNs mostly neglects the latent collaborative information hidden in its connected node pairs. To address this issue, this study proposes to model the node collaborations via a symmetric latent factor analysis model, and then regards it as a node-collaboration module for supplementing the collaboration loss in a GCN. Based on this idea, a Node-collaboration-informed Graph Convolutional Network (NGCN) is proposed with three-fold ideas: a) Learning latent collaborative information from the interaction of node pairs via a node-collaboration module; b) Building the residual connection and weighted representation propagation to obtain high representation capacity; and c) Implementing the model optimization in an end-to-end fashion to achieve precise representation to the target UWG. Empirical studies on UWGs emerging from real applications demonstrate that owing to its efficient incorporation of node-collaborations, the proposed NGCN significantly outperforms state-of-the-art GCNs in addressing the task of missing weight estimation. Meanwhile, its good scalability ensures its compatibility with more advanced GCN extensions, which will be further investigated in our future studies.

cs.LG↗

arXiv2022

FedVMR: A New Federated Learning method for Video Moment Retrieval

Despite the great success achieved, existing video moment retrieval (VMR) methods are developed under the assumption that data are centralizedly stored. However, in real-world applications, due to the inherent nature of data generation and privacy concerns, data are often distributed on different silos, bringing huge challenges to effective large-scale training. In this work, we try to overcome above limitation by leveraging the recent success of federated learning. As the first that is explored in VMR field, the new task is defined as video moment retrieval with distributed data. Then, a novel federated learning method named FedVMR is proposed to facilitate large-scale and secure training of VMR models in decentralized environment. Experiments on benchmark datasets demonstrate its effectiveness. This work is the very first attempt to enable safe and efficient VMR training in decentralized scene, which is hoped to pave the way for further study in the related research field.

cs.CV↗

arXiv2022

Prototype-Based Layered Federated Cross-Modal Hashing

Recently, deep cross-modal hashing has gained increasing attention. However, in many practical cases, data are distributed and cannot be collected due to privacy concerns, which greatly reduces the cross-modal hashing performance on each client. And due to the problems of statistical heterogeneity, model heterogeneity, and forcing each client to accept the same parameters, applying federated learning to cross-modal hash learning becomes very tricky. In this paper, we propose a novel method called prototype-based layered federated cross-modal hashing. Specifically, the prototype is introduced to learn the similarity between instances and classes on server, reducing the impact of statistical heterogeneity (non-IID) on different clients. And we monitor the distance between local and global prototypes to further improve the performance. To realize personalized federated learning, a hypernetwork is deployed on server to dynamically update different layers' weights of local model. Experimental results on benchmark datasets show that our method outperforms state-of-the-art methods.

cs.LG↗

arXiv2022

Adaptive Divergence-based Non-negative Latent Factor Analysis

High-Dimensional and Incomplete (HDI) data are frequently found in various industrial applications with complex interactions among numerous nodes, which are commonly non-negative for representing the inherent non-negativity of node interactions. A Non-negative Latent Factor (NLF) model is able to extract intrinsic features from such data efficiently. However, existing NLF models all adopt a static divergence metric like Euclidean distance or α-\b{eta} divergence to build its learning objective, which greatly restricts its scalability of accurately representing HDI data from different domains. Aiming at addressing this issue, this study presents an Adaptive Divergence-based Non-negative Latent Factor (ADNLF) model with three-fold ideas: a) generalizing the objective function with the α-\b{eta}-divergence to expand its potential of representing various HDI data; b) adopting a non-negative bridging function to connect the optimization variables with output latent factors for fulfilling the non-negativity constraints constantly; and c) making the divergence parameters adaptive through particle swarm optimization, thereby facilitating adaptive divergence in the learning objective to achieve high scalability. Empirical studies are conducted on four HDI datasets from real applications, whose results demonstrate that in comparison with state-of-the-art NLF models, an ADNLF model achieves significantly higher estimation accuracy for missing data of an HDI dataset with high computational efficiency.

cs.LG↗

arXiv2022

PI-NLF: A Proportional-Integral Approach for Non-negative Latent Factor Analysis

A high-dimensional and incomplete (HDI) matrix frequently appears in various big-data-related applications, which demonstrates the inherently non-negative interactions among numerous nodes. A non-negative latent factor (NLF) model performs efficient representation learning to an HDI matrix, whose learning process mostly relies on a single latent factor-dependent, non-negative and multiplicative update (SLF-NMU) algorithm. However, an SLF-NMU algorithm updates a latent factor based on the current update increment only without appropriate considerations of past learning information, resulting in slow convergence. Inspired by the prominent success of a proportional-integral (PI) controller in various applications, this paper proposes a Proportional-Integral-incorporated Non-negative Latent Factor (PI-NLF) model with two-fold ideas: a) establishing an Increment Refinement (IR) mechanism via considering the past update increments following the principle of a PI controller; and b) designing an IR-based SLF-NMU (ISN) algorithm to accelerate the convergence rate of a resultant model. Empirical studies on four HDI datasets demonstrate that a PI-NLF model outperforms the state-of-the-art models in both computational efficiency and estimation accuracy for missing data of an HDI matrix. Hence, this study unveils the feasibility of boosting the performance of a non-negative learning algorithm through an error feedback controller.

cs.LG↗

arXiv2022

A New Calibration Method for Industrial Robot Based on Step-Size Levenberg-Marquardt Algorithm

Industrial robots play a vital role in automatic production, which have been widely utilized in industrial production activities, like handling and welding. However, due to an uncalibrated robot with machining tolerance and assembly tolerance, it suffers from low absolute positioning accuracy, which cannot satisfy the requirements of high-precision manufacture. To address this hot issue, we propose a novel calibration method based on an unscented Kalman filter and variable step-size Levenberg-Marquardt algorithm. This work has three ideas: a) proposing a novel variable step-size Levenberg-Marquardt algorithm to addresses the issue of local optimum in a Levenberg-Marquardt algorithm; b) employing an unscented Kalman filter to reduce the influence of the measurement noises; and c) developing a novel calibration method incorporating an unscented Kalman filter with a variable step-size Levenberg-Marquardt algorithm. Furthermore, we conduct enough experiments on an ABB IRB 120 industrial robot. From the experimental results, the proposed method achieves much higher calibration accuracy than some state-of-the-art calibration methods. Hence, this work is an important milestone in the field of robot calibration.

cs.RO↗

arXiv2022

Data-driven evolutionary algorithm for oil reservoir well-placement and control optimization

Optimal well placement and well injection-production are crucial for the reservoir development to maximize the financial profits during the project lifetime. Meta-heuristic algorithms have showed good performance in solving complex, nonlinear and non-continuous optimization problems. However, a large number of numerical simulation runs are involved during the optimization process. In this work, a novel and efficient data-driven evolutionary algorithm, called generalized data-driven differential evolutionary algorithm (GDDE), is proposed to reduce the number of simulation runs on well-placement and control optimization problems. Probabilistic neural network (PNN) is adopted as the classifier to select informative and promising candidates, and the most uncertain candidate based on Euclidean distance is prescreened and evaluated with a numerical simulator. Subsequently, local surrogate model is built by radial basis function (RBF) and the optimum of the surrogate, found by optimizer, is evaluated by the numerical simulator to accelerate the convergence. It is worth noting that the shape factors of RBF model and PNN are optimized via solving hyper-parameter sub-expensive optimization problem. The results show the optimization algorithm proposed in this study is very promising for a well-placement optimization problem of two-dimensional reservoir and joint optimization of Egg model.

cs.NE↗

arXiv2022

A Differential Evolution-Enhanced Latent Factor Analysis Model for High-dimensional and Sparse Data

High-dimensional and sparse (HiDS) matrices are frequently adopted to describe the complex relationships in various big data-related systems and applications. A Position-transitional Latent Factor Analysis (PLFA) model can accurately and efficiently represent an HiDS matrix. However, its involved latent factors are optimized by stochastic gradient descent with the specific gradient direction step-by-step, which may cause a suboptimal solution. To address this issue, this paper proposes a Sequential-Group-Differential- Evolution (SGDE) algorithm to refine the latent factors optimized by a PLFA model, thereby achieving a highly-accurate SGDE-PLFA model to HiDS matrices. As demonstrated by the experiments on four HiDS matrices, a SGDE-PLFA model outperforms the state-of-the-art models.

cs.LG↗

arXiv2022

Graph-incorporated Latent Factor Analysis for High-dimensional and Sparse Matrices

A High-dimensional and sparse (HiDS) matrix is frequently encountered in a big data-related application like an e-commerce system or a social network services system. To perform highly accurate representation learning on it is of great significance owing to the great desire of extracting latent knowledge and patterns from it. Latent factor analysis (LFA), which represents an HiDS matrix by learning the low-rank embeddings based on its observed entries only, is one of the most effective and efficient approaches to this issue. However, most existing LFA-based models perform such embeddings on a HiDS matrix directly without exploiting its hidden graph structures, thereby resulting in accuracy loss. To address this issue, this paper proposes a graph-incorporated latent factor analysis (GLFA) model. It adopts two-fold ideas: 1) a graph is constructed for identifying the hidden high-order interaction (HOI) among nodes described by an HiDS matrix, and 2) a recurrent LFA structure is carefully designed with the incorporation of HOI, thereby improving the representa-tion learning ability of a resultant model. Experimental results on three real-world datasets demonstrate that GLFA outperforms six state-of-the-art models in predicting the missing data of an HiDS matrix, which evidently supports its strong representation learning ability to HiDS data.

cs.LG↗

arXiv2022

A Multi-Metric Latent Factor Model for Analyzing High-Dimensional and Sparse data

High-dimensional and sparse (HiDS) matrices are omnipresent in a variety of big data-related applications. Latent factor analysis (LFA) is a typical representation learning method that extracts useful yet latent knowledge from HiDS matrices via low-rank approximation. Current LFA-based models mainly focus on a single-metric representation, where the representation strategy designed for the approximation Loss function, is fixed and exclusive. However, real-world HiDS matrices are commonly heterogeneous and inclusive and have diverse underlying patterns, such that a single-metric representation is most likely to yield inferior performance. Motivated by this, we in this paper propose a multi-metric latent factor (MMLF) model. Its main idea is two-fold: 1) two vector spaces and three Lp-norms are simultaneously employed to develop six variants of LFA model, each of which resides in a unique metric representation space, and 2) all the variants are ensembled with a tailored, self-adaptive weighting strategy. As such, our proposed MMLF enjoys the merits originated from a set of disparate metric spaces all at once, achieving the comprehensive and unbiased representation of HiDS matrices. Theoretical study guarantees that MMLF attains a performance gain. Extensive experiments on eight real-world HiDS datasets, spanning a wide range of industrial and science domains, verify that our MMLF significantly outperforms ten state-of-the-art, shallow and deep counterparts.

cs.LG↗

arXiv2022

A New Robot Arm Calibration Method Based on Cubic Interpolated Beetle Antennae Search Approach

Industrial robot arms are extensively important for intelligent manufacturing. An industrial robot arm commonly enjoys its high repetitive positioning accuracy while suffering from its low absolute positioning accuracy, which greatly restricts its application in high-precision manufacture, like automobile manufacture. Aiming at addressing this hot issue, this work proposes a novel robot arm calibration method based on cubic interpolated beetle antennae search (CIBAS). This study has three ideas: a) developing a novel CIBAS algorithm, which can effectively addresses the issue of local optimum in a Beetle Antennae Search algorithm; b) utilizing a particle filter to reduce the influence of non-Gaussian noises; and c) proposing a new calibration method incorporating CIBAS algorithm and particle filter to searching the optimal kinematic parameters. Experimental results on an ABB IRB120 industrial robot arm demonstrate that the proposed method achieves much higher calibration accuracy than several state-of-the-art calibration methods.

cs.RO↗

arXiv2022

An Advancing Ensemble with Diversified Algorithms for Robot Arm Calibration

Recently, industrial robots plays a significant role in intelligent manufacturing. Hence, it is an urgent issue to ensure the robot with the high positioning precision. To address this hot issue, a novel calibration method based on an powerful ensemble with various algorithms is proposed. This paper has two ideas: a) developing eight calibration methods to identify the kinematic parameter errors; 2) establishing an effective ensemble to search calibrated kinematic parameters. Enough experimental results show that this ensemble can achieve: 1) higher calibration accuracy for the robot; 2) model diversity; 3) strong generalization ability.

cs.RO↗

arXiv2022

A Novel Quadratic Interpolated Beetle Antennae Search for Manipulator Calibration

Over the past decades, industrial manipulators play a vital role in in various fields, like aircraft manufacturing and automobile manufacturing. However, an industrial manipulator without calibration suffers from its low absolute positioning accuracy, which extensively restricts its application in high-precision intelligent manufacture. Recent manipulator calibration methods are developed to address this issue, while they frequently encounter long-tail convergence and low calibration accuracy. To address this thorny issue, this work proposes a novel manipulator calibration method incorporating an extended Kalman filter with a Quadratic Interpolated Beetle Antennae Search algorithm. This paper has three-fold ideas: a) proposing a new Quadratic Interpolated Beetle Antennae Search algorithm to deal with the issue of local optimum and low convergence rate in a Beetle Antennae Search algorithm; b) adopting an extended Kalman filter algorithm to suppress non-Gaussian noises and c) developing a new manipulator calibration method incorporating an extended Kalman filter with a Quadratic Interpolated Beetle Antennae Search algorithm to calibrating a manipulator. Extensively experimental results on an ABB IRB120 industrial manipulator demonstrate that the proposed method achieves much higher calibration accuracy than several state-of-the-art calibration methods.

cs.RO↗

arXiv2022

Three-Stream Joint Network for Zero-Shot Sketch-Based Image Retrieval

The Zero-Shot Sketch-based Image Retrieval (ZS-SBIR) is a challenging task because of the large domain gap between sketches and natural images as well as the semantic inconsistency between seen and unseen categories. Previous literature bridges seen and unseen categories by semantic embedding, which requires prior knowledge of the exact class names and additional extraction efforts. And most works reduce domain gap by mapping sketches and natural images into a common high-level space using constructed sketch-image pairs, which ignore the unpaired information between images and sketches. To address these issues, in this paper, we propose a novel Three-Stream Joint Training Network (3JOIN) for the ZS-SBIR task. To narrow the domain differences between sketches and images, we extract edge maps for natural images and treat them as a bridge between images and sketches, which have similar content to images and similar style to sketches. For exploiting a sufficient combination of sketches, natural images, and edge maps, a novel three-stream joint training network is proposed. In addition, we use a teacher network to extract the implicit semantics of the samples without the aid of other semantics and transfer the learned knowledge to unseen classes. Extensive experiments conducted on two real-world datasets demonstrate the superiority of our proposed method.

cs.CV↗

arXiv2022

An Adaptive Alternating-direction-method-based Nonnegative Latent Factor Model

An alternating-direction-method-based nonnegative latent factor model can perform efficient representation learning to a high-dimensional and incomplete (HDI) matrix. However, it introduces multiple hyper-parameters into the learning process, which should be chosen with care to enable its superior performance. Its hyper-parameter adaptation is desired for further enhancing its scalability. Targeting at this issue, this paper proposes an Adaptive Alternating-direction-method-based Nonnegative Latent Factor (A2NLF) model, whose hyper-parameter adaptation is implemented following the principle of particle swarm optimization. Empirical studies on nonnegative HDI matrices generated by industrial applications indicate that A2NLF outperforms several state-of-the-art models in terms of computational and storage efficiency, as well as maintains highly competitive estimation accuracy for an HDI matrix's missing data.

cs.LG↗