arXiv subjects

Shengxin Zhu

Publications and source records attributed to Shengxin Zhu.

31 records · Page 2Linked to original sources

arXiv2024

EchoMamba4Rec: Harmonizing Bidirectional State Space Models with Spectral Filtering for Advanced Sequential Recommendation

Predicting user preferences and sequential dependencies based on historical behavior is the core goal of sequential recommendation. Although attention-based models have shown effectiveness in this field, they often struggle with inference inefficiency due to the quadratic computational complexity inherent in attention mechanisms, especially with long-range behavior sequences. Drawing inspiration from the recent advancements of state space models (SSMs) in control theory, which provide a robust framework for modeling and controlling dynamic systems, we introduce EchoMamba4Rec. Control theory emphasizes the use of SSMs for managing long-range dependencies and maintaining inferential efficiency through structured state matrices. EchoMamba4Rec leverages these control relationships in sequential recommendation and integrates bi-directional processing with frequency-domain filtering to capture complex patterns and dependencies in user interaction data more effectively. Our model benefits from the ability of state space models (SSMs) to learn and perform parallel computations, significantly enhancing computational efficiency and scalability. It features a bi-directional Mamba module that incorporates both forward and reverse Mamba components, leveraging information from both past and future interactions. Additionally, a filter layer operates in the frequency domain using learnable Fast Fourier Transform (FFT) and learnable filters, followed by an inverse FFT to refine item embeddings and reduce noise. We also integrate Gate Linear Units (GLU) to dynamically control information flow, enhancing the model's expressiveness and training stability. Experimental results demonstrate that EchoMamba significantly outperforms existing models, providing more accurate and personalized recommendations.

cs.LG↗

arXiv2023

Native spaces and generalization of Wu functions

We prove that the native space of a Wu function is a dense subspace of a Sobolev space. An explicit characterization of the native spaces of Wu functions is given. Three definitions of Wu functions are introduced and proven to be equivalent. Based on these new equivalent definitions and the so called $f$-form tricks, we can generalize the Wu functions into the even-dimensional spaces $\R^{2k}$, while the original Wu functions are only defined in the odd-dimensional spaces $\R^{2k+1}$. Such functions in even-dimensional spaces are referred to as the `missing Wu functions'. Furthermore we can generalize the Wu functions into `fractional'-dimensional spaces. We call all these Wu functions the generalized Wu functions. The closed form of the generalized Wu functions are given in terms of hypergeometric functions. Finally we prove that the Wu functions and the missing Wu functions can be written as linear combinations of the generalized Wendland functions.

math.CA↗

arXiv2023

Suboptimal subspace construction for log-determinant approximation

Variance reduction is a crucial idea for Monte Carlo simulation and the stochastic Lanczos quadrature method is a dedicated method to approximate the trace of a matrix function. Inspired by their advantages, we combine these two techniques to approximate the log-determinant of large-scale symmetric positive definite matrices. Key questions to be answered for such a method are how to construct or choose an appropriate projection subspace and derive guaranteed theoretical analysis. This paper applies some probabilistic approaches including the projection-cost-preserving sketch and matrix concentration inequalities to construct a suboptimal subspace. Furthermore, we provide some insights on choosing design parameters in the underlying algorithm by deriving corresponding approximation error and probabilistic error estimations. Numerical experiments demonstrate our method's effectiveness and illustrate the quality of the derived error bounds.

math.NA↗

arXiv2022

Learning with linear mixed model for group recommendation systems

Accurate prediction of users' responses to items is one of the main aims of many computational advising applications. Examples include recommending movies, news articles, songs, jobs, clothes, books and so forth. Accurate prediction of inactive users' responses still remains a challenging problem for many applications. In this paper, we explore the linear mixed model in recommendation system. The recommendation process is naturally modelled as the mixed process between objective effects (fixed effects) and subjective effects (random effects). The latent association between the subjective effects and the users' responses can be mined through the restricted maximum likelihood method. It turns out the linear mixed models can collaborate items' attributes and users' characteristics naturally and effectively. While this model cannot produce the most precisely individual level personalized recommendation, it is relative fast and accurate for group (users)/class (items) recommendation. Numerical examples on GroupLens benchmark problems are presented to show the effectiveness of this method.

stat.AP↗

arXiv2021

Generalized Rough Polyharmonic Splines for Multiscale PDEs with Rough Coefficients

In this paper, we demonstrate the construction of generalized Rough Polyhamronic Splines (GRPS) within the Bayesian framework, in particular, for multiscale PDEs with rough coefficients. The optimal coarse basis can be derived automatically by the randomization of the original PDEs with a proper prior distribution and the conditional expectation given partial information on edge or derivative measurements. We prove the (quasi)-optimal localization and approximation properties of the obtained bases, and justify the theoretical results with numerical experiments.

math.NA↗

arXiv2020

AIMS:Average Information Matrix Splitting

For linear mixed models with co-variance matrices which are not linearly dependent on variance component parameters, we prove that the average of the observed information and the Fisher information can be split into two parts. The essential part enjoys a simple and computational friendly formula, while the other part which involves a lot of computations is a random zero matrix and thus is negligible.

stat.ME↗

arXiv2020

Summation of Gaussian shifts as Jacobi's third Theta function

A proper choice of parameters of the Jacobi modular identity (Jacobi Imaginary transformation) implies that the summation of Gaussian shifts on infinity periodic grids can be represented as the Jacobi's third Theta function. As such, connection between summation of Gaussian shifts and the solution to a Schrödinger equation is explicitly shown. A concise and controllable upper bound of the saturation error for approximating constant functions with summation of Gaussian shifts can be immediately obtained in terms of the underlying shape parameter of the Gaussian. This shed light on how to choose a shape parameter and provides further understanding on using Gaussians with increasingly flatness.

math.NA↗

arXiv2020

Fabricated Pictures Detection with Graph Matching

Fabricating experimental pictures in research work is a serious academic misconduct, which should better be detected in the reviewing process. However, due to large number of submissions, the detection whether a picture is fabricated or reused is laborious for reviewers, and sometimes is indistinct with human eyes. A tool for detecting similarity between images may help to alleviate this problem. Some methods based on local feature points matching work for most of the time, while these methods may result in mess of matchings due to ignorance of global relationship between features. We present a framework to detect similar, or perhaps fabricated, pictures with the graph matching techniques. A new iterative method is proposed, and experiments show that such a graph matching technique is better than the methods based only on local features for some cases.

cs.CV↗

arXiv2019

Sparse inversion for derivative of log determinant

Algorithms for Gaussian process, marginal likelihood methods or restricted maximum likelihood methods often require derivatives of log determinant terms. These log determinants are usually parametric with variance parameters of the underlying statistical models. This paper demonstrates that, when the underlying matrix is sparse, how to take the advantage of sparse inversion---selected inversion which share the same sparsity as the original matrix---to accelerate evaluating the derivative of log determinant.

stat.CO↗

arXiv2018

Essential formulae for restricted maximum likelihood and its derivatives associated with the linear mixed models

The restricted maximum likelihood method enhances popularity of maximum likelihood methods for variance component analysis on large scale unbalanced data. As the high throughput biological data sets and the emerged science on uncertainty quantification, such a method receives increasing attention. Estimating the unknown variance parameters with restricted maximum likelihood method usually requires an nonlinear iterative method. Therefore proper formulae for the log-likelihood function and its derivatives play an essential role in practical algorithm design. It is our aim to provide a mathematical introduction to this method, and supply a self-contained derivation on some available formulae used in practical algorithms. Some new proof are supplied.

stat.CO↗

arXiv2016

Computing log-likelihood and its derivatives for restricted maximum likelihood methods

Recent large scale genome wide association analysis involves large scale linear mixed models. Quantifying (co)-variance parameters in the mixed models with a restricted maximum likelihood method results in a score function which is the first derivative of a log-likelihood. To obtain a statistically efficient estimate of the variance parameters, one needs to find the root of the score function via the Newton method. Most elements of the Jacobian matrix of the score involve a trace term of four parametric matrix-matrix multiplications. It is computationally prohibitively for large scale data sets. By a serial matrix transforms and an averaged information splitting technique, an approximate Jacobian matrix can be obtained by splitting the average of the Jacobian matrix and its expected value. In the approximated Jacobian, its elements only involve Four matrix vector multiplications which can be efficiently evaluated by solving sparse linear systems with the multi-frontal factorization method.

math.NA↗

arXiv2016

Information Splitting for Big Data Analytics

Many statistical models require an estimation of unknown (co)-variance parameter(s) in a model. The estimation usually obtained by maximizing a log-likelihood which involves log determinant terms. In principle, one requires the \emph{observed information}--the negative Hessian matrix or the second derivative of the log-likelihood---to obtain an accurate maximum likelihood estimator according to the Newton method. When one uses the \emph{Fisher information}, the expect value of the observed information, a simpler algorithm than the Newton method is obtained as the Fisher scoring algorithm. With the advance in high-throughput technologies in the biological sciences, recommendation systems and social networks, the sizes of data sets---and the corresponding statistical models---have suddenly increased by several orders of magnitude. Neither the observed information nor the Fisher information is easy to obtained for these big data sets. This paper introduces an information splitting technique to simplify the computation. After splitting the mean of the observed information and the Fisher information, an simpler approximate Hessian matrix for the log-likelihood can be obtained. This approximated Hessian matrix can significantly reduce computations, and makes the linear mixed model applicable for big data sets. Such a spitting and simpler formulas heavily depends on matrix algebra transforms, and applicable to large scale breeding model, genetics wide association analysis.

stat.CO↗

arXiv2016

Computing Overlaps of Two Quadrilateral Mesh Of the Same Connectivity

An exact conservative remapping scheme requires overlaps between two meshes and a reconstruction scheme on the old cells (Lagrangian mesh). While the are intensive discussion on reconstruction schemes, there are relative sparse discussion on how to calculate overlaps of two grids. Computing the exact overlaps was believed complicated and often be avoided. This paper introduces the mathematical formulation of such a problem and tools to solve the problem. We propose methods to calculate the overlaps of two dismissable general quadrilateral mesh of the same logically structure in a planar domain. The quadrilateral polygon intersection problem is reduced to as a problem that how an edge in a new mesh intersects with a local frame which consists at most 7 connected edges in the old mesh. As such, locality of the method is persevered. The alternative direction technique is applied to reduce the dimension of the searching space. It reduces more than 256 possible intersections between a new cell with the old tessellation to 34 (17 when considering symmetry) programmable intersections between an edge and an local frame whenever the intersection between the old and new cell does not degenerate. At the same time, we shall how the computational amount of the overlaps depends on the underlying problem in terms of singular intersection points. A simple and detailed classification on the type of overlaps is presented, according to classification, degeneracy of an overlap can be easily identified.

math.NA↗