Searcharxiv⌕ Search

arXiv subjects

Hufei Zhu

Publications and source records attributed to Hufei Zhu.

16 recordsLinked to original sources
arXiv2024

Two Ridge Solutions for the Incremental Broad Learning System on Added Nodes

The original Broad Learning System (BLS) on new added nodes and its existing efficient implementation both assume the ridge parameter lambda -> 0 in the ridge inverse to approximate the generalized inverse, and compute the generalized inverse solution for the output weights. In this paper, we propose two ridge solutions for the output weights in the BLS on added nodes, where lambda -> 0 is no longer assumed, and lambda can be any positive real number. One of the proposed ridge solutions computes the output weights from the inverse Cholesky factor, which is updated efficiently by extending the existing inverse Cholesky factorization. The other proposed ridge solution computes the output weights from the ridge inverse, and updates the ridge inverse by extending the Greville's method that is a classical tool to compute the generalized inverse of partitioned matrices. For the proposed efficient ridge solution based on the inverse Cholesky factor, we also develop another implementation that is numerically more stable when the ridge parameter lambda is very small. The proposed ridge solution based on the ridge inverse and the numerically more stable implementation of the proposed efficient ridge solution require the same complexity as the original BLS and the existing efficient BLS, respectively. Moreover, the speedups of the proposed efficient ridge solution to the original BLS and the existing efficient BLS are 3 and more than 1.67 respectively, when the computational complexities for each update are compared, and the speedups are 1.99 - 2.52 and 1.31 - 1.58, respectively, when the total training time is compared by numerical experiments. On the other hand, our numerical experiments show that both the proposed ridge solutions for BLS achieve better testing accuracies than the original BLS and the existing efficient BLS.

cs.LG↗
arXiv2023

Rethinking Soft Interference Cancellation (IC) for MIMO: A Hard-Decision IC Inspired Recursive Scheme

Multiple-input multiple-output (MIMO) technology has been regarded as one of the most important technologies to enable emerging applications in current and next generation wireless communication systems, for which signal detection methods have been endowed with higher requirements, such as finer bit-error ratio (BER) performance, lower complexity, and smaller memory. Existing detectors mainly include hard-decision-based ordered successive interference cancellation (HD-OSIC) schemes with relatively simple implementation, and linear-minimum-mean-squareerror-based iterative soft interference cancellation (LMMSE-ISIC) schemes exhibiting near-optimal BER performance, whose advantages are combined by the detector developed in this paper. Specifically, we first elaborate that the LMMSE-ISIC scheme is the extension of the HD-OSIC counterpart, via comparing our proposed reordered description based on the equivalent channel matrix for the LMMSE-ISIC detection process with the other. Then, we propose a recursive scheme with speed advantage and memory saving for LMMSE-ISIC by extending that for HDOSIC, where the LMMSE-ISIC estimate and the filtering bias are updated highly efficiently. Compared to the existing best low-complexity LMMSE-ISIC scheme, theoretically, the required computations and memory units in each iteration of our proposed scheme decrease by at least 87.50% and 80.00%, respectively, and simulation results demonstrate that our proposed scheme always yields identical BER performance.

eess.SP↗
arXiv2023

On Efficient V-BLAST Detection: Recursive and Hybrid Schemes

This paper investigates low-complexity and memoryefficient detection for vertical Bell Laboratories layered spacetime architecture (V-BLAST), where recursive and square-root implementations exhibit different tradeoffs in computational complexity. Two new improvements are first developed for the recursive implementation. The first adopts a more efficient partitioned-matrix inverse relation, reducing the dominant complexity of the inverse-construction step by a factor of 1.67. The second reformulates symbol estimation and interference cancellation using quantities available from the detection-error covariance matrix, thereby avoiding the need to retain the inverse covariance matrix during recursion. The resulting recursive algorithm reduces the dominant complexity of the existing speedoriented implementation by a factor of 1.3, while achieving a speedup of approximately 1.86 and reducing the principal matrix-storage requirement by about one half relative to the existing memory-saving implementation. Since the proposed recursive algorithm offers lower worst-case complexity, whereas the inverse-Cholesky square-root implementation provides lower best-case and average complexities, a hybrid scheme is further developed to switch adaptively between them. The hybrid scheme preserves the worst-case dominant-complexity bound of the recursive implementation, attains the best-case complexity of the square-root implementation, and slightly reduces the average processing cost. Analytical operation counts and numerical results verify these gains and demonstrate robustness to moderate variations in a switching threshold.

eess.SP↗
arXiv2023

Two Efficient Ridge Solutions for the Incremental Broad Learning System on Added Inputs

This paper proposes the recursive and square-root BLS algorithms to improve the original BLS for new added inputs, which utilize the inverse and inverse Cholesky factor of the Hermitian matrix in the ridge inverse, respectively, to update the ridge solution. The recursive BLS updates the inverse by the matrix inversion lemma, while the square-root BLS updates the upper-triangular inverse Cholesky factor by multiplying it with an upper-triangular intermediate matrix. When the added p training samples are more than the total k nodes in the network, i.e., p>k, the inverse of a sum of matrices is applied to take a smaller matrix inversion or inverse Cholesky factorization. For the distributed BLS with data-parallelism, we introduce the parallel implementation of the square-root BLS, which is deduced from the parallel implementation of the inverse Cholesky factorization. The original BLS based on the generalized inverse with the ridge regression assumes the ridge parameter lamda->0 in the ridge inverse. When lambda->0 is not satisfied, the numerical experiments on the MNIST and NORB datasets show that both the proposed ridge solutions improve the testing accuracy of the original BLS, and the improvement becomes more significant as lambda is bigger. On the other hand, compared to the original BLS, both the proposed BLS algorithms theoretically require less complexities, and are significantly faster in the simulations on the MNIST dataset. The speedups in total training time of the recursive and square-root BLS algorithms over the original BLS are 4.41 and 6.92 respectively when p > k, and are 2.80 and 1.59 respectively when p < k.

cs.LG↗
arXiv2022

Reducing the Computational Complexity of Pseudoinverse for the Incremental Broad Learning System on Added Inputs

In this brief, we improve the Broad Learning System (BLS) [7] by reducing the computational complexity of the incremental learning for added inputs. We utilize the inverse of a sum of matrices in [8] to improve a step in the pseudoinverse of a row-partitioned matrix. Accordingly we propose two fast algorithms for the cases of q > k and q < k, respectively, where q and k denote the number of additional training samples and the total number of nodes, respectively. Specifically, when q > k, the proposed algorithm computes only a k * k matrix inverse, instead of a q * q matrix inverse in the existing algorithm. Accordingly it can reduce the complexity dramatically. Our simulations, which follow those for Table V in [7], show that the proposed algorithm and the existing algorithm achieve the same testing accuracy, while the speedups in BLS training time of the proposed algorithm over the existing algorithm are 1.24 - 1.30.

cs.LG↗
arXiv2022

Efficient and Stable Algorithms to Extend Greville's Method to Partitioned Matrices Based on Inverse Cholesky Factorization

Greville's method has been utilized in (Broad Learn-ing System) BLS to propose an effective and efficient incremental learning system without retraining the whole network from the beginning. For a column-partitioned matrix where the second part consists of p columns, Greville's method requires p iterations to compute the pseudoinverse of the whole matrix from the pseudoinverse of the first part. The incremental algorithms in BLS extend Greville's method to compute the pseudoinverse of the whole matrix from the pseudoinverse of the first part by just 1 iteration, which have neglected some possible cases, and need further improvements in efficiency and numerical stability. In this paper, we propose an efficient and numerical stable algorithm from Greville's method, to compute the pseudoinverse of the whole matrix from the pseudoinverse of the first part by just 1 iteration, where all possible cases are considered, and the recently proposed inverse Cholesky factorization can be applied to further reduce the computational complexity. Finally, we give the whole algorithm for column-partitioned matrices in BLS. On the other hand, we also give the proposed algorithm for row-partitioned matrices in BLS.

math.NA↗
arXiv2021

Low-Memory Implementations of Ridge Solutions for Broad Learning System with Incremental Learning

The existing low-memory BLS implementation proposed recently avoids the need for storing and inverting large matrices, to achieve efficient usage of memories. However, the existing low-memory BLS implementation sacrifices the testing accuracy as a price for efficient usage of memories, since it can no longer obtain the generalized inverse or ridge solution for the output weights during incremental learning, and it cannot work under the very small ridge parameter that is utilized in the original BLS. Accordingly, it is required to develop the low-memory BLS implementations, which can work under very small ridge parameters and compute the generalized inverse or ridge solution for the output weights in the process of incremental learning. In this paper, firstly we propose the low-memory implementations for the recently proposed recursive and square-root BLS algorithms on added inputs and the recently proposed squareroot BLS algorithm on added nodes, by simply processing a batch of inputs or nodes in each recursion. Since the recursive BLS implementation includes the recursive updates of the inverse matrix that may introduce numerical instabilities after a large number of iterations, and needs the extra computational load to decompose the inverse matrix into the Cholesky factor when cooperating with the proposed low-memory implementation of the square-root BLS algorithm on added nodes, we only improve the low-memory implementations of the square-root BLS algorithms on added inputs and nodes, to propose the full lowmemory implementation of the square-root BLS algorithm. All the proposed low-memory BLS implementations compute the ridge solution for the output weights in the process of incremental learning, and most of them can work under very small ridge parameters.

cs.LG↗
arXiv2020

Complexity Comparison between Two Optimal-Ordered SIC MIMO Detectors Based on Matlab Simulations

This paper firstly introduces our shared Matlab source code that simulates the two optimal-ordered SIC detectors proposed in [1] and [2]. Based on our shared Matlab code, we compare the computational complexities between the two detectors in [1] and [2] by theoretical complexity calculations and numerical experiments. We carry out theoretical complexity calculations to obtain the worst-case complexities for the two detectors in [1] and [2]. Then from the theoretical worst-case complexities, we make the conclusion that the detector proposed in [2] requires 9N 2 more floating-point operations (flops) than the detector proposed in [1], where N is the number of transmit antennas. Our numerical experiments also show that the detector in [2] requires more worst-case and average complexities than the detector in [1].

eess.SP↗
arXiv2020

Efficient Inverse-Free Incremental and Decremental Algorithms for Multiple Hidden Nodes in Extreme Learning Machine

The inverse-free extreme learning machine (ELM) algorithm proposed in [4] was based on an inverse-free algorithm to compute the regularized pseudo-inverse, which was deduced from an inverse-free recursive algorithm to update the inverse of a Hermitian matrix. Before that recursive algorithm was applied in [4], its improved version had been utilized in previous literatures [9], [10]. Accordingly from the improved recursive algorithm [9], [10], several efficient inverse-free algorithms for ELM were proposed in [13] to reduce the computational complexity. In this paper, we propose two inverse-free algorithms for ELM with Tikhonov regularization, which can increase multiple hidden nodes in an iteration. On the other hand, we also propose two efficient decremental learning algorithms for ELM with Tikhonov regularization, which can remove multiple redundant nodes in an iteration.

cs.LG↗
arXiv2020

Comments on "Low-Complexity SIC Detection Algorithms for Multiple-Input Multiple-Output Systems"

In the above paper, the optimal-ordered successive interference cancellation (SIC) detector proposed for multiple input multiple output (MIMO) systems was claimed to require a lower computational complexity than the optimal-ordered SIC detector proposed in the paper "An Improved Square-Root Algorithm for V-BLAST Based on Efficient Inverse Cholesky Factorization" (IEEE Trans. Wireless Commun., vol. 10, no. 1, Jan. 2011), since several incorrect complexities were quoted or claimed. In this comment, we revise the incorrect complexities, to draw the conclusion that the above-mentioned two detectors actually require the same dominant complexity.

eess.SP↗
arXiv2020

An Improved Square-root Algorithm for V-BLAST Based on Efficient Inverse Cholesky Factorization

A fast algorithm for inverse Cholesky factorization is proposed, to compute a triangular square-root of the estimation error covariance matrix for Vertical Bell Laboratories Layered Space-Time architecture (V-BLAST). It is then applied to propose an improved square-root algorithm for V-BLAST, which speedups several steps in the previous one, and can offer further computational savings in MIMO Orthogonal Frequency Division Multiplexing (OFDM) systems. Compared to the conventional inverse Cholesky factorization, the proposed one avoids the back substitution (of the Cholesky factor), and then requires only half divisions. The proposed V-BLAST algorithm is faster than the existing efficient V-BLAST algorithms. The expected speedups of the proposed square-root V-BLAST algorithm over the previous one and the fastest known recursive V-BLAST algorithm are 3.9~5.2 and 1.05~1.4, respectively.

eess.SP↗
arXiv2020

A Fast Recursive Algorithm for G-STBC

This paper proposes a fast recursive algorithm for Group-wise Space-Time Block Code (G-STBC), which takes full advantage of the Alamouti structure in the equivalent channel matrix to reduce the computational complexity. With respect to the existing efficient algorithms for G-STBC, the proposed algorithm achieves better performance and usually requires less computational complexity.

eess.SP↗
arXiv2019

Efficient Decremental Learning Algorithms for Broad Learning System

The decremented learning algorithms are required in machine learning, to prune redundant nodes and remove obsolete inline training samples. In this paper, an efficient decremented learning algorithm to prune redundant nodes is deduced from the incremental learning algorithm 1 proposed in [9] for added nodes, and two decremented learning algorithms to remove training samples are deduced from the two incremental learning algorithms proposed in [10] for added inputs. The proposed decremented learning algorithm for reduced nodes utilizes the inverse Cholesterol factor of the Herminia matrix in the ridge inverse, to update the output weights recursively, as the incremental learning algorithm 1 for added nodes in [9], while that inverse Cholesterol factor is updated with an unitary transformation. The proposed decremented learning algorithm 1 for reduced inputs updates the output weights recursively with the inverse of the Herminia matrix in the ridge inverse, and updates that inverse recursively, as the incremental learning algorithm 1 for added inputs in [10].

cs.LG↗
arXiv2019

Inverse LDM' and LU Factorizations of a Partitioned Matrix with the Square-root and Division Free Version for V-BLAST

This letter proposes the inverse LDM' and LU factorizations of a matrix partitioned into 2x2 blocks, which include the square-root and division free version. The proposed squareroot and division free inverse LDM' factorization is applied to compute the initial estimation error covariance matrix Q for the recursive V-BLAST algorithm, which can save K-1 divisions (where K is the number of transmit antennas), and requires about the same computational complexity as the corresponding algorithm to compute Q in the existing recursive V-BLAST algorithm [8], [9]. The proposed square-root and division free inverse LDM' factorization can also be applied to propose the square-root and division free implementation for the squareroot V-BLAST algorithm in [5], where the wide-sense Givens rotation in [14] is utilized. With respect to the existing squareroot V-BLAST algorithms [5], [6], the proposed square-root and division free V-BLAST algorithm requires about the same computational complexity, and can avoid the square-root and division operations.

eess.SP↗
arXiv2019

Efficient Inverse-Free Algorithms for Extreme Learning Machine Based on the Recursive Matrix Inverse and the Inverse LDL' Factorization

The inverse-free extreme learning machine (ELM) algorithm proposed in [4] was based on an inverse-free algorithm to compute the regularized pseudo-inverse, which was deduced from an inverse-free recursive algorithm to update the inverse of a Hermitian matrix. Before that recursive algorithm was applied in [4], its improved version had been utilized in previous literatures [9], [10]. Accordingly from the improved recursive algorithm [9], [10], we deduce a more efficient inverse-free algorithm to update the regularized pseudo-inverse, from which we develop the proposed inverse-free ELM algorithm 1. Moreover, the proposed ELM algorithm 2 further reduces the computational complexity, which computes the output weights directly from the updated inverse, and avoids computing the regularized pseudoinverse. Lastly, instead of updating the inverse, the proposed ELM algorithm 3 updates the LDLT factor of the inverse by the inverse LDLT factorization [11], to avoid numerical instabilities after a very large number of iterations [12]. With respect to the existing ELM algorithm, the proposed ELM algorithms 1, 2 and 3 are expected to require only (8+3)/M , (8+1)/M and (8+1)/M of complexities, respectively, where M is the output node number. In the numerical experiments, the standard ELM, the existing inverse-free ELM algorithm and the proposed ELM algorithms 1, 2 and 3 achieve the same performance in regression and classification, while all the 3 proposed algorithms significantly accelerate the existing inverse-free ELM algorithm

cs.LG↗