arXiv subjects

Chunfeng Cui

Publications and source records attributed to Chunfeng Cui.

At least 55 records · Page 3Linked to original sources

arXiv2024

Inertial Accelerated Stochastic Mirror Descent for Large-Scale Generalized Tensor CP Decomposition

The majority of classic tensor CP decomposition models are designed for squared loss, employing Euclidean distance as a local proximal term. However, the Euclidean distance is unsuitable for the generalized loss function applicable to various types of real-world data, such as integer and binary data. Consequently, algorithms developed under the squared loss are not easily adaptable to handle these generalized losses, partially due to the lack of the gradient Lipschitz continuity. This paper considers the generalized tensor CP decomposition. We use the Bregman distance as the proximal term and propose an inertial accelerated block randomized stochastic mirror descent algorithm (iTableSMD). Within a broader multi-block variance reduction and inertial acceleration framework, we demonstrate the sublinear convergence rate for the subsequential sequence produced by the iTableSMD algorithm. We further show that iTableSMD requires at most O(ε^{-2}) iterations in expectation to attain an ε-stationary point and establish the global convergence of the sequence. Numerical experiments on real datasets demonstrate that our proposed algorithm is efficient and achieve better performance than the existing state-of-the-art methods.

math.OC↗

arXiv2024

A Bregman Proximal Stochastic Gradient Method with Extrapolation for Nonconvex Nonsmooth Problems

In this paper, we explore a specific optimization problem that involves the combination of a differentiable nonconvex function and a nondifferentiable function. The differentiable component lacks a global Lipschitz continuous gradient, posing challenges for optimization. To address this issue and accelerate the convergence, we propose a Bregman proximal stochastic gradient method with extrapolation (BPSGE), which only requires smooth adaptivity of the differentiable part. Under the variance reduction framework, we not only analyze the subsequential and global convergence of the proposed algorithm under certain conditions, but also analyze the sublinear convergence rate of the subsequence, and the complexity of the algorithm, revealing that the BPSGE algorithm requires at most O(epsilon\^\,(-2)) iterations in expectation to attain an epsilon-stationary point. To validate the effectiveness of our proposed algorithm, we conduct numerical experiments on three real-world applications: graph regularized nonnegative matrix factorization (NMF), matrix factorization with weakly-convex regularization, and NMF with nonconvex sparsity constraints. These experiments demonstrate that BPSGE is faster than the baselines without extrapolation.

math.OC↗

arXiv2023

Quantum Computing for MIMO Beam Selection Problem: Model and Optical Experimental Solution

Massive multiple-input multiple-output (MIMO) has gained widespread popularity in recent years due to its ability to increase data rates, improve signal quality, and provide better coverage in challenging environments. In this paper, we investigate the MIMO beam selection (MBS) problem, which is proven to be NP-hard and computationally intractable. To deal with this problem, quantum computing that can provide faster and more efficient solutions to large-scale combinatorial optimization is considered. MBS is formulated in a quadratic unbounded binary optimization form and solved with Coherent Ising Machine (CIM) physical machine. We compare the performance of our solution with two classic heuristics, simulated annealing and Tabu search. The results demonstrate an average performance improvement by a factor of 261.23 and 20.6, respectively, which shows that CIM-based solution performs significantly better in terms of selecting the optimal subset of beams. This work shows great promise for practical 5G operation and promotes the application of quantum computing in solving computationally hard problems in communication.

cs.NI↗

arXiv2023

A Genuine Extension of The Moore-Penrose Inverse to Dual Matrices

The Moore-Penrose inverse is a genuine extension of the matrix inverse. Given a complex matrix, there uniquely exists another complex matrix satisfying the four Moore-Penrose conditions, and if the original matrix is nonsingular, it is exactly the inverse of that matrix. In the last one and half decade, in the study of approximate synthesis in kinematic, two generalizations of the Moore-Penrose inverse appeared for dual real matrices,including Moore-Penrose dual generalized inverse and dual Moore-Penrose generalized inverse (DMPGI). DMPGI satisfies the four Moore-Penrose conditions, but does not exist for uncountably many dual real matrices. %Another generalization, called Moore-Penrose dual generalized inverse (MPDGI), is different from DMPGI even if DMPGI exists. In this paper, based upon the singular value decomposition of dual matrices, we extend the first Moore-Penrose condition to dual matrices and introduce a genuine extension of the Moore-Penrose inverse to dual matrices, referred to as GMPI. Given a dual complex matrix, its GMPI is the unique dual complex matrix which satisfy the first extended and the other three Moore-Penrose conditions. If the original matrix is a complex matrix, its GMPI is exactly the Moore-Penrose inverse of that matrix. And if the original matrix is a dual real matrix and its DMPGI exists, its GMPI coincides with its DMPGI.

math.RA↗

arXiv2023

Vertical Ferroelectricity in Van der Waals Materials: Models and Devices

Ferroelectricity has a wide range of applications in functional electronics and is extremely important for the development of next-generation information storage technology, but it is difficult to achieve due to its special symmetry requirements. In this letter, based on van derWaals stacking, a generic model is proposed for realizing ferroelectric devices, where a freely movable center layer is packaged in two fixed and symmetrically stacked layers. In this model, the ferroelectric phase transition can be realized between the two equivalent and eccentric ground stacking-states with opposite polarizations. By means of first-principles calculations, taking the h-BN/h-BN/h-BN and h-BN/Graphene/h-BN as feasible models, we carefully evaluate the magnitude of ferroelectricity. The corresponding polarizations are estimated as 1.83 and 1.35 pC/m, respectively, which are comparable to the sliding ferroelectricity. Such a new tri-layer model of vertical ferroelectricity can be constructed by arbitrary van derWaals semiconducting materials, and usually holds low switching barrier. Optimized material combinations with remarkable polarization are highly expectable to be discovered from the huge candidate set for future information storage.

cond-mat.mtrl-sci↗

arXiv2023

Dual Number Matrices with Primitive and Irreducible Nonnegative Standard Parts

In this paper, we extend the Perron-Frobenius theory to dual number matrices with primitive and irreducible nonnegative standard parts. One motivation of our research is to consider probabilities as well as perturbation, or error bounds, or variances, in the Markov chain process. We show that such a dual number matrix always has a positive dual number eigenvalue with a positive dual number eigenvector. The standard part of this positive dual number eigenvalue is larger than or equal to the modulus of the standard part of any other eigenvalue of this dual number matrix. We present an explicit formula to compute the dual part of this positive dual number eigenvalue. The Collatz minimax theorem also holds here.The results are nontrivial as even a positive dual number matrix may have no eigenvalue at all. An algorithm based upon the Collatz minimax theorem is constructed. The convergence of the algorithm is studied. We show the upper bounds on the distance of stationary states between the dual Markov chain and the perturbed Markov chain. Numerical results on both synthetic examples and dual Markov chain including some real world examples are reported.

math.RA↗

arXiv2023

Eigenvalues and Jordan Forms of Dual Complex Matrices

Dual complex matrices have found applications in brain science. There are two different definitions of the dual complex number multiplication. One is noncommutative. Another is commutative. In this paper, we use the commutative definition. This definition is used in the research related with brain science. Under this definition, eigenvalues of dual complex matrices are defined. However, there are cases of dual complex matrices which have no eigenvalues or have infinitely many eigenvalues. We show that an $n \times n$ dual complex matrix is diagonalizable if and only if it has exactly $n$ eigenvalues with $n$ appreciably linearly independent eigenvectors. Hermitian dual complex matrices are diagonalizable. We present the Jordan form of a dual complex matrix with a diagonalizable standard part, and the Jordan form of a dual complex matrix with a Jordan block standard part. Based on these, we give a description of the eigenvalues of a general square dual complex matrix.

math.RA↗

arXiv2023

A Power Method for Computing the Dominant Eigenvalue of a Dual Quaternion Hermitian Matrix

In this paper, we first study the projections onto the set of unit dual quaternions, and the set of dual quaternion vectors with unit norms. Then we propose a power method for computing the dominant eigenvalue of a dual quaternion Hermitian matrix. For a strict dominant eigenvalue, we show the sequence generated by the power method converges to the dominant eigenvalue and its corresponding eigenvector linearly. For a general dominant eigenvalue, we show the standard part of the sequence generated by the power method converges to the standard part of the dominant eigenvalue and its corresponding eigenvector linearly. Based upon these, we reformulate the simultaneous localization and mapping (SLAM) problem as a rank-one dual quaternion completion problem. A two-block coordinate descent method is proposed to solve this problem. One block has a closed-form solution and the other block is the best rank-one approximation problem of a dual quaternion Hermitian matrix, which can be computed by the power method. Numerical experiments are presented to show the efficiency of our proposed power method.

math.OC↗

arXiv2023

Augmented Quaternion and Augmented Unit Quaternion Optimization

In this paper, we introduce and explore augmented quaternions and augmented unit quaternions, and present an augmented unit quaternion optimization model. An augmented quaternion consist of a quaternion and a translation vector. The multiplication rule of augmented quaternion is defined. An augmented unit quaternion consists of a unit quaternion and a translation vector. The augmented unit quaternions form a Lie group. By means of augmented unit quaternions, we study the error model and kinematics. Then we formulate two classical problems in robot research, i.e., the hand-eye calibration problem and the simultaneous localization and mapping (SLAM) problem as augmented unit quaternion optimization problems, which are actually real smooth spherical equality constrained optimization problems. Comparing with the corresponding unit dual quaternion optimization model, the augmented unit quaternion optimization model has less variables and removes the orthogonality constraints.

cs.RO↗

arXiv2023

Improving the generalization via coupled tensor norm regularization

In this paper, we propose a coupled tensor norm regularization that could enable the model output feature and the data input to lie in a low-dimensional manifold, which helps us to reduce overfitting. We show this regularization term is convex, differentiable, and gradient Lipschitz continuous for logistic regression, while nonconvex and nonsmooth for deep neural networks. We further analyze the convergence of the first-order method for solving this model. The numerical experiments demonstrate that our method is efficient.

math.OC↗

arXiv2022

Optimization Models and Interpretations for Three Types of Adversarial Perturbations against Support Vector Machines

Adversarial perturbations have drawn great attentions in various deep neural networks. Most of them are computed by iterations and cannot be interpreted very well. In contrast, little attentions are paid to basic machine learning models such as support vector machines. In this paper, we investigate the optimization models and the interpretations for three types of adversarial perturbations against support vector machines, including sample-adversarial perturbations (sAP), class-universal adversarial perturbations (cuAP) as well as universal adversarial perturbations (uAP). For linear binary/multi classification support vector machines (SVMs), we derive the explicit solutions for sAP, cuAP and uAP (binary case), and approximate solution for uAP of multi-classification. We also obtain the upper bound of fooling rate for uAP. Such results not only increase the interpretability of the three adversarial perturbations, but also provide great convenience in computation since iterative process can be avoided. Numerical results show that our method is fast and effective in calculating three types of adversarial perturbations.

cs.LG↗

arXiv2020

Active Subspace of Neural Networks: Structural Analysis and Universal Attacks

Active subspace is a model reduction method widely used in the uncertainty quantification community. In this paper, we propose analyzing the internal structure and vulnerability and deep neural networks using active subspace. Firstly, we employ the active subspace to measure the number of "active neurons" at each intermediate layer and reduce the number of neurons from several thousands to several dozens. This motivates us to change the network structure and to develop a new and more compact network, referred to as {ASNet}, that has significantly fewer model parameters. Secondly, we propose analyzing the vulnerability of a neural network using active subspace and finding an additive universal adversarial attack vector that can misclassify a dataset with a high probability. Our experiments on CIFAR-10 show that ASNet can achieve 23.98$\times$ parameter and 7.30$\times$ flops reduction. The universal active subspace attack vector can achieve around 20% higher attack ratio compared with the existing approach in all of our numerical experiments. The PyTorch codes for this paper are available online.

cs.LG↗

arXiv2020

Chance-Constrained and Yield-aware Optimization of Photonic ICs with Non-Gaussian Correlated Process Variations

Uncertainty quantification has become an efficient tool for uncertainty-aware prediction, but its power in yield-aware optimization has not been well explored from either theoretical or application perspectives. Yield optimization is a much more challenging task. On one side, optimizing the generally non-convex probability measure of performance metrics is difficult. On the other side, evaluating the probability measure in each optimization iteration requires massive simulation data, especially when the process variations are non-Gaussian correlated. This paper proposes a data-efficient framework for the yield-aware optimization of photonic ICs. This framework optimizes the design performance with a yield guarantee, and it consists of two modules: a modeling module that builds stochastic surrogate models for design objectives and chance constraints with a few simulation samples, and a novel yield optimization module that handles probabilistic objectives and chance constraints in an efficient deterministic way. This deterministic treatment avoids repeatedly evaluating probability measures at each iteration, thus it only requires a few simulations in the whole optimization flow. We validate the accuracy and efficiency of the whole framework by a synthetic example and two photonic ICs. Our optimization method can achieve more than $30\times$ reduction of simulation cost and better design performance on the test cases compared with a Bayesian yield optimization approach developed recently.

math.OC↗

arXiv2019

Efficient Uncertainty Modeling for System Design via Mixed Integer Programming

The post-Moore era casts a shadow of uncertainty on many aspects of computer system design. Managing that uncertainty requires new algorithmic tools to make quantitative assessments. While prior uncertainty quantification methods, such as generalized polynomial chaos (gPC), show how to work precisely under the uncertainty inherent to physical devices, these approaches focus solely on variables from a continuous domain. However, as one moves up the system stack to the architecture level many parameters are constrained to a discrete (integer) domain. This paper proposes an efficient and accurate uncertainty modeling technique, named mixed generalized polynomial chaos (M-gPC), for architectural uncertainty analysis. The M-gPC technique extends the generalized polynomial chaos (gPC) theory originally developed in the uncertainty quantification community, such that it can efficiently handle the mixed-type (i.e., both continuous and discrete) uncertainties in computer architecture design. Specifically, we employ some stochastic basis functions to capture the architecture-level impact caused by uncertain parameters in a simulator. We also develop a novel mixed-integer programming method to select a small number of uncertain parameter samples for detailed simulations. With a few highly informative simulation samples, an accurate surrogate model is constructed in place of cycle-level simulators for various architectural uncertainty analysis. In the chip-multiprocessor (CMP) model, we are able to estimate the propagated uncertainties with only 95 samples whereas Monte Carlo requires 5*10^4 samples to achieve the similar accuracy. We also demonstrate the efficiency and effectiveness of our method on a detailed DRAM subsystem.

eess.SP↗

arXiv2019

Tensor Methods for Generating Compact Uncertainty Quantification and Deep Learning Models

Tensor methods have become a promising tool to solve high-dimensional problems in the big data era. By exploiting possible low-rank tensor factorization, many high-dimensional model-based or data-driven problems can be solved to facilitate decision making or machine learning. In this paper, we summarize the recent applications of tensor computation in obtaining compact models for uncertainty quantification and deep learning. In uncertainty analysis where obtaining data samples is expensive, we show how tensor methods can significantly reduce the simulation or measurement cost. To enable the deployment of deep learning on resource-constrained hardware platforms, tensor methods can be used to significantly compress an over-parameterized neural network model or directly train a small-size model from scratch via optimization or statistical techniques. Recent Bayesian tensorized neural networks can automatically determine their tensor ranks in the training process.

math.OC↗

arXiv2019

High-Dimensional Uncertainty Quantification of Electronic and Photonic IC with Non-Gaussian Correlated Process Variations

Uncertainty quantification based on generalized polynomial chaos has been used in many applications. It has also achieved great success in variation-aware design automation. However, almost all existing techniques assume that the parameters are mutually independent or Gaussian correlated, which is rarely true in real applications. For instance, in chip manufacturing, many process variations are actually correlated. Recently, some techniques have been developed to handle non-Gaussian correlated random parameters, but they are time-consuming for high-dimensional problems. We present a new framework to solve uncertainty quantification problems with many non-Gaussian correlated uncertainties. Firstly, we propose a set of smooth basis functions to well capture the impact of non-Gaussian correlated process variations. We develop a tensor approach to compute these basis functions in a high-dimension setting. Secondly, we investigate the theoretical aspect and practical implementation of a sparse solver to compute the coefficients of all basis functions. We provide some theoretical analysis for the exact recovery condition and error bound of this sparse solver in the context of uncertainty quantification. We present three adaptive sampling approaches to improve the performance of the sparse solver. Finally, we validate our methods by synthetic and practical electronic/photonic ICs with 19 to 57 non-Gaussian correlated variation parameters. Our approach outperforms Monte Carlo by thousands of times in terms of efficiency. It can also accurately predict the output density functions with multiple peaks caused by non-Gaussian correlations, which are hard to capture by existing methods.

math.NA↗

arXiv2018

Stochastic Collocation with Non-Gaussian Correlated Process Variations: Theory, Algorithms and Applications

Stochastic spectral methods have achieved great success in the uncertainty quantification of many engineering problems, including electronic and photonic integrated circuits influenced by fabrication process variations. Existing techniques employ a generalized polynomial-chaos expansion, and they almost always assume that all random parameters are mutually independent or Gaussian correlated. However, this assumption is rarely true in real applications. How to handle non-Gaussian correlated random parameters is a long-standing and fundamental challenge. A main bottleneck is the lack of theory and computational methods to perform a projection step in a correlated uncertain parameter space. This paper presents an optimization-based approach to automatically determinate the quadrature nodes and weights required in a projection step, and develops an efficient stochastic collocation algorithm for systems with non-Gaussian correlated parameters. We also provide some theoretical proofs for the complexity and error bound of our proposed method. Numerical experiments on synthetic, electronic and photonic integrated circuit examples show the nearly exponential convergence rate and excellent efficiency of our proposed approach. Many other challenging uncertainty-related problems can be further solved based on this work.

math.NA↗

arXiv2018

Stochastic Collocation with Non-Gaussian Correlated Parameters via a New Quadrature Rule

This paper generalizes stochastic collocation methods to handle correlated non-Gaussian random parameters. The key challenge is to perform a multivariate numerical integration in a correlated parameter space when computing the coefficient of each basis function via a projection step. We propose an optimization model and a block coordinate descent solver to compute the required quadrature samples. Our method is verified with a CMOS ring oscillator and an optical ring resonator, showing 3000x speedup over Monte Carlo.

math.NA↗