Searcharxiv⌕ Search

arXiv subjects

Dongsheng Li

Publications and source records attributed to Dongsheng Li.

At least 307 records · Page 17Linked to original sources
arXiv2018

Non-ergodic Convergence Analysis of Heavy-Ball Algorithms

In this paper, we revisit the convergence of the Heavy-ball method, and present improved convergence complexity results in the convex setting. We provide the first non-ergodic O(1/k) rate result of the Heavy-ball algorithm with constant step size for coercive objective functions. For objective functions satisfying a relaxed strongly convex condition, the linear convergence is established under weaker assumptions on the step size and inertial parameter than made in the existing literature. We extend our results to multi-block version of the algorithm with both the cyclic and stochastic update rules. In addition, our results can also be extended to decentralized optimization, where the ergodic analysis is not applicable.

math.OC↗
arXiv2018

Collaborative Filtering with Stability

Collaborative filtering (CF) is a popular technique in today's recommender systems, and matrix approximation-based CF methods have achieved great success in both rating prediction and top-N recommendation tasks. However, real-world user-item rating matrices are typically sparse, incomplete and noisy, which introduce challenges to the algorithm stability of matrix approximation, i.e., small changes in the training data may significantly change the models. As a result, existing matrix approximation solutions yield low generalization performance, exhibiting high error variance on the training data, and minimizing the training error may not guarantee error reduction on the test data. This paper investigates the algorithm stability problem of matrix approximation methods and how to achieve stable collaborative filtering via stable matrix approximation. We present a new algorithm design framework, which (1) introduces new optimization objectives to guide stable matrix approximation algorithm design, and (2) solves the optimization problem to obtain stable approximation solutions with good generalization performance. Experimental results on real-world datasets demonstrate that the proposed method can achieve better accuracy compared with state-of-the-art matrix approximation methods and ensemble methods in both rating prediction and top-N recommendation tasks.

cs.LG↗
arXiv2018

A Scalable Algorithm for Privacy-Preserving Item-based Top-N Recommendation

Recommender systems have become an indispensable component in online services during recent years. Effective recommendation is essential for improving the services of various online business applications. However, serious privacy concerns have been raised on recommender systems requiring the collection of users' private information for recommendation. At the same time, the success of e-commerce has generated massive amounts of information, making scalability a key challenge in the design of recommender systems. As such, it is desirable for recommender systems to protect users' privacy while achieving high-quality recommendations with low-complexity computations. This paper proposes a scalable privacy-preserving item-based top-N recommendation solution, which can achieve high-quality recommendations with reduced computation complexity while ensuring that users' private information is protected. Furthermore, the computation complexity of the proposed method increases slowly as the number of users increases, thus providing high scalability for privacy-preserving recommender systems. More specifically, the proposed approach consists of two key components: (1) MinHash-based similarity estimation and (2) client-side privacy-preserving prediction generation. Our theoretical and experimental analysis using real-world data demonstrates the efficiency and effectiveness of the proposed approach.

cs.CR↗
arXiv2018

Attention-Guided Answer Distillation for Machine Reading Comprehension

Despite that current reading comprehension systems have achieved significant advancements, their promising performances are often obtained at the cost of making an ensemble of numerous models. Besides, existing approaches are also vulnerable to adversarial attacks. This paper tackles these problems by leveraging knowledge distillation, which aims to transfer knowledge from an ensemble model to a single model. We first demonstrate that vanilla knowledge distillation applied to answer span prediction is effective for reading comprehension systems. We then propose two novel approaches that not only penalize the prediction on confusing answers but also guide the training with alignment information distilled from the ensemble. Experiments show that our best student model has only a slight drop of 0.4% F1 on the SQuAD test set compared to the ensemble teacher, while running 12x faster during inference. It even outperforms the teacher on adversarial SQuAD datasets and NarrativeQA benchmark.

cs.CL↗
arXiv2018

An Efficient ADMM-Based Algorithm to Nonconvex Penalized Support Vector Machines

Support vector machines (SVMs) with sparsity-inducing nonconvex penalties have received considerable attentions for the characteristics of automatic classification and variable selection. However, it is quite challenging to solve the nonconvex penalized SVMs due to their nondifferentiability, nonsmoothness and nonconvexity. In this paper, we propose an efficient ADMM-based algorithm to the nonconvex penalized SVMs. The proposed algorithm covers a large class of commonly used nonconvex regularization terms including the smooth clipped absolute deviation (SCAD) penalty, minimax concave penalty (MCP), log-sum penalty (LSP) and capped-$\ell_1$ penalty. The computational complexity analysis shows that the proposed algorithm enjoys low computational cost. Moreover, the convergence of the proposed algorithm is guaranteed. Extensive experimental evaluations on five benchmark datasets demonstrate the superior performance of the proposed algorithm to other three state-of-the-art approaches.

stat.ML↗
arXiv2018

Simulation of Heterogeneous Atom Probe Tip Shapes Evolution during Field Evaporation Using a Level Set Method and Different Evaporation Models

In atom probe tomography (APT), accurate reconstruction of the spatial positions of field evaporated ions from measured detector patterns depends upon a correct understanding of the dynamic tip shape evolution and evaporation laws of component atoms. Artifacts in APT reconstructions of heterogeneous materials can be attributed to the assumption of homogeneous evaporation of all the elements in the material in addition to the assumption of a steady state hemispherical dynamic tip shape evolution. A level set method based specimen shape evolution model is developed in this study to simulate the evaporation of synthetic layered-structured APT tips. The simulation results of the shape evolution by the level set model qualitatively agree with the finite element method and the literature data using the finite difference method. The asymmetric evolving shape predicted by the level set model demonstrates the complex evaporation behavior of heterogeneous tip and the interface curvature can potentially lead to the artifacts in the APT reconstruction of such materials. Compared with other APT simulation methods, the new method provides smoother interface representation with the aid of the intrinsic sub-grid accuracy. Two evaporation models (linear and exponential evaporation laws) are implemented in the level set simulations and the effect of evaporation laws on the tip shape evolution is also presented.

physics.comp-ph↗
arXiv2018

Modeling Selective Intergranular Oxidation of Binary Alloys

Intergranular attack of alloys under hydrothermal conditions is a complex problem that depends on metal and oxygen transport kinetics via solid-state and channel-like pathways to an advancing oxidation front. Experiments reveal very different rates of intergranular attack and minor element depletion distances ahead of the oxidation front for nickel-based binary alloys depending on the minor element. For example, a significant Cr depletion up to 9 microns ahead of grain boundary crack tips were documented for Ni-5Cr binary alloy, in contrast to relatively moderate Al depletion for Ni-5Al (~100s of nm). We present a mathematical kinetics model that adapts Wagner's model for thick film growth to intergranular attack of binary alloys. The transport coefficients of elements O, Ni, Cr, and Al in bulk alloys and along grain boundaries were estimated from the literature. For planar surface oxidation, a critical concentration of the minor element can be determined from the model where the oxide of minor element becomes dominant over the major element. This generic model for simple grain boundary oxidation can predict oxidation penetration velocities and minor element depletion distances ahead of the advancing front that are comparable to experimental data. The significant distance of depletion of Cr in Ni-5Cr in contrast to the localized Al depletion in Ni-5Al can be explained by the model due to the combination of the relatively faster diffusion of Cr along the grain boundary and slower diffusion in bulk grains, relative to Al.

physics.chem-ph↗
arXiv2018

Symmetry and Approximate Symmetry of a Nonlinear Elliptic Problem over a Ring

A singularly perturbed free boundary problem arising from a real problem associated with a Radiographic Integrated Test Stand concerns a solution of the equation $Δu = f(u)$ in a domain $Ω$ subject to constant boundary data, where the function $f$ in general is not monotone. When the domain $Ω$ is a perfect ring, we incorporate a new idea of radial correction into the classical moving plane method to prove the radial symmetry of a solution. When the domain is slightly shifted from a ring, we establish the stability of the solution by showing the approximate radial symmetry of the free boundary and the solution. For this purpose, we complete the proof via an evolutionary point of view, as an elliptic comparison principle is false, nevertheless a parabolic one holds.

math.AP↗
arXiv2018

Loss Rank Mining: A General Hard Example Mining Method for Real-time Detectors

Modern object detectors usually suffer from low accuracy issues, as foregrounds always drown in tons of backgrounds and become hard examples during training. Compared with those proposal-based ones, real-time detectors are in far more serious trouble since they renounce the use of region-proposing stage which is used to filter a majority of backgrounds for achieving real-time rates. Though foregrounds as hard examples are in urgent need of being mined from tons of backgrounds, a considerable number of state-of-the-art real-time detectors, like YOLO series, have yet to profit from existing hard example mining methods, as using these methods need detectors fit series of prerequisites. In this paper, we propose a general hard example mining method named Loss Rank Mining (LRM) to fill the gap. LRM is a general method for real-time detectors, as it utilizes the final feature map which exists in all real-time detectors to mine hard examples. By using LRM, some elements representing easy examples in final feature map are filtered and detectors are forced to concentrate on hard examples during training. Extensive experiments validate the effectiveness of our method. With our method, the improvements of YOLOv2 detector on auto-driving related dataset KITTI and more general dataset PASCAL VOC are over 5% and 2% mAP, respectively. In addition, LRM is the first hard example mining strategy which could fit YOLOv2 perfectly and make it better applied in series of real scenarios where both real-time rates and accurate detection are strongly demanded.

cs.CV↗
arXiv2018

Diagonalwise Refactorization: An Efficient Training Method for Depthwise Convolutions

Depthwise convolutions provide significant performance benefits owing to the reduction in both parameters and mult-adds. However, training depthwise convolution layers with GPUs is slow in current deep learning frameworks because their implementations cannot fully utilize the GPU capacity. To address this problem, in this paper we present an efficient method (called diagonalwise refactorization) for accelerating the training of depthwise convolution layers. Our key idea is to rearrange the weight vectors of a depthwise convolution into a large diagonal weight matrix so as to convert the depthwise convolution into one single standard convolution, which is well supported by the cuDNN library that is highly-optimized for GPU computations. We have implemented our training method in five popular deep learning frameworks. Evaluation results show that our proposed method gains $15.4\times$ training speedup on Darknet, $8.4\times$ on Caffe, $5.4\times$ on PyTorch, $3.5\times$ on MXNet, and $1.4\times$ on TensorFlow, compared to their original implementations of depthwise convolutions.

cs.CV↗
arXiv2017

Crack detection in beam structures with a novel Laplace based Wavelet Finite Element method

Beam structure is one of the most widely used structures in mechanical engineering and civil engineering. Ultrasonic guided wave based crack identification is one of the most important and accepted approaches applied to detect unseen small flaws in structures. Numerical simulations of ultrasonic guided wave propagation have caught more and more attention due to the fast development of hardware and software in the last few years. From all the numerical simulation methods, wavelet based finite element method has been proved to be one of the most efficient methods due to its better spatial resolution, which means it needs fewer elements to get the same accuracy and it can improve the calculation cost significantly. However, it needs a very small time interval. Laplace transform can easily convert the time domain into a frequency domain and then revert it back to a time domain. Laplace transform has thus the advantage of finding better results with a very large time interval. which can save a lot of time cost. This paper will present an innovative method combining Laplace transform and the B-spline wavelet on interval (BSWI) finite element method. This novel method allows to get results with the same accuracy and with a significantly lower time cost, which would not only decrease the total number of elements in the structure but also increase the time integration interval. The numerical Laplace transform and BSWI finite element will be introduced. Moreover, this innovative method is applied to simulate the ultrasonic wave propagation in a beam structure in different materials. Numerical examples for crack identification in beam structures have been studied for verification.

cs.CE↗
arXiv2017

Wave analysis in one dimensional structures with a wavelet finite element model and precise integration method

Numerical simulation of ultrasonic wave propagation provides an efficient tool for crack identification in structures, while it requires a high resolution and expensive time calculation cost in both time integration and spatial discretization. Wavelet finite element model provides a highorder finite element model and gives a higher accuracy on spatial discretization, B-Spline wavelet interval (BSWI) has been proved to be one of the most commonly used wavelet finite element model with the advantage of getting the same accuracy but with fewer element so that the calculation cost is much lower than traditional finite element method and other high-order element methods. Precise Integration Method provides a higher resolution in time integration and has been proved to be a stable time integration method with a much lower cut-off error for same and even smaller time step. In this paper, a wavelet finite element model combined with precise integration method is presented for the numerical simulation of ultrasonic wave propagation and crack identification in 1D structures. Firstly, the wavelet finite element based on BSWI is constructed for rod and beam structures. Then Precise Integrated Method is introduced with application for the wave propagation in 1D structures. Finally, numerical examples of ultrasonic wave propagation in rod and beam structures are conducted for verification. Moreover, crack identification in both rod and beam structures are studied based on the new model.

cs.CE↗
arXiv2017

Global $W^{2,δ}$ estimates for singular fully nonlinear elliptic equations with $L^n$ right hand side terms

We establish in this paper \emph{a priori} global $W^{2,δ}$ estimates for singular fully nonlinear elliptic equations with $L^n$ right hand side terms. The method is to slide paraboloids and barrier functions vertically to touch the solution of the equation, and then to estimate the measure of the contact set in terms of the measure of the vertex point set. To derive global estimates from $L^n$ data, the Hardy-Littlewood maximal functions, appropriate localizations and a new type of covering argument are adopted. These methods also provide us a more direct proof of the $W^{2,δ}$ estimates for (nonsingular) fully nonlinear elliptic equations established by L. A. Caffarelli and X. Cabré.

math.AP↗
arXiv2017

On the exterior Dirichlet problem for Hessian quotient equations

In this paper, we establish the existence and uniqueness theorem for solutions of the exterior Dirichlet problem for Hessian quotient equations with prescribed asymptotic behavior at infinity. This extends the previous related results on the Monge-Ampère equations and on the Hessian equations, and rearranges them in a systematic way. Based on the Perron's method, the main ingredient of this paper is to construct some appropriate subsolutions of the Hessian quotient equation, which is realized by introducing some new quantities about the elementary symmetric functions and using them to analyze the corresponding ordinary differential equation related to the generalized radially symmetric subsolutions of the original equation.

math.AP↗
arXiv2017

A Bernstein problem for special Lagrangian equations in exterior domains

We establish quadratic asymptotics for solutions to special Lagrangian equations with supercritical phases in exterior domains. The method is based on an exterior Liouville type result for general fully nonlinear elliptic equations toward constant asymptotics of bounded Hessian, and also certain rotation arguments toward Hessian bound. Our unified approach also leads to quadratic asymptotics for convex solutions to Monge-Ampère equations (previously known), quadratic Hessian equations, and inverse harmonic Hessian equations over exterior domains.

math.AP↗
arXiv2017

Global $W^{2,δ}$ estimates for a type of singular fully nonlinear elliptic equations

We obtain global $W^{2,δ}$ estimates for a type of singular fully nonlinear elliptic equations where the right hand side term belongs to $L^\infty$. The main idea of the proof is to slide paraboloids from below and above to touch the solution of the equation, and then to estimate the low bound of the measure of the set of contact points by the measure of the set of vertex points.

math.AP↗