arXiv subjects

Tran Thi Phuong

Publications and source records attributed to Tran Thi Phuong.

7 recordsLinked to original sources

arXiv2021

Communication-Efficient Distributed SGD with Error-Feedback, Revisited

We show that the convergence proof of a recent algorithm called dist-EF-SGD for distributed stochastic gradient descent with communication efficiency using error-feedback of Zheng et al. (NeurIPS 2019) is problematic mathematically. Concretely, the original error bound for arbitrary sequences of learning rate is unfortunately incorrect, leading to an invalidated upper bound in the convergence theorem for the algorithm. As evidences, we explicitly provide several counter-examples, for both convex and non-convex cases, to show the incorrectness of the error bound. We fix the issue by providing a new error bound and its corresponding proof, leading to a new convergence theorem for the dist-EF-SGD algorithm, and therefore recovering its mathematical analysis.

math.OC↗

arXiv2019

On the Convergence Proof of AMSGrad and a New Version

The adaptive moment estimation algorithm Adam (Kingma and Ba) is a popular optimizer in the training of deep neural networks. However, Reddi et al. have recently shown that the convergence proof of Adam is problematic and proposed a variant of Adam called AMSGrad as a fix. In this paper, we show that the convergence proof of AMSGrad is also problematic. Concretely, the problem in the convergence proof of AMSGrad is in handling the hyper-parameters, treating them as equal while they are not. This is also the neglected issue in the convergence proof of Adam. We provide an explicit counter-example of a simple convex optimization setting to show this neglected issue. Depending on manipulating the hyper-parameters, we present various fixes for this issue. We provide a new convergence proof for AMSGrad as the first fix. We also propose a new version of AMSGrad called AdamX as another fix. Our experiments on the benchmark dataset also support our theoretical results.

cs.LG↗

arXiv2019

Privacy-Preserving Deep Learning via Weight Transmission

This paper considers the scenario that multiple data owners wish to apply a machine learning method over the combined dataset of all owners to obtain the best possible learning output but do not want to share the local datasets owing to privacy concerns. We design systems for the scenario that the stochastic gradient descent (SGD) algorithm is used as the machine learning method because SGD (or its variants) is at the heart of recent deep learning techniques over neural networks. Our systems differ from existing systems in the following features: {\bf (1)} any activation function can be used, meaning that no privacy-preserving-friendly approximation is required; {\bf (2)} gradients computed by SGD are not shared but the weight parameters are shared instead; and {\bf (3)} robustness against colluding parties even in the extreme case that only one honest party exists. We prove that our systems, while privacy-preserving, achieve the same learning accuracy as SGD and hence retain the merit of deep learning with respect to accuracy. Finally, we conduct several experiments using benchmark datasets, and show that our systems outperform previous system in terms of learning accuracies.

cs.LG↗

arXiv2015

Sequentially Cohen-Macaulay Rees algebras

This paper studies the question of when the Rees algebras associated to arbitrary filtration of ideals are sequentially Cohen-Macaulay. Although this problem has been already investigated by N. T. Cuong, S. Goto and H. L. Truong, their situation is quite a bit of restricted, so we are eager to try the generalization of their results.

math.AC↗

arXiv2015

Topics on sequentially Cohen-Macaulay modules

In this paper, we study the two different topics related to sequentially Cohen-Macaulay modules. The questions are when the sequentially Cohen-Macaulay property preserve the localization and the module-finite extension of rings.

math.AC↗

arXiv2011

Almost Gorenstein rings

The notion of almost Gorenstein ring given by Barucci and Fr{ö}berg \cite{BF} in the case where the local rings are analytically unramified is generalized, so that it works well also in the case where the rings are analytically ramified. As a sequel, the problem of when the endomorphism algebra $\m : \m$ of $\m$ is a Gorenstein ring is solved in full generality, where $\m$ denotes the maximal ideal in a given Cohen-Macaulay local ring of dimension one. Characterizations of almost Gorenstein rings are given in connection with the principle of idealization. Examples are explored.

math.AC↗

arXiv2008

Quasi-socle ideals in local rings with Gorenstein tangent cones

Quasi-socle ideals, that is the ideals $I$ of the form $I= Q : \mathfrak{m}^q$ in a Noetherian local ring $(A, \mathfrak{m})$ with the Gorenstein tangent cone $\mathrm{G}(\mathfrak{m}) = \bigoplus_{n \geq 0}{\mathfrak{m}}^n/{\mathfrak{m}}^{n+1}$ are explored, where $q \geq 1$ is an integer and $Q$ is a parameter ideal of $A$ generated by monomials of a system $x_1, x_2, ..., x_d$ of elements in $A$ such that $(x_1, x_2, ..., x_d)$ is a reduction of $\mathfrak{m}$. The questions of when $I$ is integral over $Q$ and of when the graded rings $\mathrm{G}(I) = \bigoplus_{n \geq 0}I^n/I^{n+1}$ and $\mathrm{F}(I) = \bigoplus_{n \ge 0}I^n/\mathfrak{m} I^n$ are Cohen-Macaulay are answered. Criteria for $\mathrm{G} (I)$ and $\mathcal{R} (I) = \bigoplus_{n \geq 0}I^n$ to be Gorenstein rings are given.

math.AC↗