arXiv subjects

Tuan Nguyen

Publications and source records attributed to Tuan Nguyen.

At least 55 records · Page 3Linked to original sources

arXiv2024

Towards Efficient Communication and Secure Federated Recommendation System via Low-rank Training

Federated Recommendation (FedRec) systems have emerged as a solution to safeguard users' data in response to growing regulatory concerns. However, one of the major challenges in these systems lies in the communication costs that arise from the need to transmit neural network models between user devices and a central server. Prior approaches to these challenges often lead to issues such as computational overheads, model specificity constraints, and compatibility issues with secure aggregation protocols. In response, we propose a novel framework, called Correlated Low-rank Structure (CoLR), which leverages the concept of adjusting lightweight trainable parameters while keeping most parameters frozen. Our approach substantially reduces communication overheads without introducing additional computational burdens. Critically, our framework remains fully compatible with secure aggregation protocols, including the robust use of Homomorphic Encryption. The approach resulted in a reduction of up to 93.75% in payload size, with only an approximate 8% decrease in recommendation performance across datasets. Code for reproducing our experiments can be found at https://github.com/NNHieu/CoLR-FedRec.

cs.LG↗

arXiv2024

High-temperature stability of ambient-cured one-part alkali-activated materials incorporating graphene for thermal energy storage

In this research, the ambient cured one part alkali activated material (AAM) containing graphene nanoplatelets (GNPs), fly ash, slag and silica fume has been investigated after high temperature exposure to 200 to 800oC. Their compressive strength, thermal properties, microstructure, pore structure were characterised through visual observation, isothermal calorimetry, TGA, XRD, SEM-EDS and X-ray CT. The research findings indicated high strength characteristics of the developed AAM (80 MPa) at ambient condition, which could further reach to approx. 100 MPa after being heated up to 400oC. GNPs provided nucleation effects for promoting geopolymerisation and crystallisation. As observed from X-ray CT, a high extent of severe cracks initiated from the core and propagated towards the surface. From SEM-EDS analysis, high Na-Al and Na-Si ratios or low Si-Al and Ca-Si ratios highly correlated to thermal stability. Overall, the research outcomes implied the promising use of the nano-engineered AAMs for thermal energy storage (TES) at 400 to 600oC.

cond-mat.mtrl-sci↗

arXiv2024

A Class-aware Optimal Transport Approach with Higher-Order Moment Matching for Unsupervised Domain Adaptation

Unsupervised domain adaptation (UDA) aims to transfer knowledge from a labeled source domain to an unlabeled target domain. In this paper, we introduce a novel approach called class-aware optimal transport (OT), which measures the OT distance between a distribution over the source class-conditional distributions and a mixture of source and target data distribution. Our class-aware OT leverages a cost function that determines the matching extent between a given data example and a source class-conditional distribution. By optimizing this cost function, we find the optimal matching between target examples and source class-conditional distributions, effectively addressing the data and label shifts that occur between the two domains. To handle the class-aware OT efficiently, we propose an amortization solution that employs deep neural networks to formulate the transportation probabilities and the cost function. Additionally, we propose minimizing class-aware Higher-order Moment Matching (HMM) to align the corresponding class regions on the source and target domains. The class-aware HMM component offers an economical computational approach for accurately evaluating the HMM distance between the two distributions. Extensive experiments on benchmark datasets demonstrate that our proposed method significantly outperforms existing state-of-the-art baselines.

cs.LG↗

arXiv2023

p-Laplacian Transformer

$p$-Laplacian regularization, rooted in graph and image signal processing, introduces a parameter $p$ to control the regularization effect on these data. Smaller values of $p$ promote sparsity and interpretability, while larger values encourage smoother solutions. In this paper, we first show that the self-attention mechanism obtains the minimal Laplacian regularization ($p=2$) and encourages the smoothness in the architecture. However, the smoothness is not suitable for the heterophilic structure of self-attention in transformers where attention weights between tokens that are in close proximity and non-close ones are assigned indistinguishably. From that insight, we then propose a novel class of transformers, namely the $p$-Laplacian Transformer (p-LaT), which leverages $p$-Laplacian regularization framework to harness the heterophilic features within self-attention layers. In particular, low $p$ values will effectively assign higher attention weights to tokens that are in close proximity to the current token being processed. We empirically demonstrate the advantages of p-LaT over the baseline transformers on a wide range of benchmark datasets.

cs.LG↗

arXiv2023

Sparse*BERT: Sparse Models Generalize To New tasks and Domains

Large Language Models have become the core architecture upon which most modern natural language processing (NLP) systems build. These models can consistently deliver impressive accuracy and robustness across tasks and domains, but their high computational overhead can make inference difficult and expensive. To make using these models less costly, recent work has explored leveraging structured and unstructured pruning, quantization, and distillation to improve inference speed and decrease size. This paper studies how models pruned using Gradual Unstructured Magnitude Pruning can transfer between domains and tasks. Our experimentation shows that models that are pruned during pretraining using general domain masked language models can transfer to novel domains and tasks without extensive hyperparameter exploration or specialized approaches. We demonstrate that our general sparse model Sparse*BERT can become SparseBioBERT simply by pretraining the compressed architecture on unstructured biomedical text. Moreover, we show that SparseBioBERT can match the quality of BioBERT with only 10\% of the parameters.

cs.CL↗

arXiv2023

Backdoor Attacks and Defenses in Federated Learning: Survey, Challenges and Future Research Directions

Federated learning (FL) is a machine learning (ML) approach that allows the use of distributed data without compromising personal privacy. However, the heterogeneous distribution of data among clients in FL can make it difficult for the orchestration server to validate the integrity of local model updates, making FL vulnerable to various threats, including backdoor attacks. Backdoor attacks involve the insertion of malicious functionality into a targeted model through poisoned updates from malicious clients. These attacks can cause the global model to misbehave on specific inputs while appearing normal in other cases. Backdoor attacks have received significant attention in the literature due to their potential to impact real-world deep learning applications. However, they have not been thoroughly studied in the context of FL. In this survey, we provide a comprehensive survey of current backdoor attack strategies and defenses in FL, including a comprehensive analysis of different approaches. We also discuss the challenges and potential future directions for attacks and defenses in the context of FL.

cs.LG↗

arXiv2023

Federated Learning for ASR based on Wav2vec 2.0

This paper presents a study on the use of federated learning to train an ASR model based on a wav2vec 2.0 model pre-trained by self supervision. Carried out on the well-known TED-LIUM 3 dataset, our experiments show that such a model can obtain, with no use of a language model, a word error rate of 10.92% on the official TED-LIUM 3 test set, without sharing any data from the different users. We also analyse the ASR performance for speakers depending to their participation to the federated learning. Since federated learning was first introduced for privacy purposes, we also measure its ability to protect speaker identity. To do that, we exploit an approach to analyze information contained in exchanged models based on a neural network footprint on an indicator dataset. This analysis is made layer-wise and shows which layers in an exchanged wav2vec 2.0 based model bring the speaker identity information.

eess.AS↗

arXiv2022

The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models

Transformer-based language models have become a key building block for natural language processing. While these models are extremely accurate, they can be too large and computationally intensive to run on standard deployments. A variety of compression methods, including distillation, quantization, structured and unstructured pruning are known to decrease model size and increase inference speed, with low accuracy loss. In this context, this paper's contributions are two-fold. We perform an in-depth study of the accuracy-compression trade-off for unstructured weight pruning of BERT models. We introduce Optimal BERT Surgeon (oBERT), an efficient and accurate weight pruning method based on approximate second-order information, which we show to yield state-of-the-art results in both stages of language tasks: pre-training and fine-tuning. Specifically, oBERT extends existing work on unstructured second-order pruning by allowing for pruning blocks of weights, and by being applicable at the BERT scale. Second, we investigate the impact of this pruning method when compounding compression approaches to obtain highly compressed but accurate models for deployment on edge devices. These models significantly push boundaries of the current state-of-the-art sparse BERT models with respect to all metrics: model size, inference speed and task accuracy. For example, relative to the dense BERT-base, we obtain 10x model size compression (in MB) with < 1% accuracy drop, 10x CPU-inference speedup with < 2% accuracy drop, and 29x CPU-inference speedup with < 7.5% accuracy drop. Our code, fully integrated with Transformers and SparseML, is available at https://github.com/neuralmagic/sparseml/tree/main/research/optimal_BERT_surgeon_oBERT.

cs.CL↗

arXiv2022

On Label Shift in Domain Adaptation via Wasserstein Distance

We study the label shift problem between the source and target domains in general domain adaptation (DA) settings. We consider transformations transporting the target to source domains, which enable us to align the source and target examples. Through those transformations, we define the label shift between two domains via optimal transport and develop theory to investigate the properties of DA under various DA settings (e.g., closed-set, partial-set, open-set, and universal settings). Inspired from the developed theory, we propose Label and Data Shift Reduction via Optimal Transport (LDROT) which can mitigate the data and label shifts simultaneously. Finally, we conduct comprehensive experiments to verify our theoretical findings and compare LDROT with state-of-the-art baselines.

cs.LG↗

arXiv2021

Riemannian manifold hybrid Monte Carlo in lattice QCD

Critical slowing down presents a critical obstacle to lattice QCD calculation at the smaller lattice spacings made possible by Exascale computers. Inspired by the concept of Fourier acceleration, we study a version of the Riemannian Manifold HMC (RMHMC) algorithm in which the canonical mass term of the HMC algorithm is replaced by a rational function of the SU(3) gauge covariant Laplacian. We have developed a suite of tools using Chebyshev filters based on the SU(3) gauge covariant Laplacian that provides the power spectra of both the gauge and fermion forces and determines the spectral dependence of the resulting RMHMC evolution of long- and short-distance QCD observables. These tools can be used to optimize the RMHMC mass term and to monitor the resulting acceleration mode-wise.

hep-lat↗

arXiv2021

Coulomb corrections to pi-pi scattering

The relationship between finite volume multi-hadron energy levels and matrix elements and two particle scattering phase shifts and decays is well known, but the inclusion of long range interactions such as QED is non-trivial. Inclusion of QED is an important systematic error correction to $K\toππ$ decays. In this talk, we present a method of including a truncated, finite-range Coulomb interaction in a finite-volume lattice QCD calculation. We show how the omission caused by the truncation can be restored by an infinite-volume analytic calculation so that the final result contains no power-law finite-volume errors beyond those usually present in Luscher's finite-volume phase shift determination. This approach allows us to calculate the QED corrected infinite-volume phase shift for $ππ$ scattering in Coulomb gauge, a necessary ingredient to $K\toππ$, while neglecting the transverse radiation for now.

hep-lat↗

arXiv2021

$π-π$ scattering, QED and finite-volume quantization

Using the Coulomb gauge formulation of QED we present a lattice QCD procedure to calculate the $π^+π^+$ scattering phase shift including the effects of the Coulomb potential which appears in this formulation. The approach described here incorporates the effects of relativity and avoids finite-volume corrections that vanish as a power of the volume in which the lattice calculation is performed. This is the first step in developing a complete lattice QCD calculation of the electromagnetic and isospin-breaking light-quark mass contributions to $\varepsilon'$, the parameter describing direct CP violating effects in $K_L\toππ$ decay.

hep-lat↗

arXiv2021

SP-GPT2: Semantics Improvement in Vietnamese Poetry Generation

Automatic text generation has garnered growing attention in recent years as an essential step towards computer creativity. Generative Pretraining Transformer 2 (GPT2) is one of the state of the art approaches that have excellent successes. In this paper, we took the first step to investigate the power of GPT2 in traditional Vietnamese poetry generation. In the earlier time, our experiment with base GPT2 was quite good at generating the poem in the proper template. Though it can learn the patterns, including rhyme and tone rules, from the training data, like almost all other text generation approaches, the poems generated still has a topic drift and semantic inconsistency. To improve the cohesion within the poems, we proposed a new model SP-GPT2 (semantic poem GPT2) which was built on the top GPT2 model and an additional loss to constrain context throughout the entire poem. For better evaluation, we examined the methods by both automatic quantitative evaluation and human evaluation. Both automatic and human evaluation demonstrated that our approach can generate poems that have better cohesion without losing the quality due to additional loss. At the same time, we are the pioneers of this topic. We released the first computational scoring module for poems generated in the template containing the style rule dictionary. Additionally, we are the first to publish a Luc-Bat dataset, including 87609 Luc Bat poems, which is equivalent to about 2.6 million sentences, combined with about 83579 poems in other styles was also published for further exploration. The code is available at https://github.com/fsoft-ailab/Poem-Generator

cs.CL↗

arXiv2019

Small Target Detection for Search and Rescue Operations using Distributed Deep Learning and Synthetic Data Generation

It is important to find the target as soon as possible for search and rescue operations. Surveillance camera systems and unmanned aerial vehicles (UAVs) are used to support search and rescue. Automatic object detection is important because a person cannot monitor multiple surveillance screens simultaneously for 24 hours. Also, the object is often too small to be recognized by the human eye on the surveillance screen. This study used UAVs around the Port of Houston and fixed surveillance cameras to build an automatic target detection system that supports the US Coast Guard (USCG) to help find targets (e.g., person overboard). We combined image segmentation, enhancement, and convolution neural networks to reduce detection time to detect small targets. We compared the performance between the auto-detection system and the human eye. Our system detected the target within 8 seconds, but the human eye detected the target within 25 seconds. Our systems also used synthetic data generation and data augmentation techniques to improve target detection accuracy. This solution may help the search and rescue operations of the first responders in a timely manner.

cs.CV↗

arXiv2018

The Impacts of Three Flamelet Burning Regimes in Nonlinear Combustion Dynamics

Axisymmetric simulations of a liquid rocket engine are performed using a delayed detached-eddy-simulation (DDES) turbulence model with the Compressible Flamelet Progress Variable (CFPV) combustion model. Three different pressure instability domains are simulated: completely unstable, semi-stable, and fully stable. The different instability domains are found by varying the combustion chamber and oxidizer post length. Laminar flamelet solutions with a detailed chemical mechanism are examined. The $β$ Probability Density Function (PDF) for the mixture fraction and Dirac $δ$ PDF for both the pressure and the progress variable are used. A coupling mechanism between the Heat Release Rate (HRR) and the pressure in an unstable cycle is demonstrated. Local extinction and reignition is investigated for all the instability domains using the full S-curve approach. A monotonic decrease in the amount of local extinctions and reignitions occurs when pressure oscillation amplitude becomes smaller. The flame index is used to distinguish between the premixed and non-premixed burning mode in different stability domains. An additional simulation of the unstable pressure oscillation case using only the stable flamelet burning branch of the S-curve is performed. Better agreement with experiments in terms of pressure oscillation amplitude is found when the full S-curve is used.

physics.flu-dyn↗

arXiv2014

Radio Resource Allocation for Scalable Video Services over Wireless Cellular Networks

Good quality video services always require higher bandwidth. Hence, to provide the video services e.g., multicast/broadcast services (MBS) and unicast services along with the existing voice, internet, and other background traffic services over the wireless cellular networks, it is required to efficiently manage the wireless resources in order to reduce the overall forced call termination probability, to maximize the overall service quality, and to maximize the revenue. Fixed bandwidth allocation for the MBS sessions either reduces the quality of the MBS videos and bandwidth utilization or increases the overall forced call termination probability and of course the handover call dropping probability as well. Scalable Video Coding (SVC) technique allows the variable bit rate allocation for the video services. In this paper, we propose a bandwidth allocation scheme that efficiently allocates bandwidth among the MBS sessions and the non-MBS traffic calls (e.g., voice, unicast, internet, and other background traffic). The proposed scheme reduces the bandwidth allocation for the MBS sessions during the congested traffic condition only to accommodate more calls in the system. Instead of allocating fixed bandwidths for the BMS sessions and the non-MBS traffic, our scheme allocates variable bandwidths for them. However, the minimum quality of the videos is guaranteed by allocating minimum bandwidth for them. Using the mathematical and numerical analyses, we show that the proposed scheme maximizes the bandwidth utilization and significantly reduces the overall forced call termination probability as well as the handover call dropping probability.

cs.MM↗

arXiv2014

Optimal Regularization Parameters for General-Form Tikhonov Regularization

In this work we consider the problem of finding optimal regularization parameters for general-form Tikhonov regularization using training data. We formulate the general-form Tikhonov solution as a spectral filtered solution using the generalized singular value decomposition of the matrix of the forward model and a given regularization matrix. Then, we find the optimal regularization parameter by minimizing the average of the errors between the filtered solutions and the true data. We extend the approach to the multi-parameter Tikhonov problem for the case where all the matrices involved are simultaneously diagonalizable. For problems where this is not the case, we describe an approach to compute optimal or near-optimal regularization parameters by using operator approximations for the original problem. Several tests are performed for 1D and 2D examples using different norms on the errors, showing the effectiveness of this approach.

math.NA↗

arXiv2012

Keeler's theorem and products of distinct transpositions

An episode of Futurama features a two-body mind-switching machine which will not work more than once on the same pair of bodies. After the Futurama community engages in a mind-switching spree, the question is asked, "Can the switching be undone so as to restore all minds to their original bodies?" Ken Keeler found an algorithm that undoes any mind-scrambling permutation with the aid of two "outsiders." We refine Keeler's result by providing a more efficient algorithm that uses the smallest possible number of switches. We also present best possible algorithms for undoing two natural sequences of switches, each sequence effecting a cyclic mind-scrambling permutation in the symmetric group S_n. Finally, we give necessary and sufficient conditions on m and n for the identity permutation to be expressible as a product of m distinct transpositions in S_n.

math.GR↗