arXiv subjects

Son Nguyen

Publications and source records attributed to Son Nguyen.

At least 37 records · Page 2Linked to original sources

arXiv2025

VDInstruct: Zero-Shot Key Information Extraction via Content-Aware Vision Tokenization

Key Information Extraction (KIE) underpins the understanding of visual documents (e.g., receipts and contracts) by extracting precise semantic content and accurately capturing spatial structure. Yet existing multimodal large language models (MLLMs) often perform poorly on dense documents and rely on vision tokenization approaches that scale with image size, leading to redundant computation and memory inefficiency. To address these challenges, we introduce VDInstruct, an MLLM that separates spatial region detection from semantic feature extraction. Central to our model is a content-aware tokenization strategy: rather than fragmenting the entire image uniformly, it generates tokens in proportion to document complexity, preserving critical structure while eliminating wasted tokens. Leveraging a three-stage training paradigm, our model achieves state-of-the-art (SOTA) results on KIE benchmarks, matching or exceeding the accuracy of leading approaches while reducing the number of image tokens by roughly 3.6x. In zero-shot evaluations, VDInstruct surpasses strong baselines-such as DocOwl 1.5-by +5.5 F1 points, highlighting its robustness to unseen documents. These findings show that content-aware tokenization combined with explicit layout modeling offers a promising direction forward for document understanding. Data, source code, and model weights will be made publicly available.

cs.CV↗

arXiv2025

Shuffle Tableaux, Littlewood--Richardson Coefficients, and Schur Log-Concavity

We give a new formula for the Littlewood--Richardson coefficients in terms of peelable tableaux compatible with shuffle tableaux, in the same fashion as Remmel--Whitney rule. This gives an efficient way to compute generalized Littlewood--Richardson coefficients for Temperley--Lieb immanants of Jacobi--Trudi matrices. We will also show that our rule behaves well with Bender--Knuth involutions, recovering the symmetry of Littlewood--Richardson coefficients. As an application, we use our rule to prove a special case of a Schur log-concavity conjecture by Lam--Postnikov--Pylyavskyy.

math.CO↗

arXiv2025

Improving Adaptive Moment Optimization via Preconditioner Diagonalization

Modern adaptive optimization methods, such as Adam and its variants, have emerged as the most widely used tools in deep learning over recent years. These algorithms offer automatic mechanisms for dynamically adjusting the update step based on estimates of gradient statistics. Compared to traditional algorithms like Stochastic Gradient Descent, these adaptive methods are typically more robust to model scale and hyperparameter tuning. However, the gradient statistics employed by these methods often do not leverage sufficient gradient covariance information, leading to suboptimal updates in certain directions of the parameter space and potentially slower convergence. In this work, we keep track of such covariance statistics in the form of a structured preconditioner matrix. Unlike other works, our approach does not apply direct approximations to estimate this matrix. We instead implement an invertible transformation that maps the preconditioner matrix into a new space where it becomes approximately diagonal. This enables a diagonal approximation of the preconditioner matrix in the transformed space, offering several computational advantages. Empirical results show that our approach can substantially enhance the convergence speed of modern adaptive optimizers. Notably, for large language models like LLaMA, we can achieve a speedup of 2x compared to the baseline Adam. Additionally, our method can be integrated with memory-efficient optimizers like Adafactor to manage computational overhead.

cs.LG↗

arXiv2024

Fairness-enhancing mixed effects deep learning improves fairness on in- and out-of-distribution clustered (non-iid) data

Traditional deep learning (DL) models have two ubiquitous limitations. First, they assume training samples are independent and identically distributed (i.i.d), an assumption often violated in real-world datasets where samples have additional correlation due to repeat measurements (e.g., on the same participants in a longitudinal study or cells from the same sequencer). This leads to performance degradation, limited generalization, and covariate confounding, which induces Type I and Type II errors. Second, DL models typically prioritize overall accuracy, favoring accuracy on the majority while sacrificing performance for underrepresented subpopulations, leading to unfair, biased models. This is critical to remediate, particularly in models which influence decisions regarding loan approvals and healthcare. To address these issues, we propose the Fair Mixed Effects Deep Learning (Fair MEDL) framework. This framework quantifies cluster-invariant fixed effects (FE) and cluster-specific random effects (RE) through: 1) a cluster adversary for learning invariant FE, 2) a Bayesian neural network for RE, and 3) a mixing function combining FE and RE for final predictions. Fairness is enhanced through architectural and loss function changes introduced by an adversarial debiasing network. We formally define and demonstrate improved fairness across three metrics: equalized odds, demographic parity, and counterfactual fairness, for both classification and regression tasks. Our method also identifies and de-weights confounded covariates, mitigating Type I and II errors. The framework is comprehensively evaluated across three datasets spanning two industries, including finance and healthcare. The Fair MEDL framework improves fairness by 86.4% for Age, 64.9% for Race, 57.8% for Sex, and 36.2% for Marital status, while maintaining robust predictive performance.

cs.LG↗

arXiv2024

Generating Critical Scenarios for Testing Automated Driving Systems

Autonomous vehicles (AVs) have demonstrated significant potential in revolutionizing transportation, yet ensuring their safety and reliability remains a critical challenge, especially when exposed to dynamic and unpredictable environments. Real-world testing of an Autonomous Driving System (ADS) is both expensive and risky, making simulation-based testing a preferred approach. In this paper, we propose AVASTRA, a Reinforcement Learning (RL)-based approach to generate realistic critical scenarios for testing ADSs in simulation environments. To capture the complexity of driving scenarios, AVASTRA comprehensively represents the environment by both the internal states of an ADS under-test (e.g., the status of the ADS's core components, speed, or acceleration) and the external states of the surrounding factors in the simulation environment (e.g., weather, traffic flow, or road condition). AVASTRA trains the RL agent to effectively configure the simulation environment that places the AV in dangerous situations and potentially leads it to collisions. We introduce a diverse set of actions that allows the RL agent to systematically configure both environmental conditions and traffic participants. Additionally, based on established safety requirements, we enforce heuristic constraints to ensure the realism and relevance of the generated test scenarios. AVASTRA is evaluated on two popular simulation maps with four different road configurations. Our results show AVASTRA's ability to outperform the state-of-the-art approach by generating 30% to 115% more collision scenarios. Compared to the baseline based on Random Search, AVASTRA achieves up to 275% better performance. These results highlight the effectiveness of AVASTRA in enhancing the safety testing of AVs through realistic comprehensive critical scenario generation.

cs.RO↗

arXiv2024

Reinforcement Learning-Based REST API Testing with Multi-Coverage

REST (Representational State Transfer) APIs have become integral for data communication and exchange due to their simplicity, scalability, and compatibility with web standards. However, ensuring REST APIs' reliability through rigorous testing poses significant challenges, given the complexities of operations, parameters, inputs, dependencies, and call sequences. In this paper, we introduce MUCOREST, a novel Reinforcement Learning (RL)-based API testing approach that leverages Q-learning to maximize code coverage and output coverage, thereby improving bug discovery. By focusing on these proximate objectives rather than the abstract goal of maximizing failures, MUCOREST effectively discovers critical code areas and diverse API behaviors. The experimental results on a benchmark of 10 services show that MUCOREST significantly outperforms state-of-the-art API testing approaches by 11.6-261.1% in the number of discovered API bugs. MUCOREST can generate much fewer API calls to discover the same number of bugs compared to the other approaches. Furthermore, 12.17%-64.09% of the bugs discovered by the other techniques can also be found by MUCOREST.

cs.SE↗

arXiv2024

Memory-Efficient Optimization with Factorized Hamiltonian Descent

Modern deep learning heavily depends on adaptive optimizers such as Adam and its variants, which are renowned for their capacity to handle model scaling and streamline hyperparameter tuning. However, these algorithms typically experience high memory overhead caused by the accumulation of optimization states, leading to a critical challenge in training large-scale network models. In this study, we introduce a novel adaptive optimizer, H-Fac, which incorporates a memory-efficient factorization approach to address this challenge. By employing a rank-1 parameterization for both momentum and scaling parameter estimators, H-Fac reduces memory costs to a sublinear level while maintaining competitive performance across a wide range of architectures. We develop our algorithms based on principles derived from Hamiltonian dynamics, providing robust theoretical underpinnings in optimization dynamics and convergence guarantees. These optimization algorithms are designed to be both straightforward and adaptable, facilitating easy implementation in diverse settings.

cs.LG↗

arXiv2024

Minimal Matchings for dP3 Cluster Variables

In previous work [LM17], Tri Lai and the second author studied a family of subgraphs of the dP3 brane tiling, called Aztec castles, whose dimer partition functions provide combinatorial formulas for cluster variables resulting from mutations of the quiver associated with the del Pezzo surface dP3. In our paper, we investigate a variant of the dP3 quiver by considering a second alphabet of variables that breaks the symmetries of the relevant recurrences. This deformation is motivated by the theory of cluster algebras with principal coefficients introduced by Fomin and Zelevinsky. Our main result gives an explicit formula extending previously known generating functions for dP3 cluster variables by using Aztec castles and constructing their associated minimal matchings.

math.CO↗

arXiv2024

Automated Description Generation for Software Patches

Software patches are pivotal in refining and evolving codebases, addressing bugs, vulnerabilities, and optimizations. Patch descriptions provide detailed accounts of changes, aiding comprehension and collaboration among developers. However, manual description creation poses challenges in terms of time consumption and variations in quality and detail. In this paper, we propose PATCHEXPLAINER, an approach that addresses these challenges by framing patch description generation as a machine translation task. In PATCHEXPLAINER, we leverage explicit representations of critical elements, historical context, and syntactic conventions. Moreover, the translation model in PATCHEXPLAINER is designed with an awareness of description similarity. Particularly, the model is explicitly trained to recognize and incorporate similarities present in patch descriptions clustered into groups, improving its ability to generate accurate and consistent descriptions across similar patches. The dual objectives maximize similarity and accurately predict affiliating groups. Our experimental results on a large dataset of real-world software patches show that PATCHEXPLAINER consistently outperforms existing methods, with improvements up to 189% in BLEU, 5.7X in Exact Match rate, and 154% in Semantic Similarity, affirming its effectiveness in generating software patch descriptions.

cs.SE↗

arXiv2024

Venomancer: Towards Imperceptible and Target-on-Demand Backdoor Attacks in Federated Learning

Federated Learning (FL) is a distributed machine learning approach that maintains data privacy by training on decentralized data sources. Similar to centralized machine learning, FL is also susceptible to backdoor attacks, where an attacker can compromise some clients by injecting a backdoor trigger into local models of those clients, leading to the global model's behavior being manipulated as desired by the attacker. Most backdoor attacks in FL assume a predefined target class and require control over a large number of clients or knowledge of benign clients' information. Furthermore, they are not imperceptible and are easily detected by human inspection due to clear artifacts left on the poison data. To overcome these challenges, we propose Venomancer, an effective backdoor attack that is imperceptible and allows target-on-demand. Specifically, imperceptibility is achieved by using a visual loss function to make the poison data visually indistinguishable from the original data. Target-on-demand property allows the attacker to choose arbitrary target classes via conditional adversarial training. Additionally, experiments showed that the method is robust against state-of-the-art defenses such as Norm Clipping, Weak DP, Krum, Multi-Krum, RLR, FedRAD, Deepsight, and RFLBAT. The source code is available at https://github.com/nguyenhongson1902/Venomancer.

cs.CV↗

arXiv2024

An identity involving $h$-polynomials of poset associahedra and type B Narayana polynomials

For any finite connected poset $P$, Galashin introduced a simple convex $(|P|-2)$-dimensional polytope $\mathscr{A}(P)$ called the poset associahedron. Let $P$ be a poset with a proper autonomous subposet $S$ that is a chain of size $n$. For $1\leq i \leq n$, let $P_i$ be the poset obtained from $P$ by replacing $S$ by an antichain of size $i$. We show that the $h$-polynomial of $\mathscr{A}(P)$ can be written in terms of the $h$-polynomials of $\mathscr{A}(P_i)$ and type B Narayana polynomials. We then use the identity to deduce several identities involving Narayana polynomials, Eulerian polynomials, and stack-sorting preimages.

math.CO↗

arXiv2024

An Empirical Study on Capability of Large Language Models in Understanding Code Semantics

Large Language Models for Code (code LLMs) have demonstrated remarkable performance across various software engineering (SE) tasks, increasing the application of code LLMs in software development. Despite the success of code LLMs, there remain significant concerns about the actual capabilities and reliability of these models, "whether these models really learn the semantics of code from the training data and leverage the learned knowledge to perform the SE tasks". In this paper, we introduce EMPICA, a comprehensive framework designed to systematically and empirically evaluate the capabilities of code LLMs in understanding code semantics. Specifically, EMPICA systematically introduces controlled modifications/transformations into the input code and examines the models' responses. Generally, code LLMs must be robust to semantically equivalent code inputs and be sensitive to non-equivalent ones for all SE tasks. Specifically, for every SE task, given an input code snippet c and its semantic equivalent variants, code LLMs must robustly produce consistent/equivalent outputs while they are expected to generate different outputs for c and its semantic non-equivalent variants. Our experimental results on three representative code understanding tasks, including code summarization, method name prediction, and output prediction, reveal that the robustness and sensitivity of the state-of-the-art code LLMs to code transformations vary significantly across tasks and transformation operators. In addition, the code LLMs exhibit better robustness to the semantic preserving transformations than their sensitivity to the semantic non-preserving transformations. These results highlight a need to enhance the model's capabilities of understanding code semantics, especially the sensitivity property.

cs.SE↗

arXiv2024

Bender-Knuth involutions on linear extensions of posets

We study the permutation group $\mathcal{BK}_P$ generated by Bender-Knuth moves on linear extensions of a poset $P$, an analog of the Berenstein-Kirillov group on column-strict tableaux. We explore the group relations, with an emphasis on identifying posets $P$ for which the cactus relations hold in $\mathcal{BK}_P$. We also examine $\mathcal{BK}_P$ as a subgroup of the symmetric group $\mathfrak{S}_{\mathcal{L}(P)}$ on the set of linear extensions of $P$ with the focus on analyzing posets $P$ for which $\mathcal{BK}_P = \mathfrak{S}_{\mathcal{L}(P)}$.

math.CO↗

arXiv2024

Temperley-Lieb Crystals

Elements of Lusztig's dual canonical bases are Schur-positive when evaluated on (generalized) Jacobi-Trudi matrices. This deep property was proved by Rhoades and Skandera, relying on a result of Haiman, and ultimately on the (proof of) Kazhdan-Lusztig conjecture. For a particularly tractable part of the dual canonical basis - called Temperley-Lieb immanants - we give a generalization of Littlewood-Richardson rule: we provide a combinatorial interpretation for the coefficient of a particular Schur function in the evaluation of a particular Temperley-Lieb immanant on a particular Jacobi-Trudi matrix. For this we introduce shuffle tableaux, and apply Stembridge's axioms to show that certain graphs on shuffle tableaux are type $A$ Kashiwara crystals.

math.CO↗

arXiv2023

Growth Diagrams for Schubert RSK

Motivated by classical combinatorial Schubert calculus on the Grassmannian, Huang--Pylyavskyy introduced a generalized theory of Robinson-Schensted-Knuth (RSK) correspondence for studying Schubert calculus on the complete flag variety via insertion algorithms. The inputs of the correspondence are certain biwords, the insertion objects are bumpless pipe dreams, and the recording objects are certain chains in Bruhat order. In particular, they defined plactic biwords and showed that classical Knuth relations can be generalized to plactic biwords. In this paper, we give an analogue of Fomin's growth diagrams for this generalized RSK correspondence on plactic biwords. We show that this growth diagram recovers the bijection between pipe dreams and bumpless pipe dreams of Gao--Huang.

math.CO↗

arXiv2023

Instance Segmentation under Occlusions via Location-aware Copy-Paste Data Augmentation

Occlusion is a long-standing problem in computer vision, particularly in instance segmentation. ACM MMSports 2023 DeepSportRadar has introduced a dataset that focuses on segmenting human subjects within a basketball context and a specialized evaluation metric for occlusion scenarios. Given the modest size of the dataset and the highly deformable nature of the objects to be segmented, this challenge demands the application of robust data augmentation techniques and wisely-chosen deep learning architectures. Our work (ranked 1st in the competition) first proposes a novel data augmentation technique, capable of generating more training samples with wider distribution. Then, we adopt a new architecture - Hybrid Task Cascade (HTC) framework with CBNetV2 as backbone and MaskIoU head to improve segmentation performance. Furthermore, we employ a Stochastic Weight Averaging (SWA) training strategy to improve the model's generalization. As a result, we achieve a remarkable occlusion score (OM) of 0.533 on the challenge dataset, securing the top-1 position on the leaderboard. Source code is available at this https://github.com/nguyendinhson-kaist/MMSports23-Seg-AutoID.

cs.CV↗

arXiv2023

Labeled Chip-firing on Binary Trees with $2^n-1$ Chips

We study labeled chip-firing on binary trees and some of its modifications. We prove a sorting property of terminal configurations of the process. We also analyze the endgame moves poset and prove that this poset is a modular lattice.

math.CO↗

arXiv2023

Poset Associahedra and Stack-sorting

For any finite connected poset $P$, Galashin introduced a simple convex $(|P|-2)$-dimensional polytope $\mathscr{A}(P)$ called the poset associahedron. For a certain family of posets, whose poset associahedra interpolate between the classical permutohedron and associahedron, we give a simple combinatorial interpretation of the $h$-vector. Our interpretation relates to the theory of stack-sorting of permutations. It also allows us to prove real-rootedness of some of their $h$-polynomials.

math.CO↗