Searcharxiv⌕ Search

arXiv subjects

Minh Le

Publications and source records attributed to Minh Le.

At least 37 records · Page 2Linked to original sources
arXiv2025

Impact of Artificial Intelligence on Environmental Quality through Technical Change: A Free Dynamic Equilibrium Approach

In the times we live in today, humanity faces unprecedented environmental challenges. The emergence of artificial intelligence (AI) has opened new doors in our collective efforts to address our planet's pressing problems; however, many have doubts on the actual extent of impact that AI have on the environment. In particular, AI also assisting dirty production is a drawback that is largely absent from the literature. To investigate the impact of AI on the environment, we establish mathematical models to model the economy and the production process of goods based on outdated and advanced technologies. The secondary results are stated in the form of lemmas, the main results are stated in the form of theorems. From the theorems we conclude that AI may not on its own prevent an environmental disaster, a reason of which is its concurrent contribution to dirty production. With temporary government intervention, however, AI is able to avert an environmental disaster.

econ.TH↗
arXiv2025

On Zero-Initialized Attention: Optimal Prompt and Gating Factor Estimation

The LLaMA-Adapter has recently emerged as an efficient fine-tuning technique for LLaMA models, leveraging zero-initialized attention to stabilize training and enhance performance. However, despite its empirical success, the theoretical foundations of zero-initialized attention remain largely unexplored. In this paper, we provide a rigorous theoretical analysis, establishing a connection between zero-initialized attention and mixture-of-expert models. We prove that both linear and non-linear prompts, along with gating functions, can be optimally estimated, with non-linear prompts offering greater flexibility for future applications. Empirically, we validate our findings on the open LLM benchmarks, demonstrating that non-linear prompts outperform linear ones. Notably, even with limited training data, both prompt types consistently surpass vanilla attention, highlighting the robustness and adaptability of zero-initialized attention.

cs.LG↗
arXiv2025

RepLoRA: Reparameterizing Low-Rank Adaptation via the Perspective of Mixture of Experts

Low-rank Adaptation (LoRA) has emerged as a powerful method for fine-tuning large-scale foundation models. Despite its popularity, the theoretical understanding of LoRA has remained limited. This paper presents a theoretical analysis of LoRA by examining its connection to the Mixture of Experts models. Under this framework, we show that simple reparameterizations of the LoRA matrices can notably accelerate the low-rank matrix estimation process. In particular, we prove that reparameterization can reduce the data needed to achieve a desired estimation error from an exponential to a polynomial scale. Motivated by this insight, we propose Reparameterized Low-Rank Adaptation (RepLoRA), which incorporates lightweight MLPs to reparameterize the LoRA matrices. Extensive experiments across multiple domains demonstrate that RepLoRA consistently outperforms vanilla LoRA. Notably, with limited data, RepLoRA surpasses LoRA by a margin of up to 40.0% and achieves LoRA's performance with only 30.0% of the training data, highlighting both the theoretical and empirical robustness of our PEFT method.

cs.LG↗
arXiv2025

Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts

Prompt-based techniques, such as prompt-tuning and prefix-tuning, have gained prominence for their efficiency in fine-tuning large pre-trained models. Despite their widespread adoption, the theoretical foundations of these methods remain limited. For instance, in prefix-tuning, we observe that a key factor in achieving performance parity with full fine-tuning lies in the reparameterization strategy. However, the theoretical principles underpinning the effectiveness of this approach have yet to be thoroughly examined. Our study demonstrates that reparameterization is not merely an engineering trick but is grounded in deep theoretical foundations. Specifically, we show that the reparameterization strategy implicitly encodes a shared structure between prefix key and value vectors. Building on recent insights into the connection between prefix-tuning and mixture of experts models, we further illustrate that this shared structure significantly improves sample efficiency in parameter estimation compared to non-shared alternatives. The effectiveness of prefix-tuning across diverse tasks is empirically confirmed to be enhanced by the shared structure, through extensive experiments in both visual and language domains. Additionally, we uncover similar structural benefits in prompt-tuning, offering new perspectives on its success. Our findings provide theoretical and empirical contributions, advancing the understanding of prompt-based methods and their underlying mechanisms.

cs.LG↗
arXiv2025

Global boundedness in the higher-dimensional fully parabolic chemotaxis with weak singular sensitivity and logistic source

We consider the following chemotaxis system under homogeneous Neumann boundary conditions in a smooth, open, bounded domain $Ω\subset \mathbb{R}^n$ with $n \geq 3$: \begin{equation*} \begin{cases} u_t = Δu - χ\nabla \cdot \left( \frac{u}{v^k} \nabla v \right) + ru - μu^2, & \text{in } Ω\times (0,T_{\rm max}), v_t = Δv - αv + βu, & \text{in } Ω\times (0,T_{\rm max}), \end{cases} \end{equation*} where $k \in (0,1)$, and $χ, r, μ, α, β$ are positive parameters. In this paper, we demonstrate that for suitably smooth initial data, the problem admits a unique nonnegative classical solution that remains globally bounded in time when $μ$ is sufficiently large.

math.AP↗
arXiv2025

Leveraging Hierarchical Taxonomies in Prompt-based Continual Learning

Humans perceive the world as a series of sequential events, which can be hierarchically organized with different levels of abstraction based on conceptual knowledge. Drawing inspiration from human learning behaviors, this work proposes a novel approach to mitigate catastrophic forgetting in Prompt-based Continual Learning models by exploiting the relationships between continuously emerging class data. We find that applying human habits of organizing and connecting information can serve as an efficient strategy when training deep learning models. Specifically, by building a hierarchical tree structure based on the expanding set of labels, we gain fresh insights into the data, identifying groups of similar classes could easily cause confusion. Additionally, we delve deeper into the hidden connections between classes by exploring the original pretrained model's behavior through an optimal transport-based approach. From these insights, we propose a novel regularization loss function that encourages models to focus more on challenging knowledge areas, thereby enhancing overall performance. Experimentally, our method demonstrated significant superiority over the most robust state-of-the-art models on various benchmarks.

cs.LG↗
arXiv2025

Absence of blow-up in a fully parabolic chemotaxis system with weak singular sensitivity and logistic damping in dimension two

It is shown in this paper that blow-up does not occur in the following chemotaxis system under homogeneous Neumann boundary conditions in a smooth, open, bounded domain \(Ω\subset \mathbb{R}^2\): \begin{equation*} \begin{cases} u_t = Δu - χ\nabla \cdot \left( \frac{u}{v^k} \nabla v \right) + ru - μu^2, \qquad &\text{in } Ω\times (0,T_{\rm max}), v_t = Δv - αv + βu, \qquad &\text{in } Ω\times (0,T_{\rm max}), \end{cases} \end{equation*} where \( k \in (0,1) \), and \(χ, r, μ, α, β\) are positive parameters. Known results have already established the same conclusion for the parabolic-elliptic case. Here, we complement these findings by extending the result to the fully parabolic case.

math.AP↗
arXiv2025

Adaptive Prompting for Continual Relation Extraction: A Within-Task Variance Perspective

To address catastrophic forgetting in Continual Relation Extraction (CRE), many current approaches rely on memory buffers to rehearse previously learned knowledge while acquiring new tasks. Recently, prompt-based methods have emerged as potent alternatives to rehearsal-based strategies, demonstrating strong empirical performance. However, upon analyzing existing prompt-based approaches for CRE, we identified several critical limitations, such as inaccurate prompt selection, inadequate mechanisms for mitigating forgetting in shared parameters, and suboptimal handling of cross-task and within-task variances. To overcome these challenges, we draw inspiration from the relationship between prefix-tuning and mixture of experts, proposing a novel approach that employs a prompt pool for each task, capturing variations within each task while enhancing cross-task variances. Furthermore, we incorporate a generative model to consolidate prior knowledge within shared parameters, eliminating the need for explicit data storage. Extensive experiments validate the efficacy of our approach, demonstrating superior performance over state-of-the-art prompt-based and rehearsal-free methods in continual relation extraction.

cs.CL↗
arXiv2025

Mixture of Experts Meets Prompt-Based Continual Learning

Exploiting the power of pre-trained models, prompt-based approaches stand out compared to other continual learning solutions in effectively preventing catastrophic forgetting, even with very few learnable parameters and without the need for a memory buffer. While existing prompt-based continual learning methods excel in leveraging prompts for state-of-the-art performance, they often lack a theoretical explanation for the effectiveness of prompting. This paper conducts a theoretical analysis to unravel how prompts bestow such advantages in continual learning, thus offering a new perspective on prompt design. We first show that the attention block of pre-trained models like Vision Transformers inherently encodes a special mixture of experts architecture, characterized by linear experts and quadratic gating score functions. This realization drives us to provide a novel view on prefix tuning, reframing it as the addition of new task-specific experts, thereby inspiring the design of a novel gating mechanism termed Non-linear Residual Gates (NoRGa). Through the incorporation of non-linear activation and residual connection, NoRGa enhances continual learning performance while preserving parameter efficiency. The effectiveness of NoRGa is substantiated both theoretically and empirically across diverse benchmarks and pretraining paradigms. Our code is publicly available at https://github.com/Minhchuyentoancbn/MoE_PromptCL

cs.LG↗
arXiv2024

Global existence of solutions to the fully parabolic chemotaxis system with logistic source under nonlinear Neumann boundary conditions

We study the existence of global boundedness solutions to the fully parabolic chemotaxis systems with logistic sources, $ru- μu^2$, under nonlinear Neumann boundary conditions, $\frac{\partial u}{\partial ν}= |u|^p$ where $p >1 $ in smooth bounded domain $Ω\subset \mathbb{R}^n$ with $n \geq 2$. A recent study by Le (2023) has shown that the logistic sources can ensure that solutions are global and bounded when $n =2$ with $p < \frac{3}{2}$ and $n=3$ with $p <\frac{7}{5}$. In this paper, we extend the previous findings by demonstrating the existence of global bounded solutions when $p< \frac{3}{2}$ in any spatial dimension $n \geq 2$.

math.AP↗
arXiv2024

Boundedness in a chemotaxis system with weakly singular sensitivity in dimension two with arbitrary sub-quadratic degradation sources

We study the global existence and boundedness of solutions to a chemotaxis system with weakly singular sensitivity and sub-logistic sources in a two dimensional domain. X. Zhao (Nonlinearity; 2023; 36; 3909-3938 ) showed that the logistic degradation, $-μu^2$, can prevent blow-up under the largeness assumption on $μ$. In this paper, we improve the result by replacing the quadratic degradation by sub-logistic one, $-\frac{μu^2}{\ln^β(u+e)}$ with $β\in (0,1)$, and removing the largeness assumption on $μ$.

math.AP↗
arXiv2023

Application of Orlicz regularity theory in two dimensional chemotaxis systems with logistic sources

The objective is to investigate the global existence of solutions for a degenerate chemotaxis system with logistic sources in a two-dimensional domain. It is demonstrated that the inclusion of logistic sources can exclude the occurrence of blow-up solutions, even in the presence of superlinear growth in the cross-diffusion rate. Our proof relies on the application of elliptic and parabolic regularity in Orlicz spaces and variational approach.

math.AP↗
arXiv2023

Efficacy of Neural Prediction-Based Zero-Shot NAS

In prediction-based Neural Architecture Search (NAS), performance indicators derived from graph convolutional networks have shown remarkable success. These indicators, achieved by representing feed-forward structures as component graphs through one-hot encoding, face a limitation: their inability to evaluate architecture performance across varying search spaces. In contrast, handcrafted performance indicators (zero-shot NAS), which use the same architecture with random initialization, can generalize across multiple search spaces. Addressing this limitation, we propose a novel approach for zero-shot NAS using deep learning. Our method employs Fourier sum of sines encoding for convolutional kernels, enabling the construction of a computational feed-forward graph with a structure similar to the architecture under evaluation. These encodings are learnable and offer a comprehensive view of the architecture's topological information. An accompanying multi-layer perceptron (MLP) then ranks these architectures based on their encodings. Experimental results show that our approach surpasses previous methods using graph convolutional networks in terms of correlation on the NAS-Bench-201 dataset and exhibits a higher convergence rate. Moreover, our extracted feature representation trained on each NAS benchmark is transferable to other NAS benchmarks, showing promising generalizability across multiple search spaces. The code is available at: https://github.com/minh1409/DFT-NPZS-NAS

cs.LG↗
arXiv2023

Global existence of solutions to the chemotaxis system with logistic source under nonlinear Neumann boundary condition

We consider classical solutions to the chemotaxis system with logistic source $f(u) := au-μu^2$ under nonlinear Neumann boundary condition $\frac{\partial u}{ \partial ν} = |u|^{p}$ with $p>1$ in a smooth convex bounded domain $Ω\subset \mathbb{R}^n$ where $n \geq 2$. This paper aims to show that if $p<\frac{3}{2}$, and $μ>0$, $n=2$, or $μ$ is sufficiently large when $n\geq 3$, then the parabolic-elliptic chemotaxis system admits a unique positive global-in-time classical solution that is bounded in $Ω\times (0, \infty)$. The similar result is also true if $p<\frac{3}{2}$, $n=2$, and $μ>0$ or $p<\frac{7}{5}$, $n=3$, and $μ$ is sufficiently large for the parabolic-parabolic chemotaxis system.

math.AP↗
arXiv2023

Blow-up prevention by sub-logistic sources in 2d Keller-Segel system

This paper investigates the global existence of solutions to Keller-Segel systems with sub-logistic sources using the test function method. Prior work demonstrated that sub-logistic sources $f(u)=ru -μ\frac{u^2}{\ln^p(u+e)}$ with $p\in(0,1)$ can prevent blow-up solutions for the 2D minimal Keller-Segel chemotaxis model. Our study extends this result by showing that when $p=1$, sub-logistic sources can still prevent the occurrence of finite time blow-up solutions. Additionally, we provide a concise proof for a known result that the equi-integrability of $\left \{ \int_Ωu^{\frac{n}{2}}(\cdot,t) \right \}_{t\in (0,T_{\rm max})}$ can avoid blow-up.

math.AP↗
arXiv2023

Blow-up prevention by sub-logistic sources in Keller-Segel cross diffusion type system

The focus of this paper is on solutions to a two-dimensional Keller-Segel system containing sub-logistic sources. We show that the presence of sub-logistic terms is adequate to prevent blow-up phenomena even in strongly degenerate Keller-Segel systems. Our proof relies on several techniques, including parabolic regularity theory in Orlicz spaces, variational arguments, interpolation inequalities, and the Moser iteration method.

math.AP↗
arXiv2020

Revisiting Edge Detection in Convolutional Neural Networks

The ability to detect edges is a fundamental attribute necessary to truly capture visual concepts. In this paper, we prove that edges cannot be represented properly in the first convolutional layer of a neural network, and further show that they are poorly captured in popular neural network architectures such as VGG-16 and ResNet. The neural networks are found to rely on color information, which might vary in unexpected ways outside of the datasets used for their evaluation. To improve their robustness, we propose edge-detection units and show that they reduce performance loss and generate qualitatively different representations. By comparing various models, we show that the robustness of edge detection is an important factor contributing to the robustness of models against color noise.

cs.CV↗