arXiv subjects

Nam Nguyen

Publications and source records attributed to Nam Nguyen.

At least 55 records · Page 3Linked to original sources

arXiv2023

TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecasting

Transformers have gained popularity in time series forecasting for their ability to capture long-sequence interactions. However, their high memory and computing requirements pose a critical bottleneck for long-term forecasting. To address this, we propose TSMixer, a lightweight neural architecture exclusively composed of multi-layer perceptron (MLP) modules for multivariate forecasting and representation learning on patched time series. Inspired by MLP-Mixer's success in computer vision, we adapt it for time series, addressing challenges and introducing validated components for enhanced accuracy. This includes a novel design paradigm of attaching online reconciliation heads to the MLP-Mixer backbone, for explicitly modeling the time-series properties such as hierarchy and channel-correlations. We also propose a novel Hybrid channel modeling and infusion of a simple gating approach to effectively handle noisy channel interactions and generalization across diverse datasets. By incorporating these lightweight components, we significantly enhance the learning capability of simple MLP structures, outperforming complex Transformer models with minimal computing usage. Moreover, TSMixer's modular design enables compatibility with both supervised and masked self-supervised learning methods, making it a promising building block for time-series Foundation Models. TSMixer outperforms state-of-the-art MLP and Transformer models in forecasting by a considerable margin of 8-60%. It also outperforms the latest strong benchmarks of Patch-Transformer models (by 1-2%) with a significant reduction in memory and runtime (2-3X). The source code of our model is officially released as PatchTSMixer in the HuggingFace. Model: https://huggingface.co/docs/transformers/main/en/model_doc/patchtsmixer Examples: https://github.com/ibm/tsfm/#notebooks-links

cs.LG↗

arXiv2023

Subspace methods for electronic structure simulations on quantum computers

Quantum subspace methods (QSMs) are a class of quantum computing algorithms where the time-independent Schrodinger equation for a quantum system is projected onto a subspace of the underlying Hilbert space. This projection transforms the Schrodinger equation into an eigenvalue problem determined by measurements carried out on a quantum device. The eigenvalue problem is then solved on a classical computer, yielding approximations to ground- and excited-state energies and wavefunctions. QSMs are examples of hybrid quantum-classical methods, where a quantum device supported by classical computational resources is employed to tackle a problem. QSMs are rapidly gaining traction as a strategy to simulate electronic wavefunctions on quantum computers, and thus their design, development, and application is a key research field at the interface between quantum computation and electronic structure. In this review, we provide a self-contained introduction to QSMs, with emphasis on their application to the electronic structure of molecules. We present the theoretical foundations and applications of QSMs, and we discuss their implementation on quantum hardware, illustrating the impact of noise on their performance.

quant-ph↗

arXiv2023

A Supervised Contrastive Learning Pretrain-Finetune Approach for Time Series

Foundation models have recently gained attention within the field of machine learning thanks to its efficiency in broad data processing. While researchers had attempted to extend this success to time series models, the main challenge is effectively extracting representations and transferring knowledge from pretraining datasets to the target finetuning dataset. To tackle this issue, we introduce a novel pretraining procedure that leverages supervised contrastive learning to distinguish features within each pretraining dataset. This pretraining phase enables a probabilistic similarity metric, which assesses the likelihood of a univariate sample being closely related to one of the pretraining datasets. Subsequently, using this similarity metric as a guide, we propose a fine-tuning procedure designed to enhance the accurate prediction of the target data by aligning it more closely with the learned dynamics of the pretraining datasets. Our experiments have shown promising results which demonstrate the efficacy of our approach.

cs.LG↗

arXiv2023

Quantum Computation of Reactions on Surfaces Using Local Embedding

Modeling electronic systems is an important application for quantum computers. In the context of materials science, an important open problem is the computational description of chemical reactions on surfaces. In this work, we outline a workflow to model the adsorption and reaction of molecules on surfaces using quantum computing algorithms. We develop and compare two local embedding methods for the systematic determination of active spaces. These methods are automated and based on the physics of molecule-surface interactions and yield systematically improvable active spaces. Furthermore, to reduce the quantum resources required for the simulation of the selected active spaces using quantum algorithms, we introduce a technique for exact and automated circuit simplification. This technique is applicable to a broad class of quantum circuits and critical to enable demonstration on near-term quantum devices. We apply the proposed combination of active-space selection and circuit simplification to the dissociation of water on a magnesium surface using classical simulators and quantum hardware. Our study identifies reactions of molecules on surfaces, in conjunction with the proposed algorithmic workflow, as a promising research direction in the field of quantum computing applied to materials science.

quant-ph↗

arXiv2023

AI Foundation Models for Weather and Climate: Applications, Design, and Implementation

Machine learning and deep learning methods have been widely explored in understanding the chaotic behavior of the atmosphere and furthering weather forecasting. There has been increasing interest from technology companies, government institutions, and meteorological agencies in building digital twins of the Earth. Recent approaches using transformers, physics-informed machine learning, and graph neural networks have demonstrated state-of-the-art performance on relatively narrow spatiotemporal scales and specific tasks. With the recent success of generative artificial intelligence (AI) using pre-trained transformers for language modeling and vision with prompt engineering and fine-tuning, we are now moving towards generalizable AI. In particular, we are witnessing the rise of AI foundation models that can perform competitively on multiple domain-specific downstream tasks. Despite this progress, we are still in the nascent stages of a generalizable AI model for global Earth system models, regional climate models, and mesoscale weather models. Here, we review current state-of-the-art AI approaches, primarily from transformer and operator learning literature in the context of meteorology. We provide our perspective on criteria for success towards a family of foundation models for nowcasting and forecasting weather and climate predictions. We also discuss how such models can perform competitively on downstream tasks such as downscaling (super-resolution), identifying conditions conducive to the occurrence of wildfires, and predicting consequential meteorological phenomena across various spatiotemporal scales such as hurricanes and atmospheric rivers. In particular, we examine current AI methodologies and contend they have matured enough to design and implement a weather foundation model.

cs.LG↗

arXiv2023

Treatment-Response Analysis of Tumor as A Quantum Particle

In this article, I present a novel and computational-efficient approach for treatment-response modeling of tumor progression-free survival (PFS) probability using the physical phenomenon of a quantum particle walking on a one-dimensional lattice with the presence of a proximate trap.

physics.bio-ph↗

arXiv2023

An End-to-End Time Series Model for Simultaneous Imputation and Forecast

Time series forecasting using historical data has been an interesting and challenging topic, especially when the data is corrupted by missing values. In many industrial problem, it is important to learn the inference function between the auxiliary observations and target variables as it provides additional knowledge when the data is not fully observed. We develop an end-to-end time series model that aims to learn the such inference relation and make a multiple-step ahead forecast. Our framework trains jointly two neural networks, one to learn the feature-wise correlations and the other for the modeling of temporal behaviors. Our model is capable of simultaneously imputing the missing entries and making a multiple-step ahead prediction. The experiments show good overall performance of our framework over existing methods in both imputation and forecasting tasks.

cs.LG↗

arXiv2023

Long-Range Transformers for Dynamic Spatiotemporal Forecasting

Multivariate time series forecasting focuses on predicting future values based on historical context. State-of-the-art sequence-to-sequence models rely on neural attention between timesteps, which allows for temporal learning but fails to consider distinct spatial relationships between variables. In contrast, methods based on graph neural networks explicitly model variable relationships. However, these methods often rely on predefined graphs that cannot change over time and perform separate spatial and temporal updates without establishing direct connections between each variable at every timestep. Our work addresses these problems by translating multivariate forecasting into a "spatiotemporal sequence" formulation where each Transformer input token represents the value of a single variable at a given time. Long-Range Transformers can then learn interactions between space, time, and value information jointly along this extended sequence. Our method, which we call Spacetimeformer, achieves competitive results on benchmarks from traffic forecasting to electricity demand and weather prediction while learning spatiotemporal relationships purely from data.

cs.LG↗

arXiv2023

Translational Quantum Machine Intelligence for Modeling Tumor Dynamics in Oncology

Quantifying the dynamics of tumor burden reveals useful information about cancer evolution concerning treatment effects and drug resistance, which play a crucial role in advancing model-informed drug developments (MIDD) towards personalized medicine and precision oncology. The emergence of Quantum Machine Intelligence offers unparalleled insights into tumor dynamics via a quantum mechanics perspective. This paper introduces a novel hybrid quantum-classical neural architecture named $η-$Net that enables quantifying quantum dynamics of tumor burden concerning treatment effects. We evaluate our proposed neural solution on two major use cases, including cohort-specific and patient-specific modeling. In silico numerical results show a high capacity and expressivity of $η-$Net to the quantified biological problem. Moreover, the close connection to representation learning - the foundation for successes of modern AI, enables efficient transferability of empirical knowledge from relevant cohorts to targeted patients. Finally, we leverage Bayesian optimization to quantify the epistemic uncertainty of model predictions, paving the way for $η-$Net towards reliable AI in decision-making for clinical usages.

q-bio.OT↗

arXiv2022

On matched asymptotic expansions of backreacting metastable anti-branes

We construct analytically a perturbative supergravity solution that captures the backreaction of a metastable state of anti-branes in the background of a particular modification of the Klebanov-Strassler throat in a long-wavelength approximation. Our solution, which has no unphysical singularities, describes how non-supersymmetric spherical NS5-branes with dissolved anti-D3 brane charge backreact in a fluxed throat geometry. It supports previous claims that there is a well-behaved supergravity description of the metastable state of wrapped NS5-branes proposed years ago by Kachru, Pearson, and Verlinde.

hep-th↗

arXiv2021

Quantum Embedding Search for Quantum Machine Learning

This paper introduces a novel quantum embedding search algorithm (QES, pronounced as "quest"), enabling search for optimal quantum embedding design for a specific dataset of interest. First, we establish the connection between the structures of quantum embedding and the representations of directed multi-graphs, enabling a well-defined search space. Second, we instigate the entanglement level to reduce the cardinality of the search space to a feasible size for practical implementations. Finally, we mitigate the cost of evaluating the true loss function by using surrogate models via sequential model-based optimization. We demonstrate the feasibility of our proposed approach on synthesis and Iris datasets, which empirically shows that found quantum embedding architecture by QES outperforms manual designs whereas achieving comparable performance to classical machine learning models.

quant-ph↗

arXiv2021

Contrastive Self-supervised Neural Architecture Search

This paper proposes a novel cell-based neural architecture search algorithm (NAS), which completely alleviates the expensive costs of data labeling inherited from supervised learning. Our algorithm capitalizes on the effectiveness of self-supervised learning for image representations, which is an increasingly crucial topic of computer vision. First, using only a small amount of unlabeled train data under contrastive self-supervised learning allow us to search on a more extensive search space, discovering better neural architectures without surging the computational resources. Second, we entirely relieve the cost for labeled data (by contrastive loss) in the search stage without compromising architectures' final performance in the evaluation phase. Finally, we tackle the inherent discrete search space of the NAS problem by sequential model-based optimization via the tree-parzen estimator (SMBO-TPE), enabling us to reduce the computational expense response surface significantly. An extensive number of experiments empirically show that our search algorithm can achieve state-of-the-art results with better efficiency in data labeling cost, searching time, and accuracy in final validation.

cs.CV↗

arXiv2021

COVID-19 Pneumonia Severity Prediction using Hybrid Convolution-Attention Neural Architectures

This study proposed a novel framework for COVID-19 severity prediction, which is a combination of data-centric and model-centric approaches. First, we propose a data-centric pre-training for extremely scare data scenarios of the investigating dataset. Second, we propose two hybrid convolution-attention neural architectures that leverage the self-attention from the Transformer and the Dense Associative Memory (Modern Hopfield networks). Our proposed approach achieves significant improvement from the conventional baseline approach. The best model from our proposed approach achieves $R^2 = 0.85 \pm 0.05$ and Pearson correlation coefficient $ρ= 0.92 \pm 0.02$ in geographic extend and $R^2 = 0.72 \pm 0.09, ρ= 0.85\pm 0.06$ in opacity prediction.

eess.IV↗

arXiv2021

An effective theory for higher-dimensional black holes and applications to metastable antibranes

Despite their consequential applications, metastable states of antibranes in warped throats are not yet fully understood. In this thesis, we provide new information on various aspects of these metastable antibranes through applications of the blackfold effective theory for higher-dimensional black holes. As concrete examples, we study the conjectured metastable state of polarised anti-D3 branes at the tip of the Klebanov-Strassler (KS) throat in type IIB supergravity and the analogous state of polarised anti-M2 branes at the tip of the Cvetic-Gibbons-Lu-Pope (CGLP) throat in eleven-dimensional supergravity. For anti-D3 branes in KS throat, we provide novel evidence for the existence of the metastable state exactly where no-go theorems are lifted. In the extremal limit, we recover directly in supergravity the metastable states originally discovered by Kachru, Pearson, and Verlinde (KPV). Away from extremality, we uncover a metastable wrapped black NS5 state. We observe that such metastability is lost when the wrapped NS5 is heated sufficiently that its horizon geometry resembles that of a black anti-D3. We study the classical stability of the KPV state under generic long-wavelength deformations. We observe that, with regards to considered perturbations and regime of parameters, the state is classically stable. A study of anti-M2 branes in CGLP throat reveals many similarities to that of the anti-D3 branes. We recover directly in supergravity the Klebanov-Pufu (KP) state at extremality, and our finite temperature results fit suggestively well with known, complementary no-go theorems. However, we discover an unexpected, exotic pattern of thermal transitions of the KP state different from that of the KPV. This thesis contains also a pedagogical introduction to the blackfold formalism, focusing on aspects immediately relevant to applications to metastable antibranes.

hep-th↗

arXiv2021

ICDAR 2021 Competition on Historical Map Segmentation

This paper presents the final results of the ICDAR 2021 Competition on Historical Map Segmentation (MapSeg), encouraging research on a series of historical atlases of Paris, France, drawn at 1/5000 scale between 1894 and 1937. The competition featured three tasks, awarded separately. Task~1 consists in detecting building blocks and was won by the L3IRIS team using a DenseNet-121 network trained in a weakly supervised fashion. This task is evaluated on 3 large images containing hundreds of shapes to detect. Task~2 consists in segmenting map content from the larger map sheet, and was won by the UWB team using a U-Net-like FCN combined with a binarization method to increase detection edge accuracy. Task~3 consists in locating intersection points of geo-referencing lines, and was also won by the UWB team who used a dedicated pipeline combining binarization, line detection with Hough transform, candidate filtering, and template matching for intersection refinement. Tasks~2 and~3 are evaluated on 95 map sheets with complex content. Dataset, evaluation tools and results are available under permissive licensing at \url{https://icdar21-mapseg.github.io/}.

cs.CV↗

arXiv2021

Identifying Hubs in Undergraduate Course Networks Based on Scaled Co-Enrollments: Extended Version

Understanding course enrollment patterns is valuable to predict upcoming demands for future courses, and to provide student with realistic courses to pursue given their current backgrounds. This study uses undergraduate student enrollment data to form networks of courses where connections are based on student co-enrollments. The course networks generated in this paper are based on eight years of undergraduate course enrollment data from a large metropolitan university. The networks are analyzed to identify "hub" courses often taken with many other courses. Two notions of hubs are considered: one focused on raw popularity across all students, and one focused on proportional likelihoods of co-enrollment with other courses. A variety of network metrics are calculated to evaluate the course networks. Academic departments and high-level academic categories, such as Humanities vs STEM, are studied for their influence over course groupings. The identification of hub courses has practical applications, since it can help better predict the impact of changes in course offerings and in course popularity, and in the case of interdisciplinary hub courses, can be used to increase or decrease interest and enrollments in specific academic departments and areas.

cs.SI↗

arXiv2021

Temporal Latent Auto-Encoder: A Method for Probabilistic Multivariate Time Series Forecasting

Probabilistic forecasting of high dimensional multivariate time series is a notoriously challenging task, both in terms of computational burden and distribution modeling. Most previous work either makes simple distribution assumptions or abandons modeling cross-series correlations. A promising line of work exploits scalable matrix factorization for latent-space forecasting, but is limited to linear embeddings, unable to model distributions, and not trainable end-to-end when using deep learning forecasting. We introduce a novel temporal latent auto-encoder method which enables nonlinear factorization of multivariate time series, learned end-to-end with a temporal deep learning latent space forecast model. By imposing a probabilistic latent space model, complex distributions of the input series are modeled via the decoder. Extensive experiments demonstrate that our model achieves state-of-the-art performance on many popular multivariate datasets, with gains sometimes as high as $50\%$ for several standard metrics.

cs.LG↗

arXiv2020

Comments on the stability of the KPV state

Using the blackfold approach, we study the classical stability of the KPV (Kachru-Pearson-Verlinde) state of anti-D3 branes at the tip of the Klebanov-Strassler throat. With regards to generic long-wavelength deformations considered, we found no instabilities. We comment on the relation of our results to existing results on the stability of the KPV state.

hep-th↗