Searcharxiv⌕ Search

arXiv subjects

Yue Yu

Publications and source records attributed to Yue Yu.

At least 289 records · Page 16Linked to original sources
arXiv2023

Personalized Federated Learning via Amortized Bayesian Meta-Learning

Federated learning is a decentralized and privacy-preserving technique that enables multiple clients to collaborate with a server to learn a global model without exposing their private data. However, the presence of statistical heterogeneity among clients poses a challenge, as the global model may struggle to perform well on each client's specific task. To address this issue, we introduce a new perspective on personalized federated learning through Amortized Bayesian Meta-Learning. Specifically, we propose a novel algorithm called \emph{FedABML}, which employs hierarchical variational inference across clients. The global prior aims to capture representations of common intrinsic structures from heterogeneous clients, which can then be transferred to their respective tasks and aid in the generation of accurate client-specific approximate posteriors through a few local updates. Our theoretical analysis provides an upper bound on the average generalization error and guarantees the generalization performance on unseen data. Finally, several empirical results are implemented to demonstrate that \emph{FedABML} outperforms several competitive baselines.

cs.LG↗
arXiv2023

Non-degenerate surface pair density wave in the Kagome superconductor CsV$_3$Sb$_5$ -- application to vestigial orders

On the Sb-layer of the Kagome superconductor CsV$_3$Sb$_5$, pair density wave states have been observed. When the high-temperature charge orderings are treated as static backgrounds, these PDW states exhibit the same wavevector in the effective 2D Brillouin zone. Interestingly, these PDW states break the same symmetry on the surface. Considering the presence of this non-degenerate PDW, we investigate the implications for the possible existence of a vestigial charge-4e phase with a non-zero center-of-mass momentum. To distinguish between different vestigial phases, we propose scanning tunneling microscopy experiments. We aim to provide insights into the nature of the vestigial phases and their distinct characteristics in CsV$_3$Sb$_5$. This research sheds light on the interplay between PDW states, charge orderings, and superconductivity of the Kagome superconductor.

cond-mat.supr-con↗
arXiv2023

Linearizations via Dirac delta function

This note is to show that the position-space embedding in \cite{ESP2021embedding} in the position and occupation bases can be obtained by considering the dynamics of Dirac delta function $$δ(\mathbf{x}- \mathbf{z}(t)) = δ(x_1-z_1(t))\cdots δ(x_d-z_d(t)),$$ where $\mathbf{z}(t)\in \mathbb{R}^d$ is the solution of a nonlinear dynamical system and $\mathbf{x}\in \mathbb{R}^d$ is a variable in the position space.

math.DS↗
arXiv2023

Practical Privacy-Preserving Gaussian Process Regression via Secret Sharing

Gaussian process regression (GPR) is a non-parametric model that has been used in many real-world applications that involve sensitive personal data (e.g., healthcare, finance, etc.) from multiple data owners. To fully and securely exploit the value of different data sources, this paper proposes a privacy-preserving GPR method based on secret sharing (SS), a secure multi-party computation (SMPC) technique. In contrast to existing studies that protect the data privacy of GPR via homomorphic encryption, differential privacy, or federated learning, our proposed method is more practical and can be used to preserve the data privacy of both the model inputs and outputs for various data-sharing scenarios (e.g., horizontally/vertically-partitioned data). However, it is non-trivial to directly apply SS on the conventional GPR algorithm, as it includes some operations whose accuracy and/or efficiency have not been well-enhanced in the current SMPC protocol. To address this issue, we derive a new SS-based exponentiation operation through the idea of 'confusion-correction' and construct an SS-based matrix inversion algorithm based on Cholesky decomposition. More importantly, we theoretically analyze the communication cost and the security of the proposed SS-based operations. Empirical results show that our proposed method can achieve reasonable accuracy and efficiency under the premise of preserving data privacy.

cs.CR↗
arXiv2023

ToolQA: A Dataset for LLM Question Answering with External Tools

Large Language Models (LLMs) have demonstrated impressive performance in various NLP tasks, but they still suffer from challenges such as hallucination and weak numerical reasoning. To overcome these challenges, external tools can be used to enhance LLMs' question-answering abilities. However, current evaluation methods do not distinguish between questions that can be answered using LLMs' internal knowledge and those that require external information through tool use. To address this issue, we introduce a new dataset called ToolQA, which is designed to faithfully evaluate LLMs' ability to use external tools for question answering. Our development of ToolQA involved a scalable, automated process for dataset curation, along with 13 specialized tools designed for interaction with external knowledge in order to answer questions. Importantly, we strive to minimize the overlap between our benchmark data and LLMs' pre-training data, enabling a more precise evaluation of LLMs' tool-use reasoning abilities. We conducted an in-depth diagnosis of existing tool-use LLMs to highlight their strengths, weaknesses, and potential improvements. Our findings set a new benchmark for evaluating LLMs and suggest new directions for future advancements. Our data and code are freely available to the broader scientific community on GitHub.

cs.CL↗
arXiv2023

The Sparse-Grid-Based Adaptive Spectral Koopman Method

The adaptive spectral Koopman (ASK) method was introduced to numerically solve autonomous dynamical systems that lay the foundation of numerous applications across different fields in science and engineering. Although ASK achieves high accuracy, it is computationally more expensive for multi-dimensional systems compared with conventional time integration schemes like Runge-Kutta. In this work, we combine the sparse grid and ASK to accelerate the computation for multi-dimensional systems. This sparse-grid-based ASK (SASK) method uses the Smolyak structure to construct multi-dimensional collocation points as well as associated polynomials that are used to approximate eigenfunctions of the Koopman operator of the system. In this way, the number of collocation points is reduced compared with using the tensor product rule. We demonstrate that SASK can be used to solve partial differential equations based-on their semi-discrete forms. Numerical experiments illustrate that SASK balances the accuracy with the computational cost, and hence accelerates ASK.

math.DS↗
arXiv2023

DyGen: Learning from Noisy Labels via Dynamics-Enhanced Generative Modeling

Learning from noisy labels is a challenge that arises in many real-world applications where training data can contain incorrect or corrupted labels. When fine-tuning language models with noisy labels, models can easily overfit the label noise, leading to decreased performance. Most existing methods for learning from noisy labels use static input features for denoising, but these methods are limited by the information they can provide on true label distributions and can result in biased or incorrect predictions. In this work, we propose the Dynamics-Enhanced Generative Model (DyGen), which uses dynamic patterns in the embedding space during the fine-tuning process of language models to improve noisy label predictions. DyGen uses the variational auto-encoding framework to infer the posterior distributions of true labels from noisy labels and training dynamics. Additionally, a co-regularization mechanism is used to minimize the impact of potentially noisy labels and priors. DyGen demonstrates an average accuracy improvement of 3.10% on two synthetic noise datasets and 1.48% on three real-world noise datasets compared to the previous state-of-the-art. Extensive experiments and analyses show the effectiveness of each component in DyGen. Our code is available for reproducibility on GitHub.

cs.CL↗
arXiv2023

Time complexity analysis of quantum algorithms via linear representations for nonlinear ordinary and partial differential equations

We construct quantum algorithms to compute the solution and/or physical observables of nonlinear ordinary differential equations (ODEs) and nonlinear Hamilton-Jacobi equations (HJE) via linear representations or exact mappings between nonlinear ODEs/HJE and linear partial differential equations (the Liouville equation and the Koopman-von Neumann equation). The connection between the linear representations and the original nonlinear system is established through the Dirac delta function or the level set mechanism. We compare the quantum linear systems algorithms based methods and the quantum simulation methods arising from different numerical approximations, including the finite difference discretisations and the Fourier spectral discretisations for the two different linear representations, with the result showing that the quantum simulation methods usually give the best performance in time complexity. We also propose the Schrödinger framework to solve the Liouville equation for the HJE with the Hamiltonian formulation of classical mechanics, since it can be recast as the semiclassical limit of the Wigner transform of the Schrödinger equation. Comparsion between the Schrödinger and the Liouville framework will also be made.

quant-ph↗
arXiv2023

Weakly-Supervised Scientific Document Classification via Retrieval-Augmented Multi-Stage Training

Scientific document classification is a critical task for a wide range of applications, but the cost of obtaining massive amounts of human-labeled data can be prohibitive. To address this challenge, we propose a weakly-supervised approach for scientific document classification using label names only. In scientific domains, label names often include domain-specific concepts that may not appear in the document corpus, making it difficult to match labels and documents precisely. To tackle this issue, we propose WANDER, which leverages dense retrieval to perform matching in the embedding space to capture the semantics of label names. We further design the label name expansion module to enrich the label name representations. Lastly, a self-training step is used to refine the predictions. The experiments on three datasets show that WANDER outperforms the best baseline by 11.9% on average. Our code will be published at https://github.com/ritaranx/wander.

cs.CL↗
arXiv2023

Local Boosting for Weakly-Supervised Learning

Boosting is a commonly used technique to enhance the performance of a set of base models by combining them into a strong ensemble model. Though widely adopted, boosting is typically used in supervised learning where the data is labeled accurately. However, in weakly supervised learning, where most of the data is labeled through weak and noisy sources, it remains nontrivial to design effective boosting approaches. In this work, we show that the standard implementation of the convex combination of base learners can hardly work due to the presence of noisy labels. Instead, we propose $\textit{LocalBoost}$, a novel framework for weakly-supervised boosting. LocalBoost iteratively boosts the ensemble model from two dimensions, i.e., intra-source and inter-source. The intra-source boosting introduces locality to the base learners and enables each base learner to focus on a particular feature regime by training new base learners on granularity-varying error regions. For the inter-source boosting, we leverage a conditional function to indicate the weak source where the sample is more likely to appear. To account for the weak labels, we further design an estimate-then-modify approach to compute the model weights. Experiments on seven datasets show that our method significantly outperforms vanilla boosting methods and other weakly-supervised methods.

cs.LG↗
arXiv2023

R-Mixup: Riemannian Mixup for Biological Networks

Biological networks are commonly used in biomedical and healthcare domains to effectively model the structure of complex biological systems with interactions linking biological entities. However, due to their characteristics of high dimensionality and low sample size, directly applying deep learning models on biological networks usually faces severe overfitting. In this work, we propose R-MIXUP, a Mixup-based data augmentation technique that suits the symmetric positive definite (SPD) property of adjacency matrices from biological networks with optimized training efficiency. The interpolation process in R-MIXUP leverages the log-Euclidean distance metrics from the Riemannian manifold, effectively addressing the swelling effect and arbitrarily incorrect label issues of vanilla Mixup. We demonstrate the effectiveness of R-MIXUP with five real-world biological network datasets on both regression and classification tasks. Besides, we derive a commonly ignored necessary condition for identifying the SPD matrices of biological networks and empirically study its influence on the model performance. The code implementation can be found in Appendix E.

cs.LG↗
arXiv2023

Superconductivity of anomalous pseudospin

Spin-orbit coupling driven by broken inversion symmetry ($I$) is known to lead to unusual magnetic response of superconductors, including extremely large critical fields for spin-singlet superconductors. This unusual response is also known to appear in materials that have $I$, provided there is local $I$-breaking: fermions participating in superconductivity reside on crystal sites that lack $I$. Here we show that this unusual response exists even when the crystal sites preserve $I$. Indeed, we argue that the symmetry of Kramers degenerate fermionic pseudospin is more relevant than the local crystal site symmetry. We examine and classify non-symmorphic materials with momentum space spin-textures that exhibit an anomalous pseudospin with different symmetry properties than usual spin-1/2. We find that this anomalous pseudospin does not depend on the existence of local $I$ breaking crystal sites and it optimizes the unusual magnetic response traditionally associated with locally noncentrosymmetric superconductors, dramatically extending the range of relevant materials. We further show this anomalous pseudospin leads to fully gapped `nodal' superconductors and provides additional insight into the breakdown of Blount's theorem for pseudospin triplet superconductors. We apply our results to UPt$_3$, BiS$_2$-based superconductors, Fe-based superconductors, and paramagnetic UCoGe.

cond-mat.supr-con↗
arXiv2023

Real-Time Sequential Conic Optimization for Multi-Phase Rocket Landing Guidance

We introduce a multi-phase rocket landing guidance framework that can handle nonlinear dynamics and does not mandate any additional mixed-integer or nonconvex constraints to handle discrete temporal events/switching. To achieve this, we first introduce sequential conic optimization (SeCO), a new paradigm for solving nonconvex optimal control problems that is entirely devoid of matrix factorizations and inversions. This framework combines sequential convex programming (SCP) and first-order conic optimization and can solve unified multi-phase trajectory optimization problems in real-time. The novel features of this framework are: (1) time-interval dilation, which enables multi-phase trajectory optimization with free-transition-time; (2) single-crossing compound state-triggered constraints, which are entirely convex if the trigger and constraint conditions are convex; (3) virtual state, which is a new approach to handling artificial infeasibility in SCP methods that preserves the shapes of the constraint sets; and, (4) the use of the proportional-integral projected gradient method (PIPG), a high-performance first-order conic optimization solver, in tandem with the penalized trust region (PTR) SCP algorithm. We demonstrate the efficacy and real-time capability of SeCO by solving a relevant multi-phase rocket landing guidance problem with nonlinear dynamics and convex constraints only, and observe that our solver is 2.7 times faster than a state-of-the-art convex optimization solver.

math.OC↗
arXiv2023

Higher Order Gauge Equivariant CNNs on Riemannian Manifolds and Applications

With the advent of group equivariant convolutions in deep networks literature, spherical CNNs with $\mathsf{SO}(3)$-equivariant layers have been developed to cope with data that are samples of signals on the sphere $S^2$. One can implicitly obtain $\mathsf{SO}(3)$-equivariant convolutions on $S^2$ with significant efficiency gains by explicitly requiring gauge equivariance w.r.t. $\mathsf{SO}(2)$. In this paper, we build on this fact by introducing a higher order generalization of the gauge equivariant convolution, whose implementation is dubbed a gauge equivariant Volterra network (GEVNet). This allows us to model spatially extended nonlinear interactions within a given receptive field while still maintaining equivariance to global isometries. We prove theoretical results regarding the equivariance and construction of higher order gauge equivariant convolutions. Then, we empirically demonstrate the parameter efficiency of our model, first on computer vision benchmark data (e.g. spherical MNIST), and then in combination with a convolutional kernel network (CKN) on neuroimaging data. In the neuroimaging data experiments, the resulting two-part architecture (CKN + GEVNet) is used to automatically discriminate between patients with Lewy Body Disease (DLB), Alzheimer's Disease (AD) and Parkinson's Disease (PD) from diffusion magnetic resonance images (dMRI). The GEVNet extracts micro-architectural features within each voxel, while the CKN extracts macro-architectural features across voxels. This compound architecture is uniquely poised to exploit the intra- and inter-voxel information contained in the dMRI data, leading to improved performance over the classification results obtained from either of the individual components.

cs.CV↗
arXiv2023

Cost Design in Atomic Routing Games

An atomic routing game is a multiplayer game on a directed graph. Each player in the game chooses a path -- a sequence of links that connect its origin node to its destination node -- with the lowest cost, where the cost of each link is a function of all players' choices. We develop a novel numerical method to design the link cost function in atomic routing games such that the players' choices at the Nash equilibrium minimize a given smooth performance function. This method first approximates the nonsmooth Nash equilibrium conditions with smooth ones, then iteratively improves the link cost function via implicit differentiation. We demonstrate the application of this method to atomic routing games that model noncooperative agents navigating in grid worlds.

cs.GT↗
arXiv2023

ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval

With the development of large language models (LLMs), zero-shot learning has attracted much attention for various NLP tasks. Different from prior works that generate training data with billion-scale natural language generation (NLG) models, we propose a retrieval-enhanced framework to create training data from a general-domain unlabeled corpus. To realize this, we first conduct contrastive pretraining to learn an unsupervised dense retriever for extracting the most relevant documents using class-descriptive verbalizers. We then further propose two simple strategies, namely Verbalizer Augmentation with Demonstrations and Self-consistency Guided Filtering to improve the topic coverage of the dataset while removing noisy examples. Experiments on nine datasets demonstrate that REGEN achieves 4.3% gain over the strongest baselines and saves around 70% of the time compared to baselines using large NLG models. Besides, REGEN can be naturally integrated with recently proposed large language models to boost performance.

cs.CL↗
arXiv2023

Dynamic Routing in Stochastic Urban Air Mobility Networks: A Markov Decision Process Approach

Urban air mobility (UAM) is an emerging concept in short-range aviation transportation, where the aircraft will take off, land, and charge their batteries at a set of vertistops, and travel only through a set of flight corridors connecting these vertistops. We study the problem of routing an electric aircraft from its origin vertistop to its destination vertistop with the minimal expected total travel time. We first introduce a UAM network model that accounts for the limited battery capacity of aircraft, stochastic travel times of flight corridors, stochastic queueing delays, and a limited number of battery-charging stations at vertistops. Based on this model, we provide a sufficient condition for the existence of a routing strategy that avoids battery exhaustion. Furthermore, we show how to compute such a strategy by computing the optimal policy in a Markov decision process, a mathematical framework for decision-making in a stochastic dynamic environment. We illustrate our results using a case study with 29 vertistops and 137 flight corridors.

eess.SY↗
arXiv2023

Cold-Start Data Selection for Few-shot Language Model Fine-tuning: A Prompt-Based Uncertainty Propagation Approach

Large Language Models have demonstrated remarkable few-shot performance, but the performance can be sensitive to the selection of few-shot instances. We propose PATRON, a new method that uses prompt-based uncertainty estimation for data selection for pre-trained language model fine-tuning under cold-start scenarios, i.e., no initial labeled data are available. In PATRON, we design (1) a prompt-based uncertainty propagation approach to estimate the importance of data points and (2) a partition-then-rewrite (PTR) strategy to promote sample diversity when querying for annotations. Experiments on six text classification datasets show that PATRON outperforms the strongest cold-start data selection baselines by up to 6.9%. Besides, with 128 labels only, PATRON achieves 91.0% and 92.1% of the fully supervised performance based on vanilla fine-tuning and prompt-based learning respectively. Our implementation of PATRON is available at \url{https://github.com/yueyu1030/Patron}.

cs.CL↗