arXiv subjects

Yu Yang

Publications and source records attributed to Yu Yang.

At least 163 records · Page 9Linked to original sources

arXiv2024

Quantifying angular distributions in multinucleon transfer reactions with a semi-classical method

The multinucleon transfer (MNT) process in low-energy heavy ion collisions can be utilized to produce unknown nuclei far beyond the stability line. However, the reaction products exhibit broad angular and energy distributions, which could lower the experimental detection efficiency. We present a classical approach that employs a parameterized angular distribution to describe the complex issue. By analyzing limited experimental data on angular distribution, we proposed a three-parameter formula to calculate the angular distribution and identified the dependencies of the parameters. We also discuss the sensitivity of these parameters within this method. A comprehensive comparison with microscopic models and experimental data across a wide range of conditions is conducted. The proposed formula offers an efficient and straightforward way to determine the angular distribution of MNT products.

nucl-th↗

arXiv2024

Brevity is the soul of wit: Pruning long files for code generation

Data curation is commonly considered a "secret-sauce" for LLM training, with higher quality data usually leading to better LLM performance. Given the scale of internet-scraped corpora, data pruning has become a larger and larger focus. Specifically, many have shown that de-duplicating data, or sub-selecting higher quality data, can lead to efficiency or performance improvements. Generally, three types of methods are used to filter internet-scale corpora: embedding-based, heuristic-based, and classifier-based. In this work, we contrast the former two in the domain of finetuning LLMs for code generation. We find that embedding-based methods are often confounded by length, and that a simple heuristic--pruning long files--outperforms other methods in compute-limited regimes. Our method can yield up to a 2x efficiency benefit in training (while matching performance) or a 3.5% absolute performance improvement on HumanEval (while matching compute). However, we find that perplexity on held-out long files can increase, begging the question of whether optimizing data mixtures for common coding benchmarks (HumanEval, MBPP) actually best serves downstream use cases. Overall, we hope our work builds useful intuitions about code data (specifically, the low quality of extremely long code files) provides a compelling heuristic-based method for data pruning, and brings to light questions in how we evaluate code generation models.

cs.CL↗

arXiv2024

Self-assessment, Exhibition, and Recognition: a Review of Personality in Large Language Models

As large language models (LLMs) appear to behave increasingly human-like in text-based interactions, more and more researchers become interested in investigating personality in LLMs. However, the diversity of psychological personality research and the rapid development of LLMs have led to a broad yet fragmented landscape of studies in this interdisciplinary field. Extensive studies across different research focuses, different personality psychometrics, and different LLMs make it challenging to have a holistic overview and further pose difficulties in applying findings to real-world applications. In this paper, we present a comprehensive review by categorizing current studies into three research problems: self-assessment, exhibition, and recognition, based on the intrinsic characteristics and external manifestations of personality in LLMs. For each problem, we provide a thorough analysis and conduct in-depth comparisons of their corresponding solutions. Besides, we summarize research findings and open challenges from current studies and further discuss their underlying causes. We also collect extensive publicly available resources to facilitate interested researchers and developers. Lastly, we discuss the potential future research directions and application scenarios. Our paper is the first comprehensive survey of up-to-date literature on personality in LLMs. By presenting a clear taxonomy, in-depth analysis, promising future directions, and extensive resource collections, we aim to provide a better understanding and facilitate further advancements in this emerging field.

cs.CL↗

arXiv2024

AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies

We present a comprehensive AI risk taxonomy derived from eight government policies from the European Union, United States, and China and 16 company policies worldwide, making a significant step towards establishing a unified language for generative AI safety evaluation. We identify 314 unique risk categories organized into a four-tiered taxonomy. At the highest level, this taxonomy encompasses System & Operational Risks, Content Safety Risks, Societal Risks, and Legal & Rights Risks. The taxonomy establishes connections between various descriptions and approaches to risk, highlighting the overlaps and discrepancies between public and private sector conceptions of risk. By providing this unified framework, we aim to advance AI safety through information sharing across sectors and the promotion of best practices in risk mitigation for generative AI models and systems.

cs.CY↗

arXiv2024

Urban-Focused Multi-Task Offline Reinforcement Learning with Contrastive Data Sharing

Enhancing diverse human decision-making processes in an urban environment is a critical issue across various applications, including ride-sharing vehicle dispatching, public transportation management, and autonomous driving. Offline reinforcement learning (RL) is a promising approach to learn and optimize human urban strategies (or policies) from pre-collected human-generated spatial-temporal urban data. However, standard offline RL faces two significant challenges: (1) data scarcity and data heterogeneity, and (2) distributional shift. In this paper, we introduce MODA -- a Multi-Task Offline Reinforcement Learning with Contrastive Data Sharing approach. MODA addresses the challenges of data scarcity and heterogeneity in a multi-task urban setting through Contrastive Data Sharing among tasks. This technique involves extracting latent representations of human behaviors by contrasting positive and negative data pairs. It then shares data presenting similar representations with the target task, facilitating data augmentation for each task. Moreover, MODA develops a novel model-based multi-task offline RL algorithm. This algorithm constructs a robust Markov Decision Process (MDP) by integrating a dynamics model with a Generative Adversarial Network (GAN). Once the robust MDP is established, any online RL or planning algorithm can be applied. Extensive experiments conducted in a real-world multi-task urban setting validate the effectiveness of MODA. The results demonstrate that MODA exhibits significant improvements compared to state-of-the-art baselines, showcasing its capability in advancing urban decision-making processes. We also made our code available to the research community.

cs.LG↗

arXiv2024

Toward Structure Fairness in Dynamic Graph Embedding: A Trend-aware Dual Debiasing Approach

Recent studies successfully learned static graph embeddings that are structurally fair by preventing the effectiveness disparity of high- and low-degree vertex groups in downstream graph mining tasks. However, achieving structure fairness in dynamic graph embedding remains an open problem. Neglecting degree changes in dynamic graphs will significantly impair embedding effectiveness without notably improving structure fairness. This is because the embedding performance of high-degree and low-to-high-degree vertices will significantly drop close to the generally poorer embedding performance of most slightly changed vertices in the long-tail part of the power-law distribution. We first identify biased structural evolutions in a dynamic graph based on the evolving trend of vertex degree and then propose FairDGE, the first structurally Fair Dynamic Graph Embedding algorithm. FairDGE learns biased structural evolutions by jointly embedding the connection changes among vertices and the long-short-term evolutionary trend of vertex degrees. Furthermore, a novel dual debiasing approach is devised to encode fair embeddings contrastively, customizing debiasing strategies for different biased structural evolutions. This innovative debiasing strategy breaks the effectiveness bottleneck of embeddings without notable fairness loss. Extensive experiments demonstrate that FairDGE achieves simultaneous improvement in the effectiveness and fairness of embeddings.

cs.LG↗

arXiv2024

More Efficient Randomized Exploration for Reinforcement Learning via Approximate Sampling

Thompson sampling (TS) is one of the most popular exploration techniques in reinforcement learning (RL). However, most TS algorithms with theoretical guarantees are difficult to implement and not generalizable to Deep RL. While the emerging approximate sampling-based exploration schemes are promising, most existing algorithms are specific to linear Markov Decision Processes (MDP) with suboptimal regret bounds, or only use the most basic samplers such as Langevin Monte Carlo. In this work, we propose an algorithmic framework that incorporates different approximate sampling methods with the recently proposed Feel-Good Thompson Sampling (FGTS) approach (Zhang, 2022; Dann et al., 2021), which was previously known to be computationally intractable in general. When applied to linear MDPs, our regret analysis yields the best known dependency of regret on dimensionality, surpassing existing randomized algorithms. Additionally, we provide explicit sampling complexity for each employed sampler. Empirically, we show that in tasks where deep exploration is necessary, our proposed algorithms that combine FGTS and approximate sampling perform significantly better compared to other strong baselines. On several challenging games from the Atari 57 suite, our algorithms achieve performance that is either better than or on par with other strong baselines from the deep RL literature.

cs.LG↗

arXiv2024

A mechanical qubit

Strong nonlinear interactions between quantized excitations are an important resource for quantum technologies based on bosonic oscillator modes. However, most electromagnetic and mechanical nonlinearities arising from intrinsic material properties are far too weak compared to dissipation in the system to allow for nonlinear effects to be observed on the single-quantum level. To overcome this limitation, electromagnetic resonators in both the optical and microwave frequency regimes have been coupled to other strongly nonlinear quantum systems such as atoms and superconducting qubits, allowing for the demonstration of effects such as photon blockade and coherent quantum protocols using the Kerr effect. Here, we demonstrate the realization of the single-phonon nonlinear regime in a solid-state mechanical system. The single-phonon anharmonicity in our system exceeds the decoherence rate by a factor of 6.8, allowing us to use the lowest two energy levels of the resonator as a mechanical qubit, for which we show initialization, readout, and a complete set of direct single qubit gates. Our work adds another unique capability to a powerful quantum acoustics platform for quantum simulations, sensing, and information processing.

quant-ph↗

arXiv2024

Moving Sampling Physics-informed Neural Networks induced by Moving Mesh PDE

In this work, we propose an end-to-end adaptive sampling neural network (MMPDE-Net) based on the moving mesh method, which can adaptively generate new sampling points by solving the moving mesh PDE. This model focuses on improving the quality of sampling points generation. Moreover, we develop an iterative algorithm based on MMPDE-Net, which makes the sampling points more precise and controllable. Since MMPDE-Net is a framework independent of the deep learning solver, we combine it with physics-informed neural networks (PINN) to propose moving sampling PINN (MS-PINN) and demonstrate its effectiveness by error analysis under some assumptions. Finally, we demonstrate the performance improvement of MS-PINN compared to PINN through numerical experiments of four typical examples, which numerically verify the effectiveness of our method.

math.NA↗

arXiv2024

On the Stability of Expressive Positional Encodings for Graphs

Designing effective positional encodings for graphs is key to building powerful graph transformers and enhancing message-passing graph neural networks. Although widespread, using Laplacian eigenvectors as positional encodings faces two fundamental challenges: (1) \emph{Non-uniqueness}: there are many different eigendecompositions of the same Laplacian, and (2) \emph{Instability}: small perturbations to the Laplacian could result in completely different eigenspaces, leading to unpredictable changes in positional encoding. Despite many attempts to address non-uniqueness, most methods overlook stability, leading to poor generalization on unseen graph structures. We identify the cause of instability to be a ``hard partition'' of eigenspaces. Hence, we introduce Stable and Expressive Positional Encodings (SPE), an architecture for processing eigenvectors that uses eigenvalues to ``softly partition'' eigenspaces. SPE is the first architecture that is (1) provably stable, and (2) universally expressive for basis invariant functions whilst respecting all symmetries of eigenvectors. Besides guaranteed stability, we prove that SPE is at least as expressive as existing methods, and highly capable of counting graph structures. Finally, we evaluate the effectiveness of our method on molecular property prediction, and out-of-distribution generalization tasks, finding improved generalization compared to existing positional encoding methods. Our code is available at \url{https://github.com/Graph-COM/SPE}.

cs.LG↗

arXiv2024

Few-shot Adaptation to Distribution Shifts By Mixing Source and Target Embeddings

Pretrained machine learning models need to be adapted to distribution shifts when deployed in new target environments. When obtaining labeled data from the target distribution is expensive, few-shot adaptation with only a few examples from the target distribution becomes essential. In this work, we propose MixPro, a lightweight and highly data-efficient approach for few-shot adaptation. MixPro first generates a relatively large dataset by mixing (linearly combining) pre-trained embeddings of large source data with those of the few target examples. This process preserves important features of both source and target distributions, while mitigating the specific noise in the small target data. Then, it trains a linear classifier on the mixed embeddings to effectively adapts the model to the target distribution without overfitting the small target data. Theoretically, we demonstrate the advantages of MixPro over previous methods. Our experiments, conducted across various model architectures on 8 datasets featuring different types of distribution shifts, reveal that MixPro can outperform baselines by up to 7\%, with only 2-4 target examples.

cs.LG↗

arXiv2024

LLaMA-Reg: Using LLaMA 2 for Unsupervised Medical Image Registration

Medical image registration is an essential topic in medical image analysis. In this paper, we propose a method for medical image registration using a pretrained large language model. We find that using the pretrained large language model to encode deep features of the medical images in the registration model can effectively improve image registration accuracy, indicating the great potential of the large language model in medical image registration tasks. We use dual encoders to perform deep feature extraction on image pairs and then input the features into the pretrained large language model. To adapt the large language model to our registration task, the weights of the large language model are frozen in the registration model, and an adapter is utilized to fine-tune the large language model, which aims at (a) mapping the visual tokens to the language space before the large language model computing, (b) project the modeled language tokens output from the large language model to the visual space. Our method combines output features from the fine-tuned large language model with the features output from each encoder layer to gradually generate the deformation fields required for registration in the decoder. To demonstrate the effectiveness of the large prediction model in registration tasks, we conducted experiments on knee and brain MRI and achieved state-of-the-art results.

cs.CV↗

arXiv2024

Is a direct numerical simulation (DNS) of Navier-Stokes equations with small enough grid spacing and time-step definitely reliable/correct?

Traditionally, results given by the direct numerical simulation (DNS) of Navier-Stokes equations are widely regarded as reliable benchmark solutions of turbulence, as long as grid spacing is fine enough (i.e. less than the minimum Kolmogorov scale) and time-step is small enough, say, satisfying the Courant-Friedrichs-Lewy condition. Is this really true? In this paper a two-dimensional sustained turbulent Kolmogorov flow is investigated numerically by the two numerical methods with detailed comparisons: one is the traditional `direct numerical simulation' (DNS), the other is the `clean numerical simulation' (CNS). The results given by DNS are a kind of mixture of the false numerical noise and the true physical solution, which however are mostly at the same order of magnitude due to the butterfly-effect of chaos. On the contrary, the false numerical noise of the results given by CNS is much smaller than the true physical solution of turbulence in a long enough interval of time so that a CNS result is very close to the true physical solution and thus can be used as a benchmark solution. It is found that numerical noise as a kind of artificial tiny disturbances can lead to huge deviations at large scale on the two-dimensional Kolmogorov turbulence, not only quantitatively (even in statistics) but also qualitatively (such as symmetry of flow). Thus, fine enough spatial grid spacing with small enough time-step alone cannot guarantee the validity of the DNS: it is only a necessary condition but not sufficient. This finding might challenge some assumptions in investigation of turbulence. So, DNS results of a few sustained turbulent flows might have huge deviations on both of small and large scales from the true solution of Navier-Stokes equations even in statistics. Hopefully, CNS as a new tool to investigate turbulent flows more accurately than DNS could bring us some new discoveries.

physics.flu-dyn↗

arXiv2024

Affective-NLI: Towards Accurate and Interpretable Personality Recognition in Conversation

Personality Recognition in Conversation (PRC) aims to identify the personality traits of speakers through textual dialogue content. It is essential for providing personalized services in various applications of Human-Computer Interaction (HCI), such as AI-based mental therapy and companion robots for the elderly. Most recent studies analyze the dialog content for personality classification yet overlook two major concerns that hinder their performance. First, crucial implicit factors contained in conversation, such as emotions that reflect the speakers' personalities are ignored. Second, only focusing on the input dialog content disregards the semantic understanding of personality itself, which reduces the interpretability of the results. In this paper, we propose Affective Natural Language Inference (Affective-NLI) for accurate and interpretable PRC. To utilize affectivity within dialog content for accurate personality recognition, we fine-tuned a pre-trained language model specifically for emotion recognition in conversations, facilitating real-time affective annotations for utterances. For interpretability of recognition results, we formulate personality recognition as an NLI problem by determining whether the textual description of personality labels is entailed by the dialog content. Extensive experiments on two daily conversation datasets suggest that Affective-NLI significantly outperforms (by 6%-7%) state-of-the-art approaches. Additionally, our Flow experiment demonstrates that Affective-NLI can accurately recognize the speaker's personality in the early stages of conversations by surpassing state-of-the-art methods with 22%-34%.

cs.CL↗

arXiv2024

Sieve: Multimodal Dataset Pruning Using Image Captioning Models

Vision-Language Models (VLMs) are pretrained on large, diverse, and noisy web-crawled datasets. This underscores the critical need for dataset pruning, as the quality of these datasets is strongly correlated with the performance of VLMs on downstream tasks. Using CLIPScore from a pretrained model to only train models using highly-aligned samples is one of the most successful methods for pruning. We argue that this approach suffers from multiple limitations including: false positives and negatives due to CLIP's pretraining on noisy labels. We propose a pruning signal, Sieve, that employs synthetic captions generated by image-captioning models pretrained on small, diverse, and well-aligned image-text pairs to evaluate the alignment of noisy image-text pairs. To bridge the gap between the limited diversity of generated captions and the high diversity of alternative text (alt-text), we estimate the semantic textual similarity in the embedding space of a language model pretrained on unlabeled text corpus. Using DataComp, a multimodal dataset filtering benchmark, when evaluating on 38 downstream tasks, our pruning approach, surpasses CLIPScore by 2.6\% and 1.7\% on medium and large scale respectively. In addition, on retrieval tasks, Sieve leads to a significant improvement of 2.7% and 4.5% on medium and large scale respectively.

cs.CV↗

arXiv2024

Identifying Spurious Biases Early in Training through the Lens of Simplicity Bias

Neural networks trained with (stochastic) gradient descent have an inductive bias towards learning simpler solutions. This makes them highly prone to learning spurious correlations in the training data, that may not hold at test time. In this work, we provide the first theoretical analysis of the effect of simplicity bias on learning spurious correlations. Notably, we show that examples with spurious features are provably separable based on the model's output early in training. We further illustrate that if spurious features have a small enough noise-to-signal ratio, the network's output on the majority of examples is almost exclusively determined by the spurious features, leading to poor worst-group test accuracy. Finally, we propose SPARE, which identifies spurious correlations early in training and utilizes importance sampling to alleviate their effect. Empirically, we demonstrate that SPARE outperforms state-of-the-art methods by up to 21.1% in worst-group accuracy, while being up to 12x faster. We also show that SPARE is a highly effective but lightweight method to discover spurious correlations.

cs.LG↗

arXiv2024

Attention Is Not the Only Choice: Counterfactual Reasoning for Path-Based Explainable Recommendation

Compared with only pursuing recommendation accuracy, the explainability of a recommendation model has drawn more attention in recent years. Many graph-based recommendations resort to informative paths with the attention mechanism for the explanation. Unfortunately, these attention weights are intentionally designed for model accuracy but not explainability. Recently, some researchers have started to question attention-based explainability because the attention weights are unstable for different reproductions, and they may not always align with human intuition. Inspired by the counterfactual reasoning from causality learning theory, we propose a novel explainable framework targeting path-based recommendations, wherein the explainable weights of paths are learned to replace attention weights. Specifically, we design two counterfactual reasoning algorithms from both path representation and path topological structure perspectives. Moreover, unlike traditional case studies, we also propose a package of explainability evaluation solutions with both qualitative and quantitative methods. We conduct extensive experiments on three real-world datasets, the results of which further demonstrate the effectiveness and reliability of our method.

cs.IR↗

arXiv2024

Phonon-lithium ion interactions: A case study of LiM(SeO3)2

Li ion diffusion is fundamentally a thermally activated ion hopping process. Recently, soft lattice, anharmonic phonon and paddlewheel mechanism have been proposed to potentially benefit the ion transport, while the understanding of vibrational couplings of mobile ion and anions is still limited but essential. Herein, we access the ionic conductivity, the stability and the lattice dynamics in LiM(SeO3)2 (M =Al, Ga, In, Sc, Y, and La) with two types of oxygen anions within LiO4 polyhedron, namely edge-shared and corner-shared, the prototype of which, LiGa(SeO3)2, has been experimentally synthesized. We studied in detail the anharmonic and harmonic phonon interactions, as well as couplings between vibrations of edge-bonded or corner-bonded anions in Li polyanions and Li ion diffusion. As M changing from Sc to La, anharmonic phonons increase alongside reduced activation energy for Li diffusion. Phonon modes involving edge-bonded oxygen anions contribute more to Li migration than corner-bonded oxygen anions, owing to greater atomic interactions between Li ions and edge-bonded anions. Thus, rather than the overall lattice softness, attentions shall be paid to reduce the frequency of the critical phonons contributing to Li ion diffusions as well as to increase the anharmonicity, for the design of Li ion superionic conductors for all-solid-state-batteries.

cond-mat.mtrl-sci↗