arXiv subjects

Yan Liu

Publications and source records attributed to Yan Liu.

At least 235 records · Page 13Linked to original sources

arXiv2024

Microscopic Insights into Fatigue Mechanism in Wurtzite Ferroelectric Al$_{0.65}$Sc$_{0.35}$N: Oxygen Infiltration Enabled Grain Amorphization Spanning Boundary to Bulk

For the first time, the fatigue behavior involving external oxygen in highly Sc-doped AlN ferroelectric film was observed using transmission electron microscope techniques. Despite increasing the Sc composition in AlScN film contributes to reducing the device operation voltage, the inherent affinity of Sc for oxygen introduces instability in device performance. In this study, oxygen incorporation at top electrode edges and grain boundaries accompanied with an increase in current leakage and the disappearance of ferroelectric properties, was observed in nanoscale after long-term field cycling. This observation indicates the emergence of non-ferroelectric and even amorphous states. This presented work revealed solid experimental evidence of an oxygen-involved fatigue mechanism, providing valuable insights into the physical nature of the ferroelectric properties of AlScN films.

cond-mat.mtrl-sci↗

arXiv2024

Interpretable Detection of Out-of-Context Misinformation with Neural-Symbolic-Enhanced Large Multimodal Model

Recent years have witnessed the sustained evolution of misinformation that aims at manipulating public opinions. Unlike traditional rumors or fake news editors who mainly rely on generated and/or counterfeited images, text and videos, current misinformation creators now more tend to use out-of-context multimedia contents (e.g. mismatched images and captions) to deceive the public and fake news detection systems. This new type of misinformation increases the difficulty of not only detection but also clarification, because every individual modality is close enough to true information. To address this challenge, in this paper we explore how to achieve interpretable cross-modal de-contextualization detection that simultaneously identifies the mismatched pairs and the cross-modal contradictions, which is helpful for fact-check websites to document clarifications. The proposed model first symbolically disassembles the text-modality information to a set of fact queries based on the Abstract Meaning Representation of the caption and then forwards the query-image pairs into a pre-trained large vision-language model select the ``evidences" that are helpful for us to detect misinformation. Extensive experiments indicate that the proposed methodology can provide us with much more interpretable predictions while maintaining the accuracy same as the state-of-the-art model on this task.

cs.CL↗

arXiv2024

TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting

The past decade has witnessed significant advances in time series modeling with deep learning. While achieving state-of-the-art results, the best-performing architectures vary highly across applications and domains. Meanwhile, for natural language processing, the Generative Pre-trained Transformer (GPT) has demonstrated impressive performance via training one general-purpose model across various textual datasets. It is intriguing to explore whether GPT-type architectures can be effective for time series, capturing the intrinsic dynamic attributes and leading to significant accuracy improvements. In this paper, we propose a novel framework, TEMPO, that can effectively learn time series representations. We focus on utilizing two essential inductive biases of the time series task for pre-trained models: (i) decomposition of the complex interaction between trend, seasonal and residual components; and (ii) introducing the design of prompts to facilitate distribution adaptation in different types of time series. TEMPO expands the capability for dynamically modeling real-world temporal phenomena from data within diverse domains. Our experiments demonstrate the superior performance of TEMPO over state-of-the-art methods on zero shot setting for a number of time series benchmark datasets. This performance gain is observed not only in scenarios involving previously unseen datasets but also in scenarios with multi-modal inputs. This compelling finding highlights TEMPO's potential to constitute a foundational model-building framework.

cs.LG↗

arXiv2024

Chiral magnetic waves in strongly coupled Weyl semimetals

Propagating chiral magnetic waves (CMW) are expected to exist in chiral plasmas due to the interplay between the chiral magnetic and chiral separation effects induced by the presence of a chiral anomaly. Unfortunately, it was pointed out that, because of the effects of electric conductivity and dissipation, CMW are overdamped and therefore their signatures are unlikely to be seen in heavy-ion collision experiments and in the quark gluon plasma. Nonetheless, the chiral anomaly plays a fundamental role in Weyl semimetals and their anomalous transport properties as well. Hence, CMW could be potentially observed in topological semimetals using table-top experiments. By using a holographic model for strongly coupled Weyl semimetals, we investigate in detail the nature of CMW in presence of Coulomb interactions and axial charge relaxation and estimate whether, and in which regimes, CMW could be observed as underdamped collective excitations in topological materials.

hep-th↗

arXiv2024

How accurately can quantitative imaging methods be ranked without ground truth: An upper bound on no-gold-standard evaluation

Objective evaluation of quantitative imaging (QI) methods with patient data, while important, is typically hindered by the lack of gold standards. To address this challenge, no-gold-standard evaluation (NGSE) techniques have been proposed. These techniques have demonstrated efficacy in accurately ranking QI methods without access to gold standards. The development of NGSE methods has raised an important question: how accurately can QI methods be ranked without ground truth. To answer this question, we propose a Cramer-Rao bound (CRB)-based framework that quantifies the upper bound in ranking QI methods without any ground truth. We present the application of this framework in guiding the use of a well-known NGSE technique, namely the regression-without-truth (RWT) technique. Our results show the utility of this framework in quantifying the performance of this NGSE technique for different patient numbers. These results provide motivation towards studying other applications of this upper bound.

physics.med-ph↗

arXiv2024

XAIport: A Service Framework for the Early Adoption of XAI in AI Model Development

In this study, we propose the early adoption of Explainable AI (XAI) with a focus on three properties: Quality of explanation, the explanation summaries should be consistent across multiple XAI methods; Architectural Compatibility, for effective integration in XAI, the architecture styles of both the XAI methods and the models to be explained must be compatible with the framework; Configurable operations, XAI explanations are operable, akin to machine learning operations. Thus, an explanation for AI models should be reproducible and tractable to be trustworthy. We present XAIport, a framework of XAI microservices encapsulated into Open APIs to deliver early explanations as observation for learning model quality assurance. XAIport enables configurable XAI operations along with machine learning development. We quantify the operational costs of incorporating XAI with three cloud computer vision services on Microsoft Azure Cognitive Services, Google Cloud Vertex AI, and Amazon Rekognition. Our findings show comparable operational costs between XAI and traditional machine learning, with XAIport significantly improving both cloud AI model performance and explanation stability.

cs.AI↗

arXiv2024

LoRAMoE: Alleviate World Knowledge Forgetting in Large Language Models via MoE-Style Plugin

Supervised fine-tuning (SFT) is a crucial step for large language models (LLMs), enabling them to align with human instructions and enhance their capabilities in downstream tasks. Increasing instruction data substantially is a direct solution to align the model with a broader range of downstream tasks or notably improve its performance on a specific task. However, we find that large-scale increases in instruction data can damage the world knowledge previously stored in LLMs. To address this challenge, we propose LoRAMoE, a novelty framework that introduces several low-rank adapters (LoRA) and integrates them by using a router network, like a plugin version of Mixture of Experts (MoE). It freezes the backbone model and forces a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks, to alleviate world knowledge-edge forgetting. Experimental results show that, as the instruction data increases, LoRAMoE can significantly improve the ability to process downstream tasks, while maintaining the world knowledge stored in the LLM.

cs.CL↗

arXiv2024

Resonant Quantum Magnetodielectric Effect in Multiferroic Metal-Organic Framework [CH3NH3]Co(HCOO)3

We report the observation of both resonant quantum tunneling of magnetization (RQTM) and resonant quantum magnetodielectric (RQMD) effect in the perovskite multiferroic metal-organic framework [CH3NH3]Co(HCOO)3. An intrinsic magnetic phase separation emerges at low temperatures due to hydrogen-bond-modified long range super-exchange interaction, leading to the coexistence of canted antiferromagnetic order and single-ion magnet. Subsequently, a stair-shaped magnetic hysteresis loop along the [101] direction characterizing the RQTM appears below the magnetic blocking temperature. More interestingly, the magnetic field dependence of dielectric permittivity exhibits pronounced negative peaks at the critical fields corresponding to the RQTM, a phenomenon termed the RQMD effect which enables electrical detection of the RQTM. These intriguing properties make the multiferroic metal-organic framework a promising candidate for solid-state quantum computing.

cond-mat.mtrl-sci↗

arXiv2024

Validity of energy conditions of matter in traversable wormholes under the $f(Q)$ modified gravity theory

In the framework of the theory of general relativity, in order to obtain stable traversable wormholes, matter needs to violate the null energy condition. It is well known that the violation of the energy condition (EC) of matter leads to various physical problems. To address this issue, researchers have turned their attention to exploring modified theories of gravity, aiming to avoid the violation of ECs by introducing geometric terms. In this paper, within the framework of the $f(Q)$ modified gravitational theory, we investigate the effectiveness of ECs for matter in traversable wormholes. We examine the compliance of four types of energy conditions (weak energy condition, null energy condition, dominant energy condition, and strong energy condition) in the model by selecting a power-law model for $f(Q)$ and considering different shape functions $b(r)$. Our study reveals that for traversable wormholes realized through the $f(Q)$ modified gravity theory using the power-law model $f(Q)=a(-Q)^n$, all four types of ECs for matter can be satisfied. There is no need to introduce exotic matter (violating the null energy condition) or special matter (violating other energy conditions) artificially in the physics of wormholes.

gr-qc↗

arXiv2024

COSTAR: Improved Temporal Counterfactual Estimation with Self-Supervised Learning

Estimation of temporal counterfactual outcomes from observed history is crucial for decision-making in many domains such as healthcare and e-commerce, particularly when randomized controlled trials (RCTs) suffer from high cost or impracticality. For real-world datasets, modeling time-dependent confounders is challenging due to complex dynamics, long-range dependencies and both past treatments and covariates affecting the future outcomes. In this paper, we introduce Counterfactual Self-Supervised Transformer (COSTAR), a novel approach that integrates self-supervised learning for improved historical representations. We propose a component-wise contrastive loss tailored for temporal treatment outcome observations and explain its effectiveness from the view of unsupervised domain adaptation. COSTAR yields superior performance in estimation accuracy and generalization to out-of-distribution data compared to existing models, as validated by empirical results on both synthetic and real-world datasets.

cs.LG↗

arXiv2024

A Survey of Privacy Threats and Defense in Vertical Federated Learning: From Model Life Cycle Perspective

Vertical Federated Learning (VFL) is a federated learning paradigm where multiple participants, who share the same set of samples but hold different features, jointly train machine learning models. Although VFL enables collaborative machine learning without sharing raw data, it is still susceptible to various privacy threats. In this paper, we conduct the first comprehensive survey of the state-of-the-art in privacy attacks and defenses in VFL. We provide taxonomies for both attacks and defenses, based on their characterizations, and discuss open challenges and future research directions. Specifically, our discussion is structured around the model's life cycle, by delving into the privacy threats encountered during different stages of machine learning and their corresponding countermeasures. This survey not only serves as a resource for the research community but also offers clear guidance and actionable insights for practitioners to safeguard data privacy throughout the model's life cycle.

cs.CR↗

arXiv2024

StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback

The advancement of large language models (LLMs) has significantly propelled the field of code generation. Previous work integrated reinforcement learning (RL) with compiler feedback for exploring the output space of LLMs to enhance code generation quality. However, the lengthy code generated by LLMs in response to complex human requirements makes RL exploration a challenge. Also, since the unit tests may not cover the complicated code, optimizing LLMs by using these unexecuted code snippets is ineffective. To tackle these challenges, we introduce StepCoder, a novel RL framework for code generation, consisting of two main components: CCCS addresses the exploration challenge by breaking the long sequences code generation task into a Curriculum of Code Completion Subtasks, while FGO only optimizes the model by masking the unexecuted code segments to provide Fine-Grained Optimization. In addition, we furthermore construct the APPS+ dataset for RL training, which is manually verified to ensure the correctness of unit tests. Experimental results show that our method improves the ability to explore the output space and outperforms state-of-the-art approaches in corresponding benchmarks. Our dataset APPS+ and StepCoder are available online.

cs.SE↗

arXiv2024

The Analytical Solutions of Equatorial Geodesic Motion in Kerr Spacetime

The study of Kerr geodesics has a long history, particularly for those occurring within the equatorial plane, which is generally well-understood. However, upon comparison with the classification introduced by one of us https://journals.aps.org/prd/abstract/10.1103/PhysRevD.105.024075 , it becomes apparent that certain classes of geodesics, such as trapped orbits, are still lacking analytical solutions. Thus, in this study, we provide explicit analytical solutions for equatorial timelike geodesics in Kerr spacetime, including solutions of trapped orbits, which capture the characteristics of special geodesics, such as the positions and conserved quantities of circular orbits, bound orbits, and deflecting orbits. Specifically, we determine the precise location at which retrograde orbits undergo a transition from counter-rotating to prograde motion due to the strong gravitational effects near the rotating black hole. Interestingly, we observe that for orbits with negative energy, the trajectory remains prograde despite the negative angular momentum. Furthermore, we investigate the intriguing phenomenon of deflecting orbits exhibiting an increased number of revolutions around the black hole as the turning point approaches the turning point of the trapped orbit. Additionally, we find that only prograde marginal deflecting geodesics are capable of traversing through the ergoregion. In summary, our findings present explicit solutions for equatorial timelike geodesics and offer insights into the dynamics of particle motion in the vicinity of a rotating black hole.

gr-qc↗

arXiv2024

APPT: Boosting Automated Patch Correctness Prediction via Fine-tuning Pre-trained Models

Automated program repair (APR) aims to fix software bugs automatically without human debugging efforts and plays a crucial role in software development and maintenance. Despite promising, APR is still challenged by a long-standing overfitting problem (i.e., the generated patch is plausible but overfitting). Various techniques have thus been proposed to address the overfitting problem. Recently, researchers have employed BERT to extract code features, which are then used to train a classifier for patch correctness prediction. However, BERT is restricted to feature extraction for classifier training without benefiting from the training process, potentially generating sub-optimal vector representations for patched code snippets. In this paper, we propose APPT, a pre-trained model-based automated patch correctness assessment technique by both pre-training and fine-tuning. APPT adopts a pre-trained model as the encoder stack, followed by an LSTM stack and a deep learning classifier. More importantly, the pre-trained model is fine-tuned in conjunction with other components as a whole pipeline to fully adapt it specifically for reasoning about patch correctness. We conduct an extensive experiment on 1,183 Defects4J patches and the experimental results show that APPT achieves prediction accuracy of 79.7% and recall of 83.2%, outperforming CACHE by 4.3% and 6.7%. Our additional investigation on 49,694 real-world patches shows that APPT achieves the optimum performance compared with existing representation learning techniques. We further investigate the impact of each component and find that they all positively contribute to APPT, e.g., the fine-tuning process and the LSTM stack increase F1-score by 10.22% and 4.11%, respectively. We also prove that adopting advanced pre-trained models can further provide substantial advancement, highlighting the generalizability of APPT.

cs.SE↗

arXiv2024

Internal structure of hairy rotating black holes in three dimensions

We construct hairy rotating black hole solutions in three dimensional Einstein gravity coupled to a complex scalar field. When we turn on a real and uniform source on the dual CFT, the black hole is stationary with two Killing vectors and we show that there is no inner horizon for the black hole and the system evolves smoothly into a Kasner universe. When we turn on a complex and periodic driving source on the dual CFT with a phase velocity equal to the angular velocity of the black hole, we have a time-dependent black hole with only one Killing vector. We show that inside the black hole, after a rapid collapse of the Einstein-Rosen bridge, oscillations of the scalar field follow. Then the system evolves into the Kasner epoch with possible Kasner inversion, which occurs in most of the parameter regimes. In both cases, one of the metric fields obeys a simple relation between its value at the horizon and in the Kasner epoch.

hep-th↗

arXiv2024

Secrets of RLHF in Large Language Models Part II: Reward Modeling

Reinforcement Learning from Human Feedback (RLHF) has become a crucial technology for aligning language models with human values and intentions, enabling models to produce more helpful and harmless responses. Reward models are trained as proxies for human preferences to drive reinforcement learning optimization. While reward models are often considered central to achieving high performance, they face the following challenges in practical applications: (1) Incorrect and ambiguous preference pairs in the dataset may hinder the reward model from accurately capturing human intent. (2) Reward models trained on data from a specific distribution often struggle to generalize to examples outside that distribution and are not suitable for iterative RLHF training. In this report, we attempt to address these two issues. (1) From a data perspective, we propose a method to measure the strength of preferences within the data, based on a voting mechanism of multiple reward models. Experimental results confirm that data with varying preference strengths have different impacts on reward model performance. We introduce a series of novel methods to mitigate the influence of incorrect and ambiguous preferences in the dataset and fully leverage high-quality preference data. (2) From an algorithmic standpoint, we introduce contrastive learning to enhance the ability of reward models to distinguish between chosen and rejected responses, thereby improving model generalization. Furthermore, we employ meta-learning to enable the reward model to maintain the ability to differentiate subtle differences in out-of-distribution samples, and this approach can be utilized for iterative RLHF optimization.

cs.AI↗

arXiv2023

Lightweight Improved Residual Network for Efficient Inverse Tone Mapping

The display devices like HDR10 televisions are increasingly prevalent in our daily life for visualizing high dynamic range (HDR) images. But the majority of media images on the internet remain in 8-bit standard dynamic range (SDR) format. Therefore, converting SDR images to HDR ones by inverse tone mapping (ITM) is crucial to unlock the full potential of abundant media images. However, existing ITM methods are usually developed with complex network architectures requiring huge computational costs. In this paper, we propose a lightweight Improved Residual Network (IRNet) by enhancing the power of popular residual block for efficient ITM. Specifically, we propose a new Improved Residual Block (IRB) to extract and fuse multi-layer features for fine-grained HDR image reconstruction. Experiments on three benchmark datasets demonstrate that our IRNet achieves state-of-the-art performance on both the ITM and joint SR-ITM tasks. The code, models and data will be publicly available at https://github.com/ThisisVikki/ITM-baseline.

cs.CV↗

arXiv2023

TextGenSHAP: Scalable Post-hoc Explanations in Text Generation with Long Documents

Large language models (LLMs) have attracted huge interest in practical applications given their increasingly accurate responses and coherent reasoning abilities. Given their nature as black-boxes using complex reasoning processes on their inputs, it is inevitable that the demand for scalable and faithful explanations for LLMs' generated content will continue to grow. There have been major developments in the explainability of neural network models over the past decade. Among them, post-hoc explainability methods, especially Shapley values, have proven effective for interpreting deep learning models. However, there are major challenges in scaling up Shapley values for LLMs, particularly when dealing with long input contexts containing thousands of tokens and autoregressively generated output sequences. Furthermore, it is often unclear how to effectively utilize generated explanations to improve the performance of LLMs. In this paper, we introduce TextGenSHAP, an efficient post-hoc explanation method incorporating LM-specific techniques. We demonstrate that this leads to significant increases in speed compared to conventional Shapley value computations, reducing processing times from hours to minutes for token-level explanations, and to just seconds for document-level explanations. In addition, we demonstrate how real-time Shapley values can be utilized in two important scenarios, providing better understanding of long-document question answering by localizing important words and sentences; and improving existing document retrieval systems through enhancing the accuracy of selected passages and ultimately the final responses.

cs.CL↗