arXiv subjects

Zhaowei Zhang

Publications and source records attributed to Zhaowei Zhang.

At least 37 records · Page 2Linked to original sources

arXiv2024

ValueDCG: Measuring Comprehensive Human Value Understanding Ability of Language Models

Personal values are a crucial factor behind human decision-making. Considering that Large Language Models (LLMs) have been shown to impact human decisions significantly, it is essential to make sure they accurately understand human values to ensure their safety. However, evaluating their grasp of these values is complex due to the value's intricate and adaptable nature. We argue that truly understanding values in LLMs requires considering both "know what" and "know why". To this end, we present a comprehensive evaluation metric, ValueDCG (Value Discriminator-Critique Gap), to quantitatively assess the two aspects with an engineering implementation. We assess four representative LLMs and provide compelling evidence that the growth rates of LLM's "know what" and "know why" capabilities do not align with increases in parameter numbers, resulting in a decline in the models' capacity to understand human values as larger amounts of parameters. This may further suggest that LLMs might craft plausible explanations based on the provided context without truly understanding their inherent value, indicating potential risks.

cs.CL↗

arXiv2024

Efficient Model-agnostic Alignment via Bayesian Persuasion

With recent advancements in large language models (LLMs), alignment has emerged as an effective technique for keeping LLMs consensus with human intent. Current methods primarily involve direct training through Supervised Fine-tuning (SFT) or Reinforcement Learning from Human Feedback (RLHF), both of which require substantial computational resources and extensive ground truth data. This paper explores an efficient method for aligning black-box large models using smaller models, introducing a model-agnostic and lightweight Bayesian Persuasion Alignment framework. We formalize this problem as an optimization of the signaling strategy from the small model's perspective. In the persuasion process, the small model (Advisor) observes the information item (i.e., state) and persuades large models (Receiver) to elicit improved responses. The Receiver then generates a response based on the input, the signal from the Advisor, and its updated belief about the information item. Through training using our framework, we demonstrate that the Advisor can significantly enhance the performance of various Receivers across a range of tasks. We theoretically analyze our persuasion framework and provide an upper bound on the Advisor's regret, confirming its effectiveness in learning the optimal signaling strategy. Our Empirical results demonstrates that GPT-2 can significantly improve the performance of various models, achieving an average enhancement of 16.1% in mathematical reasoning ability and 13.7% in code generation. We hope our work can provide an initial step toward rethinking the alignment framework from the Bayesian Persuasion perspective.

cs.CL↗

arXiv2024

CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents

The generalization of decision-making agents encompasses two fundamental elements: learning from past experiences and reasoning in novel contexts. However, the predominant emphasis in most interactive environments is on learning, often at the expense of complexity in reasoning. In this paper, we introduce CivRealm, an environment inspired by the Civilization game. Civilization's profound alignment with human history and society necessitates sophisticated learning, while its ever-changing situations demand strong reasoning to generalize. Particularly, CivRealm sets up an imperfect-information general-sum game with a changing number of players; it presents a plethora of complex features, challenging the agent to deal with open-ended stochastic environments that require diplomacy and negotiation skills. Within CivRealm, we provide interfaces for two typical agent types: tensor-based agents that focus on learning, and language-based agents that emphasize reasoning. To catalyze further research, we present initial results for both paradigms. The canonical RL-based agents exhibit reasonable performance in mini-games, whereas both RL- and LLM-based agents struggle to make substantial progress in the full game. Overall, CivRealm stands as a unique learning and reasoning challenge for decision-making agents. The code is available at https://github.com/bigai-ai/civrealm.

cs.AI↗

arXiv2024

Heterogeneous Value Alignment Evaluation for Large Language Models

The emergent capabilities of Large Language Models (LLMs) have made it crucial to align their values with those of humans. However, current methodologies typically attempt to assign value as an attribute to LLMs, yet lack attention to the ability to pursue value and the importance of transferring heterogeneous values in specific practical applications. In this paper, we propose a Heterogeneous Value Alignment Evaluation (HVAE) system, designed to assess the success of aligning LLMs with heterogeneous values. Specifically, our approach first brings the Social Value Orientation (SVO) framework from social psychology, which corresponds to how much weight a person attaches to the welfare of others in relation to their own. We then assign the LLMs with different social values and measure whether their behaviors align with the inducing values. We conduct evaluations with new auto-metric \textit{value rationality} to represent the ability of LLMs to align with specific values. Evaluating the value rationality of five mainstream LLMs, we discern a propensity in LLMs towards neutral values over pronounced personal values. By examining the behavior of these LLMs, we contribute to a deeper insight into the value alignment of LLMs within a heterogeneous value system.

cs.CL↗

arXiv2024

ProAgent: Building Proactive Cooperative Agents with Large Language Models

Building agents with adaptive behavior in cooperative tasks stands as a paramount goal in the realm of multi-agent systems. Current approaches to developing cooperative agents rely primarily on learning-based methods, whose policy generalization depends heavily on the diversity of teammates they interact with during the training phase. Such reliance, however, constrains the agents' capacity for strategic adaptation when cooperating with unfamiliar teammates, which becomes a significant challenge in zero-shot coordination scenarios. To address this challenge, we propose ProAgent, a novel framework that harnesses large language models (LLMs) to create proactive agents capable of dynamically adapting their behavior to enhance cooperation with teammates. ProAgent can analyze the present state, and infer the intentions of teammates from observations. It then updates its beliefs in alignment with the teammates' subsequent actual behaviors. Moreover, ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various of coordination scenarios. Experimental evaluations conducted within the Overcooked-AI environment unveil the remarkable performance superiority of ProAgent, outperforming five methods based on self-play and population-based training when cooperating with AI agents. Furthermore, in partnered with human proxy models, its performance exhibits an average improvement exceeding 10% compared to the current state-of-the-art method. For more information about our project, please visit~\url{https://pku-proagent.github.io}.

cs.AI↗

arXiv2024

STAS: Spatial-Temporal Return Decomposition for Multi-agent Reinforcement Learning

Centralized Training with Decentralized Execution (CTDE) has been proven to be an effective paradigm in cooperative multi-agent reinforcement learning (MARL). One of the major challenges is credit assignment, which aims to credit agents by their contributions. While prior studies have shown great success, their methods typically fail to work in episodic reinforcement learning scenarios where global rewards are revealed only at the end of the episode. They lack the functionality to model complicated relations of the delayed global reward in the temporal dimension and suffer from inefficiencies. To tackle this, we introduce Spatial-Temporal Attention with Shapley (STAS), a novel method that learns credit assignment in both temporal and spatial dimensions. It first decomposes the global return back to each time step, then utilizes the Shapley Value to redistribute the individual payoff from the decomposed global reward. To mitigate the computational complexity of the Shapley Value, we introduce an approximation of marginal contribution and utilize Monte Carlo sampling to estimate it. We evaluate our method on an Alice & Bob example and MPE environments across different scenarios. Our results demonstrate that our method effectively assigns spatial-temporal credit, outperforming all state-of-the-art baselines.

cs.AI↗

arXiv2023

Quantum metric-induced nonlinear transport in a topological antiferromagnet

The Berry curvature and quantum metric are the imaginary part and real part, respectively, of the quantum geometric tensor which characterizes the topology of quantum states. The former is known to generate a zoo of important discoveries such as quantum Hall effect and anomalous Hall effect (AHE), while the consequences of the quantum metric have rarely been probed by transport. In this work, we observed quantum metric induced nonlinear transport, including both nonlinear AHE and diode-like nonreciprocal longitudinal response, in thin films of a topological antiferromagnet, MnBi$_2$Te$_4$. Our observation reveals that the transverse and longitudinal nonlinear conductivities reverse signs when reversing the antiferromagnetic order, diminish above the Néel temperature, and are insensitive to disorder scattering, thus verifying their origin in the band structure topology. They also flip signs between electron and hole-doped regions, in agreement with theoretical calculations. Our work provides a pathway to probe the quantum metric through nonlinear transport and to design magnetic nonlinear devices.

cond-mat.mes-hall↗

arXiv2023

Interlayer donor-acceptor pair excitons in MoSe2/WSe2 moiré heterobilayer

Localized interlayer excitons (LIXs) in two-dimensional moiré superlattices exhibit sharp and dense emission peaks, making them promising as highly tunable single-photon sources. However, the fundamental nature of these LIXs is still elusive. Here, we show the donor-acceptor pair (DAP) mechanism as one of the origins of these excitonic peaks. Numerical simulation results of the DAP model agree with the experimental photoluminescence spectra of LIX in the moiré MoSe2/WSe2 heterobilayer. In particular, we find that the emission energy-lifetime correlation and the nonmonotonic power dependence of the lifetime agree well with the DAP IX model. Our results provide insight into the physical mechanism of LIX formation in moiré heterostructures and pave new directions for engineering interlayer exciton properties in moiré superlattices.

cond-mat.mes-hall↗

arXiv2022

Diet Code Is Healthy: Simplifying Programs for Pre-trained Models of Code

Pre-trained code representation models such as CodeBERT have demonstrated superior performance in a variety of software engineering tasks, yet they are often heavy in complexity, quadratically with the length of the input sequence. Our empirical analysis of CodeBERT's attention reveals that CodeBERT pays more attention to certain types of tokens and statements such as keywords and data-relevant statements. Based on these findings, we propose DietCode, which aims at lightweight leverage of large pre-trained models for source code. DietCode simplifies the input program of CodeBERT with three strategies, namely, word dropout, frequency filtering, and an attention-based strategy which selects statements and tokens that receive the most attention weights during pre-training. Hence, it gives a substantial reduction in the computational cost without hampering the model performance. Experimental results on two downstream tasks show that DietCodeBERT provides comparable results to CodeBERT with 40% less computational cost in fine-tuning and testing.

cs.SE↗

arXiv2022

Contextual Transformer for Offline Meta Reinforcement Learning

The pretrain-finetuning paradigm in large-scale sequence models has made significant progress in natural language processing and computer vision tasks. However, such a paradigm is still hindered by several challenges in Reinforcement Learning (RL), including the lack of self-supervised pretraining algorithms based on offline data and efficient fine-tuning/prompt-tuning over unseen downstream tasks. In this work, we explore how prompts can improve sequence modeling-based offline reinforcement learning (offline-RL) algorithms. Firstly, we propose prompt tuning for offline RL, where a context vector sequence is concatenated with the input to guide the conditional policy generation. As such, we can pretrain a model on the offline dataset with self-supervised loss and learn a prompt to guide the policy towards desired actions. Secondly, we extend our framework to Meta-RL settings and propose Contextual Meta Transformer (CMT); CMT leverages the context among different tasks as the prompt to improve generalization on unseen tasks. We conduct extensive experiments across three different offline-RL settings: offline single-agent RL on the D4RL dataset, offline Meta-RL on the MuJoCo benchmark, and offline MARL on the SMAC benchmark. Superior results validate the strong performance, and generality of our methods.

cs.LG↗

arXiv2022

Continuous Decomposition of Granularity for Neural Paraphrase Generation

While Transformers have had significant success in paragraph generation, they treat sentences as linear sequences of tokens and often neglect their hierarchical information. Prior work has shown that decomposing the levels of granularity~(e.g., word, phrase, or sentence) for input tokens has produced substantial improvements, suggesting the possibility of enhancing Transformers via more fine-grained modeling of granularity. In this work, we propose a continuous decomposition of granularity for neural paraphrase generation (C-DNPG). In order to efficiently incorporate granularity into sentence encoding, C-DNPG introduces a granularity-aware attention (GA-Attention) mechanism which extends the multi-head self-attention with: 1) a granularity head that automatically infers the hierarchical structure of a sentence by neurally estimating the granularity level of each input token; and 2) two novel attention masks, namely, granularity resonance and granularity scope, to efficiently encode granularity into attention. Experiments on two benchmarks, including Quora question pairs and Twitter URLs have shown that C-DNPG outperforms baseline models by a remarkable margin and achieves state-of-the-art results in terms of many metrics. Qualitative analysis reveals that C-DNPG indeed captures fine-grained levels of granularity with effectiveness.

cs.CL↗

arXiv2022

Non-reciprocal charge transport in an intrinsic magnetic topological insulator MnBi2Te4

Symmetries, quantum geometries and electronic correlations are among the most important ingredients of condensed matters, and lead to nontrivial phenomena in experiments, for example, non-reciprocal charge transport. Here we report the non-reciprocal charge transport in the intrinsic magnetic topological insulator MnBi2Te4. The current direction relevant resistance is observed at chiral edges, which is magnetically switchable, edge position sensitive and septuple layer number controllable. Applying gate voltage can effectively tune the non-reciprocal resistance. The observation and manipulation of non-reciprocal charge transport indicate the fundamental role of chirality in charge transport of MnBi2Te4, and pave ways to develop van der Waals spintronic devices by chirality engineering.

cond-mat.mes-hall↗

arXiv2022

Optical signature of cascade transitions between moiré interlayer excitons

Cascade transition between energy levels has important applications, such as in quantum information protocols and quantum cascade lasers. In two-dimensional heterostructure, the moiré superlattice potential can result in multiple interlayer exciton (IX) energy levels. We demonstrate the cascade transitions between such moiré IXs by performing time- and energy-resolved photoluminescence measurements. We show that the lower-energy moiré IX can be excited to higher-energy ones, facilitating IX population inversion.

cond-mat.mes-hall↗

arXiv2022

Nonlinear Hall effect with non-centrosymmetric topological phase in ZrTe$_5$

The non-centrosymmetric topological material has attracted intense attention due to its superior characters as compared to the centrosymmetric one. On one side, the topological phase coming from global geometric properties of the quantum wave function remains unchanged, on the other side, abundant exotic phenomena are predicted to be merely emerged in non-centrosymmetric ones, due to the redistribution of local quantum geometry. Whereas, probing the local quantum geometry in non-centrosymmetric topological material remains challenging. Here, we report a non-centrosymmetric topological phase in ZrTe$_5$, probed by the nonlinear Hall (NLH) effect. The angle-resolved and temperature-dependent NLH measurement reveals the inversion and ab-plane mirror symmetries breaking under 30 K, consistent with our theoretical calculation. Our findings identify a new non-centrosymmetric phase of ZrTe$_5$ and provide a platform to probe and control local quantum geometry via crystal symmetries.

cond-mat.mtrl-sci↗

arXiv2021

Layer Hall effect in a 2D topological Axion antiferromagnet

While ferromagnets have been known and exploited for millennia, antiferromagnets (AFMs) were only discovered in the 1930s. The elusive nature indicates AFMs' unique properties: At large scale, due to the absence of global magnetization, AFMs may appear to behave like any non-magnetic material; However, such a seemingly mundane macroscopic magnetic property is highly nontrivial at microscopic level, where opposite spin alignment within the AFM unit cell forms a rich internal structure. In topological AFMs, such an internal structure leads to a new possibility, where topology and Berry phase can acquire distinct spatial textures. Here, we study this exciting possibility in an AFM Axion insulator, even-layered MnBi$_2$Te$_4$ flakes, where spatial degrees of freedom correspond to different layers. Remarkably, we report the observation of a new type of Hall effect, the layer Hall effect, where electrons from the top and bottom layers spontaneously deflect in opposite directions. Specifically, under no net electric field, even-layered MnBi$_2$Te$_4$ shows no anomalous Hall effect (AHE); However, applying an electric field isolates the response from one layer and leads to the surprising emergence of a large layer-polarized AHE (~50%$\frac{e^2}{h}$). Such a layer Hall effect uncovers a highly rare layer-locked Berry curvature, which serves as a unique character of the space-time $\mathcal{PT}$-symmetric AFM topological insulator state. Moreover, we found that the layer-locked Berry curvature can be manipulated by the Axion field, E$\cdot$B, which drives the system between the opposite AFM states. Our results achieve previously unavailable pathways to detect and manipulate the rich internal spatial structure of fully-compensated topological AFMs. The layer-locked Berry curvature represents a first step towards spatial engineering of Berry phase, such as through layer-specific moiré potential.

cond-mat.mtrl-sci↗

arXiv2020

Cosmic rays escaping from Galactic starburst-driven superbubbles

We calculate spectra of escaping cosmic rays (CRs) accelerated at shocks produced by expanding Galactic superbubbles powered by multiple supernovae producing a continuous energy outflow in star-forming galaxies. We solve the generalized Kompaneets equations adapted to expansion in various external density profiles, including exponential and power-law shapes, and take into account that escaping CRs are dominated by those around their maximum energies. We find that the escaping CR spectrum largely depends on the specific density profiles and power source properties, and the results are compared to and constrained by the observed CR spectrum. As a generic demonstration, we apply the scheme to a superbubble occurring in the centre of the Milky Way, and find that under specific parameter sets the CRs produced in our model can explain the observed CR flux and spectrum around the second knee at $10^{17}$ eV.

astro-ph.HE↗

arXiv2019

Magnetic Proximity Effect in 2D Ferromagnetic CrBr3/Graphene van der Waals Heterostructures

Two-dimensional (2D) van der Waals heterostructures serve as a promising platform to exploit various physical phenomena in a diverse range of novel spintronic device applications. The efficient spin injection is the prerequisite for these devices. The recent discovery of magnetic 2D materials leads to the possibility of fully 2D van der Waals spintronics devices by implementing spin injection through magnetic proximity effect (MPE). Here, we report the investigation of magnetic proximity effect in 2D CrBr3/graphene van der Waals heterostructures, which is probed by Zeeman spin Hall effect through non-local measurements. Zeeman splitting field estimation demonstrates a significant magnetic proximity exchange field even in a low magnetic field. Furthermore, the observed anomalous longitudinal resistance changes at the Dirac point R_(XX,D)with increasing magnetic field at ν = 0 may attribute to the MPE induced new ground state phases. This MPE revealed in our CrBr3/graphene van der Waals heterostructures therefore provides a solid physics basis and key functionality for next generation 2D spin logic and memory devices.

cond-mat.mes-hall↗

arXiv2019

Direct photoluminescence probing of ferromagnetism in monolayer two-dimensional CrBr3

Atomically thin magnets are the key element to build up spintronics based on two-dimensional materials. The surface nature of two-dimensional ferromagnet opens up opportunities to improve the device performance efficiently. Here, we report the intrinsic ferromagnetism in atomically thin monolayer CrBr3, directly probed by polarization resolved magneto-photoluminescence. The spontaneous magnetization persists in monolayer CrBr3 with a Curie temperature of 34 K. The development of magnons by the thermal excitation is in line with the spin-wave theory. We attribute the layer-number dependent hysteresis loops in thick layers to the magnetic domain structures. As a stable monolayer material in air, CrBr3 provides a convenient platform for fundamental physics and pushes the potential applications of the two-dimensional ferromagnetism.

cond-mat.mes-hall↗