arXiv subjects

Ning Ding

Publications and source records attributed to Ning Ding.

At least 163 records · Page 9Linked to original sources

arXiv2023

Enhancing Chat Language Models by Scaling High-quality Instructional Conversations

Fine-tuning on instruction data has been widely validated as an effective practice for implementing chat language models like ChatGPT. Scaling the diversity and quality of such data, although straightforward, stands a great chance of leading to improved performance. This paper aims to improve the upper bound of open-source models further. We first provide a systematically designed, diverse, informative, large-scale dataset of instructional conversations, UltraChat, which does not involve human queries. Our objective is to capture the breadth of interactions that a human might have with an AI assistant and employs a comprehensive framework to generate multi-turn conversation iteratively. UltraChat contains 1.5 million high-quality multi-turn dialogues and covers a wide range of topics and instructions. Our statistical analysis of UltraChat reveals its superiority in various key metrics, including scale, average length, diversity, coherence, etc., solidifying its position as a leading open-source dataset. Building upon UltraChat, we fine-tune a LLaMA model to create a powerful conversational model, UltraLLaMA. Our evaluations indicate that UltraLLaMA consistently outperforms other open-source models, including Vicuna, the previously recognized state-of-the-art open-source model. The dataset and the model will be publicly released\footnote{\url{https://github.com/thunlp/UltraChat}}.

cs.CL↗

arXiv2023

EEG-Fest: Few-shot based Attention Network for Driver's Vigilance Estimation with EEG Signals

A lack of driver's vigilance is the main cause of most vehicle crashes. Electroencephalography(EEG) has been reliable and efficient tool for drivers' drowsiness estimation. Even though previous studies have developed accurate and robust driver's vigilance detection algorithms, these methods are still facing challenges on following areas: (a) small sample size training, (b) anomaly signal detection, and (c) subject-independent classification. In this paper, we propose a generalized few-shot model, namely EEG-Fest, to improve aforementioned drawbacks. The EEG-Fest model can (a) classify the query sample's drowsiness with a few samples, (b) identify whether a query sample is anomaly signals or not, and (c) achieve subject independent classification. The proposed algorithm achieves state-of-the-art results on the SEED-VIG dataset and the SADT dataset. The accuracy of the drowsy class achieves 92% and 94% for 1-shot and 5-shot support samples in the SEED-VIG dataset, and 62% and 78% for 1-shot and 5-shot support samples in the SADT dataset.

cs.LG↗

arXiv2023

Electrical tuning of robust layered antiferromagnetism in MXene monolayer

A-type antiferromagnetism, with an in-plane ferromagnetic order and the interlayer antiferromagnetic coupling, owns inborn advantages for electrical manipulations but is naturally rare in real materials except in those artificial antiferromagnetic heterostructures. Here, a robust layered antiferromagnetism with a high Néel temperature is predicted in a MXene Cr$_2$CCl$_2$ monolayer, which provides an ideal platform as a magnetoelectric field effect transistor. Based on first-principles calculations, we demonstrate that an electric field can induce the band splitting between spin-up and spin-down channels. Although no net magnetization is generated, the inversion symmetry between the lower Cr layer and the upper Cr layer is broken via electronic cloud distortions. Moreover, this electric field can be replaced by a proximate ferroelectric layer for nonvolatility. The magneto-optic Kerr effect can be used to detect this magnetoelectricity, even if it is a collinear antiferromagnet with zero magnetization.

cond-mat.mtrl-sci↗

arXiv2023

Adsorption tuning of polarity and magnetism in AgCr2S4 monolayer

As a recent successfully exfoliated non van der Waals layered material, AgCrS2 has received a lot of attentions. Motivated by its structure related magnetic and ferroelectric behavior, a theoretical study on its exfoliated monolayer AgCr2S4 has been carried out in the present work. Based on density functional theory, the ground state and magnetic order of monolayer AgCr2S4 have been determined. The centrosymmetry emerges upon two-dimensional confinement and thus eliminates the bulk polarity. Moreover, two-dimensional ferromagnetism appears in the CrS2 layer of AgCr2S4 and can persist up to room temperature. The surface adsorption has also been taken into consideration, which shows a nonmonotonic effect on the ionic conductivity through ion displacement of the interlayer Ag, but has little impact on the layered magnetic structure.

cond-mat.mtrl-sci↗

arXiv2023

Magneto-optical Kerr effect in ferroelectric antiferromagnetic two-dimensional heterostructures

We study the magneto-optical Kerr effect (MOKE) of the two-dimensional heterostructure CrI3/In2Se3/CrI3 by using density functional theory calculations and symmetry analysis. The spontaneous polarization in the In2Se3 ferroelectric layer and the antiferromagnetic ordering in CrI3 layers break the mirror symmetry and the timereversal symmetry, thus activating MOKE. We show that the Kerr angle can be switched by either the polarization or the antiferromagnetic order parameter. Our results suggest that ferroelectric and antiferromagnetic 2D heterostructures could be exploited for ultra-compact information storage devices, where the information is encoded by the two ferroelectric or the two time-reversed antiferromagnetic states, and the read-out performed optically by MOKE.

cond-mat.mtrl-sci↗

arXiv2023

Direct observation of geometric and sliding ferroelectricity in an amphidynamic crystal

Sliding ferroelectricity is a recently observed polarity existing in two-dimensional materials. However, due to their weak polarization and poor electrical insulation in these materials, all available experimental evidence till now are indirect, with most based on transport properties in the nanoscale or piezoresponse force microscopy. We report the direct observation of sliding ferroelectricity, using a high-quality amphidynamic single crystal, (15-Crown-5)Cd$_3$Cl$_6$, which possesses a large band-gap and so allows direct measurement of P-E hysteresis. This coordination polymer is a van der Waals material, which is composed of inorganic stators and organic rotators as measured using XRD and NMR characterisation. From DFT calculations, we find that after the freezing of rotators an electric dipole is generated in each layer driven by the geometric mechanism, meanwhile a comparable ferroelectric polarization originates from the interlayer sliding. The net polarization of these two components can be directly measured and manipulated. Our finding provides insight into low-dimensional ferroelectrics, especially the controlling of synchronous dynamics of rotating molecules and sliding layers in solids.

cond-mat.mtrl-sci↗

arXiv2022

ProQA: Structural Prompt-based Pre-training for Unified Question Answering

Question Answering (QA) is a longstanding challenge in natural language processing. Existing QA works mostly focus on specific question types, knowledge domains, or reasoning skills. The specialty in QA research hinders systems from modeling commonalities between tasks and generalization for wider applications. To address this issue, we present ProQA, a unified QA paradigm that solves various tasks through a single model. ProQA takes a unified structural prompt as the bridge and improves the QA-centric ability by structural prompt-based pre-training. Through a structurally designed prompt-based input schema, ProQA concurrently models the knowledge generalization for all QA tasks while keeping the knowledge customization for every specific QA task. Furthermore, ProQA is pre-trained with structural prompt-formatted large-scale synthesized corpus, which empowers the model with the commonly-required QA ability. Experimental results on 11 QA benchmarks demonstrate that ProQA consistently boosts performance on both full data fine-tuning, few-shot learning, and zero-shot testing scenarios. Furthermore, ProQA exhibits strong ability in both continual learning and transfer learning by taking the advantages of the structural prompt.

cs.CL↗

arXiv2022

Exploring Universal Intrinsic Task Subspace via Prompt Tuning

Why can pre-trained language models (PLMs) learn universal representations and effectively adapt to broad NLP tasks differing a lot superficially? In this work, we empirically find evidence indicating that the adaptations of PLMs to various few-shot tasks can be reparameterized as optimizing only a few free parameters in a unified low-dimensional intrinsic task subspace, which may help us understand why PLMs could easily adapt to various NLP tasks with small-scale data. To find such a subspace and examine its universality, we propose an analysis pipeline called intrinsic prompt tuning (IPT). Specifically, we resort to the recent success of prompt tuning and decompose the soft prompts of multiple NLP tasks into the same low-dimensional nonlinear subspace, then we learn to adapt the PLM to unseen data or tasks by only tuning parameters in this subspace. In the experiments, we study diverse few-shot NLP tasks and surprisingly find that in a 250-dimensional subspace found with 100 tasks, by only tuning 250 free parameters, we can recover 97% and 83% of the full prompt tuning performance for 100 seen tasks (using different training data) and 20 unseen tasks, respectively, showing great generalization ability of the found intrinsic task subspace. Besides being an analysis tool, IPT could further help us improve the prompt tuning stability.

cs.CL↗

arXiv2022

MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extraction

The diverse relationships among real-world events, including coreference, temporal, causal, and subevent relations, are fundamental to understanding natural languages. However, two drawbacks of existing datasets limit event relation extraction (ERE) tasks: (1) Small scale. Due to the annotation complexity, the data scale of existing datasets is limited, which cannot well train and evaluate data-hungry models. (2) Absence of unified annotation. Different types of event relations naturally interact with each other, but existing datasets only cover limited relation types at once, which prevents models from taking full advantage of relation interactions. To address these issues, we construct a unified large-scale human-annotated ERE dataset MAVEN-ERE with improved annotation schemes. It contains 103,193 event coreference chains, 1,216,217 temporal relations, 57,992 causal relations, and 15,841 subevent relations, which is larger than existing datasets of all the ERE tasks by at least an order of magnitude. Experiments show that ERE on MAVEN-ERE is quite challenging, and considering relation interactions with joint learning can improve performances. The dataset and source codes can be obtained from https://github.com/THU-KEG/MAVEN-ERE.

cs.CL↗

arXiv2022

Few-shot Classification with Hypersphere Modeling of Prototypes

Metric-based meta-learning is one of the de facto standards in few-shot learning. It composes of representation learning and metrics calculation designs. Previous works construct class representations in different ways, varying from mean output embedding to covariance and distributions. However, using embeddings in space lacks expressivity and cannot capture class information robustly, while statistical complex modeling poses difficulty to metric designs. In this work, we use tensor fields (``areas'') to model classes from the geometrical perspective for few-shot learning. We present a simple and effective method, dubbed hypersphere prototypes (HyperProto), where class information is represented by hyperspheres with dynamic sizes with two sets of learnable parameters: the hypersphere's center and the radius. Extending from points to areas, hyperspheres are much more expressive than embeddings. Moreover, it is more convenient to perform metric-based classification with hypersphere prototypes than statistical modeling, as we only need to calculate the distance from a data point to the surface of the hypersphere. Following this idea, we also develop two variants of prototypes under other measurements. Extensive experiments and analysis on few-shot learning tasks across NLP and CV and comparison with 20+ competitive baselines demonstrate the effectiveness of our approach.

cs.LG↗

arXiv2022

Different Tunes Played with Equal Skill: Exploring a Unified Optimization Subspace for Delta Tuning

Delta tuning (DET, also known as parameter-efficient tuning) is deemed as the new paradigm for using pre-trained language models (PLMs). Up to now, various DETs with distinct design elements have been proposed, achieving performance on par with fine-tuning. However, the mechanisms behind the above success are still under-explored, especially the connections among various DETs. To fathom the mystery, we hypothesize that the adaptations of different DETs could all be reparameterized as low-dimensional optimizations in a unified optimization subspace, which could be found by jointly decomposing independent solutions of different DETs. Then we explore the connections among different DETs by conducting optimization within the subspace. In experiments, we find that, for a certain DET, conducting optimization simply in the subspace could achieve comparable performance to its original space, and the found solution in the subspace could be transferred to another DET and achieve non-trivial performance. We also visualize the performance landscape of the subspace and find that there exists a substantial region where different DETs all perform well. Finally, we extend our analysis and show the strong connections between fine-tuning and DETs.

cs.CL↗

arXiv2022

Stacking dependent ferroelectricity and antiferroelectricity in quasi-one-dimensional oxyhalides NbO$X_3$

Low-dimensional ferroelectricity and polar materials have attracted considerable attentions for their fascinating physics and potential applications. Based on first-principles calculations, here we investigate the stacking modes and polar properties of a typical series of quasi-one-dimensional ferroelectrics: double-chain oxyhalides NbO$X_3$ ($X$=Cl, Br, I). The geometry of their double-chains allows both the interchain/intrachain permutation. Thus, different stacking modes of double-chains lead to a variety of ferroelectric and antiferroelectric phases in both the tetragonal and monoclinic crystals. The proximate energies of these phases may lead to multiphase coexistence in real materials, as well as the hydrostatic pressure driving structural phase transition. Their spontaneous polarizations and piezoelectricity of the ferroelectric phases are prominent, comparable to commercially used ferroelectric BaTiO$_3$ and piezoelectric ZnO, respectively. Our work demonstrates that the van der Waals NbO$X_3$ are promising materials for exploring quasi-one-dimensional ferroelectricity and antiferroelectricity.

cond-mat.mtrl-sci↗

arXiv2022

Magneto-optical Kerr effect and magnetoelasticity in weak ferromagnetic RuF$_4$ monolayer

Considerable research interest has been attracted to noncollinear magnetic structures for their intriguing physics and promising applications. In this work, based on relativistic density functional theory, we reveal the interesting magnetic order and relevant properties in monolayer RuF$_4$, which can be exfoliated from its bulk phase. Although the spins on Ru ions are almost antiferromagnetically aligned between nearest-neighbors, weak ferromagnetism is generated because of the antisymmetric Dzyaloshinskii-Moriya interaction as well as the single-ion anisotropy. A prominent magneto-optical Kerr effect can be observed for this antiferromagnet, similar to those of regular strong ferromagnets. In addition, a uniaxial strain can induce a ferroelastic switching together with the in-plane rotation of spin direction, giving rise to a strong intrinsic magnetoelasticity. Our work not only suggests an alternative direction for two-dimensional magnetic materials, but also provides hints to future devices based on antiferromagnetic magnetoelastic or magneto-optical materials.

cond-mat.mtrl-sci↗

arXiv2022

Two-Dimensional Semiconducting Metal Organic Frameworks with Auxetic Effect, Room Temperature Ferrimagnetism, Chiral Ferroelectricity, Bipolar Spin Polarization and Topological Nodal Lines/Points

Two-dimensional (2D) semiconductors integrated with two or more functions are the cornerstone for constructing multifunctional nanodevices, but remain largely limited. Here, by tuning the spin state of organic linkers and the symmetry/topology of crystal lattice, we predict a class of unprecedented multifunctional semiconductors in 2D Cr(II) five-membered heterocyclic metal organic frameworks that simultaneously possess auxetic effect, room temperature ferrimagnetism, chiral ferroe-lectricity, electrically reversible spin polarization and topological nodal lines/points. Taking 2D Cr(TDZ)$_2$ (TDZ=1.2.5-thiadiazole) as an exemplification, the auxetic effect is produced by the anti-tetra-chiral lattice structure. The high temperature ferrimagnetism originates from the strong d-p direct magnetic exchange interaction between Cr cations and TDZ doublet radical anions. Meanwhile, the clockwise-counterclockwise alignment of TDZ' dipoles results in unique 2D chiral ferroelectricity with atomic-scale vortex-antivortex states. 2D Cr(TDZ)$_2$ is an intrinsic bipolar magnetic semiconductor where half-metallic conduction with switchable spin-polarization direction can be induced by applying a gate voltage. Besides, the symmetry of the little group C$_4$ of lattice structure endows 2D Cr(TDZ)$_2$ with topological nodal lines and a quadratic nodal point in the Brillouin zone near the Fermi level.

cond-mat.mtrl-sci↗

arXiv2022

Improving Task Generalization via Unified Schema Prompt

Task generalization has been a long standing challenge in Natural Language Processing (NLP). Recent research attempts to improve the task generalization ability of pre-trained language models by mapping NLP tasks into human-readable prompted forms. However, these approaches require laborious and inflexible manual collection of prompts, and different prompts on the same downstream task may receive unstable performance. We propose Unified Schema Prompt, a flexible and extensible prompting method, which automatically customizes the learnable prompts for each task according to the task input schema. It models the shared knowledge between tasks, while keeping the characteristics of different task schema, and thus enhances task generalization ability. The schema prompt takes the explicit data structure of each task to formulate prompts so that little human effort is involved. To test the task generalization ability of schema prompt at scale, we conduct schema prompt-based multitask pre-training on a wide variety of general NLP tasks. The framework achieves strong zero-shot and few-shot generalization performance on 16 unseen downstream tasks from 8 task types (e.g., QA, NLI, etc). Furthermore, comprehensive analyses demonstrate the effectiveness of each component in the schema prompt, its flexibility in task compositionality, and its ability to improve performance under a full-data fine-tuning setting.

cs.CL↗

arXiv2022

Sparse Structure Search for Parameter-Efficient Tuning

Adapting large pre-trained models (PTMs) through fine-tuning imposes prohibitive computational and storage burdens. Recent studies of parameter-efficient tuning (PET) find that only optimizing a small portion of parameters conditioned on PTMs could yield on-par performance compared to conventional fine-tuning. Generally, PET methods exquisitely design parameter-efficient modules (PET modules) which could be applied to arbitrary fine-grained positions inside PTMs. However, the effectiveness of these fine-grained positions largely relies on sophisticated manual designation, thereby usually producing sub-optimal results. In contrast to the manual designation, we explore constructing PET modules in an automatic manner. We automatically \textbf{S}earch for the \textbf{S}parse \textbf{S}tructure of \textbf{P}arameter-\textbf{E}fficient \textbf{T}uning (S$^3$PET). Based on a unified framework of various PET methods, S$^3$PET conducts the differentiable PET structure search through bi-level optimization and proposes shifted global sigmoid method to explicitly control the number of trainable parameters. Extensive experiments show that S$^3$PET surpasses manual and random structures with less trainable parameters. The searched structures preserve more than 99\% fine-tuning performance with 0.01\% trainable parameters. Moreover, the advantage of S$^3$PET is amplified with extremely low trainable parameters budgets (0.0009\%$\sim$0.01\%). The searched structures are transferable and explainable, providing suggestions and guidance for the future design of PET methods.

cs.CL↗

arXiv2022

A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications

To understand human behaviors, action recognition based on videos is a common approach. Compared with image-based action recognition, videos provide much more information. Reducing the ambiguity of actions and in the last decade, many works focused on datasets, novel models and learning approaches have improved video action recognition to a higher level. However, there are challenges and unsolved problems, in particular in sports analytics where data collection and labeling are more sophisticated, requiring sport professionals to annotate data. In addition, the actions could be extremely fast and it becomes difficult to recognize them. Moreover, in team sports like football and basketball, one action could involve multiple players, and to correctly recognize them, we need to analyse all players, which is relatively complicated. In this paper, we present a survey on video action recognition for sports analytics. We introduce more than ten types of sports, including team sports, such as football, basketball, volleyball, hockey and individual sports, such as figure skating, gymnastics, table tennis, tennis, diving and badminton. Then we compare numerous existing frameworks for sports analysis to present status quo of video action recognition in both team sports and individual sports. Finally, we discuss the challenges and unsolved problems in this area and to facilitate sports analytics, we develop a toolbox using PaddlePaddle, which supports football, basketball, table tennis and figure skating action recognition.

cs.CV↗

arXiv2022

Gadolinium halide monolayers: a fertile family of two-dimensional 4f magnets

Two-dimensional (2D) magnets have great potentials for applications in next-generation information devices. Since the recent experimental discovery of intrinsic 2D magnetism in monolayer CrI$_3$ and few-layer Cr$_2$Ge$_2$Te$_6$, intensive studies have been stimulated in pursuing more 2D magnets and revealing their intriguing physical properties. In comparison to the magnetism based on $3d$ electrons, $4f$ electrons can provide larger magnetic moments and stronger spin-orbit coupling, but have been much less studied in the 2D forms. Only in very recent years, some exciting results have been obtained in this area. In this mini-review, we will introduce some recent progress in 2D Gd halides from a theoretical aspect. It is noteworthy that $4f$ and $5d$ orbitals of Gd both play key roles in these materials. For Gd$X_2$ ($X$=I, Br, Cl and F) monolayers and related Janus monolayers, robust ferromagnetism with large exchanges comes from the $4f^7$+$5d^1$ hybridization of Gd$^{2+}$. The spatially expanded $5d$ electrons act as a bridge to couple localized $4f$ spins. For Gd$X_3$ monolayers, the intercalation of metal atoms can dope electrons into Gd's $5d$ orbitals, which leads to numerous intriguing physical properties, such as ferroelasticity, ferromagnetism, and anisotropic conductance. In brief, Gd halides establish an effective strategy to take advantage of $f$-electron magnetism in 2D materials.

cond-mat.mtrl-sci↗