arXiv subjects

Jian Sun

Publications and source records attributed to Jian Sun.

At least 217 records · Page 12Linked to original sources

arXiv2023

Excitonic Insulator to Superconductor Phase Transition in Ultra-Compressed Helium

Helium, the second most abundant element in the universe, exhibits an extremely large electronic band gap of about $20$ eV at low pressures ($\le 0.1$ GPa). While the metallization pressure of hcp helium has been accurately predicted, thus far little attention has been paid to the specific mechanisms driving the band-gap closure and electronic properties of this quantum crystal in the terapascal regime (1 TPa $= 1,000$ GPa). Here, we employ state-of-the-art density functional theory and many-body perturbation theory calculations to fill up this knowledge gap. It is found that prior to reaching metallicity bulk solid helium becomes an excitonic insulator (EI), an exotic state of matter typically observed in low-dimensional systems in which electrostatically bound electron-hole pairs form spontaneously. Furthermore, it is shown that electron-phonon coupling (EPC) is significantly enhanced across the EI to metal phase transition as signaled by prominent phonon softening and giant EPC strength values ($λ\sim 10-100$) estimated at specific reciprocal space points. Accordingly, we predict metallic helium to be a superconductor with a critical temperature of $\approx 30$ K at $20$ TPa and of $\approx 100$ K at $100$ TPa. These unforeseen phenomena have important consequences on the elastic, thermodynamic and transport properties of metallic helium hence may be critical for improving our fundamental understanding and modelling of celestial bodies.

cond-mat.mtrl-sci↗

arXiv2023

Dynamic Grained Encoder for Vision Transformers

Transformers, the de-facto standard for language modeling, have been recently applied for vision tasks. This paper introduces sparse queries for vision transformers to exploit the intrinsic spatial redundancy of natural images and save computational costs. Specifically, we propose a Dynamic Grained Encoder for vision transformers, which can adaptively assign a suitable number of queries to each spatial region. Thus it achieves a fine-grained representation in discriminative regions while keeping high efficiency. Besides, the dynamic grained encoder is compatible with most vision transformer frameworks. Without bells and whistles, our encoder allows the state-of-the-art vision transformers to reduce computational complexity by 40%-60% while maintaining comparable performance on image classification. Extensive experiments on object detection and segmentation further demonstrate the generalizability of our approach. Code is available at https://github.com/StevenGrove/vtpack.

cs.CV↗

arXiv2023

Pressure-Induced Superconductivity in Topological Heterostructure (PbSe)5(Bi2Se3)6

Recently, the natural heterostructure of (PbSe)5(Bi2Se3)6 has been theoretically predicted and experimentally confirmed as a topological insulator. In this work, we induce superconductivity in (PbSe)5(Bi2Se3)6 by implementing high pressure. As increasing pressure up to 10 GPa, superconductivity with Tc ~ 4.6 K suddenly appears, followed by an abrupt decrease. Remarkably, upon further compression above 30 GPa, a new superconducting state arises, where pressure raises the Tc to an unsaturated 6.0 K within the limit of our research. Combining XRD and Raman spectroscopies, we suggest that the emergence of two distinct superconducting states occurs concurrently with the pressure-induced structural transition in this topological heterostructure (PbSe)5(Bi2Se3)6.

cond-mat.supr-con↗

arXiv2022

Prediction of surface reconstructions using MAGUS

In this paper, we present a new module to predict the potential surface reconstruction configurations of given surface structures in the framework of our machine learning and graph theory assisted universal structure searcher (MAGUS). In addition to random structures generated with specific lattice symmetry, we made full use of bulk materials to obtain a better distribution of population energy, namely, randomly appending atoms to a surface cleaved from bulk structures or moving/removing some of the atoms on the surface, which is inspired by natural surface reconstruction processes. In addition, we borrowed ideas from cluster predictions to spread structures better between different compositions, considering that surface models of different atom numbers usually have some building blocks in common. To validate this newly developed module, we tested it with studies on the surface reconstructions of Si (100), Si (111) and 4H-SiC(1-102)-c(2x2), respectively. We successfully gave the known ground states as well as a new SiC surface model in an extremely Si-rich environment.

cond-mat.mtrl-sci↗

arXiv2022

Novel superhard structures of high-pressure C-N compounds

Through machine learning force field accelerated structure search combined with first-principles calculations, we have studied the structures of new C-N compounds with different stoichiometric ratios, and found twelve new superhard C-N compounds, the energies of these structures are similar to c-C3N4 , which is possibly synthesized by high pressure experiment, the XRD of Pa-3(C4N) P3(C4N) and C2/m(C2N) are consistent with previous experimental data and can be used as the structural candidate. According to the macro hardness model, they are all superhard structures, with Vickers hardness over 40GPa, even, the hardness of Pa-3 (C4N) as high as 82.2GPa, and Pa-3 (C4N) combines high tensile and shear resistance. Compared with the hardness calculated by macro hardness model and bond resistance model, we obtained the relationship between the hardness and chemical concentration of C-N compounds under the two models, besides that, we also calculated the fracture toughness of these structures. According to Niu's model, P2_1/c(C4N) has the best fracture toughness, which is higher than WC in calculation, This also indicated the superior mechanical properties of the novel C-N compounds. Moreover, for nitrogen-rich structures, they have the potential to be used as high energy density, the energy density of Pa-3(CN3), P-3c1 (CN4), and I-42d (CN4) are 7.076kJ/g, 7.742kJ/g and 8.045kJ/g, which is close or higher than CL-20, therefore, the C-N compounds synthesized under high pressure have great potential as ideally superhard materials and high energy density materials(HEDMs).

cond-mat.mtrl-sci↗

arXiv2022

Noval Si-C Compounds with High Thermal Conductivity under TPa in Planetary Interior

Silicon carbide has excellent physical properties, such as high stability, thermal conductivity, and mechanical strength. It has been widely used in high-power devices, catalysis, material processing, and other fields and is of great significance in basic discipline research. We used crystal structure search and first principles calculation to predict four new silicon carbide structures stabilized within 3TPa. Pnma (SiC) replaces B1 (SiC) above 2.6TPa, and the new component Si3C2 becomes the most stable after 2.5TPa. P4/mbm phase Si3C2 under high pressure has electrode characteristics. The silicon-carbide structure is metallic in the study pressure range, and the electrons contribute most of the thermal conductivity, which is of great significance for the thermal evolution of silicon-carbide-like terrestrial planets. In addition, we propose a new silicon carbide planetary model and calculate the sound velocity of Si3C2 under TPa. Therefore, our research has deepened the understanding of silicon-carbide terrestrial planets' internal structure and thermal evolution, explored the complex silicon-carbide phase space, and enriched the silicon-carbide phase diagram.

cond-mat.mtrl-sci↗

arXiv2022

Towards Generalized Open Information Extraction

Open Information Extraction (OpenIE) facilitates the open-domain discovery of textual facts. However, the prevailing solutions evaluate OpenIE models on in-domain test sets aside from the training corpus, which certainly violates the initial task principle of domain-independence. In this paper, we propose to advance OpenIE towards a more realistic scenario: generalizing over unseen target domains with different data distributions from the source training domains, termed Generalized OpenIE. For this purpose, we first introduce GLOBE, a large-scale human-annotated multi-domain OpenIE benchmark, to examine the robustness of recent OpenIE models to domain shifts, and the relative performance degradation of up to 70% implies the challenges of generalized OpenIE. Then, we propose DragonIE, which explores a minimalist graph expression of textual fact: directed acyclic graph, to improve the OpenIE generalization. Extensive experiments demonstrate that DragonIE beats the previous methods in both in-domain and out-of-domain settings by as much as 6.0% in F1 score absolutely, but there is still ample room for improvement.

cs.CL↗

arXiv2022

Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialogue

Lifelong learning (LL) is vital for advanced task-oriented dialogue (ToD) systems. To address the catastrophic forgetting issue of LL, generative replay methods are widely employed to consolidate past knowledge with generated pseudo samples. However, most existing generative replay methods use only a single task-specific token to control their models. This scheme is usually not strong enough to constrain the generative model due to insufficient information involved. In this paper, we propose a novel method, prompt conditioned VAE for lifelong learning (PCLL), to enhance generative replay by incorporating tasks' statistics. PCLL captures task-specific distributions with a conditional variational autoencoder, conditioned on natural language prompts to guide the pseudo-sample generation. Moreover, it leverages a distillation process to further consolidate past knowledge by alleviating the noise in pseudo samples. Experiments on natural language understanding tasks of ToD systems demonstrate that PCLL significantly outperforms competitive baselines in building LL models.

cs.CL↗

arXiv2022

Semi-Supervised Lifelong Language Learning

Lifelong learning aims to accumulate knowledge and alleviate catastrophic forgetting when learning tasks sequentially. However, existing lifelong language learning methods only focus on the supervised learning setting. Unlabeled data, which can be easily accessed in real-world scenarios, are underexplored. In this paper, we explore a novel setting, semi-supervised lifelong language learning (SSLL), where a model learns sequentially arriving language tasks with both labeled and unlabeled data. We propose an unlabeled data enhanced lifelong learner to explore SSLL. Specially, we dedicate task-specific modules to alleviate catastrophic forgetting and design two modules to exploit unlabeled data: (1) a virtual supervision enhanced task solver is constructed on a teacher-student framework to mine the underlying knowledge from unlabeled data; and (2) a backward augmented learner is built to encourage knowledge transfer from newly arrived unlabeled data to previous tasks. Experimental results on various language tasks demonstrate our model's effectiveness and superiority over competitive baselines under the new setting SSLL.

cs.CL↗

arXiv2022

CGoDial: A Large-Scale Benchmark for Chinese Goal-oriented Dialog Evaluation

Practical dialog systems need to deal with various knowledge sources, noisy user expressions, and the shortage of annotated data. To better solve the above problems, we propose CGoDial, new challenging and comprehensive Chinese benchmark for multi-domain Goal-oriented Dialog evaluation. It contains 96,763 dialog sessions and 574,949 dialog turns totally, covering three datasets with different knowledge sources: 1) a slot-based dialog (SBD) dataset with table-formed knowledge, 2) a flow-based dialog (FBD) dataset with tree-formed knowledge, and a retrieval-based dialog (RBD) dataset with candidate-formed knowledge. To bridge the gap between academic benchmarks and spoken dialog scenarios, we either collect data from real conversations or add spoken features to existing datasets via crowd-sourcing. The proposed experimental settings include the combinations of training with either the entire training set or a few-shot training set, and testing with either the standard test set or a hard test subset, which can assess model capabilities in terms of general prediction, fast adaptability and reliable robustness.

cs.CL↗

arXiv2022

Doc2Bot: Accessing Heterogeneous Documents via Conversational Bots

This paper introduces Doc2Bot, a novel dataset for building machines that help users seek information via conversations. This is of particular interest for companies and organizations that own a large number of manuals or instruction books. Despite its potential, the nature of our task poses several challenges: (1) documents contain various structures that hinder the ability of machines to comprehend, and (2) user information needs are often underspecified. Compared to prior datasets that either focus on a single structural type or overlook the role of questioning to uncover user needs, the Doc2Bot dataset is developed to target such challenges systematically. Our dataset contains over 100,000 turns based on Chinese documents from five domains, larger than any prior document-grounded dialog dataset for information seeking. We propose three tasks in Doc2Bot: (1) dialog state tracking to track user intentions, (2) dialog policy learning to plan system actions and contents, and (3) response generation which generates responses based on the outputs of the dialog policy. Baseline methods based on the latest deep learning models are presented, indicating that our proposed tasks are challenging and worthy of further research.

cs.CL↗

arXiv2022

PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images

In this paper, we propose PETRv2, a unified framework for 3D perception from multi-view images. Based on PETR, PETRv2 explores the effectiveness of temporal modeling, which utilizes the temporal information of previous frames to boost 3D object detection. More specifically, we extend the 3D position embedding (3D PE) in PETR for temporal modeling. The 3D PE achieves the temporal alignment on object position of different frames. A feature-guided position encoder is further introduced to improve the data adaptability of 3D PE. To support for multi-task learning (e.g., BEV segmentation and 3D lane detection), PETRv2 provides a simple yet effective solution by introducing task-specific queries, which are initialized under different spaces. PETRv2 achieves state-of-the-art performance on 3D object detection, BEV segmentation and 3D lane detection. Detailed robustness analysis is also conducted on PETR framework. We hope PETRv2 can serve as a strong baseline for 3D perception. Code is available at \url{https://github.com/megvii-research/PETR}.

cs.CV↗

arXiv2022

Estimating Soft Labels for Out-of-Domain Intent Detection

Out-of-Domain (OOD) intent detection is important for practical dialog systems. To alleviate the issue of lacking OOD training samples, some works propose synthesizing pseudo OOD samples and directly assigning one-hot OOD labels to these pseudo samples. However, these one-hot labels introduce noises to the training process because some hard pseudo OOD samples may coincide with In-Domain (IND) intents. In this paper, we propose an adaptive soft pseudo labeling (ASoul) method that can estimate soft labels for pseudo OOD samples when training OOD detectors. Semantic connections between pseudo OOD samples and IND intents are captured using an embedding graph. A co-training framework is further introduced to produce resulting soft labels following the smoothness assumption, i.e., close samples are likely to have similar labels. Extensive experiments on three benchmark datasets show that ASoul consistently improves the OOD detection performance and outperforms various competitive baselines.

cs.CL↗

arXiv2022

Distributed Event-Triggered Algorithm for Convex Optimization with Coupled Constraints

This paper develops a distributed primal-dual algorithm via event-triggered mechanism to solve a class of convex optimization problems subject to local set constraints, coupled equality and inequality constraints. Different from some existing distributed algorithms with the diminishing step-sizes, our algorithm uses the constant step-sizes, and is shown to achieve an exact convergence to an optimal solution with an $O(1/k)$ convergence rate for general convex objective functions, where $k>0$ is the iteration number. Moreover, by applying event-triggered communication mechanism, the proposed algorithm can effectively reduce the communication cost without sacrificing the convergence rate. Finally, a numerical example is presented to verify the effectiveness of the proposed algorithm.

math.OC↗

arXiv2022

Fully Convolutional Networks for Panoptic Segmentation with Point-based Supervision

In this paper, we present a conceptually simple, strong, and efficient framework for fully- and weakly-supervised panoptic segmentation, called Panoptic FCN. Our approach aims to represent and predict foreground things and background stuff in a unified fully convolutional pipeline, which can be optimized with point-based fully or weak supervision. In particular, Panoptic FCN encodes each object instance or stuff category with the proposed kernel generator and produces the prediction by convolving the high-resolution feature directly. With this approach, instance-aware and semantically consistent properties for things and stuff can be respectively satisfied in a simple generate-kernel-then-segment workflow. Without extra boxes for localization or instance separation, the proposed approach outperforms the previous box-based and -free models with high efficiency. Furthermore, we propose a new form of point-based annotation for weakly-supervised panoptic segmentation. It only needs several random points for both things and stuff, which dramatically reduces the annotation cost of human. The proposed Panoptic FCN is also proved to have much superior performance in this weakly-supervised setting, which achieves 82% of the fully-supervised performance with only 20 randomly annotated points per instance. Extensive experiments demonstrate the effectiveness and efficiency of Panoptic FCN on COCO, VOC 2012, Cityscapes, and Mapillary Vistas datasets. And it sets up a new leading benchmark for both fully- and weakly-supervised panoptic segmentation. Our code and models are made publicly available at https://github.com/dvlab-research/PanopticFCN.

cs.CV↗

arXiv2022

Unifying Voxel-based Representation with Transformer for 3D Object Detection

In this work, we present a unified framework for multi-modality 3D object detection, named UVTR. The proposed method aims to unify multi-modality representations in the voxel space for accurate and robust single- or cross-modality 3D detection. To this end, the modality-specific space is first designed to represent different inputs in the voxel feature space. Different from previous work, our approach preserves the voxel space without height compression to alleviate semantic ambiguity and enable spatial connections. To make full use of the inputs from different sensors, the cross-modality interaction is then proposed, including knowledge transfer and modality fusion. In this way, geometry-aware expressions in point clouds and context-rich features in images are well utilized for better performance and robustness. The transformer decoder is applied to efficiently sample features from the unified space with learnable positions, which facilitates object-level interactions. In general, UVTR presents an early attempt to represent different modalities in a unified framework. It surpasses previous work in single- or multi-modality entries. The proposed method achieves leading performance in the nuScenes test set for both object detection and the following object tracking task. Code is made publicly available at https://github.com/dvlab-research/UVTR.

cs.CV↗

arXiv2022

Distributed stochastic proximal algorithm with random reshuffling for non-smooth finite-sum optimization

The non-smooth finite-sum minimization is a fundamental problem in machine learning. This paper develops a distributed stochastic proximal-gradient algorithm with random reshuffling to solve the finite-sum minimization over time-varying multi-agent networks. The objective function is a sum of differentiable convex functions and non-smooth regularization. Each agent in the network updates local variables with a constant step-size by local information and cooperates to seek an optimal solution. We prove that local variable estimates generated by the proposed algorithm achieve consensus and are attracted to a neighborhood of the optimal solution in expectation with an $\mathcal{O}(\frac{1}{T}+\frac{1}{\sqrt{T}})$ convergence rate, where $T$ is the total number of iterations. Finally, some comparative simulations are provided to verify the convergence performance of the proposed algorithm.

math.OC↗

arXiv2022

Stacking Ensemble Learning in Deep Domain Adaptation for Ophthalmic Image Classification

Domain adaptation is an attractive approach given the availability of a large amount of labeled data with similar properties but different domains. It is effective in image classification tasks where obtaining sufficient label data is challenging. We propose a novel method, named SELDA, for stacking ensemble learning via extending three domain adaptation methods for effectively solving real-world problems. The major assumption is that when base domain adaptation models are combined, we can obtain a more accurate and robust model by exploiting the ability of each of the base models. We extend Maximum Mean Discrepancy (MMD), Low-rank coding, and Correlation Alignment (CORAL) to compute the adaptation loss in three base models. Also, we utilize a two-fully connected layer network as a meta-model to stack the output predictions of these three well-performing domain adaptation models to obtain high accuracy in ophthalmic image classification tasks. The experimental results using Age-Related Eye Disease Study (AREDS) benchmark ophthalmic dataset demonstrate the effectiveness of the proposed model.

cs.CV↗