arXiv subjects

Jing Liu

Publications and source records attributed to Jing Liu.

At least 505 records · Page 28Linked to original sources

arXiv2022

HIRE: Distilling High-order Relational Knowledge From Heterogeneous Graph Neural Networks

Researchers have recently proposed plenty of heterogeneous graph neural networks (HGNNs) due to the ubiquity of heterogeneous graphs in both academic and industrial areas. Instead of pursuing a more powerful HGNN model, in this paper, we are interested in devising a versatile plug-and-play module, which accounts for distilling relational knowledge from pre-trained HGNNs. To the best of our knowledge, we are the first to propose a HIgh-order RElational (HIRE) knowledge distillation framework on heterogeneous graphs, which can significantly boost the prediction performance regardless of model architectures of HGNNs. Concretely, our HIRE framework initially performs first-order node-level knowledge distillation, which encodes the semantics of the teacher HGNN with its prediction logits. Meanwhile, the second-order relation-level knowledge distillation imitates the relational correlation between node embeddings of different types generated by the teacher HGNN. Extensive experiments on various popular HGNNs models and three real-world heterogeneous graphs demonstrate that our method obtains consistent and considerable performance enhancement, proving its effectiveness and generalization ability.

cs.LG↗

arXiv2022

One-off Negative Sequential Pattern Mining

Negative sequential pattern mining (SPM) is an important SPM research topic. Unlike positive SPM, negative SPM can discover events that should have occurred but have not occurred, and it can be used for financial risk management and fraud detection. However, existing methods generally ignore the repetitions of the pattern and do not consider gap constraints, which can lead to mining results containing a large number of patterns that users are not interested in. To solve this problem, this paper discovers frequent one-off negative sequential patterns (ONPs). This problem has the following two characteristics. First, the support is calculated under the one-off condition, which means that any character in the sequence can only be used once at most. Second, the gap constraint can be given by the user. To efficiently mine patterns, this paper proposes the ONP-Miner algorithm, which employs depth-first and backtracking strategies to calculate the support. Therefore, ONP-Miner can effectively avoid creating redundant nodes and parent-child relationships. Moreover, to effectively reduce the number of candidate patterns, ONP-Miner uses pattern join and pruning strategies to generate and further prune the candidate patterns, respectively. Experimental results show that ONP-Miner not only improves the mining efficiency, but also has better mining performance than the state-of-the-art algorithms. More importantly, ONP mining can find more interesting patterns in traffic volume data to predict future traffic.

cs.DB↗

arXiv2022

Dynamic Local Aggregation Network with Adaptive Clusterer for Anomaly Detection

Existing methods for anomaly detection based on memory-augmented autoencoder (AE) have the following drawbacks: (1) Establishing a memory bank requires additional memory space. (2) The fixed number of prototypes from subjective assumptions ignores the data feature differences and diversity. To overcome these drawbacks, we introduce DLAN-AC, a Dynamic Local Aggregation Network with Adaptive Clusterer, for anomaly detection. First, The proposed DLAN can automatically learn and aggregate high-level features from the AE to obtain more representative prototypes, while freeing up extra memory space. Second, The proposed AC can adaptively cluster video data to derive initial prototypes with prior information. In addition, we also propose a dynamic redundant clustering strategy (DRCS) to enable DLAN for automatically eliminating feature clusters that do not contribute to the construction of prototypes. Extensive experiments on benchmarks demonstrate that DLAN-AC outperforms most existing methods, validating the effectiveness of our method. Our code is publicly available at https://github.com/Beyond-Zw/DLAN-AC.

cs.CV↗

arXiv2022

Computationally Identifying Funneling and Focusing Questions in Classroom Discourse

Responsive teaching is a highly effective strategy that promotes student learning. In math classrooms, teachers might "funnel" students towards a normative answer or "focus" students to reflect on their own thinking, deepening their understanding of math concepts. When teachers focus, they treat students' contributions as resources for collective sensemaking, and thereby significantly improve students' achievement and confidence in mathematics. We propose the task of computationally detecting funneling and focusing questions in classroom discourse. We do so by creating and releasing an annotated dataset of 2,348 teacher utterances labeled for funneling and focusing questions, or neither. We introduce supervised and unsupervised approaches to differentiating these questions. Our best model, a supervised RoBERTa model fine-tuned on our dataset, has a strong linear correlation of .76 with human expert labels and with positive educational outcomes, including math instruction quality and student achievement, showing the model's potential for use in automated teacher feedback tools. Our unsupervised measures show significant but weaker correlations with human labels and outcomes, and they highlight interesting linguistic patterns of funneling and focusing questions. The high performance of the supervised measure indicates its promise for supporting teachers in their instruction.

cs.CY↗

arXiv2022

Tracking the nematicity in cuprate superconductors: a resistivity study under uniaxial pressure

Overshadowing the superconducting dome in hole-doped cuprates, the pseudogap state is still one of the mysteries that no consensus can be achieved. It has been suggested that the rotational symmetry is broken in this state and may result in a nematic phase transition, whose temperature seems to coincide with the onset temperature of the pseudogap state $T^*$ around optimal doping level, raising the question whether the pseudogap results from the establishment of the nematic order. Here we report results of resistivity measurements under uniaxial pressure on several hole-doped cuprates, where the normalized slope of the elastoresistivity $ζ$ can be obtained as illustrated in iron-based superconductors. The temperature dependence of $ζ$ along particular lattice axis exhibits kink feature at $T_{k}$ and shows Curie-Weiss-like behavior above it, which may suggest a spontaneous nematic transition. While $T_{k}$ seems to be the same as $T^*$ around the optimal doping and in the overdoped region, they become very different in underdoped La$_{2-x}$Sr$_{x}$CuO$_4$. Our results suggest that the nematic order, if indeed existing, is an electronic phase within the pseudogap state.

cond-mat.supr-con↗

arXiv2022

Reducing US Biofuels Requirements Mitigates Short-term Impacts of Global Population and Income Growth on Agricultural Environmental Outcomes

Biobased energy, particularly corn starch-based ethanol and other liquid renewable fuels, are a major element of federal and state energy policies in the United States. These policies are motivated by energy security and climate change mitigation objectives, but corn ethanol does not substantially reduce greenhouse gas emissions when compared to petroleum-based fuels. Corn production also imposes substantial negative externalities (e.g., nitrogen leaching, higher food prices, water scarcity, and indirect land use change). In this paper, we utilize a partial equilibrium model of corn-soy production and trade to analyze the potential of reduced US demand for corn as a biobased energy feedstock to mitigate increases in nitrogen leaching, crop production and land use associated with growing global populations and income from 2020 to 2050. We estimate that a 23% demand reduction would sustain land use and nitrogen leaching below 2020 levels through the year 2025, and a 41% reduction would do so through 2030. Outcomes are similar across major watersheds where corn and soy are intensively farmed.

econ.GN↗

arXiv2022

Multi-scale Analysis of Nitrogen Loss Mitigation in the US Corn Belt

Reducing the size of the hypoxic zone in the Gulf of Mexico has proven to be a challenging task. A variety of mitigation options have been proposed, each likely to produce markedly different patterns of mitigation with widely varying consequences for the economy. The general consensus is that no single measure alone is sufficient to achieve the EPA Task Force goal for reducing the Gulf hypoxic zone and it appears that a combination of management practices must be employed. However, absent a highly resolved, multi-scale framework for assessing these policy combinations, it has been unclear what pattern of mitigation is likely to emerge from different policies and what the consequences would be for local, regional and national land use, food prices and farm returns. We address this research gap by utilizing a novel multi-scale framework for evaluating alternative N loss management policies in the Mississippi River basin. This combines fine-scale agro-ecosystem responses with an economic model capturing domestic and international market and price linkages. We find that wetland restoration combined with improved N use efficiency, along with a leaching tax could reduce the Mississippi River N load by 30-53\% while only modestly increasing corn prices. This study underscores the value of fine-resolution analysis and the potential of combined economic and ecological instruments in tackling nonpoint source nitrate pollution.

econ.GN↗

arXiv2022

Contextual Adapters for Personalized Speech Recognition in Neural Transducers

Personal rare word recognition in end-to-end Automatic Speech Recognition (E2E ASR) models is a challenge due to the lack of training data. A standard way to address this issue is with shallow fusion methods at inference time. However, due to their dependence on external language models and the deterministic approach to weight boosting, their performance is limited. In this paper, we propose training neural contextual adapters for personalization in neural transducer based ASR models. Our approach can not only bias towards user-defined words, but also has the flexibility to work with pretrained ASR models. Using an in-house dataset, we demonstrate that contextual adapters can be applied to any general purpose pretrained ASR model to improve personalization. Our method outperforms shallow fusion, while retaining functionality of the pretrained models by not altering any of the model weights. We further show that the adapter style training is superior to full-fine-tuning of the ASR models on datasets with user-defined content.

cs.CL↗

arXiv2022

DuQM: A Chinese Dataset of Linguistically Perturbed Natural Questions for Evaluating the Robustness of Question Matching Models

In this paper, we focus on studying robustness evaluation of Chinese question matching. Most of the previous work on analyzing robustness issue focus on just one or a few types of artificial adversarial examples. Instead, we argue that it is necessary to formulate a comprehensive evaluation about the linguistic capabilities of models on natural texts. For this purpose, we create a Chinese dataset namely DuQM which contains natural questions with linguistic perturbations to evaluate the robustness of question matching models. DuQM contains 3 categories and 13 subcategories with 32 linguistic perturbations. The extensive experiments demonstrate that DuQM has a better ability to distinguish different models. Importantly, the detailed breakdown of evaluation by linguistic phenomenon in DuQM helps us easily diagnose the strength and weakness of different models. Additionally, our experiment results show that the effect of artificial adversarial examples does not work on the natural texts.

cs.CL↗

arXiv2022

Observation of three superconducting transitions in the pressurized CDW-bearing compound TaTe2

Transition metal dichalcogenides host a wide variety of lattice and electronic structures, as well as corresponding exotic physical properties, especially under certain tuning conditions. Here, we are the first to report the observation of pressure-induced three superconducting transitions in TaTe2, a charge density wave (CDW) - bearing layered transition-metal dichalcogenide that is metallic but not superconducting at ambient pressure. We find that its CDW state can be easily suppressed upon increasing pressure up to ~ 1 GPa. A superconducting state then emerges from the suppressed CDW state and persists to the pressure about 7 GPa. Unexpectedly, another superconducting state appears at ~ 11 GPa within the same monoclinic (M) structure of its ambient-pressure one. Upon further compression to 21 GPa, a third superconducting state with higher Tc appears from a high-pressure (HP) phase. Our experimental results suggest that the pressure-induced three superconducting transitions in TaTe2 are respectively driven by the suppression of the CDW state, the change of the angle in the M phase and the transition of M-to-HP phase. These results demonstrate not only the versatile nature of this correlated electron system, but also the first experimental example that shows the pressure-induced evolution from a CDW state to three superconducting states driven by different mechanisms.

cond-mat.supr-con↗

arXiv2022

NTIRE 2022 Challenge on Efficient Super-Resolution: Methods and Results

This paper reviews the NTIRE 2022 challenge on efficient single image super-resolution with focus on the proposed solutions and results. The task of the challenge was to super-resolve an input image with a magnification factor of $\times$4 based on pairs of low and corresponding high resolution images. The aim was to design a network for single image super-resolution that achieved improvement of efficiency measured according to several metrics including runtime, parameters, FLOPs, activations, and memory consumption while at least maintaining the PSNR of 29.00dB on DIV2K validation set. IMDN is set as the baseline for efficiency measurement. The challenge had 3 tracks including the main track (runtime), sub-track one (model complexity), and sub-track two (overall performance). In the main track, the practical runtime performance of the submissions was evaluated. The rank of the teams were determined directly by the absolute value of the average runtime on the validation set and test set. In sub-track one, the number of parameters and FLOPs were considered. And the individual rankings of the two metrics were summed up to determine a final ranking in this track. In sub-track two, all of the five metrics mentioned in the description of the challenge including runtime, parameter count, FLOPs, activations, and memory consumption were considered. Similar to sub-track one, the rankings of five metrics were summed up to determine a final ranking. The challenge had 303 registered participants, and 43 teams made valid submissions. They gauge the state-of-the-art in efficient single image super-resolution.

cs.CV↗

arXiv2022

MemSeg: A semi-supervised method for image surface defect detection using differences and commonalities

Under the semi-supervised framework, we propose an end-to-end memory-based segmentation network (MemSeg) to detect surface defects on industrial products. Considering the small intra-class variance of products in the same production line, from the perspective of differences and commonalities, MemSeg introduces artificially simulated abnormal samples and memory samples to assist the learning of the network. In the training phase, MemSeg explicitly learns the potential differences between normal and simulated abnormal images to obtain a robust classification hyperplane. At the same time, inspired by the mechanism of human memory, MemSeg uses a memory pool to store the general patterns of normal samples. By comparing the similarities and differences between input samples and memory samples in the memory pool to give effective guesses about abnormal regions; In the inference phase, MemSeg directly determines the abnormal regions of the input image in an end-to-end manner. Through experimental validation, MemSeg achieves the state-of-the-art (SOTA) performance on MVTec AD datasets with AUC scores of 99.56% and 98.84% at the image-level and pixel-level, respectively. In addition, MemSeg also has a significant advantage in inference speed benefiting from the end-to-end and straightforward network structure, which better meets the real-time requirement in industrial scenarios.

cs.CV↗

arXiv2022

Room-temperature printing of ultrathin Quasi-2D GaN semiconductor via liquid metal gallium surface confined nitridation reaction

Outstanding wide-bandgap semiconductor material such as gallium nitride (GaN) has been extensively utilized in power electronics, radiofrequency amplifiers, and harsh environment devices. Due to its quantum confinement effect in enabling desired deep-ultraviolet emission, excitonic impact, and electronic transport features, two-dimensional (2D) or ultrathin quasi-2D GaN semiconductors have been one of the most remarkable candidates for future growth of microelectronic devices. Here, for the first time, we reported a large area, wide bandgap, and room-temperature quasi-2D GaN synthesis and printing strategy through introducing the plasma medicated liquid metal gallium surface-confined nitridation reaction mechanism. The developed direct fabrication and compositional process is consistent with various electronics manufacturing approaches and thus opens an easy going way for cost-effective growth of the third-generation semiconductor. In particular, the fully printed field-effect transistors relying on the GaN thus made show p-type switching with an on/off ratio greater than 105, maximum field-effect hole mobility of 53 cm2/(V*s), and a small sub-threshold swing. As it was demonstrated, the present method allows to produce at room temperature the GaN with thickness spanning from 1nm to nanometers. This basic method can be further extended, generalized, and utilized for making various electronic and photoelectronic devices in the coming time.

physics.app-ph↗

arXiv2022

First operation of undoped CsI directly coupled with SiPMs at 77 Kelvin

The light yield of a small undoped cesium iodide (CsI) crystal directly coupled with two silicon photomultipliers (SiPMs) at about 77~Kelvin was measured to be $43.0 \pm 1.1$~photoelectrons (PE) per keV electron-equivalent (keV$_\text{ee}$) using $X$ and $γ$-ray peaks from an $^{241}$Am radioactive source from 18 to 60 keV. The high light yield together with some other technical advantages illustrate the great potential of this novel combination for neutrino and low-mass dark matter detection, particularly at accelerator-based neutrino sources, where random background can be highly suppressed by requiring coincident triggers between SiPMs and beam pulse timing signals. Some potential drawbacks of using cryogenic SiPMs instead of photomultiplier tubes (PMTs) were identified, such as worse energy resolution and optical cross-talks between SiPMs. Their influence to rare-event detection was discussed and possible solutions were provided.

physics.ins-det↗

arXiv2022

Multi-task RNN-T with Semantic Decoder for Streamable Spoken Language Understanding

End-to-end Spoken Language Understanding (E2E SLU) has attracted increasing interest due to its advantages of joint optimization and low latency when compared to traditionally cascaded pipelines. Existing E2E SLU models usually follow a two-stage configuration where an Automatic Speech Recognition (ASR) network first predicts a transcript which is then passed to a Natural Language Understanding (NLU) module through an interface to infer semantic labels, such as intent and slot tags. This design, however, does not consider the NLU posterior while making transcript predictions, nor correct the NLU prediction error immediately by considering the previously predicted word-pieces. In addition, the NLU model in the two-stage system is not streamable, as it must wait for the audio segments to complete processing, which ultimately impacts the latency of the SLU system. In this work, we propose a streamable multi-task semantic transducer model to address these considerations. Our proposed architecture predicts ASR and NLU labels auto-regressively and uses a semantic decoder to ingest both previously predicted word-pieces and slot tags while aggregating them through a fusion network. Using an industry scale SLU and a public FSC dataset, we show the proposed model outperforms the two-stage E2E SLU model for both ASR and NLU metrics.

cs.CL↗

arXiv2022

DS-Net: Dynamic Spatiotemporal Network for Video Salient Object Detection

As moving objects always draw more attention of human eyes, the temporal motive information is always exploited complementarily with spatial information to detect salient objects in videos. Although efficient tools such as optical flow have been proposed to extract temporal motive information, it often encounters difficulties when used for saliency detection due to the movement of camera or the partial movement of salient objects. In this paper, we investigate the complimentary roles of spatial and temporal information and propose a novel dynamic spatiotemporal network (DS-Net) for more effective fusion of spatiotemporal information. We construct a symmetric two-bypass network to explicitly extract spatial and temporal features. A dynamic weight generator (DWG) is designed to automatically learn the reliability of corresponding saliency branch. And a top-down cross attentive aggregation (CAA) procedure is designed so as to facilitate dynamic complementary aggregation of spatiotemporal features. Finally, the features are modified by spatial attention with the guidance of coarse saliency map and then go through decoder part for final saliency map. Experimental results on five benchmarks VOS, DAVIS, FBMS, SegTrack-v2, and ViSal demonstrate that the proposed method achieves superior performance than state-of-the-art algorithms. The source code is available at https://github.com/TJUMMG/DS-Net.

cs.CV↗

arXiv2022

Rapid Elastic Architecture Search under Specialized Classes and Resource Constraints

In many real-world applications, we often need to handle various deployment scenarios, where the resource constraint and the superclass of interest corresponding to a group of classes are dynamically specified. How to efficiently deploy deep models for diverse deployment scenarios is a new challenge. Previous NAS approaches seek to design architectures for all classes simultaneously, which may not be optimal for some individual superclasses. A straightforward solution is to search an architecture from scratch for each deployment scenario, which however is computation-intensive and impractical. To address this, we present a novel and general framework, called Elastic Architecture Search (EAS), permitting instant specializations at runtime for diverse superclasses with various resource constraints. To this end, we first propose to effectively train an over-parameterized network via a superclass dropout strategy during training. In this way, the resulting model is robust to the subsequent superclasses dropping at inference time. Based on the well-trained over-parameterized network, we then propose an efficient architecture generator to obtain promising architectures within a single forward pass. Experiments on three image classification datasets show that EAS is able to find more compact networks with better performance while remarkably being orders of magnitude faster than state-of-the-art NAS methods, e.g., outperforming OFA (once-for-all) by 1.3% on Top-1 accuracy at a budget around 361M #MAdds on ImageNet-10. More critically, EAS is able to find compact architectures within 0.1 second for 50 deployment scenarios.

cs.CV↗

arXiv2022

Ultrafast photothermoelectric effect in Dirac semimetallic Cd3As2 revealed by terahertz emission

The thermoelectric effects of topological semimetals have attracted tremendous research interest because many topological semimetals are excellent thermoelectric materials and thermoelectricity serves as one of their most important potential applications. In this work, we reveal the transient photothermoelectric response of Dirac semimetallic Cd3As2, namely the photo-Seebeck effect and photo-Nernst effect, by studying the terahertz (THz) emission from the transient photocurrent induced by these effects. Our excitation polarization and power dependence confirm that the observed THz emission is due to photothermoelectric effect instead of other nonlinear optical effect. Furthermore, when a weak magnetic field (~0.4 T) is applied, the response clearly indicates an order of magnitude enhancement on transient photothermoelectric current generation compared to the photo-Seebeck effect. Such enhancement supports an ambipolar transport nature of the photo-Nernst current generation in Cd3As2. These results highlight the enhancement of thermoelectric performance can be achieved in topological Dirac semimetals based on the Nernst effect, and our transient studies pave the way for thermoelectric devices applicable for high field circumstance when nonequilibrium state matters. The large THz emission due to highly efficient photothermoelectric conversion is comparable to conventional semiconductors through optical rectification and photo-Dember effect.

cond-mat.mtrl-sci↗