arXiv subjects

Dan Liu

Publications and source records attributed to Dan Liu.

At least 55 records · Page 3Linked to original sources

arXiv2022

Pruning On-the-Fly: A Recoverable Pruning Method without Fine-tuning

Most existing pruning works are resource-intensive, requiring retraining or fine-tuning of the pruned models for accuracy. We propose a retraining-free pruning method based on hyperspherical learning and loss penalty terms. The proposed loss penalty term pushes some of the model weights far from zero, while the rest weight values are pushed near zero and can be safely pruned with no need for retraining and a negligible accuracy drop. In addition, our proposed method can instantly recover the accuracy of a pruned model by replacing the pruned values with their mean value. Our method obtains state-of-the-art results in retraining-free pruning and is evaluated on ResNet-18/50 and MobileNetV2 with ImageNet dataset. One can easily get a 50\% pruned ResNet18 model with a 0.47\% accuracy drop. With fine-tuning, the experiment results show that our method can significantly boost the accuracy of the pruned models compared with existing works. For example, the accuracy of a 70\% pruned (except the first convolutional layer) MobileNetV2 model only drops 3.5\%, much less than the 7\% $\sim$ 10\% accuracy drop with conventional methods.

cs.CV↗

arXiv2022

Hyperspherical Quantization: Toward Smaller and More Accurate Models

Model quantization enables the deployment of deep neural networks under resource-constrained devices. Vector quantization aims at reducing the model size by indexing model weights with full-precision embeddings, i.e., codewords, while the index needs to be restored to 32-bit during computation. Binary and other low-precision quantization methods can reduce the model size up to 32$\times$, however, at the cost of a considerable accuracy drop. In this paper, we propose an efficient framework for ternary quantization to produce smaller and more accurate compressed models. By integrating hyperspherical learning, pruning and reinitialization, our proposed Hyperspherical Quantization (HQ) method reduces the cosine distance between the full-precision and ternary weights, thus reducing the bias of the straight-through gradient estimator during ternary quantization. Compared with existing work at similar compression levels ($\sim$30$\times$, $\sim$40$\times$), our method significantly improves the test accuracy and reduces the model size.

cs.CV↗

arXiv2022

Teacher-Student Architecture for Knowledge Learning: A Survey

Although Deep Neural Networks (DNNs) have shown a strong capacity to solve large-scale problems in many areas, such DNNs with voluminous parameters are hard to be deployed in a real-time system. To tackle this issue, Teacher-Student architectures were first utilized in knowledge distillation, where simple student networks can achieve comparable performance to deep teacher networks. Recently, Teacher-Student architectures have been effectively and widely embraced on various knowledge learning objectives, including knowledge distillation, knowledge expansion, knowledge adaption, and multi-task learning. With the help of Teacher-Student architectures, current studies are able to achieve multiple knowledge-learning objectives through lightweight and effective student networks. Different from the existing knowledge distillation surveys, this survey detailedly discusses Teacher-Student architectures with multiple knowledge learning objectives. In addition, we systematically introduce the knowledge construction and optimization process during the knowledge learning and then analyze various Teacher-Student architectures and effective learning schemes that have been leveraged to learn representative and robust knowledge. This paper also summarizes the latest applications of Teacher-Student architectures based on different purposes (i.e., classification, recognition, and generation). Finally, the potential research directions of knowledge learning are investigated on the Teacher-Student architecture design, the quality of knowledge, and the theoretical studies of regression-based learning, respectively. With this comprehensive survey, both industry practitioners and the academic community can learn insightful guidelines about Teacher-Student architectures on multiple knowledge learning objectives.

cs.LG↗

arXiv2022

Unusual electric polarization behavior in elemental quasi-2D allotropes of selenium

We investigate tunable electric polarization and electronic structure of quasi-two-dimensional (quasi-2D) allotropes of selenium, which are formed from their constituent one-dimensional (1D) structures through an inter-chain interaction facilitated by the multi-valence nature of Se. Our em ab initio calculations reveal that different quasi-2D Se allotropes display different types of electric polarization, including ferroelectric (FE) polarization normal to the chain direction in alpha and delta allotropes, non-collinear ferrielectric (FiE) polarization along the chain axis in tau-Se, and anti-ferroelectric (AFE) polarization in eta-Se. The magnitude and direction of the polarization can be changed by a previously unexplored rotation of the constituent chains. In that case, an in-plane polarization direction may change to out-of-plane in alpha-Se and delta-Se, flip its direction, and even disappear in tau-Se. Also, the band gap may be reduced and changed from indirect to direct by rotating the constituent chains about their axes in these quasi-2D Se allotropes.

cond-mat.mtrl-sci↗

arXiv2022

Ferroelectricity induced by the absorption of water molecules on double helix SnIP

We study the ferroelectricity in a one-dimensional system composed of a double helix SnIP with absorbing water molecules. Our ab initio calculations reveal two factors that are critical to the electrical polarization. The first one is the orientation of polarized water molecules staying in the R2 region of SnIP. The second one is the displacement of I atom which roots from subtle interaction with absorbed water molecules. A reasonable scenario of polarization flipping is proposed in this study. In the scenario, the water molecule is rolling-up with keeping the magnitude of its electrical dipole and changing its direction, meanwhile, the displacement of I atoms is also reversed. Highly tunable polarization can be achieved by applying strain, with 26.5% of polarization enhancement by applying tensile strain, with only 4% degradation is observed with 4% compressive strain. Finally, the direct band gap is also found to be correlated with strain.

cond-mat.mtrl-sci↗

arXiv2022

Tunable linearity of high-performance vertical dual-gate vdW phototransistor

Layered two-dimensional (2D) semiconductors have been widely exploited in photodetectors due to their excellent electronic and optoelectronic properties. To improve their performance, photogating, photoconductive, photovoltaic, photothermoelectric, and other effects have been used in phototransistors and photodiodes made with 2D semiconductors or hybrid structures. However, it is difficult to achieve the desired high responsivity and linear photoresponse simultaneously in a monopolar conduction channel or a p-n junction. Here we present dual-channel conduction with ambipolar multilayer WSe2 by employing the device concept of dual-gate phototransistor, where p-type and n-type channels are produced in the same semiconductor using opposite dual-gating. It is possible to tune the photoconductive gain using a vertical electric field, so that the gain is constant with respect to the light intensity-a linear photoresponse, with a high responsivity of ~2.5*10^4 A/W. Additionally, the 1/f noise of the device is kept at a low level under the opposite dual-gating due to the reduction of current and carrier fluctuation, resulting in a high detectivity of ~2*10^13 Jones in the linear photoresponse regime. The linear photoresponse and high performance of our dual-gate WSe2 phototransistor offer the possibility of achieving high-resolution and quantitative light detection with layered 2D semiconductors.

cond-mat.mtrl-sci↗

arXiv2022

AdaCoach: A Virtual Coach for Training Customer Service Agents

With the development of online business, customer service agents gradually play a crucial role as an interface between the companies and their customers. Most companies spend a lot of time and effort on hiring and training customer service agents. To this end, we propose AdaCoach: A Virtual Coach for Training Customer Service Agents, to promote the ability of newly hired service agents before they get to work. AdaCoach is designed to simulate real customers who seek help and actively initiate the dialogue with the customer service agents. Besides, AdaCoach uses an automated dialogue evaluation model to score the performance of the customer agent in the training process, which can provide necessary assistance when the newly hired customer service agent encounters problems. We apply recent NLP technologies to ensure efficient run-time performance in the deployed system. To the best of our knowledge, this is the first system that trains the customer service agent through human-computer interaction. Until now, the system has already supported more than 500,000 simulation training and cultivated over 1000 qualified customer service agents.

cs.CL↗

arXiv2022

Jammed disks of two sizes and weights in a channel: Alternating sequences

Disks of two sizes and weights in alternating sequence are confined to a long and narrow channel. The axis of the channel is horizontal and its plane vertical. The channel is closed off by pistons that freeze jammed microstates out of loose disk configurations subject to moderate pressure, gravity, and random agitations. Disk sizes and channel width are such that under jamming no disk remains loose and all disks touch one wall. We present exact results for the characterization of jammed macrostates including volume and entropy. The rigorous analysis divides the disk sequences of jammed microstates into overlapping tiles from which we construct a small number of species of statistically interacting particles. Jammed macrostates depend on dimensionless control parameters inferred from ratios between measures of expansion work against the pistons, gravitational potential energy, and intensity of random agitations. These control parameters enter the configurational statistics via the activation energies prior to jamming of the particles. The range of disk weights naturally divides into regimes where qualitatively different features come into play. We sketch a path toward generalizations that include random sequences under a modified jamming protocol.

cond-mat.soft↗

arXiv2021

Changes in European Solidarity Before and During COVID-19: Evidence from a Large Crowd- and Expert-Annotated Twitter Dataset

We introduce the well-established social scientific concept of social solidarity and its contestation, anti-solidarity, as a new problem setting to supervised machine learning in NLP to assess how European solidarity discourses changed before and after the COVID-19 outbreak was declared a global pandemic. To this end, we annotate 2.3k English and German tweets for (anti-)solidarity expressions, utilizing multiple human annotators and two annotation approaches (experts vs.\ crowds). We use these annotations to train a BERT model with multiple data augmentation strategies. Our augmented BERT model that combines both expert and crowd annotations outperforms the baseline BERT classifier trained with expert annotations only by over 25 points, from 58\% macro-F1 to almost 85\%. We use this high-quality model to automatically label over 270k tweets between September 2019 and December 2020. We then assess the automatically labeled data for how statements related to European (anti-)solidarity discourses developed over time and in relation to one another, before and during the COVID-19 crisis. Our results show that solidarity became increasingly salient and contested during the crisis. While the number of solidarity tweets remained on a higher level and dominated the discourse in the scrutinized time frame, anti-solidarity tweets initially spiked, then decreased to (almost) pre-COVID-19 values before rising to a stable higher level until the end of 2020.

cs.CL↗

arXiv2021

The USTC-NELSLIP Systems for Simultaneous Speech Translation Task at IWSLT 2021

This paper describes USTC-NELSLIP's submissions to the IWSLT2021 Simultaneous Speech Translation task. We proposed a novel simultaneous translation model, Cross Attention Augmented Transducer (CAAT), which extends conventional RNN-T to sequence-to-sequence tasks without monotonic constraints, e.g., simultaneous translation. Experiments on speech-to-text (S2T) and text-to-text (T2T) simultaneous translation tasks shows CAAT achieves better quality-latency trade-offs compared to \textit{wait-k}, one of the previous state-of-the-art approaches. Based on CAAT architecture and data augmentation, we build S2T and T2T simultaneous translation systems in this evaluation campaign. Compared to last year's optimal systems, our S2T simultaneous translation system improves by an average of 11.3 BLEU for all latency regimes, and our T2T simultaneous translation system improves by an average of 4.6 BLEU.

cs.CL↗

arXiv2021

Efficiency at maximum power of thermoelectric heat engines with the symmetric semiconductor superlattice

Efficiency at maximum power (EMP) is a very important specification for a heat engine to evaluate the capacity of outputting adequate power with high efficiency. It has been proved theoretically that the limit EMP of thermoelectric heat engine can be achieved with the hypothetical boxcar-shaped electron transmission, which is realized here by the resonant tunneling in the one-dimensional symmetric InP/InSe superlattice. It is found with the transfer matrix method that a symmetric mode is robust that regardless of the periodicity, and the obtained boxcar-like electron transmission stems from the strong coupling between symmetric mode and Fabry-Pérot modes inside the allowed band. High uniformity of the boxcar-like transmission and the sharp drop of the transmission edge are both beneficial to the maximum power and the EMP, which are optimized by the bias voltage and the thicknesses of barrier and well. The maximum power and EMP are extracted with the help of machine learning technique, and more than 95% of their theoretical limits can both be achieved for smaller temperature difference, smaller barrier width and larger well width. We hope the obtain results could provide some basic guidance for the future designs of high EMP thermoelectric heat engines.

cond-mat.mes-hall↗

arXiv2020

Learning to Retrieve Entity-Aware Knowledge and Generate Responses with Copy Mechanism for Task-Oriented Dialogue Systems

Task-oriented conversational modeling with unstructured knowledge access, as track 1 of the 9th Dialogue System Technology Challenges (DSTC 9), requests to build a system to generate response given dialogue history and knowledge access. This challenge can be separated into three subtasks, (1) knowledge-seeking turn detection, (2) knowledge selection, and (3) knowledge-grounded response generation. We use pre-trained language models, ELECTRA and RoBERTa, as our base encoder for different subtasks. For subtask 1 and 2, the coarse-grained information like domain and entity are used to enhance knowledge usage. For subtask 3, we use a latent variable to encode dialog history and selected knowledge better and generate responses combined with copy mechanism. Meanwhile, some useful post-processing strategies are performed on the model's final output to make further knowledge usage in the generation task. As shown in released evaluation results, our proposed system ranks second under objective metrics and ranks fourth under human metrics.

cs.CL↗

arXiv2020

Jammed disks of two sizes in a narrow channel

A granular-matter model is exactly solved, where disks of two sizes and weights in alternating sequence are confined to a narrow channel. The axis of the channel is horizontal and its plane vertical. Disk sizes and channel width are such that under jamming no disks remain loose and all disks touch one wall. Jammed microstates are characterized via statistically interacting particles constructed out of two-disk tiles. Jammed macrostates depend on measures of expansion work, gravitational potential energy, and intensity of random agitations before jamming. The dependence of configurational entropy on excess volume exhibits a critical point.

cond-mat.soft↗

arXiv2020

Edge Learning with Unmanned Ground Vehicle: Joint Path, Energy and Sample Size Planning

Edge learning (EL), which uses edge computing as a platform to execute machine learning algorithms, is able to fully exploit the massive sensing data generated by Internet of Things (IoT). However, due to the limited transmit power at IoT devices, collecting the sensing data in EL systems is a challenging task. To address this challenge, this paper proposes to integrate unmanned ground vehicle (UGV) with EL. With such a scheme, the UGV could improve the communication quality by approaching various IoT devices. However, different devices may transmit different data for different machine learning jobs and a fundamental question is how to jointly plan the UGV path, the devices' energy consumption, and the number of samples for different jobs? This paper further proposes a graph-based path planning model, a network energy consumption model and a sample size planning model that characterizes F-measure as a function of the minority class sample size. With these models, the joint path, energy and sample size planning (JPESP) problem is formulated as a large-scale mixed integer nonlinear programming (MINLP) problem, which is nontrivial to solve due to the high-dimensional discontinuous variables related to UGV movement. To this end, it is proved that each IoT device should be served only once along the path, thus the problem dimension is significantly reduced. Furthermore, to handle the discontinuous variables, a tabu search (TS) based algorithm is derived, which converges in expectation to the optimal solution to the JPESP problem. Simulation results under different task scenarios show that our optimization schemes outperform the fixed EL and the full path EL schemes.

eess.SP↗

arXiv2020

Ferromagnetic kinetic exchange interaction in magnetic insulators

The superexchange theory predicts dominant antiferromagnetic kinetic interaction when the orbitals accommodating magnetic electrons are covalently bonded through diamagnetic bridging atoms/groups. Here we show that explicit consideration of magnetic and (leading) bridging orbitals, together with the electron transfer between the former, reveals a strong ferromagnetic kinetic exchange contribution. First principle calculations show that it is comparable in strength with antiferromagnetic superexchange in a number of magnetic materials with diamagnetic metal bridges. In particular, it is responsible for a very large ferromagnetic coupling ($-10$ meV) between the iron ions in a Fe$^{3+}$-Co$^{3+}$-Fe$^{3+}$ complex.

cond-mat.str-el↗

arXiv2020

Efficient Growth and Characterization of One Dimensional Transition Metal Tellurides Inside Carbon Nanotubes

Atomically thin one dimensional (1D) van der Waals wires of transition metal monochalocogenides (TMMs) have been anticipated as promising building blocks for integrated nanoelectronics. While reliable production of TMM nanowires has eluded scientists over the past few decades, we finally demonstrated a bottom up fabrication of MoTe nanowires inside carbon nanotubes (CNTs). Still, the current synthesis method is based on vacuum annealing of reactive MoTe2, and limits access to a variety of TMMs. Here we report an expanded framework for high yield synthesis of the 1D tellurides including WTe, an unprecedented family of TMMs. Experimental and theoretical analyses revealed that the choice of suitable metal oxides as a precursor provides useful yield for their characterization. These TMM nanowires exhibit a significant optical absorption in the visible light region. More important, electronic properties of CNTs can be tuned by encapsulating different TMM nanowires.

cond-mat.mtrl-sci↗

arXiv2020

First principles study of the vibronic coupling in positively charged C$_{60}^+$

Vibronic coupling parameters for C$_{60}^{+}$ were derived via DFT calculations with hybrid B3LYP and CAM-B3LYP functional, based on which the static Jahn-Teller effect were analyzed. The global minima of adiabatic potential energy surface (APES) shows a D$_{5d}$ Jahn-Teller deformation, with stabilization energies of 110 and 129 meV (with B3LYP and CAM-B3LYP respectively), which are two times larger than that in C$_{60}^-$, suggesting the crucial role of the dynamical Jahn-Teller effect. Present results enable us to assess the actual situation of dynamical Jahn-Teller effect in C$_{60}^{+}$ and excited C$_{60}$ in combination with the established parameters for C$_{60}^{-}$.

physics.chem-ph↗

arXiv2020

Strain-Controlled Magnetic Ordering in 2D Carbon Metamaterials

We use ab initio spin-polarized density functional theory to study the magnetic order in a Kagomé-like 2D metamaterial consisting of pristine or substitutionally doped phenalenyl radicals polymerized into a nanoporous, graphene-like structure. In this and in a larger class of related structures, the constituent polyaromatic hydrocarbon molecules can be considered as quantum dots that may carry a net magnetic moment. The structure of this porous system and the coupling between the quantum dots may be changed significantly by applying moderate strain, thus allowing to control the magnetic order and the underlying electronic structure.

cond-mat.mes-hall↗