arXiv subjects

Tianyu Zhao

Publications and source records attributed to Tianyu Zhao.

104 records · Page 6Linked to original sources

arXiv2020

Multi-Referenced Training for Dialogue Response Generation

In open-domain dialogue response generation, a dialogue context can be continued with diverse responses, and the dialogue models should capture such one-to-many relations. In this work, we first analyze the training objective of dialogue models from the view of Kullback-Leibler divergence (KLD) and show that the gap between the real world probability distribution and the single-referenced data's probability distribution prevents the model from learning the one-to-many relations efficiently. Then we explore approaches to multi-referenced training in two aspects. Data-wise, we generate diverse pseudo references from a powerful pretrained model to build multi-referenced data that provides a better approximation of the real-world distribution. Model-wise, we propose to equip variational models with an expressive prior, named linear Gaussian model (LGM). Experimental results of automated evaluation and human evaluation show that the methods yield significant improvements over baselines. We will release our code and data in https://github.com/ZHAOTING/dialog-processing.

cs.CL↗

arXiv2020

DeepOPF: Deep Neural Network for DC Optimal Power Flow

We develop DeepOPF as a Deep Neural Network (DNN) approach for solving direct current optimal power flow (DC-OPF) problems. DeepOPF is inspired by the observation that solving DC-OPF for a given power network is equivalent to characterizing a high-dimensional mapping between the load inputs and the dispatch and transmission decisions. We construct and train a DNN model to learn such mapping, then we apply it to obtain optimized operating decisions upon arbitrary load inputs. We adopt uniform sampling to address the over-fitting problem common in generic DNN approaches. We leverage on a useful structure in DC-OPF to significantly reduce the mapping dimension, subsequently cutting down the size of our DNN model and the amount of training data/time needed. We also design a post-processing procedure to ensure the feasibility of the obtained solution. Simulation results of IEEE test cases show that DeepOPF always generates feasible solutions with negligible optimality loss, while speeding up the computing time by two orders of magnitude as compared to conventional approaches implemented in a state-of-the-art solver.

eess.SY↗

arXiv2020

DeepOPF: A Deep Neural Network Approach for Security-Constrained DC Optimal Power Flow

We develop DeepOPF as a Deep Neural Network (DNN) approach for solving security-constrained direct current optimal power flow (SC-DCOPF) problems, which are critical for reliable and cost-effective power system operation.DeepOPF is inspired by the observation that solving SC-DCOPF problems for a given power network is equivalent to depicting a high-dimensional mapping from the load inputs to the generation and phase angle outputs. We first train a DNN to learn the mapping and predict the generations from the load inputs. We then directly reconstruct the phase angles from the generations and loads by using the power flow equations. Such a predict-and-reconstruct approach reduces the dimension of the mapping to learn, subsequently cutting down the size of the DNN and the amount of training data needed. We further derive a condition for tuning the size of the DNN according to the desired approximation accuracy of the load-generation mapping. We develop a post-processing procedure based on $\ell_1$-projection to ensure the feasibility of the obtained solution, which can be of independent interest. Simulation results for IEEE test cases show that DeepOPF generates feasible solutions with less than 0.2% optimality loss, while speeding up the computation time by up to two orders of magnitude as compared to a state-of-the-art solver.

eess.SY↗

arXiv2020

ARPM-net: A novel CNN-based adversarial method with Markov Random Field enhancement for prostate and organs at risk segmentation in pelvic CT images

Purpose: The research is to develop a novel CNN-based adversarial deep learning method to improve and expedite the multi-organ semantic segmentation of CT images, and to generate accurate contours on pelvic CT images. Methods: Planning CT and structure datasets for 120 patients with intact prostate cancer were retrospectively selected and divided for 10-fold cross-validation. The proposed adversarial multi-residual multi-scale pooling Markov Random Field (MRF) enhanced network (ARPM-net) implements an adversarial training scheme. A segmentation network and a discriminator network were trained jointly, and only the segmentation network was used for prediction. The segmentation network integrates a newly designed MRF block into a variation of multi-residual U-net. The discriminator takes the product of the original CT and the prediction/ground-truth as input and classifies the input into fake/real. The segmentation network and discriminator network can be trained jointly as a whole, or the discriminator can be used for fine-tuning after the segmentation network is coarsely trained. Multi-scale pooling layers were introduced to preserve spatial resolution during pooling using less memory compared to atrous convolution layers. An adaptive loss function was proposed to enhance the training on small or low contrast organs. The accuracy of modeled contours was measured with the Dice similarity coefficient (DSC), Average Hausdorff Distance (AHD), Average Surface Hausdorff Distance (ASHD), and relative Volume Difference (VD) using clinical contours as references to the ground-truth. The proposed ARPM-net method was compared to several stateof-the-art deep learning methods.

eess.IV↗

arXiv2020

DeepOPF+: A Deep Neural Network Approach for DC Optimal Power Flow for Ensuring Feasibility

Deep Neural Networks (DNNs) approaches for the Optimal Power Flow (OPF) problem received considerable attention recently. A key challenge of these approaches lies in ensuring the feasibility of the predicted solutions to physical system constraints. Due to the inherent approximation errors, the solutions predicted by DNNs may violate the operating constraints, e.g., the transmission line capacities, limiting their applicability in practice. To address this challenge, we develop DeepOPF+ as a DNN approach based on the so-called "preventive" framework. Specifically, we calibrate the generation and transmission line limits used in the DNN training, thereby anticipating approximation errors and ensuring that the resulting predicted solutions remain feasible. We theoretically characterize the calibration magnitude necessary for ensuring universal feasibility. Our DeepOPF+ approach improves over existing DNN-based schemes in that it ensures feasibility and achieves a consistent speed up performance in both light-load and heavy-load regimes. Detailed simulation results on a range of test instances show that the proposed DeepOPF+ generates 100% feasible solutions with minor optimality loss. Meanwhile, it achieves a computational speedup of two orders of magnitude compared to state-of-the-art solvers.

eess.SY↗

arXiv2020

End-to-end speech-to-dialog-act recognition

Spoken language understanding, which extracts intents and/or semantic concepts in utterances, is conventionally formulated as a post-processing of automatic speech recognition. It is usually trained with oracle transcripts, but needs to deal with errors by ASR. Moreover, there are acoustic features which are related with intents but not represented with the transcripts. In this paper, we present an end-to-end model which directly converts speech into dialog acts without the deterministic transcription process. In the proposed model, the dialog act recognition network is conjunct with an acoustic-to-word ASR model at its latent layer before the softmax layer, which provides a distributed representation of word-level ASR decoding information. Then, the entire network is fine-tuned in an end-to-end manner. This allows for stable training as well as robustness against ASR errors. The model is further extended to conduct DA segmentation jointly. Evaluations with the Switchboard corpus demonstrate that the proposed method significantly improves dialog act recognition accuracy from the conventional pipeline framework.

cs.SD↗

arXiv2020

Designing Precise and Robust Dialogue Response Evaluators

Automatic dialogue response evaluator has been proposed as an alternative to automated metrics and human evaluation. However, existing automatic evaluators achieve only moderate correlation with human judgement and they are not robust. In this work, we propose to build a reference-free evaluator and exploit the power of semi-supervised training and pretrained (masked) language models. Experimental results demonstrate that the proposed evaluator achieves a strong correlation (> 0.6) with human judgement and generalizes robustly to diverse responses and corpora. We open-source the code and data in https://github.com/ZHAOTING/dialog-processing.

cs.CL↗

arXiv2019

Effective Incorporation of Speaker Information in Utterance Encoding in Dialog

In dialog studies, we often encode a dialog using a hierarchical encoder where each utterance is converted into an utterance vector, and then a sequence of utterance vectors is converted into a dialog vector. Since knowing who produced which utterance is essential to understanding a dialog, conventional methods tried integrating speaker labels into utterance vectors. We found the method problematic in some cases where speaker annotations are inconsistent among different dialogs. A relative speaker modeling method is proposed to address the problem. Experimental evaluations on dialog act recognition and response generation show that the proposed method yields superior and more consistent performances.

eess.AS↗

arXiv2019

Content Word-based Sentence Decoding and Evaluating for Open-domain Neural Response Generation

Various encoder-decoder models have been applied to response generation in open-domain dialogs, but a majority of conventional models directly learn a mapping from lexical input to lexical output without explicitly modeling intermediate representations. Utilizing language hierarchy and modeling intermediate information have been shown to benefit many language understanding and generation tasks. Motivated by Broca's aphasia, we propose to use a content word sequence as an intermediate representation for open-domain response generation. Experimental results show that the proposed method improves content relatedness of produced responses, and our models can often choose correct grammar for generated content words. Meanwhile, instead of evaluating complete sentences, we propose to compute conventional metrics on content word sequences, which is a better indicator of content relevance.

cs.CL↗

arXiv2019

Performance Analysis of Joint Transmission Schemes in Ultra-Dense Networks - An Unified Approach

Ultra-dense network (UDN) is one of the enabling technologies to achieve 1000-fold capacity increase in 5G communication systems, and the application of joint transmission (JT) is an effective method to deal with severe inter-cell interferences in UDNs. However, most works done for performance analysis on JT schemes in the literature were based largely on simulation results due to the difficulties in quantitatively identifying the numbers of desired and interfering transmitters. In this work, we are motivated to propose an analytical approach to investigate the performance of JT schemes with a unified approach based on stochastic geometry, which is in particular useful for studying different JT methods and conventional transmission schemes without JT. Using the proposed approach, we can unveil the statistic characteristics (i.e., expectation, moment generation function, variance) of desired signal and interference powers of a given user equipment (UE), and thus system performances, such as average signal-to-interference-plus-noise ratio (SINR), and area spectral efficiency, can be evaluated analytically. The simulation results are used to verify the effectiveness of the proposed unified approach.

eess.SP↗

arXiv2018

Unsupervised Degradation Learning for Single Image Super-Resolution

Deep Convolution Neural Networks (CNN) have achieved significant performance on single image super-resolution (SR) recently. However, existing CNN-based methods use artificially synthetic low-resolution (LR) and high-resolution (HR) image pairs to train networks, which cannot handle real-world cases since the degradation from HR to LR is much more complex than manually designed. To solve this problem, we propose a real-world LR images guided bi-cycle network for single image super-resolution, in which the bidirectional structural consistency is exploited to train both the degradation and SR reconstruction networks in an unsupervised way. Specifically, we propose a degradation network to model the real-world degradation process from HR to LR via generative adversarial networks, and these generated realistic LR images paired with real-world HR images are exploited for training the SR reconstruction network, forming the first cycle. Then in the second reverse cycle, consistency of real-world LR images are exploited to further stabilize the training of SR reconstruction and degradation networks. Extensive experiments on both synthetic and real-world images demonstrate that the proposed algorithm performs favorably against state-of-the-art single image SR methods.

cs.CV↗

arXiv2017

System calibration method for Fourier ptychographic microscopy

Fourier ptychographic microscopy (FPM) is a recently proposed quantitative phase imaging technique with high resolution and wide field-of-view (FOV). In current FPM imaging platforms, systematic error sources come from the aberrations, LED intensity fluctuation, parameter imperfections and noise, which will severely corrupt the reconstruction results with artifacts. Although these problems have been researched and some special methods have been proposed respectively, there is no method to solve all of them. However, the systematic error is a mixture of various sources in the real situation. It is difficult to distinguish a kind of error source from another due to the similar artifacts. To this end, we report a system calibration procedure, termed SC-FPM, based on the simulated annealing (SA) algorithm, LED intensity correction and adaptive step-size strategy, which involves the evaluation of an error matric at each iteration step, followed by the re-estimation of accurate parameters. The great performance has been achieved both in simulation and experiments. The reported system calibration scheme improves the robustness of FPM and relaxes the experiment conditions, which makes the FPM more pragmatic.

physics.ins-det↗

arXiv2010

PCA-based lung motion model

Organ motion induced by respiration may cause clinically significant targeting errors and greatly degrade the effectiveness of conformal radiotherapy. It is therefore crucial to be able to model respiratory motion accurately. A recently proposed lung motion model based on principal component analysis (PCA) has been shown to be promising on a few patients. However, there is still a need to understand the underlying reason why it works. In this paper, we present a much deeper and detailed analysis of the PCA-based lung motion model. We provide the theoretical justification of the effectiveness of PCA in modeling lung motion. We also prove that under certain conditions, the PCA motion model is equivalent to 5D motion model, which is based on physiology and anatomy of the lung. The modeling power of PCA model was tested on clinical data and the average 3D error was found to be below 1 mm.

physics.med-ph↗

arXiv2001

Column size effects of DER fluids

The static yield stress of dielectric electrorheological(DER) fluids of infinite column state and chain state are calculated from the first principle method. The results indicate that the column surface contributions to ER effects is very small and both states will give correct results to the real DER fluids.

cond-mat.soft↗