arXiv subjects

Dong Liu

Publications and source records attributed to Dong Liu.

At least 307 records · Page 17Linked to original sources

arXiv2020

Transferring and Regularizing Prediction for Semantic Segmentation

Semantic segmentation often requires a large set of images with pixel-level annotations. In the view of extremely expensive expert labeling, recent research has shown that the models trained on photo-realistic synthetic data (e.g., computer games) with computer-generated annotations can be adapted to real images. Despite this progress, without constraining the prediction on real images, the models will easily overfit on synthetic data due to severe domain mismatch. In this paper, we novelly exploit the intrinsic properties of semantic segmentation to alleviate such problem for model transfer. Specifically, we present a Regularizer of Prediction Transfer (RPT) that imposes the intrinsic properties as constraints to regularize model transfer in an unsupervised fashion. These constraints include patch-level, cluster-level and context-level semantic prediction consistencies at different levels of image formation. As the transfer is label-free and data-driven, the robustness of prediction is addressed by selectively involving a subset of image regions for model regularization. Extensive experiments are conducted to verify the proposal of RPT on the transfer of models trained on GTA5 and SYNTHIA (synthetic data) to Cityscapes dataset (urban street scenes). RPT shows consistent improvements when injecting the constraints on several neural networks for semantic segmentation. More remarkably, when integrating RPT into the adversarial-based segmentation framework, we report to-date the best results: mIoU of 53.2%/51.7% when transferring from GTA5/SYNTHIA to Cityscapes, respectively.

cs.CV↗

arXiv2020

Neural Network based Explicit Mixture Models and Expectation-maximization based Learning

We propose two neural network based mixture models in this article. The proposed mixture models are explicit in nature. The explicit models have analytical forms with the advantages of computing likelihood and efficiency of generating samples. Computation of likelihood is an important aspect of our models. Expectation-maximization based algorithms are developed for learning parameters of the proposed models. We provide sufficient conditions to realize the expectation-maximization based learning. The main requirements are invertibility of neural networks that are used as generators and Jacobian computation of functional form of the neural networks. The requirements are practically realized using a flow-based neural network. In our first mixture model, we use multiple flow-based neural networks as generators. Naturally the model is complex. A single latent variable is used as the common input to all the neural networks. The second mixture model uses a single flow-based neural network as a generator to reduce complexity. The single generator has a latent variable input that follows a Gaussian mixture distribution. We demonstrate efficiency of proposed mixture models through extensive experiments for generating samples and maximum likelihood based classification.

cs.LG↗

arXiv2020

Powering Hidden Markov Model by Neural Network based Generative Models

Hidden Markov model (HMM) has been successfully used for sequential data modeling problems. In this work, we propose to power the modeling capacity of HMM by bringing in neural network based generative models. The proposed model is termed as GenHMM. In the proposed GenHMM, each HMM hidden state is associated with a neural network based generative model that has tractability of exact likelihood and provides efficient likelihood computation. A generative model in GenHMM consists of mixture of generators that are realized by flow models. A learning algorithm for GenHMM is proposed in expectation-maximization framework. The convergence of the learning GenHMM is analyzed. We demonstrate the efficiency of GenHMM by classification tasks on practical sequential data. Code available at https://github.com/FirstHandScientist/genhmm.

cs.LG↗

arXiv2020

M-LVC: Multiple Frames Prediction for Learned Video Compression

We propose an end-to-end learned video compression scheme for low-latency scenarios. Previous methods are limited in using the previous one frame as reference. Our method introduces the usage of the previous multiple frames as references. In our scheme, the motion vector (MV) field is calculated between the current frame and the previous one. With multiple reference frames and associated multiple MV fields, our designed network can generate more accurate prediction of the current frame, yielding less residual. Multiple reference frames also help generate MV prediction, which reduces the coding cost of MV field. We use two deep auto-encoders to compress the residual and the MV, respectively. To compensate for the compression error of the auto-encoders, we further design a MV refinement network and a residual refinement network, taking use of the multiple reference frames as well. All the modules in our scheme are jointly optimized through a single rate-distortion loss function. We use a step-by-step training strategy to optimize the entire scheme. Experimental results show that the proposed method outperforms the existing learned video compression methods for low-latency mode. Our method also performs better than H.265 in both PSNR and MS-SSIM. Our code and models are publicly available.

eess.IV↗

arXiv2020

Time-lapse reconstruction of the fracture front from diffracted waves arrivals in laboratory hydraulic fracture experiments

4D acoustic imaging via an array of 32 sources / 32 receivers is used to monitor hydraulic fracture propagating in a 250~mm cubic specimen under a true-triaxial state of stress. We present a method based on the arrivals of diffracted waves to reconstruct the fracture geometry (and fluid front when distinct from the fracture front). Using Bayesian model selection, we rank different possible fracture geometries (radial, elliptical, tilted or not) and estimate model error. The imaging is repeated every 4 seconds and provide a quantitative measurement of the growth of these low velocity fractures. We test the proposed method on two experiments performed in two different rocks (marble and gabbro) under experimental conditions characteristic respectively of the fluid lag-viscosity (marble) and toughness (gabbro) dominated hydraulic fracture propagation regimes. In both experiments, about 150 to 200 source-receiver combinations exhibit clear diffracted wave arrivals. The results of the inversion indicate a radial geometry evolving slightly into an ellipse towards the end of the experiment when the fractures feel the specimen boundaries. The estimated modelling error with all models is of the order of the wave arrival picking error. Posterior estimates indicate an uncertainty of the order of a millimeter on the fracture front location for a given acquisition sequence. The reconstructed fracture evolution from diffracted waves is shown to be consistent with the analysis of $90^{\circ}$ incidence transmitted waves across the growing fracture.

physics.geo-ph↗

arXiv2020

Deep High-Resolution Representation Learning for Visual Recognition

High-resolution representations are essential for position-sensitive vision problems, such as human pose estimation, semantic segmentation, and object detection. Existing state-of-the-art frameworks first encode the input image as a low-resolution representation through a subnetwork that is formed by connecting high-to-low resolution convolutions \emph{in series} (e.g., ResNet, VGGNet), and then recover the high-resolution representation from the encoded low-resolution representation. Instead, our proposed network, named as High-Resolution Network (HRNet), maintains high-resolution representations through the whole process. There are two key characteristics: (i) Connect the high-to-low resolution convolution streams \emph{in parallel}; (ii) Repeatedly exchange the information across resolutions. The benefit is that the resulting representation is semantically richer and spatially more precise. We show the superiority of the proposed HRNet in a wide range of applications, including human pose estimation, semantic segmentation, and object detection, suggesting that the HRNet is a stronger backbone for computer vision problems. All the codes are available at~{\url{https://github.com/HRNet}}.

cs.CV↗

arXiv2020

Dual Temporal Memory Network for Efficient Video Object Segmentation

Video Object Segmentation (VOS) is typically formulated in a semi-supervised setting. Given the ground-truth segmentation mask on the first frame, the task of VOS is to track and segment the single or multiple objects of interests in the rest frames of the video at the pixel level. One of the fundamental challenges in VOS is how to make the most use of the temporal information to boost the performance. We present an end-to-end network which stores short- and long-term video sequence information preceding the current frame as the temporal memories to address the temporal modeling in VOS. Our network consists of two temporal sub-networks including a short-term memory sub-network and a long-term memory sub-network. The short-term memory sub-network models the fine-grained spatial-temporal interactions between local regions across neighboring frames in video via a graph-based learning framework, which can well preserve the visual consistency of local regions over time. The long-term memory sub-network models the long-range evolution of object via a Simplified-Gated Recurrent Unit (S-GRU), making the segmentation be robust against occlusions and drift errors. In our experiments, we show that our proposed method achieves a favorable and competitive performance on three frequently-used VOS datasets, including DAVIS 2016, DAVIS 2017 and Youtube-VOS in terms of both speed and accuracy.

cs.CV↗

arXiv2020

Is There Tradeoff between Spatial and Temporal in Video Super-Resolution?

Recent advances of deep learning lead to great success of image and video super-resolution (SR) methods that are based on convolutional neural networks (CNN). For video SR, advanced algorithms have been proposed to exploit the temporal correlation between low-resolution (LR) video frames, and/or to super-resolve a frame with multiple LR frames. These methods pursue higher quality of super-resolved frames, where the quality is usually measured frame by frame in e.g. PSNR. However, frame-wise quality may not reveal the consistency between frames. If an algorithm is applied to each frame independently (which is the case of most previous methods), the algorithm may cause temporal inconsistency, which can be observed as flickering. It is a natural requirement to improve both frame-wise fidelity and between-frame consistency, which are termed spatial quality and temporal quality, respectively. Then we may ask, is a method optimized for spatial quality also optimized for temporal quality? Can we optimize the two quality metrics jointly?

cs.CV↗

arXiv2020

SSFN -- Self Size-estimating Feed-forward Network with Low Complexity, Limited Need for Human Intervention, and Consistent Behaviour across Trials

We design a self size-estimating feed-forward network (SSFN) using a joint optimization approach for estimation of number of layers, number of nodes and learning of weight matrices. The learning algorithm has a low computational complexity, preferably within few minutes using a laptop. In addition the algorithm has a limited need for human intervention to tune parameters. SSFN grows from a small-size network to a large-size network, guaranteeing a monotonically non-increasing cost with addition of nodes and layers. The learning approach uses judicious a combination of `lossless flow property' of some activation functions, convex optimization and instance of random matrix. Consistent performance -- low variation across Monte-Carlo trials -- is found for inference performance (classification accuracy) and estimation of network size.

cs.LG↗

arXiv2020

On Dominant Interference in Random Networks and Communication Reliability

In this paper, we study the characteristics of dominant interference power with directional reception in a random network modelled by a Poisson Point Process. Additionally, the Laplace functional of cumulative interference excluding the $n$ dominant interferers is also derived, which turns out to be a generalization of omni-directional reception and complete accumulative interference. As an application of these results, we study the impact of directional receivers in random networks in terms of outage probability and error probability with queue length constraint.

cs.IT↗

arXiv2020

Will Scale-free Popularity Develop Scale-free Geo-social Networks?

Empirical results show that spatial factors such as distance, population density and communication range affect our social activities, also reflected by the development of ties in social networks. This motivates the need for social network models that take these spatial factors into account. Therefore, in this paper we propose a gravity-low-based geo-social network model, where connections develop according to the popularity of the individuals, but are constrained through their geographic distance and the surrounding population density. Specifically, we consider a power-law distributed popularity, and random node positions governed by a Poisson point process. We evaluate the characteristics of the emerging networks, considering the degree distribution, the average degree of neighbors and the local clustering coefficient. These local metrics reflect the robustness of the network, the information dissemination speed and the communication locality. We show that unless the communication range is strictly limited, the emerging networks are scale-free, with a rank exponent affected by the spatial factors. Even the average neighbor degree and the local clustering coefficient show tendencies known in non-geographic scale-free networks, at least when considering individuals with low popularity. At high-popularity values, however, the spatial constraints lead to popularity-independent average neighbor degrees and clustering coefficients.

physics.soc-ph↗

arXiv2020

Optimizing electrode positions in 2D Electrical Impedance Tomography using deep learning

Electrical Impedance Tomography (EIT) is a powerful tool for non-destructive evaluation, state estimation, and process tomography - among numerous other use cases. For these applications, and in order to reliably reconstruct images of a given process using EIT, we must obtain high-quality voltage measurements from the target of interest. As such, it is obvious that the locations of electrodes used for measuring plays a key role in this task. Yet, to date, methods for optimally placing electrodes either require knowledge on the EIT target (which is, in practice, never fully known) or are computationally difficult to implement numerically. In this paper, we circumvent these challenges and present a straightforward deep learning based approach for optimizing electrodes positions. It is found that the optimized electrode positions outperformed "standard" uniformly-distributed electrode layouts in all test cases. Further, it is found that the use of optimized electrode positions computed using the approach derived herein can reduce errors in EIT reconstructions as well as improve the distinguishability of EIT measurements.

eess.SP↗

arXiv2020

Optimizing Wireless Systems Using Unsupervised and Reinforced-Unsupervised Deep Learning

Resource allocation and transceivers in wireless networks are usually designed by solving optimization problems subject to specific constraints, which can be formulated as variable or functional optimization. If the objective and constraint functions of a variable optimization problem can be derived, standard numerical algorithms can be applied for finding the optimal solution, which however incur high computational cost when the dimension of the variable is high. To reduce the on-line computational complexity, learning the optimal solution as a function of the environment's status by deep neural networks (DNNs) is an effective approach. DNNs can be trained under the supervision of optimal solutions, which however, is not applicable to the scenarios without models or for functional optimization where the optimal solutions are hard to obtain. If the objective and constraint functions are unavailable, reinforcement learning can be applied to find the solution of a functional optimization problem, which is however not tailored to optimization problems in wireless networks. In this article, we introduce unsupervised and reinforced-unsupervised learning frameworks for solving both variable and functional optimization problems without the supervision of the optimal solutions. When the mathematical model of the environment is completely known and the distribution of environment's status is known or unknown, we can invoke unsupervised learning algorithm. When the mathematical model of the environment is incomplete, we introduce reinforced-unsupervised learning algorithms that learn the model by interacting with the environment. Our simulation results confirm the applicability of these learning frameworks by taking a user association problem as an example.

cs.LG↗

arXiv2019

Partition-Aware Adaptive Switching Neural Networks for Post-Processing in HEVC

This paper addresses neural network based post-processing for the state-of-the-art video coding standard, High Efficiency Video Coding (HEVC). We first propose a partition-aware Convolution Neural Network (CNN) that utilizes the partition information produced by the encoder to assist in the post-processing. In contrast to existing CNN-based approaches, which only take the decoded frame as input, the proposed approach considers the coding unit (CU) size information and combines it with the distorted decoded frame such that the artifacts introduced by HEVC are efficiently reduced. We further introduce an adaptive-switching neural network (ASN) that consists of multiple independent CNNs to adaptively handle the variations in content and distortion within compressed-video frames, providing further reduction in visual artifacts. Additionally, an iterative training procedure is proposed to train these independent CNNs attentively on different local patch-wise classes. Experiments on benchmark sequences demonstrate the effectiveness of our partition-aware and adaptive-switching neural networks. The source code can be found at http://min.sjtu.edu.cn/lwydemo/HEVCpostprocessing.html.

eess.IV↗

arXiv2019

Hidden Markov Models for sepsis detection in preterm infants

We explore the use of traditional and contemporary hidden Markov models (HMMs) for sequential physiological data analysis and sepsis prediction in preterm infants. We investigate the use of classical Gaussian mixture model based HMM, and a recently proposed neural network based HMM. To improve the neural network based HMM, we propose a discriminative training approach. Experimental results show the potential of HMMs over logistic regression, support vector machine and extreme learning machine.

cs.LG↗

arXiv2019

Photoluminescence mapping and time-domain thermo-photoluminescence for rapid imaging and measurement of thermal conductivity of boron arsenide

Cubic boron arsenide (BAs) is attracting greater attention due to the recent experimental demonstration of ultrahigh thermal conductivity \k{appa} above 1000 W/mK. However, its bandgap has not been settled and a simple yet effective method to probe its crystal quality is missing. Furthermore, traditional \k{appa} measurement methods are destructive and time consuming, thus they cannot meet the urgent demand for fast screening of high \k{appa} materials. After we experimentally established 1.82 eV as the indirect bandgap of BAs and observed room-temperature band-edge photoluminescence, we developed two new optical techniques that can provide rapid and non-destructive characterization of \k{appa} with little sample preparation: photoluminescence mapping (PL-mapping) and time-domain thermo-photoluminescence (TDTP). PL-mapping provides nearly real-time image of crystal quality and \k{appa} over mm-sized crystal surfaces; while TDTP allows us to pick up any spot on the sample surface and measure its \k{appa} using nanosecond laser pulses. These new techniques reveal that the apparent single crystals are not only non-uniform in \k{appa}, but also are made of domains of very distinct \k{appa}. Because PL-mapping and TDTP are based on the band-edge PL and its dependence on temperature, they can be applied to other semiconductors, thus paving the way for rapid identification and development of high-\k{appa} semiconducting materials.

cond-mat.mtrl-sci↗

arXiv2019

A Comprehensive Benchmark for Single Image Compression Artifacts Reduction

We present a comprehensive study and evaluation of existing single image compression artifacts removal algorithms, using a new 4K resolution benchmark including diversified foreground objects and background scenes with rich structures, called Large-scale Ideal Ultra high definition 4K (LIU4K) benchmark. Compression artifacts removal, as a common post-processing technique, aims at alleviating undesirable artifacts such as blockiness, ringing, and banding caused by quantization and approximation in the compression process. In this work, a systematic listing of the reviewed methods is presented based on their basic models (handcrafted models and deep networks). The main contributions and novelties of these methods are highlighted, and the main development directions, including architectures, multi-domain sources, signal structures, and new targeted units, are summarized. Furthermore, based on a unified deep learning configuration (i.e. same training data, loss function, optimization algorithm, etc.), we evaluate recent deep learning-based methods based on diversified evaluation measures. The experimental results show the state-of-the-art performance comparison of existing methods based on both full-reference, non-reference and task-driven metrics. Our survey would give a comprehensive reference source for future research on single image compression artifacts removal and inspire new directions of the related fields.

eess.IV↗

arXiv2019

Representations for three-point Lie algebras of genus zero

In this paper, we study representations for three-point Lie algebras of genus zero based on the Cox-Jurisich's presentations. We construct two functors which transform simple restricted modules with nonzero levels over the standard affine algebras into simple modules over the three-point affine algebras of genus zero. As a corollary, vertex representations are constructed for the three-point affine algebra of genus zero using vertex operators. Moreover, we construct a Fock module for certain quotient of three-point Virasoro algebra of genus zero.

math.RT↗