arXiv subjects

Ming Jiang

Publications and source records attributed to Ming Jiang.

At least 73 records · Page 4Linked to original sources

arXiv2020

Joint super-resolution image reconstruction and parameter identification in imaging operator: Analysis of bilinear operator equations, numerical solution, and application to magnetic particle imaging

One important property of imaging modalities and related applications is the resolution of image reconstructions which relies on various factors such as instrumentation or data processing. Restrictions in resolution can have manifold origins, e.g., limited resolution of available data, noise level in the data, and/or inexact model operators. In this work we investigate a novel data processing approach suited for inexact model operators. Here, two different information sources, high-dimensional model information and high-quality measurement on a lower resolution, are comprised in a hybrid approach. The joint reconstruction of a high resolution image and parameters of the imaging operator are obtained by minimizing a Tikhonov-type functional. The hybrid approach is analyzed for bilinear operator equations with respect to stability, convergence, and convergence rates. We further derive an algorithmic solution exploiting an algebraic reconstruction technique. The study is complemented by numerical results ranging from an academic test case to image reconstruction in magnetic particle imaging.

math.NA↗

arXiv2020

A CRC-aided Hybrid Decoding for Turbo Codes

Turbo codes and CRC codes are usually decoded separately according to the serially concatenated inner codes and outer codes respectively. In this letter, we propose a hybrid decoding algorithm of turbo-CRC codes, where the outer codes, CRC codes, are not used for error detection but as an assistance to improve the error correction performance. Two independent iterative decoding and reliability based decoding are carried out in a hybrid schedule, which can efficiently decode the two different codes as an entire codeword. By introducing an efficient error detecting method based on normalized Euclidean distance without CRC check, significant gain can be obtained by using the hybrid decoding method without loss of the error detection ability.

eess.SP↗

arXiv2020

Joint Shortening and Puncturing Optimization for Structured LDPC Codes

The demand for flexible broadband wireless services makes the pruning technique, including both shortening and puncturing, an indispensable component of error correcting codes. The analysis of the pruning process for structured lowdensity parity-check (LDPC) codes can be considerably simplified with their equivalent representations through base-matrices or protographs. In this letter, we evaluate the thresholds of the pruned base-matrices by using protograph based on extrinsic information transfer (PEXIT). We also provide an efficient method to optimize the pruning patterns, which can significantly improve the thresholds of both the full-length patterns and the sub-patterns. Numerical results show that the structured LDPC codes pruned by the improved patterns outperform those with the existing patterns.

eess.SP↗

arXiv2020

Lossless Attention in Convolutional Networks for Facial Expression Recognition in the Wild

Unlike the constraint frontal face condition, faces in the wild have various unconstrained interference factors, such as complex illumination, changing perspective and various occlusions. Facial expressions recognition (FER) in the wild is a challenging task and existing methods can't perform well. However, for occluded faces (containing occlusion caused by other objects and self-occlusion caused by head posture changes), the attention mechanism has the ability to focus on the non-occluded regions automatically. In this paper, we propose a Lossless Attention Model (LLAM) for convolutional neural networks (CNN) to extract attention-aware features from faces. Our module avoids decay information in the process of generating attention maps by using the information of the previous layer and not reducing the dimensionality. Sequentially, we adaptively refine the feature responses by fusing the attention map with the feature map. We participate in the seven basic expression classification sub-challenges of FG-2020 Affective Behavior Analysis in-the-wild Challenge. And we validate our method on the Aff-Wild2 datasets released by the Challenge. The total accuracy (Accuracy) and the unweighted mean (F1) of our method on the validation set are 0.49 and 0.38 respectively, and the final result is 0.42 (0.67 F1-Score + 0.33 Accuracy).

cs.CV↗

arXiv2019

Quality Assessment of In-the-Wild Videos

Quality assessment of in-the-wild videos is a challenging problem because of the absence of reference videos and shooting distortions. Knowledge of the human visual system can help establish methods for objective quality assessment of in-the-wild videos. In this work, we show two eminent effects of the human visual system, namely, content-dependency and temporal-memory effects, could be used for this purpose. We propose an objective no-reference video quality assessment method by integrating both effects into a deep neural network. For content-dependency, we extract features from a pre-trained image classification neural network for its inherent content-aware property. For temporal-memory effects, long-term dependencies, especially the temporal hysteresis, are integrated into the network with a gated recurrent unit and a subjectively-inspired temporal pooling layer. To validate the performance of our method, experiments are conducted on three publicly available in-the-wild video quality assessment databases: KoNViD-1k, CVD2014, and LIVE-Qualcomm, respectively. Experimental results demonstrate that our proposed method outperforms five state-of-the-art methods by a large margin, specifically, 12.39%, 15.71%, 15.45%, and 18.09% overall performance improvements over the second-best method VBLIINDS, in terms of SROCC, KROCC, PLCC and RMSE, respectively. Moreover, the ablation study verifies the crucial role of both the content-aware features and the modeling of temporal-memory effects. The PyTorch implementation of our method is released at https://github.com/lidq92/VSFA.

cs.MM↗

arXiv2019

REO-Relevance, Extraness, Omission: A Fine-grained Evaluation for Image Captioning

Popular metrics used for evaluating image captioning systems, such as BLEU and CIDEr, provide a single score to gauge the system's overall effectiveness. This score is often not informative enough to indicate what specific errors are made by a given system. In this study, we present a fine-grained evaluation method REO for automatically measuring the performance of image captioning systems. REO assesses the quality of captions from three perspectives: 1) Relevance to the ground truth, 2) Extraness of the content that is irrelevant to the ground truth, and 3) Omission of the elements in the images and human references. Experiments on three benchmark datasets demonstrate that our method achieves a higher consistency with human judgments and provides more intuitive evaluation results than alternative metrics.

cs.CL↗

arXiv2019

TIGEr: Text-to-Image Grounding for Image Caption Evaluation

This paper presents a new metric called TIGEr for the automatic evaluation of image captioning systems. Popular metrics, such as BLEU and CIDEr, are based solely on text matching between reference captions and machine-generated captions, potentially leading to biased evaluations because references may not fully cover the image content and natural language is inherently ambiguous. Building upon a machine-learned text-image grounding model, TIGEr allows to evaluate caption quality not only based on how well a caption represents image content, but also on how well machine-generated captions match human-generated captions. Our empirical tests show that TIGEr has a higher consistency with human judgments than alternative existing metrics. We also comprehensively assess the metric's effectiveness in caption evaluation by measuring the correlation between human judgments and metric scores.

cs.CL↗

arXiv2019

Single Image Blind Deblurring Using Multi-Scale Latent Structure Prior

Blind image deblurring is a challenging problem in computer vision, which aims to restore both the blur kernel and the latent sharp image from only a blurry observation. Inspired by the prevalent self-example prior in image super-resolution, in this paper, we observe that a coarse enough image down-sampled from a blurry observation is approximately a low-resolution version of the latent sharp image. We prove this phenomenon theoretically and define the coarse enough image as a latent structure prior of the unknown sharp image. Starting from this prior, we propose to restore sharp images from the coarsest scale to the finest scale on a blurry image pyramid, and progressively update the prior image using the newly restored sharp image. These coarse-to-fine priors are referred to as \textit{Multi-Scale Latent Structures} (MSLS). Leveraging the MSLS prior, our algorithm comprises two phases: 1) we first preliminarily restore sharp images in the coarse scales; 2) we then apply a refinement process in the finest scale to obtain the final deblurred image. In each scale, to achieve lower computational complexity, we alternately perform a sharp image reconstruction with fast local self-example matching, an accelerated kernel estimation with error compensation, and a fast non-blind image deblurring, instead of computing any computationally expensive non-convex priors. We further extend the proposed algorithm to solve more challenging non-uniform blind image deblurring problem. Extensive experiments demonstrate that our algorithm achieves competitive results against the state-of-the-art methods with much faster running speed.

cs.CV↗

arXiv2018

Parsing R-CNN for Instance-Level Human Analysis

Instance-level human analysis is common in real-life scenarios and has multiple manifestations, such as human part segmentation, dense pose estimation, human-object interactions, etc. Models need to distinguish different human instances in the image panel and learn rich features to represent the details of each instance. In this paper, we present an end-to-end pipeline for solving the instance-level human analysis, named Parsing R-CNN. It processes a set of human instances simultaneously through comprehensive considering the characteristics of region-based approach and the appearance of a human, thus allowing representing the details of instances. Parsing R-CNN is very flexible and efficient, which is applicable to many issues in human instance analysis. Our approach outperforms all state-of-the-art methods on CIHP (Crowd Instance-level Human Parsing), MHP v2.0 (Multi-Human Parsing) and DensePose-COCO datasets. Based on the proposed Parsing R-CNN, we reach the 1st place in the COCO 2018 Challenge DensePose Estimation task. Code and models are public available.

cs.CV↗

arXiv2018

Cascade-Net: a New Deep Learning Architecture for OFDM Detection

In this paper, we consider using deep neural network for OFDM symbol detection and demonstrate its performance advantages in combating large Doppler Shift. In particular, a new architecture named Cascade-Net is proposed for detection, where deep neural network is cascading with a zero-forcing preprocessor to prevent the network stucking in a saddle point or a local minimum point. In addition, we propose a sliding detection approach in order to detect OFDM symbols with large number of subcarriers. We evaluate this new architecture, as well as the sliding algorithm, using the Rayleigh channel with large Doppler spread, which could degrade detection performance in an OFDM system and is especially severe for high frequency band and mmWave communications. The numerical results of OFDM detection in SISO scenario show that cascade-net can achieve better performance than zero-forcing method while providing robustness against ill conditioned channels. We also show the better performance of the sliding cascade network (SCN) compared to sliding zero-forcing detector through numerical simulation.

eess.SP↗

arXiv2018

Theoretical analyses of the sub-grid quantities' effect on filtered Eulerian drag force

An expression of the filtered Eulerian drag force is proposed based on the second order Taylor polynomial approximation of the microscopic Eulerian drag coefficient. Theoretical computations of the expression are performed at low Reynolds numbers based on an available microscopic drag model. It shows that four sub-grid quantities, i.e., the gas drift velocity, the solid drift velocity, the scalar variance of solid volume fraction and a third-order moment, defined as the covariance of squared solid volume fraction and the slip velocity, are significant for an accurate estimation of the filtered drag force at low Reynolds numbers. The gas drift velocity is nothing but the drift velocity defined by Parmentier et al. (AIChE Journal, 2012, 58 (4): 1084-1098), and in analogy to the gas drift velocity, we defines the solid drift velocity in the present work. The great relevance of the gas drift velocity and the scalar variance of solid volume fraction to the filtered drag force have been demonstrated through numerous correlative analyses of fully resolved simulations. Present theoretical analysis shows that the gas drift velocity term is primarily responsible for the drag reduction, whereas the solid drift velocity term is prone to attenuate this reduction, especially at moderate and high solid volume fractions. The scalar variance of solid volume fraction term is found to increase the filtered drag force in the full range of solid volume fractions. The third-order term has exactly the same coefficient with that of the variance term, and it mostly attains negative values with the trend to decrease the filtered drag force.

physics.flu-dyn↗

arXiv2018

Quality Assessment for Tone-Mapped HDR Images Using Multi-Scale and Multi-Layer Information

Tone mapping operators and multi-exposure fusion methods allow us to enjoy the informative contents of high dynamic range (HDR) images with standard dynamic range devices, but also introduce distortions into HDR contents. Therefore methods are needed to evaluate tone-mapped image quality. Due to the complexity of possible distortions in a tone-mapped image, information from different scales and different levels should be considered when predicting tone-mapped image quality. So we propose a new no-reference method of tone-mapped image quality assessment based on multi-scale and multi-layer features that are extracted from a pre-trained deep convolutional neural network model. After being aggregated, the extracted features are mapped to quality predictions by regression. The proposed method is tested on the largest public database for TMIQA and compared to existing no-reference methods. The experimental results show that the proposed method achieves better performance.

cs.MM↗

arXiv2018

Exploiting High-Level Semantics for No-Reference Image Quality Assessment of Realistic Blur Images

To guarantee a satisfying Quality of Experience (QoE) for consumers, it is required to measure image quality efficiently and reliably. The neglect of the high-level semantic information may result in predicting a clear blue sky as bad quality, which is inconsistent with human perception. Therefore, in this paper, we tackle this problem by exploiting the high-level semantics and propose a novel no-reference image quality assessment method for realistic blur images. Firstly, the whole image is divided into multiple overlapping patches. Secondly, each patch is represented by the high-level feature extracted from the pre-trained deep convolutional neural network model. Thirdly, three different kinds of statistical structures are adopted to aggregate the information from different patches, which mainly contain some common statistics (i.e., the mean\&standard deviation, quantiles and moments). Finally, the aggregated features are fed into a linear regression model to predict the image quality. Experiments show that, compared with low-level features, high-level features indeed play a more critical role in resolving the aforementioned challenging problem for quality estimation. Besides, the proposed method significantly outperforms the state-of-the-art methods on two realistic blur image databases and achieves comparable performance on two synthetic blur image databases.

eess.IV↗

arXiv2018

Joint bi-modal image reconstruction of DOT and XCT with an extended Mumford-Shah functional

Feature similarity measures are indispensable for joint image reconstruction in multi-modality medical imaging, which enable joint multi-modal image reconstruction (JmmIR) by communication of feature information from one modality to another, and vice versa. In this work, we establish an image similarity measure in terms of image edges from Tversky's theory of feature similarity in psychology. For joint bi-modal image reconstruction (JbmIR), it is found that this image similarity measure is an extended Mumford-Shah functional with a-priori edge information proposed previously from the perspective of regularization approach. This image similarity measure consists of Hausdorff measures of the common and different parts of image edges from both modalities. By construction, it posits that two images are more similar if they have more common edges and fewer unique/distinctive features, and will not force the nonexistent structures to be reconstructed when applied to JbmIR. With the Gamma-approximation of the JbmIR functional, an alternating minimization method is proposed for the JbmIR of diffuse optical tomography and x-ray computed tomography. The performance of the proposed method is evaluated by three numerical phantoms. It is found that the proposed method improves the reconstructed image quality by more than 10% compared to single modality image reconstruction (SmIR) in terms of the structural similarity index measure (SSIM)

eess.IV↗

arXiv2018

A Neural Network Aided Approach for LDPC Coded DCO-OFDM with Clipping Distortion

In this paper, a neural network-aided bit-interleaved coded modulation (NN-BICM) receiver is designed to mitigate the nonlinear clipping distortion in the LDPC coded direct currentbiased optical orthogonal frequency division multiplexing (DCOOFDM) systems. Taking the cross-entropy as loss function, a feed forward network is trained by backpropagation algorithm to output the condition probability through the softmax activation function, thereby assisting in a modified log-likelihood ratio (LLR) improvement. To reduce the complexity, this feed-forward network simplifies the input layer with a single symbol and the corresponding Gaussian variance instead of focusing on the inter-carrier interference between multiple subcarriers. On the basis of the neural network-aided BICM with Gray labelling, we propose a novel stacked network architecture of the bitinterleaved coded modulation with iterative decoding (NN-BICMID). Its performance has been improved further by calculating the condition probability with the aid of a priori probability that derived from the extrinsic LLRs in the LDPC decoder at the last iteration, at the expense of customizing neural network detectors at each iteration time separately. Utilizing the optimal DC bias as the midpoint of the dynamic region, the simulation results demonstrate that both the NN-BICM and NN-BICM-ID schemes achieve noticeable performance gains than other counterparts, in which the NN-BICM-ID clearly outperforms NN-BICM with various modulation and coding schemes.

cs.LG↗

arXiv2018

Transport equation for the drift velocity in predicting the filtered Eulerian drag force: a theoretical development

The drift velocity has been proven to have significant relevance to the filtered Eulerian drag force by numerous correlative analyses of fully resolved simulations. It is a sub-grid quantity defined as the difference between the filtered gas velocity seen by the particle phase and the resolved filtered gas velocity. In literature, it is shown that various algebraic models for the drift velocity fail to give entirely satisfactory prediction of the filtered drag force. Unlike previous works, we theoretically derived the transport equation for the drift velocity from the standard two-fluid model (TFM) equations without any additional assumptions. The new approach, though requires additional closures for the new unresolved terms, is believed to have the potential to improve the precision of the estimation of this sub-grid quantity and hence give better prediction of the filtered drag force for gas-solid fluidization problems.

physics.flu-dyn↗

arXiv2018

Beyond Trade-off: Accelerate FCN-based Face Detector with Higher Accuracy

Fully convolutional neural network (FCN) has been dominating the game of face detection task for a few years with its congenital capability of sliding-window-searching with shared kernels, which boiled down all the redundant calculation, and most recent state-of-the-art methods such as Faster-RCNN, SSD, YOLO and FPN use FCN as their backbone. So here comes one question: Can we find a universal strategy to further accelerate FCN with higher accuracy, so could accelerate all the recent FCN-based methods? To analyze this, we decompose the face searching space into two orthogonal directions, `scale' and `spatial'. Only a few coordinates in the space expanded by the two base vectors indicate foreground. So if FCN could ignore most of the other points, the searching space and false alarm should be significantly boiled down. Based on this philosophy, a novel method named scale estimation and spatial attention proposal ($S^2AP$) is proposed to pay attention to some specific scales and valid locations in the image pyramid. Furthermore, we adopt a masked-convolution operation based on the attention result to accelerate FCN calculation. Experiments show that FCN-based method RPN can be accelerated by about $4\times$ with the help of $S^2AP$ and masked-FCN and at the same time it can also achieve the state-of-the-art on FDDB, AFW and MALF face detection benchmarks as well.

cs.CV↗

arXiv2017

Joint Multichannel Deconvolution and Blind Source Separation

Blind Source Separation (BSS) is a challenging matrix factorization problem that plays a central role in multichannel imaging science. In a large number of applications, such as astrophysics, current unmixing methods are limited since real-world mixtures are generally affected by extra instrumental effects like blurring. Therefore, BSS has to be solved jointly with a deconvolution problem, which requires tackling a new inverse problem: deconvolution BSS (DBSS). In this article, we introduce an innovative DBSS approach, called DecGMCA, based on sparse signal modeling and an efficient alternative projected least square algorithm. Numerical results demonstrate that the DecGMCA algorithm performs very well on simulations. It further highlights the importance of jointly solving BSS and deconvolution instead of considering these two problems independently. Furthermore, the performance of the proposed DecGMCA algorithm is demonstrated on simulated radio-interferometric data.

stat.AP↗