arXiv subjects

Chao Zeng

Publications and source records attributed to Chao Zeng.

32 records · Page 2Linked to original sources

arXiv2022

Dual Swin-Transformer based Mutual Interactive Network for RGB-D Salient Object Detection

Salient Object Detection is the task of predicting the human attended region in a given scene. Fusing depth information has been proven effective in this task. The main challenge of this problem is how to aggregate the complementary information from RGB modality and depth modality. However, conventional deep models heavily rely on CNN feature extractors, and the long-range contextual dependencies are usually ignored. In this work, we propose Dual Swin-Transformer based Mutual Interactive Network. We adopt Swin-Transformer as the feature extractor for both RGB and depth modality to model the long-range dependencies in visual inputs. Before fusing the two branches of features into one, attention-based modules are applied to enhance features from each modality. We design a self-attention-based cross-modality interaction module and a gated modality attention module to leverage the complementary information between the two modalities. For the saliency decoding, we create different stages enhanced with dense connections and keep a decoding memory while the multi-level encoding features are considered simultaneously. Considering the inaccurate depth map issue, we collect the RGB features of early stages into a skip convolution module to give more guidance from RGB modality to the final saliency prediction. In addition, we add edge supervision to regularize the feature learning process. Comprehensive experiments on five standard RGB-D SOD benchmark datasets over four evaluation metrics demonstrate the superiority of the proposed DTMINet method.

cs.CV↗

arXiv2022

Learning Transformer Features for Image Quality Assessment

Objective image quality evaluation is a challenging task, which aims to measure the quality of a given image automatically. According to the availability of the reference images, there are Full-Reference and No-Reference IQA tasks, respectively. Most deep learning approaches use regression from deep features extracted by Convolutional Neural Networks. For the FR task, another option is conducting a statistical comparison on deep features. For all these methods, non-local information is usually neglected. In addition, the relationship between FR and NR tasks is less explored. Motivated by the recent success of transformers in modeling contextual information, we propose a unified IQA framework that utilizes CNN backbone and transformer encoder to extract features. The proposed framework is compatible with both FR and NR modes and allows for a joint training scheme. Evaluation experiments on three standard IQA datasets, i.e., LIVE, CSIQ and TID2013, and KONIQ-10K, show that our proposed model can achieve state-of-the-art FR performance. In addition, comparable NR performance is achieved in extensive experiments, and the results show that the NR performance can be leveraged by the joint training scheme.

cs.CV↗

arXiv2021

Learning compliant grasping and manipulation by teleoperation with adaptive force control

In this work, we focus on improving the robot's dexterous capability by exploiting visual sensing and adaptive force control. TeachNet, a vision-based teleoperation learning framework, is exploited to map human hand postures to a multi-fingered robot hand. We augment TeachNet, which is originally based on an imprecise kinematic mapping and position-only servoing, with a biomimetic learning-based compliance control algorithm for dexterous manipulation tasks. This compliance controller takes the mapped robotic joint angles from TeachNet as the desired goal, computes the desired joint torques. It is derived from a computational model of the biomimetic control strategy in human motor learning, which allows adapting the control variables (impedance and feedforward force) online during the execution of the reference joint angle trajectories. The simultaneous adaptation of the impedance and feedforward profiles enables the robot to interact with the environment in a compliant manner. Our approach has been verified in multiple tasks in physics simulation, i.e., grasping, opening-a-door, turning-a-cap, and touching-a-mouse, and has shown more reliable performances than the existing position control and the fixed-gain-based force control approaches.

cs.RO↗

arXiv2021

Contrastive Semantic Similarity Learning for Image Captioning Evaluation with Intrinsic Auto-encoder

Automatically evaluating the quality of image captions can be very challenging since human language is quite flexible that there can be various expressions for the same meaning. Most of the current captioning metrics rely on token level matching between candidate caption and the ground truth label sentences. It usually neglects the sentence-level information. Motivated by the auto-encoder mechanism and contrastive representation learning advances, we propose a learning-based metric for image captioning, which we call Intrinsic Image Captioning Evaluation($I^2CE$). We develop three progressive model structures to learn the sentence level representations--single branch model, dual branches model, and triple branches model. Our empirical tests show that $I^2CE$ trained with dual branches structure achieves better consistency with human judgments to contemporary image captioning evaluation metrics. Furthermore, We select several state-of-the-art image captioning models and test their performances on the MS COCO dataset concerning both contemporary metrics and the proposed $I^2CE$. Experiment results show that our proposed method can align well with the scores generated from other contemporary metrics. On this concern, the proposed metric could serve as a novel indicator of the intrinsic information between captions, which may be complementary to the existing ones.

cs.CV↗

arXiv2021

A Comprehensive Survey of Incentive Mechanism for Federated Learning

Federated learning utilizes various resources provided by participants to collaboratively train a global model, which potentially address the data privacy issue of machine learning. In such promising paradigm, the performance will be deteriorated without sufficient training data and other resources in the learning process. Thus, it is quite crucial to inspire more participants to contribute their valuable resources with some payments for federated learning. In this paper, we present a comprehensive survey of incentive schemes for federate learning. Specifically, we identify the incentive problem in federated learning and then provide a taxonomy for various schemes. Subsequently, we summarize the existing incentive mechanisms in terms of the main techniques, such as Stackelberg game, auction, contract theory, Shapley value, reinforcement learning, blockchain. By reviewing and comparing some impressive results, we figure out three directions for the future study.

cs.LG↗

arXiv2021

Rank properties and computational methods for orthogonal tensor decompositions

The orthogonal decomposition factorizes a tensor into a sum of an orthogonal list of rankone tensors. We present several properties of orthogonal rank. We find that a subtensor may have a larger orthogonal rank than the whole tensor and prove the lower semicontinuity of orthogonal rank. The lower semicontinuity guarantees the existence of low orthogonal rank approximation. To fit the orthogonal decomposition, we propose an algorithm based on the augmented Lagrangian method and guarantee the orthogonality by a novel orthogonalization procedure. Numerical experiments show that the proposed method has a great advantage over the existing methods for strongly orthogonal decompositions in terms of the approximation error.

math.NA↗

arXiv2020

Intrinsic Image Captioning Evaluation

The image captioning task is about to generate suitable descriptions from images. For this task there can be several challenges such as accuracy, fluency and diversity. However there are few metrics that can cover all these properties while evaluating results of captioning models.In this paper we first conduct a comprehensive investigation on contemporary metrics. Motivated by the auto-encoder mechanism and the research advances of word embeddings we propose a learning based metrics for image captioning, which we call Intrinsic Image Captioning Evaluation(I2CE). We select several state-of-the-art image captioning models and test their performances on MS COCO dataset with respects to both contemporary metrics and the proposed I2CE. Experiment results show that our proposed method can keep robust performance and give more flexible scores to candidate captions when encountered with semantic similar expression or less aligned semantics. On this concern the proposed metric could serve as a novel indicator on the intrinsic information between captions, which may be complementary to the existing ones.

cs.CV↗

arXiv2020

Synthetic high-order PT symmetry in a single coil resonator

The exploration of non-Hermitian systems with parity-time (PT) symmetry has witnessed immense research interest both fundamentally and technologically in a wide range of subject areas in physics and engineering. One significant example of the principal emerging fields in this context is the PT symmetric wireless applications using multiple coils that are spatially separated but mutually coupled with position-dependent coupling strength. Such a spatial PT configuration limits the flexibility and miniaturization of the PT symmetric designs. As far as this is concerned, inspired by scattering induced two opposite whispering-gallery (WG) modes in an optical resonator, analogously here we experimentally demonstrate a specially constructed second-order (2-nd order) PT symmetry in a single coil resonator, whose currents with two different directions are excited by internal bypass capacitor. Our proposed structure has the following peculiar feature: First, the bypass capacitor induces coupling in spectral resonances allow us to observe a 2-nd order phase transition between symmetry regimes, without the need of a second coil in the spatial PT case. Under this circumstance, this specially constructed PT symmetry can be regarded as synthetic PT symmetry, which is enabled by coupling modes with different directions. Second, by introducing two or more internal bypass capacitors, the synthetic high-order PT symmetric system bearing such as third-order exceptional point in a single coil resonator can be realized. These results will provide a new paradigm to realize higher-order PT symmetry towards the investigation of non-Hermitian physics in a synthetic perspective, which can be extended to other physical platforms such as optics and acoustics.

physics.app-ph↗

arXiv2020

Deep learning-based air temperature mapping by fusing remote sensing, station, simulation and socioeconomic data

Air temperature (Ta) is an essential climatological component that controls and influences various earth surface processes. In this study, we make the first attempt to employ deep learning for Ta mapping mainly based on space remote sensing and ground station observations. Considering that Ta varies greatly in space and time and is sensitive to many factors, assimilation data and socioeconomic data are also included for a multi-source data fusion based estimation. Specifically, a 5-layers structured deep belief network (DBN) is employed to better capture the complicated and non-linear relationships between Ta and different predictor variables. Layer-wise pre-training process for essential features extraction and fine-tuning process for weight parameters optimization ensure the robust prediction of Ta spatio-temporal distribution. The DBN model was implemented for 0.01° daily maximum Ta mapping across China. The ten-fold cross-validation results indicate that the DBN model achieves promising results with the RMSE of 1.996°C, MAE of 1.539°C, and R of 0.986 at the national scale. Compared with multiple linear regression (MLR), back-propagation neural network (BPNN) and random forest (RF) method, the DBN model reduces the MAE values by 1.340°C, 0.387°C and 0.222°C, respectively. Further analysis on spatial distribution and temporal tendency of prediction errors both validate the great potentials of DBN in Ta estimation.

physics.ao-ph↗

arXiv2019

Spatially Continuous and High-resolution Land Surface Temperature: A Review of Reconstruction and Spatiotemporal Fusion Techniques

Remotely sensed, spatially continuous and high spatiotemporal resolution (hereafter referred to as high resolution) land surface temperature (LST) is a key parameter for studying the thermal environment and has important applications in many fields. However, difficult atmospheric conditions, sensor malfunctioning and scanning gaps between orbits frequently introduce spatial discontinuities into satellite-retri1eved LST products. For a single sensor, there is also a trade-off between temporal and spatial resolution and, therefore, it is impossible to obtain high temporal and spatial resolution simultaneously. In recent years the reconstruction and spatiotemporal fusion of LST products have become active research topics that aim at overcoming this limitation. They are two of most investigated approaches in thermal remote sensing and attract increasing attention, which has resulted in a number of different algorithms. However, to the best of our knowledge, currently no review exists that expatiates and summarizes the available LST reconstruction and spatiotemporal fusion methods and algorithms. This paper introduces the principles and theories behind LST reconstruction and spatiotemporal fusion and provides an overview of the published research and algorithms. We summarized three kinds of reconstruction methods for missing pixels (spatial, temporal and spatiotemporal methods), two kinds of reconstruction methods for cloudy pixels (Satellite Passive Microwave (PMW)-based and Surface Energy Balance (SEB)-based methods) and three kinds of spatiotemporal fusion methods (weighted function-based, unmixing-based and hybrid methods). The review concludes by summarizing validation methods and by identifying some promising future research directions for generating spatially continuous and high resolution LST products.

physics.ao-ph↗

arXiv2018

Real-time Characterization of Transient Double-Soliton Dynamics in a Mode-Locked Fiber Laser

By means of the emerging Dispersive Fourier transformation technique, we captured the pulse-resolved spectral evolution dynamics of the double-soliton (DS) states in a single-walled carbon nanotube based Er-doped fiber laser from the initial fluctuations, monitoring the evolution process up to 10 seconds (corresponding to ~260 million roundtrips) discontinuously. Two distinctly different evolutionary types of DS states have been investigated in detail: splitting from one pulse and forming simultaneously. Relaxation oscillations, beating, transient bound state, spectral broadening and pulse interval dynamics have been observed in the evolution process of the DS operation. Our study will be helpful for the further research of mode-locking operation.

physics.optics↗

arXiv2018

Missing Data Reconstruction in Remote Sensing image with a Unified Spatial-Temporal-Spectral Deep Convolutional Neural Network

Because of the internal malfunction of satellite sensors and poor atmospheric conditions such as thick cloud, the acquired remote sensing data often suffer from missing information, i.e., the data usability is greatly reduced. In this paper, a novel method of missing information reconstruction in remote sensing images is proposed. The unified spatial-temporal-spectral framework based on a deep convolutional neural network (STS-CNN) employs a unified deep convolutional neural network combined with spatial-temporal-spectral supplementary information. In addition, to address the fact that most methods can only deal with a single missing information reconstruction task, the proposed approach can solve three typical missing information reconstruction tasks: 1) dead lines in Aqua MODIS band 6; 2) the Landsat ETM+ Scan Line Corrector (SLC)-off problem; and 3) thick cloud removal. It should be noted that the proposed model can use multi-source data (spatial, spectral, and temporal) as the input of the unified framework. The results of both simulated and real-data experiments demonstrate that the proposed model exhibits high effectiveness in the three missing information reconstruction tasks listed above.

cs.CV↗

arXiv2016

Point-surface fusion of station measurements and satellite observations for mapping PM2.5 distribution in China: methods and assessment

Fine particulate matter (PM2.5) is associated with adverse human health effects, and China is currently suffering from serious PM2.5 pollution. To obtain spatially continuous ground-level PM2.5 concentrations, several models established by point-surface fusion of ground station and satellite observations have been developed. However, how well do these models perform at national scale in China? Is there space to improve the estimation accuracy of PM2.5 concentration? The contribution of this study is threefold. Firstly, taking advantage of the newly established national monitoring network, we develop a national-scale generalized regression neural network (GRNN) model to estimate PM2.5 concentrations. Secondly, different assessment experiments are undertaken in time and space, to comprehensively evaluate and compare the performance of the widely used models. Finally, to map the yearly and seasonal mean distribution of PM2.5 concentrations in China, a pixel-based merging strategy is proposed. The results indicate that the conventional models (linear regression, multiple linear regression, and semi-empirical model) do not perform well at national scale, with cross-validation R values of 0.488~0.552 and RMSEs of 30.80~31.51 μg/m3, respectively. In contrast, the more advanced models (geographically weighted regression, back-propagation neural network, and GRNN) have great advantages in PM2.5 estimation, with R values ranging from 0.610 to 0.816 and RMSEs from 20.93 to 28.68 μg/m3, respectively. In particular, the proposed GRNN model obtains the best performance. Furthermore, the mapped PM2.5 distribution retrieved from 3-km MODIS aerosol optical depth (AOD) products, agrees quite well with the station measurements. The results also show that our study has the capacity to provide reasonable information for the global monitoring of PM2.5 pollution in China.

physics.ao-ph↗

arXiv2015

Dimensions of Biquadratic and Bicubic Spline Spaces over Hierarchical T-meshes

This paper discusses the dimensions of biquadratic C1 spline spaces and bicubic C2 spline spaces over hierarchical T-meshes using the smoothing cofactor-conformality method. We obtain the dimension formula of biquadratic C1 spline spaces over hierarchical T-meshes in a concise way. In addition, we provide a dimension formula for bicubic C2 spline spaces over hierarchical T-mesh with fewer restrictions than that in the previous literature. A dimension formula for bicubic C2 spline spaces over a new type hierarchical T-mesh is also provided.

math.NA↗