arXiv subjects

Liangpei Zhang

Publications and source records attributed to Liangpei Zhang.

At least 127 records · Page 7Linked to original sources

arXiv2018

Object Tracking in Satellite Videos Based on a Multi-Frame Optical Flow Tracker

Object tracking is a hot topic in computer vision. Thanks to the booming of the very high resolution (VHR) remote sensing techniques, it is now possible to track targets of interests in satellite videos. However, since the targets in the satellite videos are usually too small compared with the entire image, and too similar with the background, most state-of-the-art algorithms failed to track the target in satellite videos with a satisfactory accuracy. Due to the fact that optical flow shows the great potential to detect even the slight movement of the targets, we proposed a multi-frame optical flow tracker (MOFT) for object tracking in satellite videos. The Lucas-Kanade optical flow method was fused with the HSV color system and integral image to track the targets in the satellite videos, while multi-frame difference method was utilized in the optical flow tracker for a better interpretation. The experiments with three VHR remote sensing satellite video datasets indicate that compared with state-of-the-art object tracking algorithms, the proposed method can track the target more accurately.

cs.CV↗

arXiv2018

Estimating regional ground-level PM2.5 directly from satellite top-of-atmosphere reflectance using deep learning

Almost all remote sensing atmospheric PM2.5 estimation methods need satellite aerosol optical depth (AOD) products, which are often retrieved from top-of-atmosphere (TOA) reflectance via an atmospheric radiative transfer model. Then, is it possible to estimate ground-level PM2.5 directly from satellite TOA reflectance without a physical model? In this study, this challenging work are achieved based on a machine learning model. Specifically, we establish the relationship between PM2.5, satellite TOA reflectance, observation angles, and meteorological factors in a deep learning architecture (denoted as Ref-PM modeling). Taking the Wuhan Urban Agglomeration (WUA) as a case study, the results demonstrate that compared with the AOD-PM modeling, the Ref-PM modeling obtains a competitive performance, with out-of-sample cross-validated R2 and RMSE values of 0.87 and 9.89 ug/m3 respectively. Also, the TOA-reflectance-derived PM2.5 have a finer resolution and larger spatial coverage than the AOD-derived PM2.5. This work updates the traditional cognition of remote sensing PM2.5 estimation and has the potential to promote the application in atmospheric environmental monitoring.

physics.ao-ph↗

arXiv2018

Learning the Synthesizability of Dynamic Texture Samples

A dynamic texture (DT) refers to a sequence of images that exhibit temporal regularities and has many applications in computer vision and graphics. Given an exemplar of dynamic texture, it is a dynamic but challenging task to generate new samples with high quality that are perceptually similar to the input exemplar, which is known to be {\em example-based dynamic texture synthesis (EDTS)}. Numerous approaches have been devoted to this problem, in the past decades, but none them are able to tackle all kinds of dynamic textures equally well. In this paper, we investigate the synthesizability of dynamic texture samples: {\em given a dynamic texture sample, how synthesizable it is by using EDTS, and which EDTS method is the most suitable to synthesize it?} To this end, we propose to learn regression models to connect dynamic texture samples with synthesizability scores, with the help of a compiled dynamic texture dataset annotated in terms of synthesizability. More precisely, we first define the synthesizability of DT samples and characterize them by a set of spatiotemporal features. Based on these features and an annotated dynamic texture dataset, we then train regression models to predict the synthesizability scores of texture samples and learn classifiers to select the most suitable EDTS methods. We further complete the selection, partition and synthesizability prediction of dynamic texture samples in a hierarchical scheme. We finally apply the learned synthesizability to detecting synthesizable regions in videos. The experiments demonstrate that our method can effectively learn and predict the synthesizability of DT samples.

cs.CV↗

arXiv2017

A Multi-Scale and Multi-Depth Convolutional Neural Network for Remote Sensing Imagery Pan-Sharpening

Pan-sharpening is a fundamental and significant task in the field of remote sensing imagery processing, in which high-resolution spatial details from panchromatic images are employed to enhance the spatial resolution of multi-spectral (MS) images. As the transformation from low spatial resolution MS image to high-resolution MS image is complex and highly non-linear, inspired by the powerful representation for non-linear relationships of deep neural networks, we introduce multi-scale feature extraction and residual learning into the basic convolutional neural network (CNN) architecture and propose the multi-scale and multi-depth convolutional neural network (MSDCNN) for the pan-sharpening of remote sensing imagery. Both the quantitative assessment results and the visual assessment confirm that the proposed network yields high-resolution MS images that are superior to the images produced by the compared state-of-the-art methods.

cs.CV↗

arXiv2017

Image Stitching by Line-guided Local Warping with Global Similarity Constraint

Low-textured image stitching remains a challenging problem. It is difficult to achieve good alignment and it is easy to break image structures due to insufficient and unreliable point correspondences. Moreover, because of the viewpoint variations between multiple images, the stitched images suffer from projective distortions. To solve these problems, this paper presents a line-guided local warping method with a global similarity constraint for image stitching. Line features which serve well for geometric descriptions and scene constraints, are employed to guide image stitching accurately. On one hand, the line features are integrated into a local warping model through a designed weight function. On the other hand, line features are adopted to impose strong geometric constraints, including line correspondence and line colinearity, to improve the stitching performance through mesh optimization. To mitigate projective distortions, we adopt a global similarity constraint, which is integrated with the projective warps via a designed weight strategy. This constraint causes the final warp to slowly change from a projective to a similarity transformation across the image. Finally, the images undergo a two-stage alignment scheme that provides accurate alignment and reduces projective distortion. We evaluate our method on a series of images and compare it with several other methods. The experimental results demonstrate that the proposed method provides a convincing stitching performance and that it outperforms other state-of-the-art methods.

cs.CV↗

arXiv2017

Deep learning in remote sensing: a review

Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.

cs.CV↗

arXiv2017

Anisotropic-Scale Junction Detection and Matching for Indoor Images

Junctions play an important role in the characterization of local geometric structures in images, the detection of which is a longstanding and challenging task. Existing junction detectors usually focus on identifying the junction locations and the orientations of the junction branches while ignoring their scales; however, these scales also contain rich geometric information. This paper presents a novel approach to junction detection and characterization that exploits the locally anisotropic geometries of a junction and estimates the scales of these geometries using an \emph{a contrario} model. The output junctions have anisotropic scales --- i.e., each branch of a junction is associated with an independent scale parameter --- and are thus termed anisotropic-scale junctions (ASJs). We then apply the newly detected ASJs for the matching of indoor images, in which there may be dramatic changes in viewpoint and the detected local visual features, e.g., key-points, are usually insufficiently distinctive. We propose to use the anisotropic geometries of our junctions to improve the matching precision for indoor images. Matching results obtained on sets of indoor images demonstrate that our approach achieves state-of-the-art performance in indoor image matching.

cs.CV↗

arXiv2017

The relationships between PM2.5 and meteorological factors in China: Seasonal and regional variations

The interactions between PM2.5 and meteorological factors play a crucial role in air pollution analysis. However, previous studies that have researched the relationships between PM2.5 concentration and meteorological conditions have been mainly confined to a certain city or district, and the correlation over the whole of China remains unclear. Whether or not spatial and seasonal variations exit deserves further research. In this study, the relationships between PM2.5 concentration and meteorological factors were investigated in 74 major cities in China for a continuous period of 22 months from February 2013 to November 2014, at season, year, city, and regional scales, and the spatial and seasonal variations were analyzed. The meteorological factors were relative humidity (RH), temperature (TEM), wind speed (WS), and surface pressure (PS). We found that spatial and seasonal variations of their relationships with PM2.5 do exist. Spatially, RH is positively correlated with PM2.5 concentration in North China and Urumqi, but the relationship turns to negative in other areas of China. WS is negatively correlated with PM2.5 everywhere expect for Hainan Island. PS has a strong positive relationship with PM2.5 concentration in Northeast China and Mid-south China, and in other areas the correlation is weak. Seasonally, the positive correlation between PM2.5 concentration and RH is stronger in winter and spring. TEM has a negative relationship with PM2.5 in autumn and the opposite in winter. PS is more positively correlated with PM2.5 in autumn than in other seasons. Our study investigated the relationships between PM2.5 and meteorological factors in terms of spatial and seasonal variations, and the conclusions about the relationships between PM2.5 and meteorological factors are more comprehensive and precise than before.

physics.ao-ph↗

arXiv2017

A 33-year NPP monitoring study in southwest China by the fusion of multi-source remote sensing and station data

Knowledge of regional net primary productivity (NPP) is important for the systematic understanding of the global carbon cycle. In this study, multi-source data were employed to conduct a 33-year regional NPP study in southwest China, at a 1-km scale. A multi-sensor fusion framework was applied to obtain a new normalized difference vegetation index (NDVI) time series from 1982 to 2014, combining the respective advantages of the different remote sensing datasets. As another key parameter for NPP modeling, the total solar radiation was calculated by the improved Yang hybrid model (YHM), using meteorological station data. The verification described in this paper proved the feasibility of all the applied data processes, and a greatly improved accuracy was obtained for the NPP calculated with the final processed NDVI. The spatio-temporal analysis results indicated that 68.07% of the study area showed an increasing NPP trend over the past three decades. Significant heterogeneity was found in the correlation between NPP and precipitation at a monthly scale, specifically, the negative correlation in the growing season and the positive correlation in the dry season. The lagged positive correlation in the growing season and no lag in the dry season indicated the important impact of precipitation on NPP.

q-bio.PE↗

arXiv2017

Estimating ground-level PM2.5 by fusing satellite and station observations: A geo-intelligent deep learning approach

Fusing satellite observations and station measurements to estimate ground-level PM2.5 is promising for monitoring PM2.5 pollution. A geo-intelligent approach, which incorporates geographical correlation into an intelligent deep learning architecture, is developed to estimate PM2.5. Specifically, it considers geographical distance and spatiotemporally correlated PM2.5 in a deep belief network (denoted as Geoi-DBN). Geoi-DBN can capture the essential features associated with PM2.5 from latent factors. It was trained and tested with data from China in 2015. The results show that Geoi-DBN performs significantly better than the traditional neural network. The cross-validation R increases from 0.63 to 0.94, and RMSE decreases from 29.56 to 13.68$μ$g/m3. On the basis of the derived PM2.5 distribution, it is predicted that over 80% of the Chinese population live in areas with an annual mean PM2.5 of greater than 35$μ$g/m3. This study provides a new perspective for air pollution monitoring in large geographic regions.

physics.ao-ph↗

arXiv2017

Boosting the accuracy of multi-spectral image pan-sharpening by learning a deep residual network

In the field of fusing multi-spectral and panchromatic images (Pan-sharpening), the impressive effectiveness of deep neural networks has been recently employed to overcome the drawbacks of traditional linear models and boost the fusing accuracy. However, to the best of our knowledge, existing research works are mainly based on simple and flat networks with relatively shallow architecture, which severely limited their performances. In this paper, the concept of residual learning has been introduced to form a very deep convolutional neural network to make a full use of the high non-linearity of deep learning models. By both quantitative and visual assessments on a large number of high quality multi-spectral images from various sources, it has been supported that our proposed model is superior to all mainstream algorithms included in the comparison, and achieved the highest spatial-spectral unified accuracy.

cs.CV↗

arXiv2017

A multi-task convolutional neural network for mega-city analysis using very high resolution satellite imagery and geospatial data

Mega-city analysis with very high resolution (VHR) satellite images has been drawing increasing interest in the fields of city planning and social investigation. It is known that accurate land-use, urban density, and population distribution information is the key to mega-city monitoring and environmental studies. Therefore, how to generate land-use, urban density, and population distribution maps at a fine scale using VHR satellite images has become a hot topic. Previous studies have focused solely on individual tasks with elaborate hand-crafted features and have ignored the relationship between different tasks. In this study, we aim to propose a universal framework which can: 1) automatically learn the internal feature representation from the raw image data; and 2) simultaneously produce fine-scale land-use, urban density, and population distribution maps. For the first target, a deep convolutional neural network (CNN) is applied to learn the hierarchical feature representation from the raw image data. For the second target, a novel CNN-based universal framework is proposed to process the VHR satellite images and generate the land-use, urban density, and population distribution maps. To the best of our knowledge, this is the first CNN-based mega-city analysis method which can process a VHR remote sensing image with such a large data volume. A VHR satellite image (1.2 m spatial resolution) of the center of Wuhan covering an area of 2606 km2 was used to evaluate the proposed method. The experimental results confirm that the proposed method can achieve a promising accuracy for land-use, urban density, and population distribution maps.

cs.CV↗

arXiv2017

Texture Characterization by Using Shape Co-occurrence Patterns

Texture characterization is a key problem in image understanding and pattern recognition. In this paper, we present a flexible shape-based texture representation using shape co-occurrence patterns. More precisely, texture images are first represented by tree of shapes, each of which is associated with several geometrical and radiometric attributes. Then four typical kinds of shape co-occurrence patterns based on the hierarchical relationship of the shapes in the tree are learned as codewords. Three different coding methods are investigated to learn the codewords, with which, any given texture image can be encoded into a descriptive vector. In contrast with existing works, the proposed method not only inherits the strong ability to depict geometrical aspects of textures and the high robustness to variations of imaging conditions from the shape-based method, but also provides a flexible way to consider shape relationships and to compute high-order statistics on the tree. To our knowledge, this is the first time to use co-occurrence patterns of explicit shapes as a tool for texture analysis. Experiments on various texture datasets and scene datasets demonstrate the efficiency of the proposed method.

cs.CV↗

arXiv2017

Multi-feature combined cloud and cloud shadow detection in GaoFen-1 wide field of view imagery

The wide field of view (WFV) imaging system onboard the Chinese GaoFen-1 (GF-1) optical satellite has a 16-m resolution and four-day revisit cycle for large-scale Earth observation. The advantages of the high temporal-spatial resolution and the wide field of view make the GF-1 WFV imagery very popular. However, cloud cover is an inevitable problem in GF-1 WFV imagery, which influences its precise application. Accurate cloud and cloud shadow detection in GF-1 WFV imagery is quite difficult due to the fact that there are only three visible bands and one near-infrared band. In this paper, an automatic multi-feature combined (MFC) method is proposed for cloud and cloud shadow detection in GF-1 WFV imagery. The MFC algorithm first implements threshold segmentation based on the spectral features and mask refinement based on guided filtering to generate a preliminary cloud mask. The geometric features are then used in combination with the texture features to improve the cloud detection results and produce the final cloud mask. Finally, the cloud shadow mask can be acquired by means of the cloud and shadow matching and follow-up correction process. The method was validated using 108 globally distributed scenes. The results indicate that MFC performs well under most conditions, and the average overall accuracy of MFC cloud detection is as high as 96.8%. In the contrastive analysis with the official provided cloud fractions, MFC shows a significant improvement in cloud fraction estimation, and achieves a high accuracy for the cloud and cloud shadow detection in the GF-1 WFV imagery with fewer spectral bands. The proposed method could be used as a preprocessing step in the future to monitor land-cover change, and it could also be easily extended to other optical satellite imagery which has a similar spectral setting.

cs.CV↗

arXiv2016

A Spatial and Temporal Non-Local Filter Based Data Fusion

The trade-off in remote sensing instruments that balances the spatial resolution and temporal frequency limits our capacity to monitor spatial and temporal dynamics effectively. The spatiotemporal data fusion technique is considered as a cost-effective way to obtain remote sensing data with both high spatial resolution and high temporal frequency, by blending observations from multiple sensors with different advantages or characteristics. In this paper, we develop the spatial and temporal non-local filter based fusion model (STNLFFM) to enhance the prediction capacity and accuracy, especially for complex changed landscapes. The STNLFFM method provides a new transformation relationship between the fine-resolution reflectance images acquired from the same sensor at different dates with the help of coarse-resolution reflectance data, and makes full use of the high degree of spatiotemporal redundancy in the remote sensing image sequence to produce the final prediction. The proposed method was tested over both the Coleambally Irrigation Area study site and the Lower Gwydir Catchment study site. The results show that the proposed method can provide a more accurate and robust prediction, especially for heterogeneous landscapes and temporally dynamic areas.

cs.CV↗

arXiv2016

AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classification

Aerial scene classification, which aims to automatically label an aerial image with a specific semantic category, is a fundamental problem for understanding high-resolution remote sensing imagery. In recent years, it has become an active task in remote sensing area and numerous algorithms have been proposed for this task, including many machine learning and data-driven approaches. However, the existing datasets for aerial scene classification like UC-Merced dataset and WHU-RS19 are with relatively small sizes, and the results on them are already saturated. This largely limits the development of scene classification algorithms. This paper describes the Aerial Image Dataset (AID): a large-scale dataset for aerial scene classification. The goal of AID is to advance the state-of-the-arts in scene classification of remote sensing images. For creating AID, we collect and annotate more than ten thousands aerial scene images. In addition, a comprehensive review of the existing aerial scene classification techniques as well as recent widely-used deep learning methods is given. Finally, we provide a performance analysis of typical aerial scene classification and deep learning approaches on AID, which can be served as the baseline results on this benchmark.

cs.CV↗

arXiv2016

Point-surface fusion of station measurements and satellite observations for mapping PM2.5 distribution in China: methods and assessment

Fine particulate matter (PM2.5) is associated with adverse human health effects, and China is currently suffering from serious PM2.5 pollution. To obtain spatially continuous ground-level PM2.5 concentrations, several models established by point-surface fusion of ground station and satellite observations have been developed. However, how well do these models perform at national scale in China? Is there space to improve the estimation accuracy of PM2.5 concentration? The contribution of this study is threefold. Firstly, taking advantage of the newly established national monitoring network, we develop a national-scale generalized regression neural network (GRNN) model to estimate PM2.5 concentrations. Secondly, different assessment experiments are undertaken in time and space, to comprehensively evaluate and compare the performance of the widely used models. Finally, to map the yearly and seasonal mean distribution of PM2.5 concentrations in China, a pixel-based merging strategy is proposed. The results indicate that the conventional models (linear regression, multiple linear regression, and semi-empirical model) do not perform well at national scale, with cross-validation R values of 0.488~0.552 and RMSEs of 30.80~31.51 μg/m3, respectively. In contrast, the more advanced models (geographically weighted regression, back-propagation neural network, and GRNN) have great advantages in PM2.5 estimation, with R values ranging from 0.610 to 0.816 and RMSEs from 20.93 to 28.68 μg/m3, respectively. In particular, the proposed GRNN model obtains the best performance. Furthermore, the mapped PM2.5 distribution retrieved from 3-km MODIS aerosol optical depth (AOD) products, agrees quite well with the station measurements. The results also show that our study has the capacity to provide reasonable information for the global monitoring of PM2.5 pollution in China.

physics.ao-ph↗

arXiv2016

Image stitching with perspective-preserving warping

Image stitching algorithms often adopt the global transformation, such as homography, and work well for planar scenes or parallax free camera motions. However, these conditions are easily violated in practice. With casual camera motions, variable taken views, large depth change, or complex structures, it is a challenging task for stitching these images. The global transformation model often provides dreadful stitching results, such as misalignments or projective distortions, especially perspective distortion. To this end, we suggest a perspective-preserving warping for image stitching, which spatially combines local projective transformations and similarity transformation. By weighted combination scheme, our approach gradually extrapolates the local projective transformations of the overlapping regions into the non-overlapping regions, and thus the final warping can smoothly change from projective to similarity. The proposed method can provide satisfactory alignment accuracy as well as reduce the projective distortions and maintain the multi-perspective view. Experiments on a variety of challenging images confirm the efficiency of the approach.

cs.CV↗