arXiv subjects

Chengyuan Zhang

Publications and source records attributed to Chengyuan Zhang.

At least 37 records · Page 2Linked to original sources

arXiv2022

Discovering Dynamic Patterns from Spatiotemporal Data with Time-Varying Low-Rank Autoregression

The problem of broad practical interest in spatiotemporal data analysis, i.e., discovering interpretable dynamic patterns from spatiotemporal data, is studied in this paper. Towards this end, we develop a time-varying reduced-rank vector autoregression (VAR) model whose coefficient matrices are parameterized by low-rank tensor factorization. Benefiting from the tensor factorization structure, the proposed model can simultaneously achieve model compression and pattern discovery. In particular, the proposed model allows one to characterize nonstationarity and time-varying system behaviors underlying spatiotemporal data. To evaluate the proposed model, extensive experiments are conducted on various spatiotemporal data representing different nonlinear dynamical systems, including fluid dynamics, sea surface temperature, USA surface temperature, and NYC taxi trips. Experimental results demonstrate the effectiveness of modeling spatiotemporal data and characterizing spatial/temporal patterns with the proposed model. In the spatial context, the spatial patterns can be automatically extracted and intuitively characterized by the spatial modes. In the temporal context, the complex time-varying system behaviors can be revealed by the temporal modes in the proposed model. Thus, our model lays an insightful foundation for understanding complex spatiotemporal data in real-world dynamical systems. The adapted datasets and Python implementation are publicly available at https://github.com/xinychen/vars.

cs.LG↗

arXiv2022

Social Interactions for Autonomous Driving: A Review and Perspectives

No human drives a car in a vacuum; she/he must negotiate with other road users to achieve their goals in social traffic scenes. A rational human driver can interact with other road users in a socially-compatible way through implicit communications to complete their driving tasks smoothly in interaction-intensive, safety-critical environments. This paper aims to review the existing approaches and theories to help understand and rethink the interactions among human drivers toward social autonomous driving. We take this survey to seek the answers to a series of fundamental questions: 1) What is social interaction in road traffic scenes? 2) How to measure and evaluate social interaction? 3) How to model and reveal the process of social interaction? 4) How do human drivers reach an implicit agreement and negotiate smoothly in social interaction? This paper reviews various approaches to modeling and learning the social interactions between human drivers, ranging from optimization theory and graphical models to social force theory and behavioral & cognitive science. We also highlight some new directions, critical challenges, and opening questions for future research.

cs.RO↗

arXiv2020

A Novel Hybrid Framework for Hourly PM2.5 Concentration Forecasting Using CEEMDAN and Deep Temporal Convolutional Neural Network

For hourly PM2.5 concentration prediction, accurately capturing the data patterns of external factors that affect PM2.5 concentration changes, and constructing a forecasting model is one of efficient means to improve forecasting accuracy. In this study, a novel hybrid forecasting model based on complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and deep temporal convolutional neural network (DeepTCN) is developed to predict PM2.5 concentration, by modelling the data patterns of historical pollutant concentrations data, meteorological data, and discrete time variables' data. Taking PM2.5 concentration of Beijing as the sample, experimental results showed that the forecasting accuracy of the proposed CEEMDAN-DeepTCN model is verified to be the highest when compared with the time series model, artificial neural network, and the popular deep learning models. The new model has improved the capability to model the PM2.5-related factor data patterns, and can be used as a promising tool for forecasting PM2.5 concentrations.

eess.SP↗

arXiv2020

Spatiotemporal Learning of Multivehicle Interaction Patterns in Lane-Change Scenarios

Interpretation of common-yet-challenging interaction scenarios can benefit well-founded decisions for autonomous vehicles. Previous research achieved this using their prior knowledge of specific scenarios with predefined models, limiting their adaptive capabilities. This paper describes a Bayesian nonparametric approach that leverages continuous (i.e., Gaussian processes) and discrete (i.e., Dirichlet processes) stochastic processes to reveal underlying interaction patterns of the ego vehicle with other nearby vehicles. Our model relaxes dependency on the number of surrounding vehicles by developing an acceleration-sensitive velocity field based on Gaussian processes. The experiment results demonstrate that the velocity field can represent the spatial interactions between the ego vehicle and its surroundings. Then, a discrete Bayesian nonparametric model, integrating Dirichlet processes and hidden Markov models, is developed to learn the interaction patterns over the temporal space by segmenting and clustering the sequential interaction data into interpretable granular patterns automatically. We then evaluate our approach in the highway lane-change scenarios using the highD dataset collected from real-world settings. Results demonstrate that our proposed Bayesian nonparametric approach provides an insight into the complicated lane-change interactions of the ego vehicle with multiple surrounding traffic participants based on the interpretable interaction patterns and their transition properties in temporal relationships. Our proposed approach sheds light on efficiently analyzing other kinds of multi-agent interactions, such as vehicle-pedestrian interactions. View the demos via https://youtu.be/z_vf9UHtdAM.

cs.RO↗

arXiv2020

A New Decomposition Ensemble Approach for Tourism Demand Forecasting: Evidence from Major Source Countries

The Asian-pacific region is the major international tourism demand market in the world, and its tourism demand is deeply affected by various factors. Previous studies have shown that different market factors influence the tourism market demand at different timescales. Accordingly, the decomposition ensemble learning approach is proposed to analyze the impact of different market factors on market demand, and the potential advantages of the proposed method on forecasting tourism demand in the Asia-pacific region are further explored. This study carefully explores the multi-scale relationship between tourist destinations and the major source countries, by decomposing the corresponding monthly tourist arrivals with noise-assisted multivariate empirical mode decomposition. With the China and Malaysia as case studies, their respective empirical results show that decomposition ensemble approach significantly better than the benchmarks which include statistical model, machine learning and deep learning model, in terms of the level forecasting accuracy and directional forecasting accuracy.

econ.GN↗

arXiv2019

A General Framework of Learning Multi-Vehicle Interaction Patterns from Videos

Semantic learning and understanding of multi-vehicle interaction patterns in a cluttered driving environment are essential but challenging for autonomous vehicles to make proper decisions. This paper presents a general framework to gain insights into intricate multi-vehicle interaction patterns from bird's-eye view traffic videos. We adopt a Gaussian velocity field to describe the time-varying multi-vehicle interaction behaviors and then use deep autoencoders to learn associated latent representations for each temporal frame. Then, we utilize a hidden semi-Markov model with a hierarchical Dirichlet process as a prior to segment these sequential representations into granular components, also called traffic primitives, corresponding to interaction patterns. Experimental results demonstrate that our proposed framework can extract traffic primitives from videos, thus providing a semantic way to analyze multi-vehicle interaction patterns, even for cluttered driving scenarios that are far messier than human beings can cope with.

cs.RO↗

arXiv2019

PAC-GAN: An Effective Pose Augmentation Scheme for Unsupervised Cross-View Person Re-identification

Person re-identification (person Re-Id) aims to retrieve the pedestrian images of a same person that captured by disjoint and non-overlapping cameras. Lots of researchers recently focuse on this hot issue and propose deep learning based methods to enhance the recognition rate in a supervised or unsupervised manner. However, two limitations that cannot be ignored: firstly, compared with other image retrieval benchmarks, the size of existing person Re-Id datasets are far from meeting the requirement, which cannot provide sufficient pedestrian samples for the training of deep model; secondly, the samples in existing datasets do not have sufficient human motions or postures coverage to provide more priori knowledges for learning. In this paper, we introduce a novel unsupervised pose augmentation cross-view person Re-Id scheme called PAC-GAN to overcome these limitations. We firstly present the formal definition of cross-view pose augmentation and then propose the framework of PAC-GAN that is a novel conditional generative adversarial network (CGAN) based approach to improve the performance of unsupervised corss-view person Re-Id. Specifically, The pose generation model in PAC-GAN called CPG-Net is to generate enough quantity of pose-rich samples from original image and skeleton samples. The pose augmentation dataset is produced by combining the synthesized pose-rich samples with the original samples, which is fed into the corss-view person Re-Id model named Cross-GAN. Besides, we use weight-sharing strategy in the CPG-Net to improve the quality of new generated samples. To the best of our knowledge, we are the first try to enhance the unsupervised cross-view person Re-Id by pose augmentation, and the results of extensive experiments show that the proposed scheme can combat the state-of-the-arts.

cs.CV↗

arXiv2018

CNN-VWII: An Efficient Approach for Large-Scale Video Retrieval by Image Queries

This paper aims to solve the problem of large-scale video retrieval by a query image. Firstly, we define the problem of top-$k$ image to video query. Then, we combine the merits of convolutional neural networks(CNN for short) and Bag of Visual Word(BoVW for short) module to design a model for video frames information extraction and representation. In order to meet the requirements of large-scale video retrieval, we proposed a visual weighted inverted index(VWII for short) and related algorithm to improve the efficiency and accuracy of retrieval process. Comprehensive experiments show that our proposed technique achieves substantial improvements (up to an order of magnitude speed up) over the state-of-the-art techniques with similar accuracy.

cs.MM↗

arXiv2018

SVS-JOIN: Efficient Spatial Visual Similarity Join over Multimedia Data

In the big data era, massive amount of multimedia data with geo-tags has been generated and collected by mobile smart devices equipped with mobile communications module and position sensor module. This trend has put forward higher request on large-scale of geo-multimedia data retrieval. Spatial similarity join is one of the important problem in the area of spatial database. Previous works focused on textual document with geo-tags, rather than geo-multimedia data such as geo-images. In this paper, we study a novel search problem named spatial visual similarity join (SVS-JOIN for short), which aims to find similar geo-image pairs in both the aspects of geo-location and visual content. We propose the definition of SVS-JOIN at the first time and present how to measure geographical similarity and visual similarity. Then we introduce a baseline inspired by the method for textual similarity join and a extension named SVS-JOIN$_G$ which applies spatial grid strategy to improve the efficiency. To further improve the performance of search, we develop a novel approach called SVS-JOIN$_Q$ which utilizes a quadtree and a global inverted index. Experimental evaluations on real geo-image datasets demonstrate that our solution has a really high performance.

cs.MM↗

arXiv2018

Efficient Multimedia Similarity Measurement Using Similar Elements

Online social networking techniques and large-scale multimedia systems are developing rapidly, which not only has brought great convenience to our daily life, but generated, collected, and stored large-scale multimedia data. This trend has put forward higher requirements and greater challenges on massive multimedia data retrieval. In this paper, we investigate the problem of image similarity measurement which is used to lots of applications. At first we propose the definition of similarity measurement of images and the related notions. Based on it we present a novel basic method of similarity measurement named SMIN. To improve the performance of calculation, we propose a novel indexing structure called SMI Temp Index (SMII for short). Besides, we establish an index of potential similar visual words off-line to solve to problem that the index cannot be reused. Experimental evaluations on two real image datasets demonstrate that our solution outperforms state-of-the-art method.

cs.MM↗

arXiv2018

Efficient Region of Visual Interests Search for Geo-multimedia Data

With the proliferation of online social networking services and mobile smart devices equipped with mobile communications module and position sensor module, massive amount of multimedia data has been collected, stored and shared. This trend has put forward higher request on massive multimedia data retrieval. In this paper, we investigate a novel spatial query named region of visual interests query (RoVIQ), which aims to search users containing geographical information and visual words. Three baseline methods are presented to introduce how to exploit existing techniques to address this problem. Then we propose the definition of this query and related notions at the first time. To improve the performance of query, we propose a novel spatial indexing structure called quadtree based inverted visual index which is a combination of quadtree, inverted index and visual words. Based on it, we design a efficient search algorithm named region of visual interests search to support RoVIQ. Experimental evaluations on real geo-image datasets demonstrate that our solution outperforms state-of-the-art method.

cs.MM↗

arXiv2018

An Efficient Approach for Geo-Multimedia Cross-Modal Retrieval

Due to the rapid development of mobile Internet techniques, cloud computation and popularity of online social networking and location-based services, massive amount of multimedia data with geographical information is generated and uploaded to the Internet. In this paper, we propose a novel type of cross-modal multimedia retrieval called geo-multimedia cross-modal retrieval which aims to search out a set of geo-multimedia objects based on geographical distance proximity and semantic similarity between different modalities. Previous studies for cross-modal retrieval and spatial keyword search cannot address this problem effectively because they do not consider multimedia data with geo-tags and do not focus on this type of query. In order to address this problem efficiently, we present the definition of $k$NN geo-multimedia cross-modal query at the first time and introduce relevant conceptions such as cross-modal semantic representation space. To bridge the semantic gap between different modalities, we propose a method named cross-modal semantic matching which contains two important component, i.e., CorrProj and LogsTran, which aims to construct a common semantic representation space for cross-modal semantic similarity measurement. Besides, we designed a framework based on deep learning techniques to implement common semantic representation space construction. In addition, a novel hybrid indexing structure named GMR-Tree combining geo-multimedia data and R-Tree is presented and a efficient $k$NN search algorithm called $k$GMCMS is designed. Comprehensive experimental evaluation on real and synthetic dataset clearly demonstrates that our solution outperforms the-state-of-the-art methods.

cs.MM↗

arXiv2018

Hierarchical One Permutation Hashing: Efficient Multimedia Near Duplicate Detection

With advances in multimedia technologies and the proliferation of smart phone, digital cameras, storage devices, there are a rapidly growing massive amount of multimedia data collected in many applications such as multimedia retrieval and management system, in which the data element is composed of text, image, video and audio. Consequently, the study of multimedia near duplicate detection has attracted significant concern from research organizations and commercial communities. Traditional solution minwish hashing (\minwise) faces two challenges: expensive preprocessing time and lower comparison speed. Thus, this work first introduce a hashing method called one permutation hashing (\oph) to shun the costly preprocessing time. Based on \oph, a more efficient strategy group based one permutation hashing (\goph) is developed to deal with the high comparison time. Based on the fact that the similarity of most multimedia data is not very high, this work design an new hashing method namely hierarchical one permutation hashing (\hoph) to further improve the performance. Comprehensive experiments on real multimedia datasets clearly show that with similar accuracy \hoph is five to seven times faster than

cs.MM↗

arXiv2018

Hierarchical Information Quadtree: Efficient Spatial Temporal Image Search for Multimedia Stream

Massive amount of multimedia data that contain times- tamps and geographical information are being generated at an unprecedented scale in many emerging applications such as photo sharing web site and social networks applications. Due to their importance, a large body of work has focused on efficiently computing various spatial image queries. In this paper,we study the spatial temporal image query which considers three important constraints during the search including time recency, spatial proximity and visual relevance. A novel index structure, namely Hierarchical Information Quadtree(\hiq), to efficiently insert/delete spatial temporal images with high arrive rates. Base on \hiq an efficient algorithm is developed to support spatial temporal image query. We show via extensive experimentation with real spatial databases clearly demonstrate the efficiency of our methods.

cs.MM↗

arXiv2018

Efficient Continuous Top-$k$ Geo-Image Search on Road Network

With the rapid development of mobile Internet and cloud computing technology, large-scale multimedia data, e.g., texts, images, audio and videos have been generated, collected, stored and shared. In this paper, we propose a novel query problem named continuous top-$k$ geo-image query on road network which aims to search out a set of geo-visual objects based on road network distance proximity and visual content similarity. Existing approaches for spatial textual query and geo-image query cannot address this problem effectively because they do not consider both of visual content similarity and road network distance proximity on road network. In order to address this challenge effectively and efficiently, firstly we propose the definition of geo-visual objects and continuous top-$k$ geo-visual objects query on road network, then develop a score function for search. To improve the query efficiency in a large-scale road network, we propose the search algorithm named geo-visual search on road network based on a novel hybrid indexing framework called VIG-Tree, which combines G-Tree and visual inverted index technique. In addition, an important notion named safe interval and results updating rule are proposed, and based on them we develop an efficient algorithm named moving monitor algorithm to solve continuous query. Experimental evaluation on real multimedia dataset and road network dataset illustrates that our solution outperforms state-of-the-art method.

cs.MM↗

arXiv2018

A Filter of Minhash for Image Similarity Measures

Image similarity measures play an important role in nearest neighbor search and duplicate detection for large-scale image datasets. Recently, Minwise Hashing (or Minhash) and its related hashing algorithms have achieved great performances in large-scale image retrieval systems. However, there are a large number of comparisons for image pairs in these applications, which may spend a lot of computation time and affect the performance. In order to quickly obtain the pairwise images that theirs similarities are higher than the specific threshold T (e.g., 0.5), we propose a dynamic threshold filter of Minwise Hashing for image similarity measures. It greatly reduces the calculation time by terminating the unnecessary comparisons in advance. We also find that the filter can be extended to other hashing algorithms, on when the estimator satisfies the binomial distribution, such as b-Bit Minwise Hashing, One Permutation Hashing, etc. In this pager, we use the Bag-of-Visual-Words (BoVW) model based on the Scale Invariant Feature Transform (SIFT) to represent the image features. We have proved that the filter is correct and effective through the experiment on real image datasets.

cs.MM↗

arXiv2018

Temporal Activity Path Based Character Correction in Social Networks

Vast amount of multimedia data contains massive and multifarious social information which is used to construct large-scale social networks. In a complex social network, a character should be ideally denoted by one and only one vertex. However, it is pervasive that a character is denoted by two or more vertices with different names, thus it is usually considered as multiple, different characters. This problem causes incorrectness of results in network analysis and mining. The factual challenge is that character uniqueness is hard to correctly confirm due to lots of complicated factors, e.g. name changing and anonymization, leading to character duplication. Early, limited research has shown that previous methods depended overly upon supplementary attribute information from databases. In this paper, we propose a novel method to merge the character vertices which refer to as the same entity but are denoted with different names. With this method, we firstly build the relationship network among characters based on records of social activities participated, which are extracted from multimedia sources. Then define temporal activity paths (TAPs) for each character over time. After that, we measure similarity of the TAPs for any two characters. If the similarity is high enough, the two vertices should be considered to the same character. Based on TAPs, we can determine whether to merge the two character vertices. Our experiments shown that this solution can accurately confirm character uniqueness in large-scale social network.

cs.SI↗

arXiv2018

Efficient Interactive Search for Geo-tagged Multimedia Data

Due to the advances in mobile computing and multimedia techniques, there are vast amount of multimedia data with geographical information collected in multifarious applications. In this paper, we propose a novel type of image search named interactive geo-tagged image search which aims to find out a set of images based on geographical proximity and similarity of visual content, as well as the preference of users. Existing approaches for spatial keyword query and geo-image query cannot address this problem effectively since they do not consider these three type of information together for query. In order to solve this challenge efficiently, we propose the definition of interactive top-$k$ geo-tagged image query and then present a framework including candidate search stage , interaction stage and termination stage. To enhance the searching efficiency in a large-scale database, we propose the candidate search algorithm named GI-SUPER Search based on a new notion called superior relationship and GIR-Tree, a novel index structure. Furthermore, two candidate selection methods are proposed for learning the preferences of the user during the interaction. At last, the termination procedure and estimation procedure are introduced in brief. Experimental evaluation on real multimedia dataset demonstrates that our solution has a really high performance.

cs.MM↗