arXiv subjects

Rong Zheng

Publications and source records attributed to Rong Zheng.

At least 37 records · Page 2Linked to original sources

arXiv2022

AttTrack: Online Deep Attention Transfer for Multi-object Tracking

Multi-object tracking (MOT) is a vital component of intelligent video analytics applications such as surveillance and autonomous driving. The time and storage complexity required to execute deep learning models for visual object tracking hinder their adoption on embedded devices with limited computing power. In this paper, we aim to accelerate MOT by transferring the knowledge from high-level features of a complex network (teacher) to a lightweight network (student) at both training and inference times. The proposed AttTrack framework has three key components: 1) cross-model feature learning to align intermediate representations from the teacher and student models, 2) interleaving the execution of the two models at inference time, and 3) incorporating the updated predictions from the teacher model as prior knowledge to assist the student model. Experiments on pedestrian tracking tasks are conducted on the MOT17 and MOT15 datasets using two different object detection backbones YOLOv5 and DLA34 show that AttTrack can significantly improve student model tracking performance while sacrificing only minor degradation of tracking speed.

cs.CV↗

arXiv2022

Efficient Action Recognition Using Confidence Distillation

Modern neural networks are powerful predictive models. However, when it comes to recognizing that they may be wrong about their predictions, they perform poorly. For example, for one of the most common activation functions, the ReLU and its variants, even a well-calibrated model can produce incorrect but high confidence predictions. In the related task of action recognition, most current classification methods are based on clip-level classifiers that densely sample a given video for non-overlapping, same-sized clips and aggregate the results using an aggregation function - typically averaging - to achieve video level predictions. While this approach has shown to be effective, it is sub-optimal in recognition accuracy and has a high computational overhead. To mitigate both these issues, we propose the confidence distillation framework to teach a representation of uncertainty of the teacher to the student sampler and divide the task of full video prediction between the student and the teacher models. We conduct extensive experiments on three action recognition datasets and demonstrate that our framework achieves significant improvements in action recognition accuracy (up to 20%) and computational efficiency (more than 40%).

cs.CV↗

arXiv2022

Individualizing Head-Related Transfer Functions for Binaural Acoustic Applications

A Head Related Transfer Function (HRTF) characterizes how a human ear receives sounds from a point in space, and depends on the shapes of one's head, pinna, and torso. Accurate estimations of HRTFs for human subjects are crucial in enabling binaural acoustic applications such as sound localization and 3D sound spatialization. Unfortunately, conventional approaches for HRTF estimation rely on specialized devices or lengthy measurement processes. This work proposes a novel lightweight method for HRTF individualization that can be implemented using commercial-off-the-shelf components and performed by average users in home settings. The proposed method has two key components: a generative neural network model that can be individualized to predict HRTFs of new subjects from sparse measurements, and a lightweight measurement procedure that collects HRTF data from spatial locations. Extensive experiments using a public dataset and in house measurement data from 10 subjects of different ages and genders, show that the individualized models significantly outperform a baseline model in the accuracy of predicted HRTFs. To further demonstrate the advantages of individualized HRTFs, we implement two prototype applications for binaural localization and acoustic spatialization. We find that the performance of a localization model is improved by 15 degree after trained with individualized HRTFs. Furthermore, in hearing tests, the success rate of correctly identifying the azimuth direction of incoming sounds increases by 183% after individualization.

cs.SD↗

arXiv2022

CROMOSim: A Deep Learning-based Cross-modality Inertial Measurement Simulator

With the prevalence of wearable devices, inertial measurement unit (IMU) data has been utilized in monitoring and assessment of human mobility such as human activity recognition (HAR). Training deep neural network (DNN) models for these tasks require a large amount of labeled data, which are hard to acquire in uncontrolled environments. To mitigate the data scarcity problem, we design CROMOSim, a cross-modality sensor simulator that simulates high fidelity virtual IMU sensor data from motion capture systems or monocular RGB cameras. It utilizes a skinned multi-person linear model (SMPL) for 3D body pose and shape representations, to enable simulation from arbitrary on-body positions. A DNN model is trained to learn the functional mapping from imperfect trajectory estimations in a 3D SMPL body tri-mesh due to measurement noise, calibration errors, occlusion and other modeling artifacts, to IMU data. We evaluate the fidelity of CROMOSim simulated data and its utility in data augmentation on various HAR datasets. Extensive experiment results show that the proposed model achieves a 6.7% improvement over baseline methods in a HAR task.

eess.SP↗

arXiv2021

DeepScale: Online Frame Size Adaptation for Multi-object Tracking on Smart Cameras and Edge Servers

In surveillance and search and rescue applications, it is important to perform multi-target tracking (MOT) in real-time on low-end devices. Today's MOT solutions employ deep neural networks, which tend to have high computation complexity. Recognizing the effects of frame sizes on tracking performance, we propose DeepScale, a model agnostic frame size selection approach that operates on top of existing fully convolutional network-based trackers to accelerate tracking throughput. In the training stage, we incorporate detectability scores into a one-shot tracker architecture so that DeepScale can learn representation estimations for different frame sizes in a self-supervised manner. During inference, it can adapt frame sizes according to the complexity of visual contents based on user-controlled parameters. To leverage computation resources on edge servers, we propose two computation partition schemes tailored for MOT, namely, edge server only with adaptive frame-size transmission and edge server-assisted tracking. Extensive experiments and benchmark tests on MOT datasets demonstrate the effectiveness and flexibility of DeepScale. Compared to a state-of-the-art tracker, DeepScale++, a variant of DeepScale achieves 1.57X accelerated with only moderate degradation ~2.3\ in tracking accuracy on the MOT15 dataset in one configuration. We have implemented and evaluated DeepScale++ and the proposed computation partition schemes on a small-scale testbed consisting of an NVIDIA Jetson TX2 board and a GPU server. The experiments reveal non-trivial trade-offs between tracking performance and latency compared to server-only or smart camera-only solutions.

cs.CV↗

arXiv2021

Ubiquitous Acoustic Sensing on Commodity IoT Devices: A Survey

With the proliferation of Internet-of-Things devices, acoustic sensing attracts much attention in recent years. It exploits acoustic transceivers such as microphones and speakers beyond their primary functions, namely recording and playing, to enable novel applications and new user experiences. In this paper, we present the first systematic survey of recent advances in active acoustic sensing using commodity hardware with a frequency range below 24~\!kHz. We propose a general framework that categorizes main building blocks of acoustic sensing systems. This framework encompasses three layers, i.e., physical layer, core technique layer, and application layer. The physical layer includes basic hardware components, acoustic platforms as well as the air-borne and structure-borne channel characteristics. The core technique layer encompasses key mechanisms to generate acoustic signals (waveforms) and to extract useful temporal, spatial and spectral information from received signals. The application layer builds upon the functions offered by the core techniques to realize different acoustic sensing applications. We highlight unique challenges due to the limitations of physical devices and acoustic channels and how they are mitigated or overcame by core processing techniques and application-specific solutions. Finally, research opportunities and future directions are discussed to spawn further in-depth investigation on acoustic sensing.

cs.SD↗

arXiv2020

Invariant Feature Learning for Sensor-based Human Activity Recognition

Wearable sensor-based human activity recognition (HAR) has been a research focus in the field of ubiquitous and mobile computing for years. In recent years, many deep models have been applied to HAR problems. However, deep learning methods typically require a large amount of data for models to generalize well. Significant variances caused by different participants or diverse sensor devices limit the direct application of a pre-trained model to a subject or device that has not been seen before. To address these problems, we present an invariant feature learning framework (IFLF) that extracts common information shared across subjects and devices. IFLF incorporates two learning paradigms: 1) meta-learning to capture robust features across seen domains and adapt to an unseen one with similarity-based data selection; 2) multi-task learning to deal with data shortage and enhance overall performance via knowledge sharing among different subjects. Experiments demonstrated that IFLF is effective in handling both subject and device diversion across popular open datasets and an in-house dataset. It outperforms a baseline model of up to 40% in test accuracy.

eess.SP↗

arXiv2020

RECOME: a New Density-Based Clustering Algorithm Using Relative KNN Kernel Density

Discovering clusters from a dataset with different shapes, densities, and scales is a known challenging problem in data clustering. In this paper, we propose the RElative COre MErge (RECOME) clustering algorithm. The core of RECOME is a novel density measure, i.e., Relative $K$ nearest Neighbor Kernel Density (RNKD). RECOME identifies core objects with unit RNKD, and {partitions} non-core objects into atom clusters by successively following higher-density neighbor relations toward core objects. Core objects and their corresponding atom clusters are then merged through $α$-reachable paths on a KNN graph. We discover that the number of clusters computed by RECOME is a step function of the $α$ parameter with jump discontinuity on a small collection of values. A fast jump discontinuity discovery (FJDD) method is proposed based on graph theory. RECOME is evaluated on both synthetic datasets and real datasets. Experimental results indicate that RECOME is able to discover clusters with different shapes, densities, and scales. It outperforms six baseline methods on both synthetic datasets and real datasets. Moreover, FJDD is shown to be effective to extract the jump discontinuity set of parameter $α$ for all tested datasets, which can ease the task of data exploration and parameter tuning.

cs.DB↗

arXiv2020

CacheNet: A Model Caching Framework for Deep Learning Inference on the Edge

The success of deep neural networks (DNN) in machine perception applications such as image classification and speech recognition comes at the cost of high computation and storage complexity. Inference of uncompressed large scale DNN models can only run in the cloud with extra communication latency back and forth between cloud and end devices, while compressed DNN models achieve real-time inference on end devices at the price of lower predictive accuracy. In order to have the best of both worlds (latency and accuracy), we propose CacheNet, a model caching framework. CacheNet caches low-complexity models on end devices and high-complexity (or full) models on edge or cloud servers. By exploiting temporal locality in streaming data, high cache hit and consequently shorter latency can be achieved with no or only marginal decrease in prediction accuracy. Experiments on CIFAR-10 and FVG have shown CacheNet is 58-217% faster than baseline approaches that run inference tasks on end devices or edge servers alone.

cs.LG↗

arXiv2020

Informative Path Planning for Mobile Sensing with Reinforcement Learning

Large-scale spatial data such as air quality, thermal conditions and location signatures play a vital role in a variety of applications. Collecting such data manually can be tedious and labour intensive. With the advancement of robotic technologies, it is feasible to automate such tasks using mobile robots with sensing and navigation capabilities. However, due to limited battery lifetime and scarcity of charging stations, it is important to plan paths for the robots that maximize the utility of data collection, also known as the informative path planning (IPP) problem. In this paper, we propose a novel IPP algorithm using reinforcement learning (RL). A constrained exploration and exploitation strategy is designed to address the unique challenges of IPP, and is shown to have fast convergence and better optimality than a classical reinforcement learning approach. Extensive experiments using real-world measurement data demonstrate that the proposed algorithm outperforms state-of-the-art algorithms in most test cases. Interestingly, unlike existing solutions that have to be re-executed when any input parameter changes, our RL-based solution allows a degree of transferability across different problem instances.

cs.RO↗

arXiv2019

SensorDrop: A Reinforcement Learning Framework for Communication Overhead Reduction on the Edge

In IoT solutions, it is usually desirable to collect data from a large number of distributed IoT sensors at a central node in the cloud for further processing. One of the main design challenges of such solutions is the high communication overhead between the sensors and the central node (especially for multimedia data). In this paper, we aim to reduce the communication overhead and propose a method that is able to determine which sensors should send their data to the central node and which to drop data. The idea is that some sensors may have data which are correlated with others and some may have data that are not essential for the operation to be performed at the central node. As such decisions are application dependent and may change over time, they should be learned during the operation of the system, for that we propose a method based on Advantage Actor-Critic (A2C) reinforcement learning which gradually learns which sensor's data is cost-effective to be sent to the central node. The proposed approach has been evaluated on a multi-view multi-camera dataset, and we observe a significant reduction in communication overhead with marginal degradation in object classification accuracy.

cs.NI↗

arXiv2018

Informative Path Planning for Location Fingerprint Collection

Fingerprint-based indoor localization methods are promising due to the high availability of deployed access points and compatibility with commercial-off-the-shelf user devices. However, to train regression models for localization, an extensive site survey is required to collect fingerprint data from the target areas. In this paper, we consider the problem of informative path planning (IPP) to find the optimal walk for site survey subject to a budget constraint. IPP for location fingerprint collection is related to the well-known orienteering problem (OP) but is more challenging due to edge-based non-additive rewards and revisits. Given the NP-hardness of IPP, we propose two heuristic approaches: a Greedy algorithm and a genetic algorithm. We show through experimental data collected from two indoor environments with different characteristics that the two algorithms have low computation complexity, can generally achieve higher utility and lower localization errors compared to the extension of two state-of-the-art approaches to OP.

cs.RO↗

arXiv2018

Logographic Subword Model for Neural Machine Translation

A novel logographic subword model is proposed to reinterpret logograms as abstract subwords for neural machine translation. Our approach drastically reduces the size of an artificial neural network, while maintaining comparable BLEU scores as those attained with the baseline RNN and CNN seq2seq models. The smaller model size also leads to shorter training and inference time. Experiments demonstrate that in the tasks of English-Chinese/Chinese-English translation, the reduction of those aspects can be from $11\%$ to as high as $77\%$. Compared to previous subword models, abstract subwords can be applied to various logographic languages. Considering most of the logographic languages are ancient and very low resource languages, these advantages are very desirable for archaeological computational linguistic applications such as a resource-limited offline hand-held Demotic-English translator.

cs.CL↗

arXiv2017

Detecting Location Fraud in Indoor Mobile Crowdsensing

Mobile crowdsensing allows a large number of mobile devices to measure phenomena of common interests and form a body of knowledge about natural and social environments. In order to get location annotations for indoor mobile crowdsensing, reference tags are usually deployed which are susceptible to tampering and compromises by attackers. In this work, we consider three types of location-related attacks including tag forgery, tag misplacement, and tag removal. Different detection algorithms are proposed to deal with these attacks. First, we introduce location-dependent fingerprints as supplementary information for better location identification. A truth discovery algorithm is then proposed to detect falsified data. Moreover, visiting patterns are utilized for the detection of tag misplacement and removal. Experiments on both crowdsensed and emulated dataset show that the proposed algorithms can detect all three types of attacks with high accuracy.

cs.CR↗

arXiv2017

TuRF: Fast Data Collection for Fingerprint-based Indoor Localization

Many infrastructure-free indoor positioning systems rely on fine-grained location-dependent fingerprints to train models for localization. The site survey process to collect fingerprints is laborious and is considered one of the major obstacles to deploying such systems. In this paper, we propose TuRF, a fast path-based fingerprint collection mechanism for site survey. We demonstrate the feasibility to collect fingerprints for indoor localization during walking along predefined paths. A step counter is utilized to accommodate the variations in walking speed. Approximate location labels inferred from the steps are then used to train a Gaussian Process regression model. Extensive experiments show that TuRF can significantly reduce the required time for site survey, without compromising the localization performance.

cs.NI↗

arXiv2017

ARABIS: an Asynchronous Acoustic Indoor Positioning System for Mobile Devices

Acoustic ranging based indoor positioning solutions have the advantage of higher ranging accuracy and better compatibility with commercial-off-the-self consumer devices. However, similar to other time-domain based approaches using Time-of-Arrival and Time-Difference-of-Arrival, they suffer from performance degradation in presence of multi-path propagation and low received signal-to-noise ratio (SNR) in indoor environments. In this paper, we improve upon our previous work on asynchronous acoustic indoor positioning and develop ARABIS, a robust and low-cost acoustic positioning system (IPS) for mobile devices. We develop a low-cost acoustic board custom-designed to support large operational ranges and extensibility. To mitigate the effects of low SNR and multi-path propagation, we devise a robust algorithm that iteratively removes possible outliers by taking advantage of redundant TDoA estimates. Experiments have been carried in two testbeds of sizes 10.67m*7.76m and 15m*15m, one in an academic building and one in a convention center. The proposed system achieves average and 95% quantile localization errors of 7.4cm and 16.0cm in the first testbed with 8 anchor nodes and average and 95% quantile localization errors of 20.4cm and 40.0cm in the second testbed with 4 anchor nodes only.

cs.NI↗

arXiv2015

Optimal Resource Allocation in Multicast Device-to-Device Communications Underlaying LTE Networks

In this paper, we present a framework for resource allocations for multicast device-to-device (D2D) communications underlaying a cellular network. The objective is to maximize the sum throughput of active cellular users (CUs) and feasible D2D groups in a cell, while meeting a certain signal-to-interferenceplus- noise ratio (SINR) constraint for both the CUs and D2D groups. We formulate the problem of power and channel allocation as a mixed integer nonlinear programming (MINLP) problem where one D2D group can reuse the channels of multiple CUs and the channel of each CU can be reused by multiple D2D groups. Distinct from existing approaches in the literature, our formulation and solution methods provide an effective and flexible means to utilize radio resources in cellular networks and share them with multicast groups without causing harmful interference to each other. A variant of the generalized bender decomposition (GBD) is applied to optimally solve the MINLP problem. A greedy algorithm and a low-complexity heuristic solution are then devised. The performance of all schemes is evaluated through extensive simulations. Numerical results demonstrate that the proposed greedy algorithm can achieve closeto- optimal performance, and the heuristic algorithm provides good performance, though inferior than that of the greedy, with much lower complexity.

cs.NI↗

arXiv2015

Device Fingerprinting in Wireless Networks: Challenges and Opportunities

Node forgery or impersonation, in which legitimate cryptographic credentials are captured by an adversary, constitutes one major security threat facing wireless networks. The fact that mobile devices are prone to be compromised and reverse engineered significantly increases the risk of such attacks in which adversaries can obtain secret keys on trusted nodes and impersonate the legitimate node. One promising approach toward thwarting these attacks is through the extraction of unique fingerprints that can provide a reliable and robust means for device identification. These fingerprints can be extracted from transmitted signal by analyzing information across the protocol stack. In this paper, the first unified and comprehensive tutorial in the area of wireless device fingerprinting for security applications is presented. In particular, we aim to provide a detailed treatment on developing novel wireless security solutions using device fingerprinting techniques. The objectives are three-fold: (i) to introduce a comprehensive taxonomy of wireless features that can be used in fingerprinting, (ii) to provide a systematic review on fingerprint algorithms including both white-list based and unsupervised learning approaches, and (iii) to identify key open research problems in the area of device fingerprinting and feature extraction, as applied to wireless security.

cs.NI↗