arXiv subjects

Jun Won Choi

Publications and source records attributed to Jun Won Choi.

53 records · Page 3Linked to original sources

arXiv2021

Joint 3D Object Detection and Tracking Using Spatio-Temporal Representation of Camera Image and LiDAR Point Clouds

In this paper, we propose a new joint object detection and tracking (JoDT) framework for 3D object detection and tracking based on camera and LiDAR sensors. The proposed method, referred to as 3D DetecTrack, enables the detector and tracker to cooperate to generate a spatio-temporal representation of the camera and LiDAR data, with which 3D object detection and tracking are then performed. The detector constructs the spatio-temporal features via the weighted temporal aggregation of the spatial features obtained by the camera and LiDAR fusion. Then, the detector reconfigures the initial detection results using information from the tracklets maintained up to the previous time step. Based on the spatio-temporal features generated by the detector, the tracker associates the detected objects with previously tracked objects using a graph neural network (GNN). We devise a fully-connected GNN facilitated by a combination of rule-based edge pruning and attention-based edge gating, which exploits both spatial and temporal object contexts to improve tracking performance. The experiments conducted on both KITTI and nuScenes benchmarks demonstrate that the proposed 3D DetecTrack achieves significant improvements in both detection and tracking performances over baseline methods and achieves state-of-the-art performance among existing methods through collaboration between the detector and tracker.

cs.CV↗

arXiv2021

LaPred: Lane-Aware Prediction of Multi-Modal Future Trajectories of Dynamic Agents

In this paper, we address the problem of predicting the future motion of a dynamic agent (called a target agent) given its current and past states as well as the information on its environment. It is paramount to develop a prediction model that can exploit the contextual information in both static and dynamic environments surrounding the target agent and generate diverse trajectory samples that are meaningful in a traffic context. We propose a novel prediction model, referred to as the lane-aware prediction (LaPred) network, which uses the instance-level lane entities extracted from a semantic map to predict the multi-modal future trajectories. For each lane candidate found in the neighborhood of the target agent, LaPred extracts the joint features relating the lane and the trajectories of the neighboring agents. Then, the features for all lane candidates are fused with the attention weights learned through a self-supervised learning task that identifies the lane candidate likely to be followed by the target agent. Using the instance-level lane information, LaPred can produce the trajectories compliant with the surroundings better than 2D raster image-based methods and generate the diverse future trajectories given multiple lane candidates. The experiments conducted on the public nuScenes dataset and Argoverse dataset demonstrate that the proposed LaPred method significantly outperforms the existing prediction models, achieving state-of-the-art performance in the benchmarks.

cs.CV↗

arXiv2020

Joint Representation of Temporal Image Sequences and Object Motion for Video Object Detection

In this paper, we propose a new video object detector (VoD) method referred to as temporal feature aggregation and motion-aware VoD (TM-VoD), which produces a joint representation of temporal image sequences and object motion. The proposed TM-VoD aggregates visual feature maps extracted by convolutional neural networks applying the temporal attention gating and spatial feature alignment. This temporal feature aggregation is performed in two stages in a hierarchical fashion. In the first stage, the visual feature maps are fused at a pixel level via gated attention model. In the second stage, the proposed method aggregates the features after aligning the object features using temporal box offset calibration and weights them according to the cosine similarity measure. The proposed TM-VoD also finds the representation of the motion of objects in two successive steps. The pixel-level motion features are first computed based on the incremental changes between the adjacent visual feature maps. Then, box-level motion features are obtained from both the region of interest (RoI)-aligned pixel-level motion features and the sequential changes of the box coordinates. Finally, all these features are concatenated to produce a joint representation of the objects for VoD. The experiments conducted on the ImageNet VID dataset demonstrate that the proposed method outperforms existing VoD methods and achieves a performance comparable to that of state-of-the-art VoDs.

cs.CV↗

arXiv2020

3D-CVF: Generating Joint Camera and LiDAR Features Using Cross-View Spatial Feature Fusion for 3D Object Detection

In this paper, we propose a new deep architecture for fusing camera and LiDAR sensors for 3D object detection. Because the camera and LiDAR sensor signals have different characteristics and distributions, fusing these two modalities is expected to improve both the accuracy and robustness of 3D object detection. One of the challenges presented by the fusion of cameras and LiDAR is that the spatial feature maps obtained from each modality are represented by significantly different views in the camera and world coordinates; hence, it is not an easy task to combine two heterogeneous feature maps without loss of information. To address this problem, we propose a method called 3D-CVF that combines the camera and LiDAR features using the cross-view spatial feature fusion strategy. First, the method employs auto-calibrated projection, to transform the 2D camera features to a smooth spatial feature map with the highest correspondence to the LiDAR features in the bird's eye view (BEV) domain. Then, a gated feature fusion network is applied to use the spatial attention maps to mix the camera and LiDAR features appropriately according to the region. Next, camera-LiDAR feature fusion is also achieved in the subsequent proposal refinement stage. The camera feature is used from the 2D camera-view domain via 3D RoI grid pooling and fused with the BEV feature for proposal refinement. Our evaluations, conducted on the KITTI and nuScenes 3D object detection datasets demonstrate that the camera-LiDAR fusion offers significant performance gain over single modality and that the proposed 3D-CVF achieves state-of-the-art performance in the KITTI benchmark.

cs.CV↗

arXiv2020

ScarfNet: Multi-scale Features with Deeply Fused and Redistributed Semantics for Enhanced Object Detection

Convolutional neural network (CNN) has led to significant progress in object detection. In order to detect the objects in various sizes, the object detectors often exploit the hierarchy of the multi-scale feature maps called feature pyramid, which is readily obtained by the CNN architecture. However, the performance of these object detectors is limited since the bottom-level feature maps, which experience fewer convolutional layers, lack the semantic information needed to capture the characteristics of the small objects. In order to address such problem, various methods have been proposed to increase the depth for the bottom-level features used for object detection. While most approaches are based on the generation of additional features through the top-down pathway with lateral connections, our approach directly fuses multi-scale feature maps using bidirectional long short term memory (biLSTM) in effort to generate deeply fused semantics. Then, the resulting semantic information is redistributed to the individual pyramidal feature at each scale through the channel-wise attention model. We integrate our semantic combining and attentive redistribution feature network (ScarfNet) with baseline object detectors, i.e., Faster R-CNN, single-shot multibox detector (SSD) and RetinaNet. Our experiments show that our method outperforms the existing feature pyramid methods as well as the baseline detectors and achieve the state of the art performances in the PASCAL VOC and COCO detection benchmarks.

cs.CV↗

arXiv2019

Downlink Pilot Precoding and Compressed Channel Feedback for FDD-Based Cell-Free Systems

Cell-free system where a group of base stations (BSs) cooperatively serves users has received much attention as a promising technology for the future wireless systems. In order to maximize the cooperation gain in the cell-free systems, acquisition of downlink channel state information (CSI) at the BSs is crucial. While this task is relatively easy for the time division duplexing (TDD) systems due to the channel reciprocity, it is not easy for the frequency division duplexing (FDD) systems due to the CSI feedback overhead. This issue is even more pronounced in the cell-free systems since the user needs to feed back the CSIs of multiple BSs. In this paper, we propose a novel feedback reduction technique for the FDD-based cell-free systems. Key feature of the proposed technique is to choose a few dominating paths and then feed back the path gain information (PGI) of the chosen paths. By exploiting the property that the angles of departure (AoDs) are quite similar in the uplink and downlink channels (this property is referred to as angle reciprocity), the BSs obtain the AoDs directly from the uplink pilot signal. From the extensive simulations, we observe that the proposed technique can achieve more than 80% of feedback overhead reduction over the conventional CSI feedback scheme.

cs.IT↗

arXiv2018

Robust Deep Multi-modal Learning Based on Gated Information Fusion Network

The goal of multi-modal learning is to use complimentary information on the relevant task provided by the multiple modalities to achieve reliable and robust performance. Recently, deep learning has led significant improvement in multi-modal learning by allowing for the information fusion in the intermediate feature levels. This paper addresses a problem of designing robust deep multi-modal learning architecture in the presence of imperfect modalities. We introduce deep fusion architecture for object detection which processes each modality using the separate convolutional neural network (CNN) and constructs the joint feature map by combining the intermediate features from the CNNs. In order to facilitate the robustness to the degraded modalities, we employ the gated information fusion (GIF) network which weights the contribution from each modality according to the input feature maps to be fused. The weights are determined through the convolutional layers followed by a sigmoid function and trained along with the information fusion network in an end-to-end fashion. Our experiments show that the proposed GIF network offers the additional architectural flexibility to achieve robust performance in handling some degraded modalities, and show a significant performance improvement based on Single Shot Detector (SSD) for KITTI dataset using the proposed fusion network and data augmentation schemes.

cs.CV↗

arXiv2018

Sequence-to-Sequence Prediction of Vehicle Trajectory via LSTM Encoder-Decoder Architecture

In this paper, we propose a deep learning based vehicle trajectory prediction technique which can generate the future trajectory sequence of surrounding vehicles in real time. We employ the encoder-decoder architecture which analyzes the pattern underlying in the past trajectory using the long short-term memory (LSTM) based encoder and generates the future trajectory sequence using the LSTM based decoder. This structure produces the $K$ most likely trajectory candidates over occupancy grid map by employing the beam search technique which keeps the $K$ locally best candidates from the decoder output. The experiments conducted on highway traffic scenarios show that the prediction accuracy of the proposed method is significantly higher than the conventional trajectory prediction techniques.

cs.LG↗

arXiv2018

Efficient Beam Training and Channel Estimation for Millimeter Wave Communications Under Mobility

In this paper, we propose an efficient beam training technique for millimeter-wave (mmWave) communications. When some mobile users are under high mobility, the beam training should be performed frequently to ensure the accurate acquisition of the channel state information. In order to reduce the resource overhead caused by frequent beam training, we introduce a dedicated beam training strategy which sends the training beams separately to a specific high mobility user (called a target user) without changing the periodicity of the conventional beam training. The dedicated beam training requires small amount of resources since the training beams can be optimized for the target user. In order to satisfy the performance requirement with low training overhead, we propose the optimal training beam selection strategy which finds the best beamforming vectors yielding the lowest channel estimation error based on the target user's probabilistic channel information. Such dedicated beam training is combined with the greedy channel estimation algorithm that accounts for sparse characteristics and temporal dynamics of the target user's channel. Our numerical evaluation demonstrates that the proposed scheme can maintain good channel estimation performance with significantly less training overhead compared to the conventional beam training protocols.

cs.IT↗

arXiv2017

Probabilistic Vehicle Trajectory Prediction over Occupancy Grid Map via Recurrent Neural Network

In this paper, we propose an efficient vehicle trajectory prediction framework based on recurrent neural network. Basically, the characteristic of the vehicle's trajectory is different from that of regular moving objects since it is affected by various latent factors including road structure, traffic rules, and driver's intention. Previous state of the art approaches use sophisticated vehicle behavior model describing these factors and derive the complex trajectory prediction algorithm, which requires a system designer to conduct intensive model optimization for practical use. Our approach is data-driven and simple to use in that it learns complex behavior of the vehicles from the massive amount of trajectory data through deep neural network model. The proposed trajectory prediction method employs the recurrent neural network called long short-term memory (LSTM) to analyze the temporal behavior and predict the future coordinate of the surrounding vehicles. The proposed scheme feeds the sequence of vehicles' coordinates obtained from sensor measurements to the LSTM and produces the probabilistic information on the future location of the vehicles over occupancy grid map. The experiments conducted using the data collected from highway driving show that the proposed method can produce reasonably good estimate of future trajectory.

cs.LG↗

arXiv2017

Autonomous Braking System via Deep Reinforcement Learning

In this paper, we propose a new autonomous braking system based on deep reinforcement learning. The proposed autonomous braking system automatically decides whether to apply the brake at each time step when confronting the risk of collision using the information on the obstacle obtained by the sensors. The problem of designing brake control is formulated as searching for the optimal policy in Markov decision process (MDP) model where the state is given by the relative position of the obstacle and the vehicle's speed, and the action space is defined as whether brake is stepped or not. The policy used for brake control is learned through computer simulations using the deep reinforcement learning method called deep Q-network (DQN). In order to derive desirable braking policy, we propose the reward function which balances the damage imposed to the obstacle in case of accident and the reward achieved when the vehicle runs out of risk as soon as possible. DQN is trained for the scenario where a vehicle is encountered with a pedestrian crossing the urban road. Experiments show that the control agent exhibits desirable control behavior and avoids collision without any mistake in various uncertain environments.

cs.AI↗

arXiv2017

New Beam Tracking Technique for Millimeter Wave-band Communications

In this paper, we propose an efficient beam tracking method for mobility scenario in mmWave-band communications. When the position of the mobile changes in mobility scenario, the base-station needs to perform beam training frequently to track the time-varying channel, thereby spending significant resources for training beams. In order to reduce the training overhead, we propose a new beam training approach called "beam tracking" which exploits the continuous nature of time varying angle of departure (AoD) for beam selection. We show that transmission of only two training beams is enough to track the time-varying AoD at good accuracy. We derive the optimal selection of beam pair which minimizes Cramer-Rao Lower Bound (CRLB) for AoD estimation averaged over statistical distribution of the AoD. Our numerical results demonstrate that the proposed beam tracking scheme produces better AoD estimation than the conventional beam training protocol with less training overhead.

cs.IT↗

arXiv2016

Compressed Sensing for Wireless Communications : Useful Tips and Tricks

As a paradigm to recover the sparse signal from a small set of linear measurements, compressed sensing (CS) has stimulated a great deal of interest in recent years. In order to apply the CS techniques to wireless communication systems, there are a number of things to know and also several issues to be considered. However, it is not easy to come up with simple and easy answers to the issues raised while carrying out research on CS. The main purpose of this paper is to provide essential knowledge and useful tips that wireless communication researchers need to know when designing CS-based wireless systems. First, we present an overview of the CS technique, including basic setup, sparse recovery algorithm, and performance guarantee. Then, we describe three distinct subproblems of CS, viz., sparse estimation, support identification, and sparse detection, with various wireless communication applications. We also address main issues encountered in the design of CS-based wireless communication systems. These include potentials and limitations of CS techniques, useful tips that one should be aware of, subtle points that one should pay attention to, and some prior knowledge to achieve better performance. Our hope is that this article will be a useful guide for wireless communication researchers and even non-experts to grasp the gist of CS techniques.

cs.IT↗

arXiv2015

Sparse Detection of Non-Sparse Signals for Large-Scale Wireless Systems

In this paper, we introduce a new detection algorithm for large-scale wireless systems, referred to as post sparse error detection (PSED) algorithm, that employs a sparse error recovery algorithm to refine the estimate of a symbol vector obtained by the conventional linear detector. The PSED algorithm operates in two steps: 1) sparse transformation converting the original non-sparse system into the sparse system whose input is an error vector caused by the symbol slicing and 2) estimation of the error vector using the sparse recovery algorithm. From the asymptotic mean square error (MSE) analysis and empirical simulations performed on large-scale systems, we show that the PSED algorithm brings significant performance gain over classical linear detectors while imposing relatively small computational overhead.

cs.IT↗

arXiv2015

Statistical Recovery of Simultaneously Sparse Time-Varying Signals from Multiple Measurement Vectors

In this paper, we propose a new sparse signal recovery algorithm, referred to as sparse Kalman tree search (sKTS), that provides a robust reconstruction of the sparse vector when the sequence of correlated observation vectors are available. The proposed sKTS algorithm builds on expectation-maximization (EM) algorithm and consists of two main operations: 1) Kalman smoothing to obtain the a posteriori statistics of the source signal vectors and 2) greedy tree search to estimate the support of the signal vectors. Through numerical experiments, we demonstrate that the proposed sKTS algorithm is effective in recovering the sparse signals and performs close to the Oracle (genie-based) Kalman estimator.

cs.IT↗

arXiv2014

Greedy Sparse Signal Recovery with Tree Pruning

Recently, greedy algorithm has received much attention as a cost-effective means to reconstruct the sparse signals from compressed measurements. Much of previous work has focused on the investigation of a single candidate to identify the support (index set of nonzero elements) of the sparse signals. Well-known drawback of the greedy approach is that the chosen candidate is often not the optimal solution due to the myopic decision in each iteration. In this paper, we propose a greedy sparse recovery algorithm investigating multiple promising candidates via the tree search. Two key ingredients of the proposed algorithm, referred to as the matching pursuit with a tree pruning (TMP), to achieve efficiency in the tree search are the {\it pre-selection} to put a restriction on columns of the sensing matrix to be investigated and the {\it tree pruning} to eliminate unpromising paths from the search tree. In our performance guarantee analysis and empirical simulations, we show that TMP is effective in recovering sparse signals in both noiseless and noisy scenarios.

cs.IT↗

arXiv2012

Low Complexity Turbo-Equalization: A Clustering Approach

We introduce a low complexity approach to iterative equalization and decoding, or "turbo equalization", that uses clustered models to better match the nonlinear relationship that exists between likelihood information from a channel decoder and the symbol estimates that arise in soft-input channel equalization. The introduced clustered turbo equalizer uses piecewise linear models to capture the nonlinear dependency of the linear minimum mean square error (MMSE) symbol estimate on the symbol likelihoods produced by the channel decoder and maintains a computational complexity that is only linear in the channel memory. By partitioning the space of likelihood information from the decoder, based on either hard or soft clustering, and using locally-linear adaptive equalizers within each clustered region, the performance gap between the linear MMSE equalizer and low-complexity, LMS-based linear turbo equalizers can be dramatically narrowed.

eess.SY↗