arXiv subjects

Chen Sun

Publications and source records attributed to Chen Sun.

At least 199 records · Page 11Linked to original sources

arXiv2020

Sum Secret Key Rate Maximization for TDD Multi-User Massive MIMO Wireless Networks

Physical-layer key generation (PKG) based on channel reciprocity has recently emerged as a new technique to establish secret keys between devices. Most works focus on pairwise communication scenarios with single or small-scale antennas. However, the fifth generation (5G) wireless communications employ massive multiple-input multiple-output (MIMO) to support multiple users simultaneously, bringing serious overhead of reciprocal channel acquisition. This paper presents a multi-user secret key generation in massive MIMO wireless networks. We provide a beam domain channel model, in which different elements represent the channel gains from different transmit directions to different receive directions. Based on this channel model, we analyze the secret key rate and derive a closed-form expression under independent channel conditions. To maximize the sum secret key rate, we provide the optimal conditions for the Kronecker product of the precoding and receiving matrices and propose an algorithm to generate these matrices with pilot reuse. The proposed optimization design can significantly reduce the pilot overhead of the reciprocal channel state information acquisition. Furthermore, we analyze the security under the channel correlation between user terminals (UTs), and propose a low overhead multi-user secret key generation with non-overlapping beams between UTs. Simulation results demonstrate the near optimal performance of the proposed precoding and receiving matrices design and the advantages of the non-overlapping beam allocation.

cs.IT↗

arXiv2020

TNT: Target-driveN Trajectory Prediction

Predicting the future behavior of moving agents is essential for real world applications. It is challenging as the intent of the agent and the corresponding behavior is unknown and intrinsically multimodal. Our key insight is that for prediction within a moderate time horizon, the future modes can be effectively captured by a set of target states. This leads to our target-driven trajectory prediction (TNT) framework. TNT has three stages which are trained end-to-end. It first predicts an agent's potential target states $T$ steps into the future, by encoding its interactions with the environment and the other agents. TNT then generates trajectory state sequences conditioned on targets. A final stage estimates trajectory likelihoods and a final compact set of trajectory predictions is selected. This is in contrast to previous work which models agent intents as latent variables, and relies on test-time sampling to generate diverse trajectories. We benchmark TNT on trajectory prediction of vehicles and pedestrians, where we outperform state-of-the-art on Argoverse Forecasting, INTERACTION, Stanford Drone and an in-house Pedestrian-at-Intersection dataset.

cs.CV↗

arXiv2020

The End-of-End-to-End: A Video Understanding Pentathlon Challenge (2020)

We present a new video understanding pentathlon challenge, an open competition held in conjunction with the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020. The objective of the challenge was to explore and evaluate new methods for text-to-video retrieval-the task of searching for content within a corpus of videos using natural language queries. This report summarizes the results of the first edition of the challenge together with the findings of the participants.

cs.CV↗

arXiv2020

Multi-modal Transformer for Video Retrieval

The task of retrieving video content relevant to natural language queries plays a critical role in effectively handling internet-scale datasets. Most of the existing methods for this caption-to-video retrieval problem do not fully exploit cross-modal cues present in video. Furthermore, they aggregate per-frame visual features with limited or no temporal information. In this paper, we present a multi-modal transformer to jointly encode the different modalities in video, which allows each of them to attend to the others. The transformer architecture is also leveraged to encode and model the temporal information. On the natural language side, we investigate the best practices to jointly optimize the language embedding together with the multi-modal transformer. This novel framework allows us to establish state-of-the-art results for video retrieval on three datasets. More details are available at http://thoth.inrialpes.fr/research/MMT.

cs.CV↗

arXiv2020

Uncertainty-Aware Weakly Supervised Action Detection from Untrimmed Videos

Despite the recent advances in video classification, progress in spatio-temporal action recognition has lagged behind. A major contributing factor has been the prohibitive cost of annotating videos frame-by-frame. In this paper, we present a spatio-temporal action recognition model that is trained with only video-level labels, which are significantly easier to annotate. Our method leverages per-frame person detectors which have been trained on large image datasets within a Multiple Instance Learning framework. We show how we can apply our method in cases where the standard Multiple Instance Learning assumption, that each bag contains at least one instance with the specified label, is invalid using a novel probabilistic variant of MIL where we estimate the uncertainty of each prediction. Furthermore, we report the first weakly-supervised results on the AVA dataset and state-of-the-art results among weakly-supervised methods on UCF101-24.

cs.CV↗

arXiv2020

Integrable multistate Landau-Zener models with parallel energy levels

We discuss solvable multistate Landau-Zener (MLZ) models whose Hamiltonians have commuting partner operators with $\sim 1/τ$-time-dependent parameters. Many already known solvable MLZ models belong precisely to this class. We derive the integrability conditions on the parameters of such commuting operators, and demonstrate how to use such conditions in order to derive new solvable cases. We show that MLZ models from this class must contain bands of parallel diabatic energy levels. The structure of the scattering matrix and other properties are found to be the same as in the previously discussed completely solvable MLZ Hamiltonians.

math-ph↗

arXiv2020

Beam-Domain Secret Key Generation for Multi-User Massive MIMO Networks

Physical-layer key generation (PKG) in multi-user massive MIMO networks faces great challenges due to the large length of pilots and the high dimension of channel matrix. To tackle these problems, we propose a novel massive MIMO key generation scheme with pilot reuse based on the beam domain channel model and derive close-form expression of secret key rate. Specifically, we present two algorithms, i.e., beam-domain based channel probing (BCP) algorithm and interference neutralization based multi-user beam allocation (IMBA) algorithm for the purpose of channel dimension reduction and multi-user pilot reuse, respectively. Numerical results verify that the proposed PKG scheme can achieve the secret key rate that approximates the perfect case, and significantly reduce the dimension of the channel estimation and pilot overhead.

cs.IT↗

arXiv2020

Analytical expressions of variable specific yield for layered soils in shallow water table environments

This paper presents analytical expressions of variable specific yield for layered soils in shallow water table environments, with introducing two distinct concepts of point specific yield (Syp) and interval average specific yield (Syi). The Syp and Syi refer to the specific yield for the water table fluctuation approaching zero infinitely and that for an interval fluctuation of water table, respectively. On the basis of specific yield definition and van Genuchten model of soil water retention, the analytical and semi-analytical expressions were respectively proposed for Syp and Syi towards layered soils. The analytical expressions are evaluated and verified by experimental data and comparison with the previous expressions. Analyses indicate our expressions for Syp and Syi could effectively reflect the changes and nonlinear properties affected by soil hydraulic properties and soil layering under shallow water table conditions. The previously confused understanding of Syp and Syi are also distinguished. The practicality and applicability for the specific yield expressions are comprehensively analyzed for the potential applications related to the subsurface water modeling and management issues.

physics.geo-ph↗

arXiv2020

VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation

Behavior prediction in dynamic, multi-agent systems is an important problem in the context of self-driving cars, due to the complex representations and interactions of road components, including moving agents (e.g. pedestrians and vehicles) and road context information (e.g. lanes, traffic lights). This paper introduces VectorNet, a hierarchical graph neural network that first exploits the spatial locality of individual road components represented by vectors and then models the high-order interactions among all components. In contrast to most recent approaches, which render trajectories of moving agents and road context information as bird-eye images and encode them with convolutional neural networks (ConvNets), our approach operates on a vector representation. By operating on the vectorized high definition (HD) maps and agent trajectories, we avoid lossy rendering and computationally intensive ConvNet encoding steps. To further boost VectorNet's capability in learning context features, we propose a novel auxiliary task to recover the randomly masked out map entities and agent trajectories based on their context. We evaluate VectorNet on our in-house behavior prediction benchmark and the recently released Argoverse forecasting dataset. Our method achieves on par or better performance than the competitive rendering approach on both benchmarks while saving over 70% of the model parameters with an order of magnitude reduction in FLOPs. It also outperforms the state of the art on the Argoverse dataset.

cs.CV↗

arXiv2020

Multitime Landau-Zener model: classification of solvable Hamiltonians

We discuss a class of models that generalize the two-state Landau-Zener (LZ) Hamiltonian to both the multistate and multitime evolution. It is already known that the corresponding quantum mechanical evolution can be understood in great detail. Here, we present an approach to classify such solvable models, namely, to identify all their independent families for a given number $N$ of interacting states and prove the absence of such families for some types of interactions. We also discuss how, within a solvable family, one can classify the scattering matrices, i.e., the system's dynamics. Due to the possibility of such a detailed classification, the multitime Landau-Zener (MTLZ) model defines a useful special function of theoretical physics.

math-ph↗

arXiv2020

Speech2Action: Cross-modal Supervision for Action Recognition

Is it possible to guess human action from dialogue alone? In this work we investigate the link between spoken words and actions in movies. We note that movie screenplays describe actions, as well as contain the speech of characters and hence can be used to learn this correlation with no additional supervision. We train a BERT-based Speech2Action classifier on over a thousand movie screenplays, to predict action labels from transcribed speech segments. We then apply this model to the speech segments of a large unlabelled movie corpus (188M speech segments from 288K movies). Using the predictions of this model, we obtain weak action labels for over 800K video clips. By training on these video clips, we demonstrate superior action recognition performance on standard action recognition benchmarks, without using a single manually labelled action example.

cs.CV↗

arXiv2020

Unsupervised Learning of Object Structure and Dynamics from Videos

Extracting and predicting object structure and dynamics from videos without supervision is a major challenge in machine learning. To address this challenge, we adopt a keypoint-based image representation and learn a stochastic dynamics model of the keypoints. Future frames are reconstructed from the keypoints and a reference frame. By modeling dynamics in the keypoint coordinate space, we achieve stable learning and avoid compounding of errors in pixel space. Our method improves upon unstructured representations both for pixel-level video prediction and for downstream tasks requiring object-level understanding of motion dynamics. We evaluate our model on diverse datasets: a multi-agent sports dataset, the Human3.6M dataset, and datasets based on continuous control tasks from the DeepMind Control Suite. The spatially structured representation outperforms unstructured representations on a range of motion-related tasks such as object tracking, action recognition and reward prediction.

cs.CV↗

arXiv2019

Learning Video Representations using Contrastive Bidirectional Transformer

This paper proposes a self-supervised learning approach for video features that results in significantly improved performance on downstream tasks (such as video classification, captioning and segmentation) compared to existing methods. Our method extends the BERT model for text sequences to the case of sequences of real-valued feature vectors, by replacing the softmax loss with noise contrastive estimation (NCE). We also show how to learn representations from sequences of visual features and sequences of words derived from ASR (automatic speech recognition), and show that such cross-modal training (when possible) helps even more.

cs.LG↗

arXiv2019

VideoBERT: A Joint Model for Video and Language Representation Learning

Self-supervised learning has become increasingly important to leverage the abundance of unlabeled data available on platforms like YouTube. Whereas most existing approaches learn low-level representations, we propose a joint visual-linguistic model to learn high-level features without any explicit supervision. In particular, inspired by its recent success in language modeling, we build upon the BERT model to learn bidirectional joint distributions over sequences of visual and linguistic tokens, derived from vector quantization of video data and off-the-shelf speech recognition outputs, respectively. We use VideoBERT in numerous tasks, including action classification and video captioning. We show that it can be applied directly to open-vocabulary classification, and confirm that large amounts of training data and cross-modal information are critical to performance. Furthermore, we outperform the state-of-the-art on video captioning, and quantitative results verify that the model learns high-level semantic features.

cs.CV↗

arXiv2019

Probing Boson Stars with Extreme Mass Ratio Inspirals

We propose to use gravitational waves from extreme mass ratio inspirals (EMRI), composed of a boson star and a supermassive black hole in the center of galaxies, as a new method to search for boson stars. Gravitational waves from EMRI have the advantage of being long-lasting within the frequency band of future space-based interferometer gravitational wave detectors and can accumulate large signal-to-noise ratio (SNR) for very sub-solar mass boson stars. Compared to gravitational waves from boson star binaries, which fall within the LIGO band, we find that much larger ranges of the mass and compactness of boson stars, as well as the underlying particle physics parameter space, can be probed by EMRI. We take tidal disruption of the boson stars into account and distinguish those which dissolve before the inner-most-stable-circular-orbit (ISCO) and those which dissolve after it. Due to the large number of cycles recorded, EMRIs can lead to a very precise mass determination of the boson star and distinguish it from standard astrophysical compact objects in the event of a signal. Tidal effects in inspiralling binary systems, as well as possible correlated electromagnetic signals, can also serve as potential discriminants.

hep-ph↗

arXiv2019

Relational Action Forecasting

This paper focuses on multi-person action forecasting in videos. More precisely, given a history of H previous frames, the goal is to detect actors and to predict their future actions for the next T frames. Our approach jointly models temporal and spatial interactions among different actors by constructing a recurrent graph, using actor proposals obtained with Faster R-CNN as nodes. Our method learns to select a subset of discriminative relations without requiring explicit supervision, thus enabling us to tackle challenging visual data. We refer to our model as Discriminative Relational Recurrent Network (DRRN). Evaluation of action prediction on AVA demonstrates the effectiveness of our proposed method compared to simpler baselines. Furthermore, we significantly improve performance on the task of early action classification on J-HMDB, from the previous SOTA of 48% to 60%.

cs.CV↗

arXiv2019

Affordance Learning In Direct Perception for Autonomous Driving

Recent development in autonomous driving involves high-level computer vision and detailed road scene understanding. Today, most autonomous vehicles are using mediated perception approach for path planning and control, which highly rely on high-definition 3D maps and real time sensors. Recent research efforts aim to substitute the massive HD maps with coarse road attributes. In this paper, we follow the direct perception based method to train a deep neural network for affordance learning in autonomous driving. Our goal in this work is to develop the affordance learning model based on freely available Google Street View panoramas and Open Street Map road vector attributes. Driving scene understanding can be achieved by learning affordances from the images captured by car-mounted cameras. Such scene understanding by learning affordances may be useful for corroborating base maps such as HD maps so that the required data storage space is minimized and available for processing in real time. We compare capability in road attribute identification between human volunteers and our model by experimental evaluation. Our results indicate that this method could act as a cheaper way for training data collection in autonomous driving. The cross validation results also indicate the effectiveness of our model.

cs.CV↗

arXiv2019

Boson Star from Repulsive Light Scalars and Gravitational Waves

We study properties of boson stars consisting of ultra-light scalar dark matter with repulsive self-interactions. We investigate the origin of the maximum mass of spherically symmetric stable stars which emerges only when solving the full equations of motion in curved space-time, but not when solving the approximated Schrödinger-Newton equations. When the repulsion is weak, the backreaction of the curvature on the scalars acts as an additional source of attraction and can overcome the repulsion, resulting in a maximum star mass and compactness. We also point out that the potential in a UV completed particle physics model of light scalar dark matter is generally more complicated than the widely used $ϕ^4$ interaction. Additional interactions beyond $ϕ^4$ in the potential can dramatically change the properties of boson stars as well as modify the prospect of LIGO gravitational wave detection for binary mergers of boson stars.

hep-ph↗