arXiv subjects

Xin Gu

Publications and source records attributed to Xin Gu.

48 records · Page 3Linked to original sources

arXiv2024

Context-Guided Spatio-Temporal Video Grounding

Spatio-temporal video grounding (or STVG) task aims at locating a spatio-temporal tube for a specific instance given a text query. Despite advancements, current methods easily suffer the distractors or heavy object appearance variations in videos due to insufficient object information from the text, leading to degradation. Addressing this, we propose a novel framework, context-guided STVG (CG-STVG), which mines discriminative instance context for object in videos and applies it as a supplementary guidance for target localization. The key of CG-STVG lies in two specially designed modules, including instance context generation (ICG), which focuses on discovering visual context information (in both appearance and motion) of the instance, and instance context refinement (ICR), which aims to improve the instance context from ICG by eliminating irrelevant or even harmful information from the context. During grounding, ICG, together with ICR, are deployed at each decoding stage of a Transformer architecture for instance context learning. Particularly, instance context learned from one decoding stage is fed to the next stage, and leveraged as a guidance containing rich and discriminative object feature to enhance the target-awareness in decoding feature, which conversely benefits generating better new instance context for improving localization finally. Compared to existing methods, CG-STVG enjoys object information in text query and guidance from mined instance visual context for more accurate target localization. In our experiments on three benchmarks, including HCSTVG-v1/-v2 and VidSTG, CG-STVG sets new state-of-the-arts in m_tIoU and m_vIoU on all of them, showing its efficacy. The code will be released at https://github.com/HengLan/CGSTVG.

cs.CV↗

arXiv2023

Local Compressed Video Stream Learning for Generic Event Boundary Detection

Generic event boundary detection aims to localize the generic, taxonomy-free event boundaries that segment videos into chunks. Existing methods typically require video frames to be decoded before feeding into the network, which contains significant spatio-temporal redundancy and demands considerable computational power and storage space. To remedy these issues, we propose a novel compressed video representation learning method for event boundary detection that is fully end-to-end leveraging rich information in the compressed domain, i.e., RGB, motion vectors, residuals, and the internal group of pictures (GOP) structure, without fully decoding the video. Specifically, we use lightweight ConvNets to extract features of the P-frames in the GOPs and spatial-channel attention module (SCAM) is designed to refine the feature representations of the P-frames based on the compressed information with bidirectional information flow. To learn a suitable representation for boundary detection, we construct the local frames bag for each candidate frame and use the long short-term memory (LSTM) module to capture temporal relationships. We then compute frame differences with group similarities in the temporal domain. This module is only applied within a local window, which is critical for event boundary detection. Finally a simple classifier is used to determine the event boundaries of video sequences based on the learned feature representation. To remedy the ambiguities of annotations and speed up the training process, we use the Gaussian kernel to preprocess the ground-truth event boundaries. Extensive experiments conducted on the Kinetics-GEBD and TAPOS datasets demonstrate that the proposed method achieves considerable improvements compared to previous end-to-end approach while running at the same speed. The code is available at https://github.com/GX77/LCVSL.

cs.CV↗

arXiv2023

Lane Change Intention Recognition and Vehicle Status Prediction for Autonomous Vehicles

Accurately detecting and predicting lane change (LC)processes of human-driven vehicles can help autonomous vehicles better understand their surrounding environment, recognize potential safety hazards, and improve traffic safety. This paper focuses on LC processes, first developing a temporal convolutional network with an attention mechanism (TCN-ATM) model to recognize LC intention. Considering the intrinsic relationship among output variables, the Multi-task Learning (MTL)framework is employed to simultaneously predict multiple LC vehicle status indicators. Furthermore, a unified modeling framework for LC intention recognition and driving status prediction (LC-IR-SP) is developed. The results indicate that the classification accuracy of LC intention was improved from 96.14% to 98.20% when incorporating the attention mechanism into the TCN model. For LC vehicle status prediction issues, three multi-tasking learning models are constructed based on MTL framework. The results indicate that the MTL-LSTM model outperforms the MTL-TCN and MTL-TCN-ATM models. Compared to the corresponding single-task model, the MTL-LSTM model demonstrates an average decrease of 26.04% in MAE and 25.19% in RMSE.

cs.LG↗

arXiv2023

Investigating the spatial heterogeneity of factors influencing speeding-related crash severities using correlated random parameter order models with heterogeneity-in-means

Speeding has been acknowledged as a critical determinant in increasing the risk of crashes and their resulting injury severities. This paper demonstrates that severe speeding-related crashes within the state of Pennsylvania have a spatial clustering trend, where four crash datasets are extracted from four hotspot districts. Two log-likelihood ratio (LR) tests were conducted to determine whether speeding-related crashes classified by hotspot districts should be modeled separately. The results suggest that separate modeling is necessary. To capture the unobserved heterogeneity, four correlated random parameter order models with heterogeneity in means are employed to explore the factors contributing to crash severity involving at least one vehicle speeding. Overall, the findings exhibit that some indicators are observed to be spatial instability, including hit pedestrian crashes, head-on crashes, speed limits, work zones, light conditions (dark), rural areas, older drivers, running stop signs, and running red lights. Moreover, drunk driving, exceeding the speed limit, and being unbelted present relative spatial stability in four district models. This paper provides insights into preventing speeding-related crashes and potentially facilitating the development of corresponding crash injury mitigation policies.

stat.AP↗

arXiv2023

Exploring differences in injury severity between occupant groups involved in fatal rear-end crashes: A correlated random parameter logit model with mean heterogeneity

Rear-end crashes are one of the most common crash types. Passenger cars involved in rear-end crashes frequently produce severe outcomes. However, no study investigated the differences in the injury severity of occupant groups when cars are involved as following and leading vehicles in rear-end crashes. Therefore, the focus of this investigation is to compare the key factors affecting the injury severity between the front- and rear-car occupant groups in rear-end crashes. First, data is extracted from the Fatality Analysis Reporting System (FARS) for two types of rear-end crashes from 2017 to 2019, including passenger cars as rear-end and rear-ended vehicles. Significant injury severity difference between front- and rear-car occupant groups is found by conducting likelihood ratio test. Moreover, the front- and rear-car occupant groups are modelled by the correlated random parameter logit model with heterogeneity in means (CRPLHM) and the random parameter logit model with heterogeneity in means (RPLHM), respectively. From the modeling, the significant factors are occupant positions, driver age, overturn, vehicle type, etc. For instance, the driving and front-right positions significantly increase the probability of severe injury when struck by another vehicle. Large truck-strike-car tends to cause severe outcomes compared to car-strike-large truck. This study provides an insightful knowledge of mechanism of occupant injury severity in rear-end crashes, and propose some effective countermeasures to mitigate the crash severity, such as implementing stricter seat belt laws, improving the coverage of the streetlights, strengthening car driver's emergency response ability.

stat.AP↗

arXiv2023

Text with Knowledge Graph Augmented Transformer for Video Captioning

Video captioning aims to describe the content of videos using natural language. Although significant progress has been made, there is still much room to improve the performance for real-world applications, mainly due to the long-tail words challenge. In this paper, we propose a text with knowledge graph augmented transformer (TextKG) for video captioning. Notably, TextKG is a two-stream transformer, formed by the external stream and internal stream. The external stream is designed to absorb additional knowledge, which models the interactions between the additional knowledge, e.g., pre-built knowledge graph, and the built-in information of videos, e.g., the salient object regions, speech transcripts, and video captions, to mitigate the long-tail words challenge. Meanwhile, the internal stream is designed to exploit the multi-modality information in videos (e.g., the appearance of video frames, speech transcripts, and video captions) to ensure the quality of caption results. In addition, the cross attention mechanism is also used in between the two streams for sharing information. In this way, the two streams can help each other for more accurate results. Extensive experiments conducted on four challenging video captioning datasets, i.e., YouCookII, ActivityNet Captions, MSRVTT, and MSVD, demonstrate that the proposed method performs favorably against the state-of-the-art methods. Specifically, the proposed TextKG method outperforms the best published results by improving 18.7% absolute CIDEr scores on the YouCookII dataset.

cs.CV↗

arXiv2023

Dual-Stream Transformer for Generic Event Boundary Captioning

This paper describes our champion solution for the CVPR2022 Generic Event Boundary Captioning (GEBC) competition. GEBC requires the captioning model to have a comprehension of instantaneous status changes around the given video boundary, which makes it much more challenging than conventional video captioning task. In this paper, a Dual-Stream Transformer with improvements on both video content encoding and captions generation is proposed: (1) We utilize three pre-trained models to extract the video features from different granularities. Moreover, we exploit the types of boundary as hints to help the model generate captions. (2) We particularly design an model, termed as Dual-Stream Transformer, to learn discriminative representations for boundary captioning. (3) Towards generating content-relevant and human-like captions, we improve the description quality by designing a word-level ensemble strategy. The promising results on the GEBC test split demonstrate the efficacy of our proposed model.

cs.CV↗

arXiv2023

Duality family of KdV equation

It is revealed that there exist duality families of the KdV type equation. A duality family consists of an infinite number of generalized KdV (GKdV) equations. A duality transformation relates the GKdV equations in a duality family. Once a family member is solved, the duality transformation presents the solutions of all other family members. We show some dualities as examples, such as the soliton solution-soliton solution duality and the periodic solution-soliton solution duality.

math-ph↗

arXiv2022

Video-Music Retrieval:A Dual-Path Cross-Modal Network

We propose a method to recommend background music for videos. Current work rarely considers the emotional information of music, which is essential for video music retrieval. To achieve this, we design two paths to process content information and emotional information between modal. Based on characteristics of video and music, we design various feature extraction schemes and common representation spaces. More importantly, we propose a way to combine content information with emotional information. Additionally, we make improvements to the classical metric loss to be more suited to this task. Experiments show that this dual path video music retrieval network can effectively merge information. Compare with existing methods, the retrieval task evaluation index: increasing Recall@1 by 3.94 and Recall@25 by 16.36.

cs.MM↗

arXiv2022

Polaritonic coherent perfect absorption based on strong critical coupling between quasi-bound state in the continuum and exciton

Enhancement of light-matter interactions is of great importance for many nanophotonic devices, and one way to achieve it is to feed energy perfectly to the strongly coupled system. Here, we propose gap-perturbed dimerized gratings based on bulk WS2 for flexible control of the strong coupling between quasi-bound state in the continuum (quasi-BIC) and exciton. The simulation results show that when a gap perturbation is introduced into the system resulting in the Brillouin zone folding, BIC transforms into quasi-BIC whose quality factor (Q-factor) is related to the value of gap perturbation. The strong coupling results in the anti-crossover behavior of the absorption spectra, and thus a Rabi splitting energy of 0.235 eV is obtained. Temporal coupled-mode theory is employed to analyze the strong coupling system, and the absorption of the system could be enhanced, by modulating the damping rate of quasi-BIC to make the system satisfy the strong critical coupling condition. Furthermore, polaritonic coherent perfect absorption is achieved by using the two-port source excitation. This work could provide ideas for enhancing light-matter interactions and a promising prospect for polariton-based light-emitting or lasing devices.

physics.optics↗

arXiv2022

Explainable and Optimally Configured Artificial Neural Networks for Attack Detection in Smart Homes

In recent years cybersecurity has become a major concern in adaptation of smart applications. Specially, in smart homes where a large number of IoT devices are used having a secure and trusted mechanisms can provide peace of mind for users. Accurate detection of cyber attacks is crucial, however precise identification of the type of attacks plays a huge role if devising the countermeasure for protecting the system. Artificial Neural Networks (ANN) have provided promising results for detecting any security attacks for smart applications. However, due to complex nature of the model used for this technique it is not easy for normal users to trust ANN based security solutions. Also, selection of right hyperparameters for ANN architecture plays a crucial role in the accurate detection of security attacks, especially when it come to identifying the subcategories of attacks. In this paper, we propose a model that considers both the issues of explainability of ANN model and the hyperparameter selection for this approach to be easily trusted and adapted by users of smart home applications. Also, our approach considers a subset of the dataset for optimal selection of hyperparamters to reduce the overhead of the process of ANN architecture design. Distinctively this paper focuses on configuration, performance and evaluation of ANN architecture for identification of five categorical attacks and nine subcategorical attacks. Using a very recent IoT dataset our approach showed high performance for intrusion detection with 99.9%, 99.7%, and 97.7% accuracy for Binary, Category, and Subcategory level classification of attacks.

cs.CR↗

arXiv2019

BFpack: Flexible Bayes Factor Testing of Scientific Theories in R

There has been a tremendous methodological development of Bayes factors for hypothesis testing in the social and behavioral sciences, and related fields. This development is due to the flexibility of the Bayes factor for testing multiple hypotheses simultaneously, the ability to test complex hypotheses involving equality as well as order constraints on the parameters of interest, and the interpretability of the outcome as the weight of evidence provided by the data in support of competing scientific theories. The available software tools for Bayesian hypothesis testing are still limited however. In this paper we present a new R-package called BFpack that contains functions for Bayes factor hypothesis testing for the many common testing problems. The software includes novel tools (i) for Bayesian exploratory testing (null vs positive vs negative effects), (ii) for Bayesian confirmatory testing (competing hypotheses with equality and/or order constraints), (iii) for common statistical analyses, such as linear regression, generalized linear models, (multivariate) analysis of (co)variance, correlation analysis, and random intercept models, (iv) using default priors, and (v) while allowing data to contain missing observations that are missing at random.

stat.CO↗