arXiv subjects

Suman Saha

Publications and source records attributed to Suman Saha.

At least 73 records · Page 4Linked to original sources

arXiv2022

Exploiting Instance-based Mixed Sampling via Auxiliary Source Domain Supervision for Domain-adaptive Action Detection

We propose a novel domain adaptive action detection approach and a new adaptation protocol that leverages the recent advancements in image-level unsupervised domain adaptation (UDA) techniques and handle vagaries of instance-level video data. Self-training combined with cross-domain mixed sampling has shown remarkable performance gain in semantic segmentation in UDA (unsupervised domain adaptation) context. Motivated by this fact, we propose an approach for human action detection in videos that transfers knowledge from the source domain (annotated dataset) to the target domain (unannotated dataset) using mixed sampling and pseudo-label-based selftraining. The existing UDA techniques follow a ClassMix algorithm for semantic segmentation. However, simply adopting ClassMix for action detection does not work, mainly because these are two entirely different problems, i.e., pixel-label classification vs. instance-label detection. To tackle this, we propose a novel action instance mixed sampling technique that combines information across domains based on action instances instead of action classes. Moreover, we propose a new UDA training protocol that addresses the long-tail sample distribution and domain shift problem by using supervision from an auxiliary source domain (ASD). For the ASD, we propose a new action detection dataset with dense frame-level annotations. We name our proposed framework as domain-adaptive action instance mixing (DA-AIM). We demonstrate that DA-AIM consistently outperforms prior works on challenging domain adaptation benchmarks. The source code is available at https://github.com/wwwfan628/DA-AIM.

cs.CV↗

arXiv2022

Transit Light-curves for Exomoons: Analytical Formalism

The photometric transit method has been the most effective method to detect and characterize exoplanets as several ground-based as well as space-based survey missions have discovered thousands of exoplanets using this method. With the advent of the upcoming next generation large telescopes, the detection of exomoons in a few of these exoplanetary systems is very plausible. In this paper, we present a comprehensive analytical formalism in order to model the transit light curves for such moon hosting exoplanets. In order to achieve analytical formalism, we have considered circular orbit of the exomoon around the host planet, which is indeed the case for tidally locked moons. The formalism uses the radius and orbital properties of both the host planet and its moon as model parameters. The co-alignment or non-coalignment of the orbits of the planet and the moon is parameterized using two angular parameters and thus can be used to model all the possible orbital alignments for a star-planet-moon system. This formalism also provides unique and direct solutions to every possible star-planet-moon three circular body alignments. Using the formula derived, a few representative light curves are also presented.

astro-ph.EP↗

arXiv2022

ROAD: The ROad event Awareness Dataset for Autonomous Driving

Humans drive in a holistic fashion which entails, in particular, understanding dynamic road events and their evolution. Injecting these capabilities in autonomous vehicles can thus take situational awareness and decision making closer to human-level performance. To this purpose, we introduce the ROad event Awareness Dataset (ROAD) for Autonomous Driving, to our knowledge the first of its kind. ROAD is designed to test an autonomous vehicle's ability to detect road events, defined as triplets composed by an active agent, the action(s) it performs and the corresponding scene locations. ROAD comprises videos originally from the Oxford RobotCar Dataset annotated with bounding boxes showing the location in the image plane of each road event. We benchmark various detection tasks, proposing as a baseline a new incremental algorithm for online road event awareness termed 3D-RetinaNet. We also report the performance on the ROAD tasks of Slowfast and YOLOv5 detectors, as well as that of the winners of the ICCV2021 ROAD challenge, which highlight the challenges faced by situation awareness in autonomous driving. ROAD is designed to allow scholars to investigate exciting tasks such as complex (road) activity detection, future event anticipation and continual learning. The dataset is available at https://github.com/gurkirt/road-dataset; the baseline can be found at https://github.com/gurkirt/3D-RetinaNet.

cs.CV↗

arXiv2022

Resilience in multiplex networks by addition of cross-repulsive links

A multiplex network of identical dynamical units becomes resilient against parameter perturbation by adding selective linear diffusive cross-coupling links. A parameter drift at any instant in one or multiple network nodes can destroy synchrony, causing failure and even collapse in the network performance. We introduced [PRE 95, 062204(2017)] a recovery strategy by selective addition of cross-coupling links to save synchrony in the network from the edge of failure due to parameter mismatch (small or large) in any nodes. This concept is extended to 2-layered multiplex networks when the emergent synchrony becomes resilient against a small or large parameter drifting. In addition, the stability of the synchronous state is enhanced from local stability to global stability of synchrony. By the addition of cross-coupling, the network revives complete synchrony in all the nodes except the perturbed nodes, which emerges into a type of generalized synchrony with all the unperturbed nodes. The generalized synchrony is manifested simply by a linear amplitude response in the state variable(s) of the perturbed node(s) by a scaling factor proportional to the mismatch. A set of systematic rules has been derived from the linear flow matrix of the dynamical system representing the nodes dynamics that helps find the connectivity matrix of the cross-coupling links. Lyapunov function stability condition is used to determine the cross-coupling link strength that, in turn, establishes global stability of synchrony of the multiplex network. We verify the efficacy of our proposed coupling scheme with analytical results and numerical simulations of two examples of multiplex networks. In the first example, we use non-local connectivity in each layer, with nodal dynamics of the FitzHugh-Nagumo neuron model.

nlin.AO↗

arXiv2021

Smallest chimeras under repulsive interactions

We present an exemplary system of three identical oscillators in a ring interacting repulsively to show up chimera patterns. The dynamics of individual oscillators is governed by the superconducting Josephson junction. Surprisingly, the repulsive interactions establish a symmetry of compelete synchrony in the ring, which is broken with increasing interactions when the junctions pass through serials of asynchronous states (periodic and chaotic), but finally emerge into chimera states. The chimera pattern appears in chaotic rotational motion of the three junctions when two junctions evolve coherently while the third junction is incoherent. For larger repulsive coupling, the junctions evolves into another chimera pattern in a periodic state when two junctions remain coherent in rotational motion and one transits to incoherent librational motion. This chimera pattern is sensitive to initial conditions, in the sense, that the chimera state flips to another pattern when two junctions switch to coherent librational motion and the third junction remains in rotational motion, but incoherent. The chimera patterns are detected by using partial and global error functions of the junctions while the librational and rotational motions are identified by a libration index. All the collective states, complete synchrony, desynchronization and two chimera patterns, are delineated in a parameter plane of the ring of junctions, where the boundaries of complete synchrony are demarcated by using the master stability function.

nlin.AO↗

arXiv2021

Critical Analysis of TESS Transit Photometric Data: Improved Physical Properties for Five Exoplanets

We present improved physical parameters for four hot Jupiters: KELT-7 b, HAT-P-14 b, WASP-29 b, WASP-95 b, and a hot Neptune: WASP-156 b, by performing critical and rigorous analysis of the time-series observations from the Transiting Exoplanet Survey Satellite (TESS). Being a space-based telescope, the transit photometric data obtained by TESS are free from any noise component due to the interference of Earth's atmosphere. In our analysis of the observed data, we have used critical noise reduction techniques, e.g., the wavelet denoising and Gaussian process regression, in order to effectively reduce the noise components that arise from other sources, such as various instrumental effects and the stellar activity and pulsations. The better quality of photometric data from TESS, combined with our state-of-the-art noise reduction and analysis technique, has resulted into more accurate and precise values of the physical properties for the target exoplanets than that reported in earlier works.

astro-ph.EP↗

arXiv2021

Learning to Relate Depth and Semantics for Unsupervised Domain Adaptation

We present an approach for encoding visual task relationships to improve model performance in an Unsupervised Domain Adaptation (UDA) setting. Semantic segmentation and monocular depth estimation are shown to be complementary tasks; in a multi-task learning setting, a proper encoding of their relationships can further improve performance on both tasks. Motivated by this observation, we propose a novel Cross-Task Relation Layer (CTRL), which encodes task dependencies between the semantic and depth predictions. To capture the cross-task relationships, we propose a neural network architecture that contains task-specific and cross-task refinement heads. Furthermore, we propose an Iterative Self-Learning (ISL) training scheme, which exploits semantic pseudo-labels to provide extra supervision on the target domain. We experimentally observe improvements in both tasks' performance because the complementary information present in these tasks is better captured. Specifically, we show that: (1) our approach improves performance on all tasks when they are complementary and mutually dependent; (2) the CTRL helps to improve both semantic segmentation and depth estimation tasks performance in the challenging UDA setting; (3) the proposed ISL training scheme further improves the semantic segmentation performance. The implementation is available at https://github.com/susaha/ctrl-uda.

cs.CV↗

arXiv2021

Multi-band transit follow up observations of five hot-Jupiters with critical noise treatments: Improved physical properties

The most challenging limitation in transit photometry arises from the noises in the photometric signal. In particular, the ground-based telescopes are heavily affected by the noise due to perturbation in the Earth's atmosphere. Use of telescopes with large apertures can improve the photometric signal-to-noise ratio (S/N) to a great extent. However, detecting a transit signal out of a noisy light curve of the host star and precisely estimating the transit parameters call for various noise reduction techniques. Here, we present multi-band transit photometric follow-up observations of five hot-Jupiters e.g., HAT-P-30 b, HAT-P-54 b, WASP-43 b, TrES-3 b and XO-2 N b, using the 2m Himalayan Chandra Telescope (HCT) at the Indian Astronomical Observatory, Hanle and the 1.3m J. C. Bhattacharya Telescope (JCBT) at the Vainu Bappu Observatory, Kavalur. Our critical noise treatment approach includes techniques such as Wavelet Denoising and Gaussian Process regression, which effectively reduce both time-correlated and time-uncorrelated noise components from our transit light curves. In addition to these techniques, use of our state-of-the-art model algorithms have allowed us to estimate the physical properties of the target exoplanets with a better accuracy and precision compared to the previous studies.

astro-ph.EP↗

arXiv2021

Unsupervised Compound Domain Adaptation for Face Anti-Spoofing

We address the problem of face anti-spoofing which aims to make the face verification systems robust in the real world settings. The context of detecting live vs. spoofed face images may differ significantly in the target domain, when compared to that of labeled source domain where the model is trained. Such difference may be caused due to new and unknown spoof types, illumination conditions, scene backgrounds, among many others. These varieties of differences make the target a compound domain, thus calling for the problem of the unsupervised compound domain adaptation. We demonstrate the effectiveness of the compound domain assumption for the task of face anti-spoofing, for the first time in this work. To this end, we propose a memory augmentation method for adapting the source model to the target domain in a domain aware manner. The adaptation process is further improved by using the curriculum learning and the domain agnostic source network training approaches. The proposed method successfully adapts to the compound target domain consisting multiple new spoof types. Our experiments on multiple benchmark datasets demonstrate the superiority of the proposed method over the state-of-the-art.

cs.CV↗

arXiv2021

Three Ways to Improve Semantic Segmentation with Self-Supervised Depth Estimation

Training deep networks for semantic segmentation requires large amounts of labeled training data, which presents a major challenge in practice, as labeling segmentation masks is a highly labor-intensive process. To address this issue, we present a framework for semi-supervised semantic segmentation, which is enhanced by self-supervised monocular depth estimation from unlabeled image sequences. In particular, we propose three key contributions: (1) We transfer knowledge from features learned during self-supervised depth estimation to semantic segmentation, (2) we implement a strong data augmentation by blending images and labels using the geometry of the scene, and (3) we utilize the depth feature diversity as well as the level of difficulty of learning depth in a student-teacher framework to select the most useful samples to be annotated for semantic segmentation. We validate the proposed model on the Cityscapes dataset, where all three modules demonstrate significant performance gains, and we achieve state-of-the-art results for semi-supervised semantic segmentation. The implementation is available at https://github.com/lhoyer/improving_segmentation_with_selfsupervised_depth.

cs.CV↗

arXiv2020

Infection spreading and recovery in a square lattice

We investigate spreading and recovery of disease in a square lattice, and in particular, emphasize the role of the initial distribution of infected patches in the network, on the progression of an endemic and initiation of a recovery process, if any, due to migration of both the susceptible and infected hosts. The disease starts in the lattice with three possible initial distribution patterns of infected and infection-free sites, infected core patches (ICP), infected peripheral patches (IPP) and randomly distributed infected patches (RDIP). Our results show that infection spreads monotonically in the lattice with increasing migration without showing any sign of recovery in the ICP case. In the IPP case, it follows a similar monotonic progression with increasing migration, however, a self-organized healing process starts for higher migration, leading the lattice to full recovery at a critical rate of migration. Encouragingly, for the initial RDIP arrangement, chances of recovery are much higher with a lower rate of critical migration. An eigenvalue based semi-analytical study is made to determine the critical migration rate for realizing a stable infection-free lattice. The initial fraction of infected patches and the force of infection play significant roles in the self-organized recovery. They follow an exponential law, for the RDIP case, that governs the recovery process. For the frustrating case of ICP arrangement, we propose a random rewiring of links in the lattice allowing long-distance migratory paths that effectively initiate a recovery process. Global prevalence of infection thereby declines and progressively improves with the rewiring probability that follows a power law with the critical migration and leads to the birth of emergent infection-free networks.

q-bio.PE↗

arXiv2020

Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interference

Multi-task networks are commonly utilized to alleviate the need for a large number of highly specialized single-task networks. However, two common challenges in developing multi-task models are often overlooked in literature. First, enabling the model to be inherently incremental, continuously incorporating information from new tasks without forgetting the previously learned ones (incremental learning). Second, eliminating adverse interactions amongst tasks, which has been shown to significantly degrade the single-task performance in a multi-task setup (task interference). In this paper, we show that both can be achieved simply by reparameterizing the convolutions of standard neural network architectures into a non-trainable shared part (filter bank) and task-specific parts (modulators), where each modulator has a fraction of the filter bank parameters. Thus, our reparameterization enables the model to learn new tasks without adversely affecting the performance of existing ones. The results of our ablation study attest the efficacy of the proposed reparameterization. Moreover, our method achieves state-of-the-art on two challenging multi-task learning benchmarks, PASCAL-Context and NYUD, and also demonstrates superior incremental learning capability as compared to its close competitors.

cs.CV↗

arXiv2020

Domain Agnostic Feature Learning for Image and Video Based Face Anti-spoofing

Nowadays, the increasingly growing number of mobile and computing devices has led to a demand for safer user authentication systems. Face anti-spoofing is a measure towards this direction for bio-metric user authentication, and in particular face recognition, that tries to prevent spoof attacks. The state-of-the-art anti-spoofing techniques leverage the ability of deep neural networks to learn discriminative features, based on cues from the training set images or video samples, in an effort to detect spoof attacks. However, due to the particular nature of the problem, i.e. large variability due to factors like different backgrounds, lighting conditions, camera resolutions, spoof materials, etc., these techniques typically fail to generalize to new samples. In this paper, we explicitly tackle this problem and propose a class-conditional domain discriminator module, that, coupled with a gradient reversal layer, tries to generate live and spoof features that are discriminative, but at the same time robust against the aforementioned variability factors. Extensive experimental analysis shows the effectiveness of the proposed method over existing image- and video-based anti-spoofing techniques, both in terms of numerical improvement as well as when visualizing the learned features.

cs.CV↗

arXiv2020

Two-Stream AMTnet for Action Detection

In this paper, we propose Two-Stream AMTnet, which leverages recent advances in video-based action representation[1] and incremental action tube generation[2]. Majority of the present action detectors follow a frame-based representation, a late-fusion followed by an offline action tube building steps. These are sub-optimal as: frame-based features barely encode the temporal relations; late-fusion restricts the network to learn robust spatiotemporal features; and finally, an offline action tube generation is not suitable for many real-world problems such as autonomous driving, human-robot interaction to name a few. The key contributions of this work are: (1) combining AMTnet's 3D proposal architecture with an online action tube generation technique which allows the model to learn stronger temporal features needed for accurate action detection and facilitates running inference online; (2) an efficient fusion technique allowing the deep network to learn strong spatiotemporal action representations. This is achieved by augmenting the previous Action Micro-Tube (AMTnet) action detection framework in three distinct ways: by adding a parallel motion stIn this paper, we propose a new deep neural network architecture for online action detection, termed ream to the original appearance one in AMTnet; (2) in opposition to state-of-the-art action detectors which train appearance and motion streams separately, and use a test time late fusion scheme to fuse RGB and flow cues, by jointly training both streams in an end-to-end fashion and merging RGB and optical flow features at training time; (3) by introducing an online action tube generation algorithm which works at video-level, and in real-time (when exploiting only appearance features). Two-Stream AMTnet exhibits superior action detection performance over state-of-the-art approaches on the standard action detection benchmarks.

cs.CV↗

arXiv2018

Predicting Action Tubes

In this work, we present a method to predict an entire `action tube' (a set of temporally linked bounding boxes) in a trimmed video just by observing a smaller subset of it. Predicting where an action is going to take place in the near future is essential to many computer vision based applications such as autonomous driving or surgical robotics. Importantly, it has to be done in real-time and in an online fashion. We propose a Tube Prediction network (TPnet) which jointly predicts the past, present and future bounding boxes along with their action classification scores. At test time TPnet is used in a (temporal) sliding window setting, and its predictions are put into a tube estimation framework to construct/predict the video long action tubes not only for the observed part of the video but also for the unobserved part. Additionally, the proposed action tube predictor helps in completing action tubes for unobserved segments of the video. We quantitatively demonstrate the latter ability, and the fact that TPnet improves state-of-the-art detection performance, on one of the standard action detection benchmarks - J-HMDB-21 dataset.

cs.CV↗

arXiv2018

TraMNet - Transition Matrix Network for Efficient Action Tube Proposals

Current state-of-the-art methods solve spatiotemporal action localisation by extending 2D anchors to 3D-cuboid proposals on stacks of frames, to generate sets of temporally connected bounding boxes called \textit{action micro-tubes}. However, they fail to consider that the underlying anchor proposal hypotheses should also move (transition) from frame to frame, as the actor or the camera does. Assuming we evaluate $n$ 2D anchors in each frame, then the number of possible transitions from each 2D anchor to the next, for a sequence of $f$ consecutive frames, is in the order of $O(n^f)$, expensive even for small values of $f$. To avoid this problem, we introduce a Transition-Matrix-based Network (TraMNet) which relies on computing transition probabilities between anchor proposals while maximising their overlap with ground truth bounding boxes across frames, and enforcing sparsity via a transition threshold. As the resulting transition matrix is sparse and stochastic, this reduces the proposal hypothesis search space from $O(n^f)$ to the cardinality of the thresholded matrix. At training time, transitions are specific to cell locations of the feature maps, so that a sparse (efficient) transition matrix is used to train the network. At test time, a denser transition matrix can be obtained either by decreasing the threshold or by adding to it all the relative transitions originating from any cell location, allowing the network to handle transitions in the test data that might not have been present in the training data, and making detection translation-invariant. Finally, we show that our network can handle sparse annotations such as those available in the DALY dataset. We report extensive experiments on the DALY, UCF101-24 and Transformed-UCF101-24 datasets to support our claims.

eess.IV↗

arXiv2018

Action Detection from a Robot-Car Perspective

We present the new Road Event and Activity Detection (READ) dataset, designed and created from an autonomous vehicle perspective to take action detection challenges to autonomous driving. READ will give scholars in computer vision, smart cars and machine learning at large the opportunity to conduct research into exciting new problems such as understanding complex (road) activities, discerning the behaviour of sentient agents, and predicting both the label and the location of future actions and events, with the final goal of supporting autonomous decision making.

cs.CV↗

arXiv2018

Incremental Tube Construction for Human Action Detection

Current state-of-the-art action detection systems are tailored for offline batch-processing applications. However, for online applications like human-robot interaction, current systems fall short, either because they only detect one action per video, or because they assume that the entire video is available ahead of time. In this work, we introduce a real-time and online joint-labelling and association algorithm for action detection that can incrementally construct space-time action tubes on the most challenging action videos in which different action categories occur concurrently. In contrast to previous methods, we solve the detection-window association and action labelling problems jointly in a single pass. We demonstrate superior online association accuracy and speed (2.2ms per frame) as compared to the current state-of-the-art offline systems. We further demonstrate that the entire action detection pipeline can easily be made to work effectively in real-time using our action tube construction algorithm.

cs.CV↗