Searcharxiv⌕ Search

arXiv subjects

Shruti Agarwal

Publications and source records attributed to Shruti Agarwal.

21 records · Page 2Linked to original sources

Watch Those Words: Video Falsification Detection Using Word-Conditioned Facial Motion

In today's era of digital misinformation, we are increasingly faced with new threats posed by video falsification techniques. Such falsifications range from cheapfakes (e.g., lookalikes or audio dubbing) to deepfakes (e.g., sophisticated AI media synthesis methods), which are becoming perceptually indistinguishable from real videos. To tackle this challenge, we propose a multi-modal semantic forensic approach to discover clues that go beyond detecting discrepancies in visual quality, thereby handling both simpler cheapfakes and visually persuasive deepfakes. In this work, our goal is to verify that the purported person seen in the video is indeed themselves by detecting anomalous facial movements corresponding to the spoken words. We leverage the idea of attribution to learn person-specific biometric patterns that distinguish a given speaker from others. We use interpretable Action Units (AUs) to capture a person's face and head movement as opposed to deep CNN features, and we are the first to use word-conditioned facial motion analysis. We further demonstrate our method's effectiveness on a range of fakes not seen in training including those without video manipulation, that were not addressed in prior work.

cs.CV↗

First Order Topological Phase Transitions and Disorder Induced Majorana Modes in Interacting Fermion Chains

Using a combination of the mean-field Bogoliubov deGennes (BdG) approach and the Density Matrix Renormalization Group (DMRG) method, we discover first order topological transitions between topological superconducting and trivial insulating phases in a sawtooth lattice of inter-site attractive fermions. Topological characterization of different phases is achieved in terms of winding numbers, Majorana edge modes and entanglement spectra. By studying the effect of disorder on the first order topological phase transitions, we establish the disorder-induced topological phase coexistence as a mechanism for generating a finite density of Majorana particles.

cond-mat.str-el↗

Detecting Deep-Fake Videos from Appearance and Behavior

Synthetically-generated audios and videos -- so-called deep fakes -- continue to capture the imagination of the computer-graphics and computer-vision communities. At the same time, the democratization of access to technology that can create sophisticated manipulated video of anybody saying anything continues to be of concern because of its power to disrupt democratic elections, commit small to large-scale fraud, fuel dis-information campaigns, and create non-consensual pornography. We describe a biometric-based forensic technique for detecting face-swap deep fakes. This technique combines a static biometric based on facial recognition with a temporal, behavioral biometric based on facial expressions and head movements, where the behavioral embedding is learned using a CNN with a metric-learning objective function. We show the efficacy of this approach across several large-scale video datasets, as well as in-the-wild deep fakes.

cs.CV↗