arXiv subjects

Andrew Brown

Publications and source records attributed to Andrew Brown.

At least 37 records · Page 2Linked to original sources

arXiv2023

Causal Video Summarizer for Video Exploration

Recently, video summarization has been proposed as a method to help video exploration. However, traditional video summarization models only generate a fixed video summary which is usually independent of user-specific needs and hence limits the effectiveness of video exploration. Multi-modal video summarization is one of the approaches utilized to address this issue. Multi-modal video summarization has a video input and a text-based query input. Hence, effective modeling of the interaction between a video input and text-based query is essential to multi-modal video summarization. In this work, a new causality-based method named Causal Video Summarizer (CVS) is proposed to effectively capture the interactive information between the video and query to tackle the task of multi-modal video summarization. The proposed method consists of a probabilistic encoder and a probabilistic decoder. Based on the evaluation of the existing multi-modal video summarization dataset, experimental results show that the proposed approach is effective with the increase of +5.4% in accuracy and +4.92% increase of F 1- score, compared with the state-of-the-art method.

cs.CV↗

arXiv2023

VoxSRC 2022: The Fourth VoxCeleb Speaker Recognition Challenge

This paper summarises the findings from the VoxCeleb Speaker Recognition Challenge 2022 (VoxSRC-22), which was held in conjunction with INTERSPEECH 2022. The goal of this challenge was to evaluate how well state-of-the-art speaker recognition systems can diarise and recognise speakers from speech obtained "in the wild". The challenge consisted of: (i) the provision of publicly available speaker recognition and diarisation data from YouTube videos together with ground truth annotation and standardised evaluation software; and (ii) a public challenge and hybrid workshop held at INTERSPEECH 2022. We describe the four tracks of our challenge along with the baselines, methods, and results. We conclude with a discussion on the new domain-transfer focus of VoxSRC-22, and on the progression of the challenge from the previous three editions.

cs.SD↗

arXiv2023

An Event-Driven Approach To Genotype Imputation On A Custom RISC-V FPGA Cluster

This paper proposes an event-driven solution to genotype imputation, a technique used to statistically infer missing genetic markers in DNA. The work implements the widely accepted Li and Stephens model, primary contributor to the computational complexity of modern x86 solutions, in an attempt to determine whether further investigation of the application is warranted in the event-driven domain. The model is implemented using graph-based Hidden Markov Modeling and executed as a customized forward/backward dynamic programming algorithm. The solution uses an event-driven paradigm to map the algorithm to thousands of concurrent cores, where events are small messages that carry both control and data within the algorithm. The design of a single processing element is discussed. This is then extended across multiple FPGAs and executed on a custom RISC-V NoC FPGA cluster called POETS. Results demonstrate how the algorithm scales over increasing hardware resources and a 48 FPGA run demonstrates a 270X reduction in wall-clock processing time when compared to a single-threaded x86 solution. Optimisation of the algorithm via linear interpolation is then introduced and tested, with results demonstrating a wall-clock reduction time of approx. 5 orders of magnitude when compared to a similarly optimised x86 solution.

cs.DC↗

arXiv2022

VoxSRC 2021: The Third VoxCeleb Speaker Recognition Challenge

The third instalment of the VoxCeleb Speaker Recognition Challenge was held in conjunction with Interspeech 2021. The aim of this challenge was to assess how well current speaker recognition technology is able to diarise and recognise speakers in unconstrained or `in the wild' data. The challenge consisted of: (i) the provision of publicly available speaker recognition and diarisation data from YouTube videos together with ground truth annotation and standardised evaluation software; and (ii) a virtual public challenge and workshop held at Interspeech 2021. This paper outlines the challenge, and describes the baselines, methods and results. We conclude with a discussion on the new multi-lingual focus of VoxSRC 2021, and on the progression of the challenge since the previous two editions.

cs.SD↗

arXiv2022

In search of strong embedding extractors for speaker diarisation

Speaker embedding extractors (EEs), which map input audio to a speaker discriminant latent space, are of paramount importance in speaker diarisation. However, there are several challenges when adopting EEs for diarisation, from which we tackle two key problems. First, the evaluation is not straightforward because the features required for better performance differ between speaker verification and diarisation. We show that better performance on widely adopted speaker verification evaluation protocols does not lead to better diarisation performance. Second, embedding extractors have not seen utterances in which multiple speakers exist. These inputs are inevitably present in speaker diarisation because of overlapped speech and speaker changes; they degrade the performance. To mitigate the first problem, we generate speaker verification evaluation protocols that mimic the diarisation scenario better. We propose two data augmentation techniques to alleviate the second problem, making embedding extractors aware of overlapped speech or speaker change input. One technique generates overlapped speech segments, and the other generates segments where two speakers utter sequentially. Extensive experimental results using three state-of-the-art speaker embedding extractors demonstrate that both proposed approaches are effective.

cs.SD↗

arXiv2022

Collisionless relaxation of a Lynden-Bell plasma

Plasmas whose Coulomb-collision rates are very small may relax on shorter time scales to non-Maxwellian quasi-equilibria, which, nevertheless, have a universal form, with dependence on initial conditions retained only via an infinite set of Casimir invariants enforcing phase-volume conservation. These are distributions derived by Lynden-Bell (1967) via a statistical-mechanical entropy-maximisation procedure, assuming perfect mixing of phase-space elements. To show that these equilibria are reached dynamically, one must derive an effective `collisionless collision integral' for which they are fixed points -- unique and inevitable provided the integral has an appropriate H-theorem. We describe how such collision integrals are derived and what assumptions are required for them to have a closed form, how to prove the H-theorems for them, and why, for a system carrying sufficiently large electric-fluctuation energy, collisionless relaxation should be fast. It is suggested that collisionless dynamics may favour maximising entropy locally in phase space before converging to global maximum-entropy states. Relaxation due to interspecies interaction is examined, leading, inter alia, to spontaneous transient generation of electron currents. The formalism also allows efficient recovery of `true' collision integrals for both classical and quantum plasmas.

physics.plasm-ph↗

arXiv2022

End-to-End Visual Editing with a Generatively Pre-Trained Artist

We consider the targeted image editing problem: blending a region in a source image with a driver image that specifies the desired change. Differently from prior works, we solve this problem by learning a conditional probability distribution of the edits, end-to-end. Training such a model requires addressing a fundamental technical challenge: the lack of example edits for training. To this end, we propose a self-supervised approach that simulates edits by augmenting off-the-shelf images in a target domain. The benefits are remarkable: implemented as a state-of-the-art auto-regressive transformer, our approach is simple, sidesteps difficulties with previous methods based on GAN-like priors, obtains significantly better edits, and is efficient. Furthermore, we show that different blending effects can be learned by an intuitive control of the augmentation process, with no other changes required to the model architecture. We demonstrate the superiority of this approach across several datasets in extensive quantitative and qualitative experiments, including human studies, significantly outperforming prior work.

cs.CV↗

arXiv2021

Face, Body, Voice: Video Person-Clustering with Multiple Modalities

The objective of this work is person-clustering in videos -- grouping characters according to their identity. Previous methods focus on the narrower task of face-clustering, and for the most part ignore other cues such as the person's voice, their overall appearance (hair, clothes, posture), and the editing structure of the videos. Similarly, most current datasets evaluate only the task of face-clustering, rather than person-clustering. This limits their applicability to downstream applications such as story understanding which require person-level, rather than only face-level, reasoning. In this paper we make contributions to address both these deficiencies: first, we introduce a Multi-Modal High-Precision Clustering algorithm for person-clustering in videos using cues from several modalities (face, body, and voice). Second, we introduce a Video Person-Clustering dataset, for evaluating multi-modal person-clustering. It contains body-tracks for each annotated character, face-tracks when visible, and voice-tracks when speaking, with their associated features. The dataset is by far the largest of its kind, and covers films and TV-shows representing a wide range of demographics. Finally, we show the effectiveness of using multiple modalities for person-clustering, explore the use of this new broad task for story understanding through character co-occurrences, and achieve a new state of the art on all available datasets for face and person-clustering.

cs.CV↗

arXiv2021

Thinking Through and Writing About Research Ethics Beyond "Broader Impact"

In March 2021, we held the first instalment of the tutorial on thinking through and writing about research ethics beyond 'Broader Impact' in conjunction with the ACM Conference on Fairness, Accountability, and Transparency (FAccT '21). The goal of this tutorial was to offer a conceptual and practical starting point for engineers and social scientists interested in thinking more expansively, holistically, and critically about research ethics. This report provides an outline of the tutorial, and contains our 'lifecourse checklist'. This was presented as part of the tutorial, and provides a practical starting point for researchers when thinking about research ethics before a project's start. We provide this to the research community, with the hope that researchers use it when considering the ethics of their research.

cs.CY↗

arXiv2021

Playing a Part: Speaker Verification at the Movies

The goal of this work is to investigate the performance of popular speaker recognition models on speech segments from movies, where often actors intentionally disguise their voice to play a character. We make the following three contributions: (i) We collect a novel, challenging speaker recognition dataset called VoxMovies, with speech for 856 identities from almost 4000 movie clips. VoxMovies contains utterances with varying emotion, accents and background noise, and therefore comprises an entirely different domain to the interview-style, emotionally calm utterances in current speaker recognition datasets such as VoxCeleb; (ii) We provide a number of domain adaptation evaluation sets, and benchmark the performance of state-of-the-art speaker recognition models on these evaluation pairs. We demonstrate that both speaker verification and identification performance drops steeply on this new data, showing the challenge in transferring models across domains; and finally (iii) We show that simple domain adaptation paradigms improve performance, but there is still large room for improvement.

cs.SD↗

arXiv2021

Automated Video Labelling: Identifying Faces by Corroborative Evidence

We present a method for automatically labelling all faces in video archives, such as TV broadcasts, by combining multiple evidence sources and multiple modalities (visual and audio). We target the problem of ever-growing online video archives, where an effective, scalable indexing solution cannot require a user to provide manual annotation or supervision. To this end, we make three key contributions: (1) We provide a novel, simple, method for determining if a person is famous or not using image-search engines. In turn this enables a face-identity model to be built reliably and robustly, and used for high precision automatic labelling; (2) We show that even for less-famous people, image-search engines can then be used for corroborative evidence to accurately label faces that are named in the scene or the speech; (3) Finally, we quantitatively demonstrate the benefits of our approach on different video domains and test settings, such as TV shows and news broadcasts. Our method works across three disparate datasets without any explicit domain adaptation, and sets new state-of-the-art results on all the public benchmarks.

cs.CV↗

arXiv2020

VoxSRC 2020: The Second VoxCeleb Speaker Recognition Challenge

We held the second installment of the VoxCeleb Speaker Recognition Challenge in conjunction with Interspeech 2020. The goal of this challenge was to assess how well current speaker recognition technology is able to diarise and recognize speakers in unconstrained or `in the wild' data. It consisted of: (i) a publicly available speaker recognition and diarisation dataset from YouTube videos together with ground truth annotation and standardised evaluation software; and (ii) a virtual public challenge and workshop held at Interspeech 2020. This paper outlines the challenge, and describes the baselines, methods used, and results. We conclude with a discussion of the progress over the first installment of the challenge.

cs.SD↗

arXiv2020

Condensed Movies: Story Based Retrieval with Contextual Embeddings

Our objective in this work is long range understanding of the narrative structure of movies. Instead of considering the entire movie, we propose to learn from the `key scenes' of the movie, providing a condensed look at the full storyline. To this end, we make the following three contributions: (i) We create the Condensed Movies Dataset (CMD) consisting of the key scenes from over 3K movies: each key scene is accompanied by a high level semantic description of the scene, character face-tracks, and metadata about the movie. The dataset is scalable, obtained automatically from YouTube, and is freely available for anybody to download and use. It is also an order of magnitude larger than existing movie datasets in the number of movies; (ii) We provide a deep network baseline for text-to-video retrieval on our dataset, combining character, speech and visual cues into a single video embedding; and finally (iii) We demonstrate how the addition of context from other video clips improves retrieval performance.

cs.CV↗

arXiv2020

Smooth-AP: Smoothing the Path Towards Large-Scale Image Retrieval

Optimising a ranking-based metric, such as Average Precision (AP), is notoriously challenging due to the fact that it is non-differentiable, and hence cannot be optimised directly using gradient-descent methods. To this end, we introduce an objective that optimises instead a smoothed approximation of AP, coined Smooth-AP. Smooth-AP is a plug-and-play objective function that allows for end-to-end training of deep networks with a simple and elegant implementation. We also present an analysis for why directly optimising the ranking based metric of AP offers benefits over other deep metric learning losses. We apply Smooth-AP to standard retrieval benchmarks: Stanford Online products and VehicleID, and also evaluate on larger-scale datasets: INaturalist for fine-grained category retrieval, and VGGFace2 and IJB-C for face retrieval. In all cases, we improve the performance over the state-of-the-art, especially for larger-scale datasets, thus demonstrating the effectiveness and scalability of Smooth-AP to real-world scenarios.

cs.CV↗

arXiv2019

4-Connected Shift Residual Networks

The shift operation was recently introduced as an alternative to spatial convolutions. The operation moves subsets of activations horizontally and/or vertically. Spatial convolutions are then replaced with shift operations followed by point-wise convolutions, significantly reducing computational costs. In this work, we investigate how shifts should best be applied to high accuracy CNNs. We apply shifts of two different neighbourhood groups to ResNet on ImageNet: the originally introduced 8-connected (8C) neighbourhood shift and the less well studied 4-connected (4C) neighbourhood shift. We find that when replacing ResNet's spatial convolutions with shifts, both shift neighbourhoods give equal ImageNet accuracy, showing the sufficiency of small neighbourhoods for large images. Interestingly, when incorporating shifts to all point-wise convolutions in residual networks, 4-connected shifts outperform 8-connected shifts. Such a 4-connected shift setup gives the same accuracy as full residual networks while reducing the number of parameters and FLOPs by over 40%. We then highlight that without spatial convolutions, ResNet's downsampling/upsampling bottleneck channel structure is no longer needed. We show a new, 4C shift-based residual network, much shorter than the original ResNet yet with a higher accuracy for the same computational cost. This network is the highest accuracy shift-based network yet shown, demonstrating the potential of shifting in deep neural networks.

cs.CV↗

arXiv2019

RMT: R-matrix with time-dependence. Solving the semi-relativistic, time-dependent Schrodinger equation for general, multi-electron atoms and molecules in intense, ultrashort, arbitrarily polarized laser pulses

RMT is a program which solves the time-dependent Schrodinger equation for general, multielectron atoms, ions and molecules interacting with laser light. As such it can be used to model ionization (single-photon, multi-photon and strong-field), recollision (high-harmonic generation, strong-field rescattering), and more generally absorption or scattering processes with a full account of the multielectron correlation effects in a time-dependent manner. Calculations can be performed for targets interacting with ultrashort, intense laser pulses of long-wavelength and arbitrary polarization. Calculations for atoms can optionally include the Breit-Pauli correction terms for the description of relativistic (in particular, spin-orbit) effects.

physics.comp-ph↗

arXiv2016

A Rn-220 source for the calibration of low-background experiments

We characterize two 40 kBq sources of electrodeposited Th-228 for use in low-background experiments. The sources efficiently emanate Rn-220, a noble gas that can diffuse in a detector volume. Rn-220 and its daughter isotopes produce alpha, beta, and gamma-radiation, which may used to calibrate a variety of detector responses and features, before decaying completely in only a few days. We perform various tests to place limits on the release of other long-lived isotopes. In particular, we find an emanation of <0.008 atoms/min/kBq (90% CL) for Th-228 and 1.53 atoms/min/kBq for Ra-224. The sources lend themselves in particular to the calibration of detectors employing liquid noble elements such as argon and xenon. With the source mounted in a noble gas system, we demonstrate that filters are highly efficient in reducing the activity of these longer-lived isotopes further. We thus confirm the suitability of these sources even for use in next-generation experiments, such as XENON1T/XENONnT, LZ, and nEXO.

physics.ins-det↗

arXiv2015

Time-resolved four-wave-mixing spectroscopy for inner-valence transitions

Non-collinear four-wave mixing (FWM) techniques at near-infrared (NIR), visible, and ultraviolet frequencies have been widely used to map vibrational and electronic couplings, typically in complex molecules. However, correlations between spatially localized inner-valence transitions among different sites of a molecule in the extreme ultraviolet (XUV) spectral range have not been observed yet. As an experimental step towards this goal we perform time-resolved FWM spectroscopy with femtosecond NIR and attosecond XUV pulses. The first two pulses (XUV-NIR) coincide in time and act as coherent excitation fields, while the third pulse (NIR) acts as a probe. As a first application we show how coupling dynamics between odd- and even-parity inner-valence excited states of neon can be revealed using a two-dimensional spectral representation. Experimentally obtained results are found to be in good agreement with ab initio time-dependent R-matrix calculations providing the full description of multi-electron interactions, as well as few-level model simulations. Future applications of this method also include site-specific probing of electronic processes in molecules.

physics.atom-ph↗