arXiv subjects

Santosh Gondi

Publications and source records attributed to Santosh Gondi.

3 recordsLinked to original sources

arXiv2023

Project Aria: A New Tool for Egocentric Multi-Modal AI Research

Egocentric, multi-modal data as available on future augmented reality (AR) devices provides unique challenges and opportunities for machine perception. These future devices will need to be all-day wearable in a socially acceptable form-factor to support always available, context-aware and personalized AI applications. Our team at Meta Reality Labs Research built the Aria device, an egocentric, multi-modal data recording and streaming device with the goal to foster and accelerate research in this area. In this paper, we describe the Aria device hardware including its sensor configuration and the corresponding software tools that enable recording and processing of such data.

cs.HC↗

arXiv2022

Wav2Vec2.0 on the Edge: Performance Evaluation

Wav2Vec2.0 is a state-of-the-art model which learns speech representations through unlabeled speech data, aka, self supervised learning. The pretrained model is then fine tuned on small amounts of labeled data to use it for speech-to-text and machine translation tasks. Wav2Vec 2.0 is a transformative solution for low resource languages as it is mainly developed using unlabeled audio data. Getting large amounts of labeled data is resource intensive and especially challenging to do for low resource languages such as Swahilli, Tatar, etc. Furthermore, Wav2Vec2.0 word-error-rate(WER) matches or surpasses the very recent supervised learning algorithms while using 100x less labeled data. Given its importance and enormous potential in enabling speech based tasks on world's 7000 languages, it is key to evaluate the accuracy, latency and efficiency of this model on low resource and low power edge devices and investigate the feasibility of using it in such devices for private, secure and reliable speech based tasks. On-device speech tasks preclude sending audio data to the server hence inherently providing privacy, reduced latency and enhanced reliability. In this paper, Wav2Vec2.0 model's accuracy and latency has been evaluated on Raspberry Pi along with the KenLM language model for speech recognition tasks. How to tune certain parameters to achieve desired level of WER rate and latency while meeting the CPU, memory and energy budgets of the product has been discussed.

cs.SD↗

arXiv2015

Protecting Memory-Performance Critical Sections in Soft Real-Time Applications

Soft real-time applications such as multimedia applications often show bursty memory access patterns---regularly requiring a high memory bandwidth for a short duration of time. Such a period is often critical for timely data processing. Hence, we call it a memory-performance critical section. Unfortunately, in multicore architecture, non-real-time applications on different cores may also demand high memory bandwidth at the same time, which can substantially increase the time spent on the memory performance critical sections. In this paper, we present BWLOCK, user-level APIs and a memory bandwidth control mechanism that can protect such memory performance critical sections of soft real-time applications. BWLOCK provides simple lock like APIs to declare memory-performance critical sections. If an application enters a memory-performance critical section, the memory bandwidth control system then dynamically limit other cores' memory access rates to protect memory performance of the application until the critical section finishes. From case studies with real-world soft real-time applications, we found (1) such memory-performance critical sections do exist and are often easy to identify; and (2) applying BWLOCK for memory critical sections significantly improve performance of the soft real-time applications at a small or no cost in throughput of non real-time applications.

cs.OS↗