arXiv subjects

François Grondin

Publications and source records attributed to François Grondin.

At least 19 recordsLinked to original sources

arXiv2026

Lend me an Ear: Speech Enhancement Using a Robotic Arm with a Microphone Array

Speech enhancement performance degrades significantly in noisy environments, limiting the deployment of speech-controlled technologies in industrial settings, such as manufacturing plants. Existing speech enhancement solutions primarily rely on advanced digital signal processing techniques, deep learning methods, or complex software optimization approaches. This paper introduces a novel speech enhancement robotic platform that can reconfigure the geometry of a microphone array and adapt to changing acoustic conditions. A sixteen-microphone array is mounted on a robotic arm manipulator with seven degrees of freedom. The microphones are divided into four groups of four, including one group positioned near the end-effector. The system reconfigures the array by adjusting the manipulator joint angles to place the end-effector microphones closer to the target speaker, thereby improving the reference signal quality. This proposed system is a multimodal sensing, reconfigurable audio capture device that integrates sound source localization techniques, computer vision, inverse kinematics, minimum variance distortionless response beamformer, and time-frequency masking using a deep neural network. Experimental results suggest that this approach outperforms other traditional recording configurations, achieving a higher average scale-invariant signal-to-distortion ratio and lower average word error rate across multiple input signal-to-noise ratio conditions.

cs.RO↗

arXiv2026

Multi-Speaker DOA Estimation in Binaural Hearing Aids using Deep Learning and Speaker Count Fusion

For extracting a target speaker voice, direction-of-arrival (DOA) estimation is crucial for binaural hearing aids operating in noisy, multi-speaker environments. Among the solutions developed for this task, a deep learning convolutional recurrent neural network (CRNN) model leveraging spectral phase differences and magnitude ratios between microphone signals is a popular option. In this paper, we explore adding source-count information for multi-sources DOA estimation. The use of dual-task training with joint multi-sources DOA estimation and source counting is first considered. We then consider using the source count as an auxiliary feature in a standalone DOA estimation system, where the number of active sources (0, 1, or 2+) is integrated into the CRNN architecture through early, mid, and late fusion strategies. Experiments using real binaural recordings are performed. Results show that the dual-task training does not improve DOA estimation performance, although it benefits source-count prediction. However, a ground-truth (oracle) source count used as an auxiliary feature significantly enhances standalone DOA estimation performance, with late fusion yielding up to 14% higher average F1-scores over the baseline CRNN. This highlights the potential of using source-count estimation for robust DOA estimation in binaural hearing aids.

eess.AS↗

arXiv2026

Acoustic Drone Package Delivery Detection

In recent years, the illicit use of unmanned aerial vehicles (UAVs) for deliveries in restricted area such as prisons became a significant security challenge. While numerous studies have focused on UAV detection or localization, little attention has been given to delivery events identification. This study presents the first acoustic package delivery detection algorithm using a ground-based microphone array. The proposed method estimates both the drone's propeller speed and the delivery event using solely acoustic features. A deep neural network detects the presence of a drone and estimates the propeller's rotation speed or blade passing frequency (BPF) from a mel spectrogram. The algorithm analyzes the BPFs to identify probable delivery moments based on sudden changes before and after a specific time. Results demonstrate a mean absolute error of the blade passing frequency estimator of 16 Hz when the drone is less than 150 meters away from the microphone array. The drone presence detection estimator has a accuracy of 97%. The delivery detection algorithm correctly identifies 96% of events with a false positive rate of 8%. This study shows that deliveries can be identified using acoustic signals up to a range of 100 meters.

cs.RO↗

arXiv2025

Sugar Shack 4.0: Implementation of a Cyber-Physical System for Logistic and Sanitary Automation in a Maple Syrup Boiling Center

This paper presents the design and deployment of a process-aware cyber-physical system that automates plant-level logistics, traceability, and sanitation in a centralized maple-syrup boiling center. The system replaces ad-hoc, manual operations with event-driven orchestration on a local server, employing reusable device abstractions and a centralized interlock with priority-based arbitration for shared piping. It implements deterministic routines for delivery, reverse osmosis integration, evaporator feed, and permeate management. The system is sensor rich: inline measurements of flow, temperature, and sugar concentration (degrees Brix) drive routing decisions and trigger systematic post-transfer rinses (cleaning-in-place), ensuring consistent hygiene and complete, immediate traceability up to the evaporator inlet. During the 2025 production season, the system queued 431 operations without incident; executed 908 \enquote{topstock} and \enquote{downstock} balancing cycles; increased usable permeate reserves from 22,712 to approximately 41,640 L through dynamic storage assignment; eliminated mid-season contamination incidents previously observed under manual practice; and reduced administrative effort for billing and reporting from more than 30 hours to roughly 1 hour through automatic documentation. These results demonstrate a practical path to modular, plant-scale automation beyond traditional architectures, and lay the groundwork for packaging reusable elements for similar plants or adjacent industries. This work is part of a larger project involving the first scientifically-documented integration of Industry 4.0 technologies in a maple syrup boiling center.

eess.SY↗

arXiv2025

Sugar Shack 4.0: Practical Demonstration of an IIoT-Based Event-Driven Automation System

This paper presents a practical alternative to programmable-logic-controller-centric automation by implementing an event-driven architecture built with industrial Internet of Things tools. A layered design on a local edge server (i) abstracts actuators, (ii) enforces mutual exclusion of shared physical resources through an interlock with priority queueing, (iii) composes deterministic singular operations, and (iv) orchestrates complete workflows as state machines in Node-RED, with communication over MQTT. The device layer uses low-cost ESP32-based gateways to interface sensors and actuators, while all automation logic is offloaded to the server side. As part of a larger project involving the first scientifically-documented integration of Industry 4.0 technologies in a maple syrup boiling center, this work demonstrates the deployment of the proposed system as a case-study. Evaluation over an entire production season shows median message time of flight around one tenth of a second, command issuance-to-motion latencies of about two to three seconds, and command completion near six seconds dominated by actuator mechanics; operation runtimes span tens of seconds to minutes. These results indicate that network and orchestration overheads are negligible relative to process dynamics, enabling modular, distributed control without compromising determinism or fault isolation. The approach reduces material and integration effort, supports portable containerized deployment, and naturally enables an edge/cloud split in which persistence and analytics are offloaded while automation remains at the edge.

eess.SY↗

arXiv2025

Sound Source Localization for Human-Robot Interaction in Outdoor Environments

This paper presents a sound source localization strategy that relies on a microphone array embedded in an unmanned ground vehicle and an asynchronous close-talking microphone near the operator. A signal coarse alignment strategy is combined with a time-domain acoustic echo cancellation algorithm to estimate a time-frequency ideal ratio mask to isolate the target speech from interferences and environmental noise. This allows selective sound source localization, and provides the robot with the direction of arrival of sound from the active operator, which enables rich interaction in noisy scenarios. Results demonstrate an average angle error of 4 degrees and an accuracy within 5 degrees of 95\% at a signal-to-noise ratio of 1dB, which is significantly superior to the state-of-the-art localization methods.

cs.RO↗

arXiv2024

Unsupervised Improved MVDR Beamforming for Sound Enhancement

Neural networks have recently become the dominant approach to sound separation. Their good performance relies on large datasets of isolated recordings. For speech and music, isolated single channel data are readily available; however the same does not hold in the multi-channel case, and with most other sound classes. Multi-channel methods have the potential to outperform single channel approaches as they can exploit both spatial and spectral features, but the lack of training data remains a challenge. We propose unsupervised improved minimum variation distortionless response (UIMVDR), which enables multi-channel separation to leverage in-the-wild single-channel data through unsupervised training and beamforming. Results show that UIMVDR generalizes well and improves separation performance compared to supervised models, particularly in cases with limited supervised data. By using data available online, it also reduces the effort required to gather data for multi-channel approaches.

cs.SD↗

arXiv2024

PlumberNet: Fixing interference leakage after GEV beamforming

Spatial filters can exploit deep-learning-based speech enhancement models to increase their reliability in scenarios with multiple speech sources scenarios. To further improve speech quality, it is common to perform postfiltering on the estimated target speech obtained with spatial filtering. In this work, Generalized Eigenvalue (GEV) beamforming is employed to provide the leakage estimation, along with the estimation of the target speech, to be later used for postfiltering. This improves the enhancement performance over a postfilter that uses the target speech and a reference microphone signal. This work also demonstrates that the spatial covariance matrices (SCMs) can be accurately estimated from the direction of arrival (DoA) of the target and a discriminative selection amongst the pairwise estimated time-frequency masks.

eess.AS↗

arXiv2024

Efficient Face Detection with Audio-Based Region Proposals for Human-Robot Interactions

Efficient face detection is critical to provide natural human-robot interactions. However, computer vision tends to involve a large computational load due to the amount of data (i.e. pixels) that needs to be processed in a short amount of time. This is undesirable on robotics platforms where multiple processes need to run in parallel and where the processing power is limited by portability constraints. Existing solutions often involve reducing image quality which can negatively impact processing. The literature also reports methods to generate regions of interest in images from pixel data. Although it is a promising idea, these methods often involve heavy vision algorithms. In this paper, we evaluate how audio can be used to generate regions of interest in optical images to reduce the number of pixels to process with computer vision. Thereby, we propose a unique attention mechanism to localize a speech source and evaluate its impact on an existing face detection algorithm. Our results show that the attention mechanism reduces the computational load and offers an interesting trade-off between speed and accuracy. The proposed pipeline is flexible and can be easily adapted to other applications such as robot surveillance, video conferences or smart glasses.

eess.AS↗

arXiv2024

Resource-Efficient Separation Transformer

Transformers have recently achieved state-of-the-art performance in speech separation. These models, however, are computationally demanding and require a lot of learnable parameters. This paper explores Transformer-based speech separation with a reduced computational cost. Our main contribution is the development of the Resource-Efficient Separation Transformer (RE-SepFormer), a self-attention-based architecture that reduces the computational burden in two ways. First, it uses non-overlapping blocks in the latent space. Second, it operates on compact latent summaries calculated from each chunk. The RE-SepFormer reaches a competitive performance on the popular WSJ0-2Mix and WHAM! datasets in both causal and non-causal settings. Remarkably, it scales significantly better than the previous Transformer-based architectures in terms of memory and inference time, making it more suitable for processing long mixtures.

eess.AS↗

arXiv2023

Fast Cross-Correlation for TDoA Estimation on Small Aperture Microphone Arrays

This paper introduces the Fast Cross-Correlation (FCC) method for Time Difference of Arrival (TDoA) Estimation for pairs of microphones on a small aperture microphone array. FCC relies on low-rank decomposition and exploits symmetry in even and odd bases to speed up computation while preserving TDoA accuracy. FCC reduces the number of flops by a factor of 4.5 and the execution speed by factors between 3.5 and 8.3 on embedded hardware, compared to the state-of-the-art Generalized Cross-Correlation (GCC) method that relies on the Fast Fourier Transform (FFT). This improvement can provide portable microphone arrays with extended battery life and allow real-time processing on low-cost hardware.

eess.SP↗

arXiv2023

Ego-noise reduction of a mobile robot using noise spatial covariance matrix learning and minimum variance distortionless response

The performance of speech and events recognition systems significantly improved recently thanks to deep learning methods. However, some of these tasks remain challenging when algorithms are deployed on robots due to the unseen mechanical noise and electrical interference generated by their actuators while training the neural networks. Ego-noise reduction as a preprocessing step therefore can help solve this issue when using pre-trained speech and event recognition algorithms on robots. In this paper, we propose a new method to reduce ego-noise using only a microphone array and less than two minute of noise recordings. Using Principal Component Analysis (PCA), the best covariance matrix candidate is selected from a dictionary created online during calibration and used with the Minimum Variance Distortionless Response (MVDR) beamformer. Results show that the proposed method runs in real-time, improves the signal-to-distortion ratio (SDR) by up to 10 dB, decreases the word error rate (WER) by 55\% in some cases and increases the Average Precision (AP) of event detection by up to 0.2.

eess.AS↗

arXiv2023

Real-time Audio Video Enhancement \\with a Microphone Array and Headphones

This paper presents a complete hardware and software pipeline for real-time speech enhancement in noisy and reverberant conditions. The device consists of a microphone array and a camera mounted on eyeglasses, connected to an embedded system that enhances speech and plays back the audio in headphones, with a latency of maximum 120 msec. The proposed approach relies on face detection, tracking and verification to enhance the speech of a target speaker using a beamformer and a postfiltering neural network. Results demonstrate the feasibility of the approach, and opens the door to the exploration and validation of a wide range of beamformer and speech enhancement methods for real-time speech enhancement.

eess.AS↗

arXiv2022

ODAS: Open embeddeD Audition System

Artificial audition aims at providing hearing capabilities to machines, computers and robots. Existing frameworks in robot audition offer interesting sound source localization, tracking and separation performance, although involve a significant amount of computations that limit their use on robots with embedded computing capabilities. This paper presents ODAS, the Open embeddeD Audition System framework, which includes strategies to reduce the computational load and perform robot audition tasks on low-cost embedded computing systems. It presents key features of ODAS, along with cases illustrating its uses in different robots and artificial audition applications.

eess.AS↗

arXiv2022

Machine Learning for Touch Localization on Ultrasonic Wave Touchscreen

Classification and regression employing a simple Deep Neural Network (DNN) are investigated to perform touch localization on a tactile surface using ultrasonic guided waves. A robotic finger first simulates the touch action and captures the data to train a model. The model is then validated with data from experiments conducted with human fingers. The localization root mean square errors (RMSE) in time and frequency domains are presented. The proposed method provides satisfactory localization results for most human-machine interactions, with a mean error of 0.47 cm and standard deviation of 0.18 cm and a computing time of 0.44 ms. The classification approach is also adapted to identify touches on an access control keypad layout, which leads to an accuracy of 97% with a computing time of 0.28 ms. These results demonstrate that DNN-based methods are a viable alternative to signal processing-based approaches for accurate and robust touch localization using ultrasonic guided waves.

eess.SP↗

arXiv2022

SMP-PHAT: Lightweight DoA Estimation by Merging Microphone Pairs

This paper introduces SMP-PHAT, which performs direction of arrival (DoA) of sound estimation with a microphone array by merging pairs of microphones that are parallel in space. This approach reduces the number of pairwise cross-correlation computations, and brings down the number of flops and memory lookups when searching for DoA. Experiments on low-cost hardware with commonly used microphone arrays show that the proposed method provides the same accuracy as the former SRP-PHAT approach, while reducing the computational load by 39% in some cases.

cs.SD↗

arXiv2022

SmartBelt: A Wearable Microphone Array for Sound Source Localization with Haptic Feedback

This paper introduces SmartBelt, a wearable microphone array on a belt that performs sound source localization and returns the direction of arrival with respect to the user waist. One of the haptic motors on the belt then vibrates in the corresponding direction to provide useful feedback to the user. We also introduce a simple calibration step to adapt the belt to different waist sizes. Experiments are performed to confirm the accuracy of this wearable sound source localization system, and results show a Mean Average Error (MAE) of 2.90 degrees, and a correct haptic motor selection with a rate of 92.3%. Results suggest the device can provide useful haptic feedback, and will be evaluated in a study with people having hearing impairments.

cs.RO↗

arXiv2022

Learning Filterbanks for End-to-End Acoustic Beamforming

Recent work on monaural source separation has shown that performance can be increased by using fully learned filterbanks with short windows. On the other hand it is widely known that, for conventional beamforming techniques, performance increases with long analysis windows. This applies also to most hybrid neural beamforming methods which rely on a deep neural network (DNN) to estimate the spatial covariance matrices. In this work we try to bridge the gap between these two worlds and explore fully end-to-end hybrid neural beamforming in which, instead of using the Short-Time-Fourier Transform, also the analysis and synthesis filterbanks are learnt jointly with the DNN. In detail, we explore two different types of learned filterbanks: fully learned and analytic. We perform a detailed analysis using the recent Clarity Challenge data and show that by using learnt filterbanks it is possible to surpass oracle-mask based beamforming for short windows.

eess.AS↗