arXiv subjects

Anton Isopoussu

Publications and source records attributed to Anton Isopoussu.

7 recordsLinked to original sources

arXiv2026

If It's Not Buggy, Don't Fix It: On the Dynamics of Iterative Bug-fixing with LLMs

Large language models (LLMs) have become ubiquitous in software development, with LLM-based automated program repair tools increasingly used during code review. In this report, we explore the iterative blind use of LLMs as bug-fixers. Across multiple models and repair environments, we find that LLMs consistently claim to detect bugs in entirely bug-free programs while the rate of repair of buggy programs is less than that of the damage to correct programs. We also explore the long-term dynamics of this iterative process, and find that this frequently reaches a pseudo-bug-fixing cycle where the same changes are added and removed again ad infinitum. Lastly, via mechanistic probing, we unveil the existence of a steering vector which controls the editing propensity, suggesting that LLMs have an internal representation of ``buggy code", and that this representation is what is falsely activated to induce pseudo-bug fixing. These results provide insight towards the dynamics of fully autonomous bug-fixing systems, as well as stopping conditions under ambiguous goals.

cs.SE↗

arXiv2026

On the Existence and Behavior of Secondary Attention Sinks

Attention sinks are tokens, often the beginning-of-sequence (BOS) token, that receive disproportionately high attention despite limited semantic relevance. In this work, we identify a class of attention sinks, which we term secondary sinks, that differ fundamentally from the sinks studied in prior works, which we term primary sinks. While prior works have identified that tokens other than BOS can sometimes become sinks, they were found to exhibit properties analogous to the BOS token. Specifically, they emerge at the same layer, persist throughout the network and draw a large amount of attention mass. Whereas, we find the existence of secondary sinks that arise primarily in middle layers and can persist for a variable number of layers, and draw a smaller, but still significant, amount of attention mass. Through extensive experiments across 11 model families, we analyze where these secondary sinks appear, their properties, how they are formed, and their impact on the attention mechanism. Specifically, we show that: (1) these sinks are formed by specific middle-layer MLP modules; these MLPs map token representations to vectors that align with the direction of the primary sink of that layer. (2) The $\ell_2$-norm of these vectors determines the sink score of the secondary sink, and also the number of layers it lasts for, thereby leading to different impacts on the attention mechanisms accordingly. (3) The primary sink weakens in middle layers, coinciding with the emergence of secondary sinks. We observe that in larger-scale models, the location and lifetime of the sinks, together referred to as sink levels, appear in a more deterministic and frequent manner. Specifically, we identify three sink levels in QwQ-32B and six levels in Qwen3-14B. We open-sourced our findings at github.com/JeffreyWong20/Secondary-Attention-Sinks.

cs.LG↗

arXiv2021

FRuDA: Framework for Distributed Adversarial Domain Adaptation

Breakthroughs in unsupervised domain adaptation (uDA) can help in adapting models from a label-rich source domain to unlabeled target domains. Despite these advancements, there is a lack of research on how uDA algorithms, particularly those based on adversarial learning, can work in distributed settings. In real-world applications, target domains are often distributed across thousands of devices, and existing adversarial uDA algorithms -- which are centralized in nature -- cannot be applied in these settings. To solve this important problem, we introduce FRuDA: an end-to-end framework for distributed adversarial uDA. Through a careful analysis of the uDA literature, we identify the design goals for a distributed uDA system and propose two novel algorithms to increase adaptation accuracy and training efficiency of adversarial uDA in distributed settings. Our evaluation of FRuDA with five image and speech datasets show that it can boost target domain accuracy by up to 50% and improve the training efficiency of adversarial uDA by at least 11 times.

cs.LG↗

arXiv2021

Deep Inertial Navigation using Continuous Domain Adaptation and Optimal Transport

In this paper, we propose a new strategy for learning inertial robotic navigation models. The proposed strategy enhances the generalisability of end-to-end inertial modelling, and is aimed at wheeled robotic deployments. Concretely, the paper describes the following. (1) Using precision robotics, we empirically characterise the effect of changing the sensor position during navigation on the distribution of raw inertial signals, as well as the corresponding impact on learnt latent spaces. (2) We propose neural architectures and algorithms to assimilate knowledge from an indexed set of sensor positions in order to enhance the robustness and generalisability of robotic inertial tracking in the field. Our scheme of choice uses continuous domain adaptation (DA) and optimal transport (OT). (3) In our evaluation, continuous OT DA outperforms a continuous adversarial DA baseline, while also showing quantifiable learning benefits over simple data augmentation. We will release our dataset to help foster future research.

cs.LG↗

arXiv2020

Mic2Mic: Using Cycle-Consistent Generative Adversarial Networks to Overcome Microphone Variability in Speech Systems

Mobile and embedded devices are increasingly using microphones and audio-based computational models to infer user context. A major challenge in building systems that combine audio models with commodity microphones is to guarantee their accuracy and robustness in the real-world. Besides many environmental dynamics, a primary factor that impacts the robustness of audio models is microphone variability. In this work, we propose Mic2Mic -- a machine-learned system component -- which resides in the inference pipeline of audio models and at real-time reduces the variability in audio data caused by microphone-specific factors. Two key considerations for the design of Mic2Mic were: a) to decouple the problem of microphone variability from the audio task, and b) put a minimal burden on end-users to provide training data. With these in mind, we apply the principles of cycle-consistent generative adversarial networks (CycleGANs) to learn Mic2Mic using unlabeled and unpaired data collected from different microphones. Our experiments show that Mic2Mic can recover between 66% to 89% of the accuracy lost due to microphone variability for two common audio tasks.

eess.AS↗

arXiv2019

On Tracking the Physicality of Wi-Fi: A Subspace Approach

Wi-Fi channel state information (CSI) has emerged as a plausible modality for sensing different human activities as a function of modulations in the wireless signal that travels between wireless devices. Until now, most research has taken a statistical approach and/or purpose-built inference pipeline. Although interesting, these approaches struggle to sustain sensing performances beyond experimental conditions. As such, the full potential of CSI as a general-purpose sensing modality is yet to be realised. We argue a universal approach with well-grounded formalisation is necessary to characterise the relationship between wireless channel modulations (spatial and temporal) and human movement. To this end, we present a formalism for quantifying the changing part of the wireless signal modulated by human motion. Grounded in this formalisation, we then present a new subspace tracking technique to describe the channel statistics in an interpretable way, which succinctly contains the human modulated part of the channel. We characterise the signal and noise subspaces for the case of uncontrolled human movement, and show that these subspaces are dynamic. Our results demonstrate that proposed channel statistics alone can robustly reproduce state-of-the-art application-specific feature engineering baseline, however, across multiple usage scenarios. We expect, our universal channel statistics will yield an effective general-purpose featurisation of wireless channel measurements and will uncover opportunities for applying CSI for a variety of human sensing applications in a robust way.

cs.IT↗

arXiv2015

K-stability of relative flag varieties

We generalise partial results about the Yau-Tian-Donaldson correspondence on ruled manifolds to bundles whose fibre is a classical flag variety. This is done using Chern class computations involving the combinatorics of Schur functors. The strongest results are obtained when working over a Riemann surface. Weaker partial results are obtained for adiabatic polarisations over a base of arbitrary dimension. We develop the notion of relative K-stability which embeds the idea of working over a base variety into the theory of K-stability. Natural constructions in filtered algebras equip the collection of test configuration with extra structures. We illustrate these constructions with several examples.

math.AG↗