arXiv subjects

Marc de Kamps

Publications and source records attributed to Marc de Kamps.

10 recordsLinked to original sources

arXiv2025

DisentangleFormer: Spatial-Channel Decoupling for Multi-Channel Vision

Vision Transformers face a fundamental limitation: standard self-attention jointly processes spatial and channel dimensions, leading to entangled representations that prevent independent modeling of structural and semantic dependencies. This problem is especially pronounced in hyperspectral imaging, from satellite hyperspectral remote sensing to infrared pathology imaging, where channels capture distinct biophysical or biochemical cues. We propose DisentangleFormer, an architecture that achieves robust multi-channel vision representation through principled spatial-channel decoupling. Motivated by information-theoretic principles of decorrelated representation learning, our parallel design enables independent modeling of structural and semantic cues while minimizing redundancy between spatial and channel streams. Our design integrates three core components: (1) Parallel Disentanglement: Independently processes spatial-token and channel-token streams, enabling decorrelated feature learning across spatial and spectral dimensions, (2) Squeezed Token Enhancer: An adaptive calibration module that dynamically fuses spatial and channel streams, and (3) Multi-Scale FFN: complementing global attention with multi-scale local context to capture fine-grained structural and semantic dependencies. Extensive experiments on hyperspectral benchmarks demonstrate that DisentangleFormer achieves state-of-the-art performance, consistently outperforming existing models on Indian Pine, Pavia University, and Houston, the large-scale BigEarthNet remote sensing dataset, as well as an infrared pathology dataset. Moreover, it retains competitive accuracy on ImageNet while reducing computational cost by 17.8% in FLOPs. The code will be made publicly available upon acceptance.

cs.CV↗

arXiv2023

GATology for Linguistics: What Syntactic Dependencies It Knows

Graph Attention Network (GAT) is a graph neural network which is one of the strategies for modeling and representing explicit syntactic knowledge and can work with pre-trained models, such as BERT, in downstream tasks. Currently, there is still a lack of investigation into how GAT learns syntactic knowledge from the perspective of model structure. As one of the strategies for modeling explicit syntactic knowledge, GAT and BERT have never been applied and discussed in Machine Translation (MT) scenarios. We design a dependency relation prediction task to study how GAT learns syntactic knowledge of three languages as a function of the number of attention heads and layers. We also use a paired t-test and F1-score to clarify the differences in syntactic dependency prediction between GAT and BERT fine-tuned by the MT task (MT-B). The experiments show that better performance can be achieved by appropriately increasing the number of attention heads with two GAT layers. With more than two layers, learning suffers. Moreover, GAT is more competitive in training speed and syntactic dependency prediction than MT-B, which may reveal a better incorporation of modeling explicit syntactic knowledge and the possibility of combining GAT and BERT in the MT tasks.

cs.CL↗

arXiv2023

Syntactic Knowledge via Graph Attention with BERT in Machine Translation

Although the Transformer model can effectively acquire context features via a self-attention mechanism, deeper syntactic knowledge is still not effectively modeled. To alleviate the above problem, we propose Syntactic knowledge via Graph attention with BERT (SGB) in Machine Translation (MT) scenarios. Graph Attention Network (GAT) and BERT jointly represent syntactic dependency feature as explicit knowledge of the source language to enrich source language representations and guide target language generation. Our experiments use gold syntax-annotation sentences and Quality Estimation (QE) model to obtain interpretability of translation quality improvement regarding syntactic knowledge without being limited to a BLEU score. Experiments show that the proposed SGB engines improve translation quality across the three MT tasks without sacrificing BLEU scores. We investigate what length of source sentences benefits the most and what dependencies are better identified by the SGB engines. We also find that learning of specific dependency relations by GAT can be reflected in the translation quality containing such relations and that syntax on the graph leads to new modeling of syntactic aspects of source sentences in the middle and bottom layers of BERT.

cs.CL↗

arXiv2022

Evaluating Muscle Synergies with EMG Data and Physics Simulation in the Neurorobotics Platform

Although we can measure muscle activity and analyze their activation patterns, we understand little about how individual muscles affect the joint torque generated. It is known that they are controlled by circuits in the spinal cord, a system much less well understood than the cortex. Knowing the contribution of the muscles towards a joint torque would improve our understanding of human limb control. We present a novel framework to examine the control of biomechanics using physics simulations informed by electromyography (EMG) data. These signals drive a virtual musculoskeletal model in the Neurorobotics Platform (NRP), which we then use to evaluate resulting joint torques. We use our framework to analyze raw EMG data collected during an isometric knee extension study to identify synergies that drive a musculoskeletal lower limb model. The resulting knee torques are used as a reference for genetic algorithms (GA) to generate new simulated activation patterns. On the platform the GA finds solutions that generate torques matching those observed. Possible solutions include synergies that are similar to those extracted from the human study. In addition, the GA finds activation patterns that are different from the the biological ones while still producing the same knee torque. The NRP forms a highly modular integrated simulation platform allowing these in silico experiments. We argue that our framework allows for research of the neurobiomechanical control of muscles during tasks, which would otherwise not be possible.

q-bio.NC↗

arXiv2020

Models Currently Implemented in MIIND

This is a living document that will be updated when appropriate. MIIND [1, 2] is a population-level neural simulator. It is based on population density techniques, just like DIPDE [3]. Contrary to DIPDE, MIIND is agnostic to the underlying neuron model used in its populations so any 1, 2 or 3 dimensional model can be set up with minimal effort. The resulting populations can then be grouped into large networks, e.g. the Potjans-Diesmann model [4]. The MIIND website http://miind.sf.net contains training materials, and helps to set up MIIND, either by using virtual machines, a DOCKER image, or directly from source code.

q-bio.NC↗

arXiv2020

Generalised linear models for prognosis and intervention: Theory, practice, and implications for machine learning

Prediction and causal explanation are fundamentally distinct tasks of data analysis. In health applications, this difference can be understood in terms of the difference between prognosis (prediction) and prevention/treatment (causal explanation). Nevertheless, these two concepts are often conflated in practice. We use the framework of generalised linear models (GLMs) to illustrate that predictive and causal queries require distinct processes for their application and subsequent interpretation of results. In particular, we identify five primary ways in which GLMs for prediction differ from GLMs for causal inference: (1) The covariates that should be considered for inclusion in (and possibly exclusion from) the model; (2) How a suitable set of covariates to include in the model is determined; (3) Which covariates are ultimately selected, and what functional form (i.e. parameterisation) they take; (4) How the model is evaluated; and (5) How the model is interpreted. We outline some of the potential consequences of failing to acknowledge and respect these differences, and additionally consider the implications for machine learning (ML) methods. We then conclude with three recommendations which we hope will help ensure that both prediction and causal modelling are used appropriately and to greatest effect in health research.

stat.AP↗

arXiv2019

Application of Cox Model to predict the survival of patients with Chronic Heart Failure: A latent class regression approach

Most prediction models that are used in medical research fail to accurately predict health outcomes due to methodological limitations. Using routinely collected patient data, we explore the use of a Cox proportional hazard (PH) model within a latent class framework to model survival of patients with chronic heart failure (CHF). We identify subgroups of patients based on their risk with the aid of available covariates. We allow each subgroup to have its own risk model.We choose an optimum number of classes based on the reported Bayesian information criteria (BIC). We assess the discriminative ability of the chosen model using an area under the receiver operating characteristic curve (AUC) for all the cross-validated and bootstrapped samples.We conduct a simulation study to compare the predictive performance of our models. Our proposed latent class model outperforms the standard one class Cox PH model.

stat.ME↗

arXiv2019

The scientific case for brain simulations

A key element of the European Union's Human Brain Project (HBP) and other large-scale brain research projects is simulation of large-scale model networks of neurons. Here we argue why such simulations will likely be indispensable for bridging the scales between the neuron and system levels in the brain, and a set of brain simulators based on neuron models at different levels of biological detail should thus be developed. To allow for systematic refinement of candidate network models by comparison with experiments, the simulations should be multimodal in the sense that they should not only predict action potentials, but also electric, magnetic, and optical signals measured at the population and system levels.

q-bio.NC↗

arXiv2017

Population Density Equations for Stochastic Processes with Memory Kernels

We present a novel method for solving population density equations (PDEs), where the populations can be subject to non-Markov noise for arbitrary distributions of jump sizes. The method combines recent developments in two different disciplines that traditionally have had limited interaction: computational neuroscience and the theory of random networks. The method uses a geometric binning scheme, based on the method of characteristics, to capture the deterministic neurodynamics of the population, separating the deterministic and stochastic process cleanly. We can independently vary the choice of the deterministic model and the model for the stochastic process, leading to a highly modular numerical solution strategy. We demonstrate this by replacing the Master equation implicit in many formulations of the PDE formalism, by a generalization called the generalized Montroll-Weiss equation - a recent result from random network theory - describing a random walker subject to transitions realized by a non-Markovian process. We demonstrate the method for leaky- (LIF) and quadratic-integrate and fire (QIF) neurons subject to spike trains with Poisson and gamma distributed spike intervals. We are able to model jump responses for both models accurately to both excitatory and inhibitory input under the assumption that all inputs are generated by one renewal process.

physics.bio-ph↗

arXiv2013

A Generic Approach to Solving Jump Diffusion Equations with Applications to Neural Populations

Diffusion processes have been applied with great success to model the dynamics of large populations throughout science, in particular biology. One advantage is that they bridge two different scales: the microscopic and the macroscopic one. Diffusion is a mathematical idealisation, however: it assumes vanishingly small state changes at the microscopic level. In real biological systems this is often not the case. The differential Chapman-Kolmogorov equation is more appropriate to model population dynamics that is not well described by drift and diffusion alone. Here, the method of characteristics is used to transform deterministic dynamics away and find a coordinate frame where this equation reduces to a Master equation. There is no longer a drift term, and solution methods there are insensitive to density gradients, making the method suitable for jump processes with arbitrary jump sizes. Moreover, its solution is universal: it no longer depends explicitly on the deterministic system. We demonstrate the technique on simple models of neuronal populations. Surprisingly, it is suitable for fast neural dynamics, even though in the new coordinate frame state space may expand rapidly towards infinity. We demonstrate universality: the method is applicable to any one dimensional neural model and show this on populations of leaky- and quadratic-integrate-and-fire neurons. In the diffusive limit, excellent approximations of Fokker-Planck equations are achieved. Nothing in the approach is particular to neuroscience, and the neural models are simple enough to serve as an example of dynamical systems that demonstrates the method.

q-bio.NC↗