arXiv subjects

Stéphane Paquelet

Publications and source records attributed to Stéphane Paquelet.

11 recordsLinked to original sources

arXiv2024

TocBERT: Medical Document Structure Extraction Using Bidirectional Transformers

Text segmentation holds paramount importance in the field of Natural Language Processing (NLP). It plays an important role in several NLP downstream tasks like information retrieval and document summarization. In this work, we propose a new solution, namely TocBERT, for segmenting texts using bidirectional transformers. TocBERT represents a supervised solution trained on the detection of titles and sub-titles from their semantic representations. This task was formulated as a named entity recognition (NER) problem. The solution has been applied on a medical text segmentation use-case where the Bio-ClinicalBERT model is fine-tuned to segment discharge summaries of the MIMIC-III dataset. The performance of TocBERT has been evaluated on a human-labeled ground truth corpus of 250 notes. It achieved an F1-score of 84.6% when evaluated on a linear text segmentation problem and 72.8% on a hierarchical text segmentation problem. It outperformed a carefully designed rule-based solution, particularly in distinguishing titles from subtitles.

cs.CL↗

arXiv2024

Anatomy of Neural Language Models

The fields of generative AI and transfer learning have experienced remarkable advancements in recent years especially in the domain of Natural Language Processing (NLP). Transformers have been at the heart of these advancements where the cutting-edge transformer-based Language Models (LMs) have led to new state-of-the-art results in a wide spectrum of applications. While the number of research works involving neural LMs is exponentially increasing, their vast majority are high-level and far from self-contained. Consequently, a deep understanding of the literature in this area is a tough task especially in the absence of a unified mathematical framework explaining the main types of neural LMs. We address the aforementioned problem in this tutorial where the objective is to explain neural LMs in a detailed, simplified and unambiguous mathematical framework accompanied by clear graphical illustrations. Concrete examples on widely used models like BERT and GPT2 are explored. Finally, since transformers pretrained on language-modeling-like tasks have been widely adopted in computer vision and time series applications, we briefly explore some examples of such solutions in order to enable readers to understand how transformers work in the aforementioned domains and compare this use with the original one in NLP.

cs.CL↗

arXiv2022

Leveraging triplet loss and nonlinear dimensionality reduction for on-the-fly channel charting

Channel charting is an unsupervised learning method that aims at mapping wireless channels to a so-called chart, preserving as much as possible spatial neighborhoods. In this paper, a model-based deep learning approach to this problem is proposed. It builds on a physically motivated distance measure to structure and initialize a neural network that is subsequently trained using a triplet loss function. The proposed structure exhibits a low number of parameters and clever initialization leads to fast training. These two features make the proposed approach amenable to on-the-fly channel charting. The method is empirically assessed on realistic synthetic channels, yielding encouraging results.

cs.NI↗

arXiv2021

Deep learning for location based beamforming with NLOS channels

Massive MIMO systems are highly efficient but critically rely on accurate channel state information (CSI) at the base station in order to determine appropriate precoders. CSI acquisition requires sending pilot symbols which induce an important overhead. In this paper, a method whose objective is to determine an appropriate precoder from the knowledge of the user's location only is proposed. Such a way to determine precoders is known as location based beamforming. It allows to reduce or even eliminate the need for pilot symbols, depending on how the location is obtained. the proposed method learns a direct mapping from location to precoder in a supervised way. It involves a neural network with a specific structure based on random Fourier features allowing to learn functions containing high spatial frequencies. It is assessed empirically and yields promising results on realistic synthetic channels. As opposed to previously proposed methods, it allows to handle both line-of-sight (LOS) and non-line-of-sight (NLOS) channels.

cs.IT↗

arXiv2021

Online unsupervised deep unfolding for MIMO channel estimation

Channel estimation is a difficult problem in MIMO systems. Using a physical model allows to ease the problem, injecting a priori information based on the physics of propagation. However, such models rest on simplifying assumptions and require to know precisely the system configuration, which is unrealistic.In this paper, we propose to perform online learning for channel estimation in a massive MIMO context, adding flexibility to physical models by unfolding a channel estimation algorithm (matching pursuit) as a neural network. This leads to a computationally efficient neural network that can be trained online when initialized with an imperfect model. The method allows a base station to automatically correct its channel estimation algorithm based on incoming data, without the need for a separate offline training phase.It is applied to realistic channels and shows great performance, achieving channel estimation error almost as low as one would get with a perfectly calibrated system.

eess.SP↗

arXiv2020

Channel estimation: unified view of optimal performance and pilot sequences

Channel estimation is of paramount importance in most communication systems in order to optimize the data rate/energy consumption tradeoff. In modern systems, the possibly large number of transmit/receive antennas and subcarriers makes this task difficult. Designing pilot sequences of reasonable size yielding good performance is thus critical. Classically, the number of pilots is reduced by viewing the channel as a random vector and assuming knowledge of its distribution. In practice, this requires estimating the channel covariance matrix, which can be computationally costly and not adapted to scenarios with high mobility. In this paper, an alternative view is considered, in which the channel is a function of unknown deterministic parameters. In this setting, the problem of designing optimal pilot sequences of smallest possible size is studied for any parametric channel model. To do so, the Cram{é}r-Rao bound (CRB) for this general channel estimation problem is given, highlighting its key dependency on the introduced variation space. Then, the minimal size of pilot sequences and minimal value of the CRB are determined. Moreover, a general strategy to build optimal minimal length power constrained pilots sequences is given, based on an estimation of the variation space. The theoretical results are finally illustrated in a massive MIMO system context. They conveniently allow to retrieve well known previous results, but also to exhibit minimal length optimal pilot sequences for a new strategy based on a nonlinear physical model.

eess.SP↗

arXiv2019

Performance of MIMO channel estimation with a physical model

Channel estimation is challenging in multi-antenna communication systems, because of the large number of parameters to estimate. One way of facilitating this task is to use a physical model describing the multiple paths constituting the channel, in the hope of reducing the number of unknowns in the problem. The achievable performance of estimation using this kind of physical model is studied theoretically. It is found that adjusting the number of estimated paths leads to a bias-variance tradeoff which is characterized. Moreover, computing the Fisher information matrix of the model allows to identify orthogonal parameters, ultimately leading to fast and asymptotically optimal algorithms as a byproduct.

eess.SP↗

arXiv2018

Massive MIMO channel estimation taking into account spherical waves

Together with millimiter waves (mmWaves), massive multiple-input multiple-output (MIMO) systems are key technological components of fifth generation (5G) wireless communication systems. In such a context, geometric considerations show that the largely adopted plane wave model (PWM) of the channel potentially loses its validity. An alternative is to consider the more accurate but more complex spherical wave model (SWM). This paper introduces an intermediate parabolic wave model (ParWM), more accurate than the PWM while less complex than the SWM. The validity domains of those three physical models are assessed in a novel way. Finally, estimation algorithms for the SWM and ParWM are proposed and compared with classical algorithms, showing a promising performance complexity trade-off.

cs.NI↗

arXiv2018

Bias-variance tradeoff in MIMO channel estimation

Channel estimation is challenging in multi-antenna communication systems, because of the large number of parameters to estimate. It is possible to facilitate this task by using a physical model describing the multiple paths constituting the channel, in the hope of reducing the number of unknowns in the problem. Adjusting the number of estimated paths leads to a bias-variance tradeoff. This paper explores this tradeoff, aiming to find the optimal number of paths to estimate. Moreover, the approach based on a physical model is compared to the classical least squares and Bayesian techniques. Finally, the impact of channel estimation error on the system data rate is assessed.

eess.SP↗

arXiv2018

MIMO Channel Hardening: A Physical Model based Analysis

In a multiple-input-multiple-output (MIMO) communication system, the multipath fading is averaged over radio links. This well-known channel hardening phenomenon plays a central role in the design of massive MIMO systems. The aim of this paper is to study channel hardening using a physical channel model in which the influences of propagation rays and antenna array topologies are highlighted. A measure of channel hardening is derived through the coefficient of variation of the channel gain. Our analyses and closed form results based on the used physical model are consistent with those of the literature relying on more abstract Rayleigh fading models, but offer further insights on the relationship with channel characteristics.

cs.NI↗

arXiv2018

Parametric channel estimation for massive MIMO

Channel state information is crucial to achieving the capacity of multi-antenna (MIMO) wireless communication systems. It requires estimating the channel matrix. This estimation task is studied, considering a sparse channel model particularly suited to millimeter wave propagation, as well as a general measurement model taking into account hybrid architectures. The contribution is twofold. First, the Cram{é}r-Rao bound in this context is derived. Second, interpretation of the Fisher Information Matrix structure allows to assess the role of system parameters, as well as to propose asymptotically optimal and computationally efficient estimation algorithms.

cs.IT↗