arXiv subjects

Jean-Claude Belfiore

Publications and source records attributed to Jean-Claude Belfiore.

At least 19 recordsLinked to original sources

arXiv2022

Topos and Stacks of Deep Neural Networks

Every known artificial deep neural network (DNN) corresponds to an object in a canonical Grothendieck's topos; its learning dynamic corresponds to a flow of morphisms in this topos. Invariance structures in the layers (like CNNs or LSTMs) correspond to Giraud's stacks. This invariance is supposed to be responsible of the generalization property, that is extrapolation from learning data under constraints. The fibers represent pre-semantic categories (Culioli, Thom), over which artificial languages are defined, with internal logics, intuitionist, classical or linear (Girard). Semantic functioning of a network is its ability to express theories in such a language for answering questions in output about input data. Quantities and spaces of semantic information are defined by analogy with the homological interpretation of Shannon's entropy of P.Baudot and D.Bennequin in 2015). They generalize the measures found by Carnap and Bar-Hillel (1952). Amazingly, the above semantical structures are classified by geometric fibrant objects in a closed model category of Quillen, then they give rise to homotopical invariants of DNNs and of their semantic functioning. Intentional type theories (Martin-Loef) organize these objects and fibrations between them. Information contents and exchanges are analyzed by Grothendieck's derivators.

math.AT↗

arXiv2021

Logical Information Cells I

In this study we explore the spontaneous apparition of visible intelligible reasoning in simple artificial networks, and we connect this experimental observation with a notion of semantic information. We start with the reproduction of a DNN model of natural neurons in monkeys, studied by Neromyliotis and Moschovakis in 2017 and 2018, to explain how "motor equivalent neurons", coding only for the action of pointing, are supplemented by other neurons for specifying the actor of the action, the eye E, the hand H, or the eye and the hand together EH. There appear inner neurons performing a logical work, making intermediary proposition, for instance E V EH. Then, we remarked that adding a second hidden layer and choosing a symmetric metric for learning, the activities of the neurons become almost quantized and more informative. Using the work of Carnap and Bar-Hillel 1952, we define a measure of the logical value for collections of such cells. The logical score growths with the depth of the layer, i.e. the information on the output decision increases, which confirms a kind of bottleneck principle. Then we study a bit more complex tasks, a priori involving predicate logic. We compare the logic and the measured weights. This shows, for groups of neurons, a neat correlation between the logical score and the size of the weights. It exhibits a form of sparsity between the layers. The most spectacular result concerns the triples which can conclude for all conditions: when applying their weight matrices to their logical matrix, we recover the classification. This shows that weights precisely perform the proofs.

cs.AI↗

arXiv2020

Multi-Kernel Polar Codes: Concept and Design Principles

In this paper, we propose a new polar code construction by employing kernels of different sizes in the Kronecker product of the transformation matrix, thus generalizing the original construction by Arikan. The proposed multi-kernel polar code allows for more flexibility in terms of the code length, moreover allowing for various new design principles. We describe in detail encoding as well as successive cancellation (SC) decoding and SC list (SCL) decoding, and we provide a novel design method for the frozen set that allows to optimise the performance under list decoding, as opposed to original relability-based code design. Finally, we numerically demonstrate the advantage of multi-kernel polar codes under the new design principles compared to punctured and shortened polar codes.

cs.IT↗

arXiv2019

Semantically Secure Lattice Codes for Compound MIMO Channels

We consider compound multi-input multi-output (MIMO) wiretap channels where minimal channel state information at the transmitter (CSIT) is assumed. Code construction is given for the special case of isotropic mutual information, which serves as a conservative strategy for general cases. Using the flatness factor for MIMO channels, we propose lattice codes universally achieving the secrecy capacity of compound MIMO wiretap channels up to a constant gap (measured in nats) that is equal to the number of transmit antennas. The proposed approach improves upon existing works on secrecy coding for MIMO wiretap channels from an error probability perspective, and establishes information theoretic security (in fact semantic security). We also give an algebraic construction to reduce the code design complexity, as well as the decoding complexity of the legitimate receiver. Thanks to the algebraic structures of number fields and division algebras, our code construction for compound MIMO wiretap channels can be reduced to that for Gaussian wiretap channels, up to some additional gap to secrecy capacity.

cs.IT↗

arXiv2018

Achieving AWGN Channel Capacity With Lattice Gaussian Coding

We propose a new coding scheme using only one lattice that achieves the $\frac{1}{2}\log(1+\SNR)$ capacity of the additive white Gaussian noise (AWGN) channel with lattice decoding, when the signal-to-noise ratio $\SNR>e-1$. The scheme applies a discrete Gaussian distribution over an AWGN-good lattice, but otherwise does not require a shaping lattice or dither. Thus, it significantly simplifies the default lattice coding scheme of Erez and Zamir which involves a quantization-good lattice as well as an AWGN-good lattice. Using the flatness factor, we show that the error probability of the proposed scheme under minimum mean-square error (MMSE) lattice decoding is almost the same as that of Erez and Zamir, for any rate up to the AWGN channel capacity. We introduce the notion of good constellations, which carry almost the same mutual information as that of continuous Gaussian inputs. We also address the implementation of Gaussian shaping for the proposed lattice Gaussian coding scheme.

cs.IT↗

arXiv2017

Minimum-Distance Based Construction of Multi-Kernel Polar Codes

In this paper, we propose a construction for multi-kernel polar codes based on the maximization of the minimum distance. Compared to the original construction based on density evolution, our new design shows particular advantages for short code lengths, where the polarization effect has less impact on the performance than the distances of the code. We introduce and compute the minimum-distance profile and provide a simple greedy algorithm for the code design. Compared to state-of-the-art punctured or shortened Arikan polar codes, multi-kernel polar codes with our new design show significantly improved error-rate performance.

cs.IT↗

arXiv2017

Compute-and-Forward over Block-Fading Channels Using Algebraic Lattices

Previous approaches to compute-and-forward (C\&F) are mostly based on quantizing channel coefficients to integers. In this work, we investigate the C\&F strategy over block fading channels using Construction A over rings, so as to allow better quantization for the channels. Advantages in decoding error probabilities and computation rates are demonstrated, and the construction is shown to outperform the C\&F strategy over the integers $\mathbb{Z}$.

cs.IT↗

arXiv2017

$β$-expansion: A Theoretical Framework for Fast and Recursive Construction of Polar Codes

In this work, we introduce $β$-expansion, a notion borrowed from number theory, as a theoretical framework to study fast construction of polar codes based on a recursive structure of universal partial order (UPO) and polarization weight (PW) algorithm. We show that polar codes can be recursively constructed from UPO by continuously solving several polynomial equations at each recursive step. From these polynomial equations, we can extract an interval for $β$, such that ranking the synthetic channels through a closed-form $β$-expansion preserves the property of nested frozen sets, which is a desired feature for low-complex construction. In an example of AWGN channels, we show that this interval for $β$ converges to a constant close to $1.1892 \approx 2^{1/4}$ when the code block-length trends to infinity. Both asymptotic analysis and simulation results validate our theoretical claims.

cs.IT↗

arXiv2017

Multi-Kernel Construction of Polar Codes

We propose a generalized construction for binary polar codes based on mixing multiple kernels of different sizes in order to construct polar codes of block lengths that are not only powers of integers. This results in a multi kernel polar code with very good performance while the encoding complexity remains low and the decoding follows the same general structure as for the original Arikan polar codes. The construction provides numerous practical advantages as more code lengths can be achieved without puncturing or shortening. We observe numerically that the error-rate performance of our construction outperforms stateof the-art constructions using puncturing methods.

cs.IT↗

arXiv2017

Low-complexity Receiver for Multi-Level Polar Coded Modulation in Non-Orthogonal Multiple Access

Non-orthogonal multiple access (NOMA) schemes have been proved to increase the multiple-access achievable rate with respect to orthogonal multiple access (OMA). In this paper we propose a novel communication system that combines multi-level coded modulation and polar codes in a NOMA scenario. Computational complexity decreases with the proposed scheme with respect to state-of-the-art solutions. We also highlight the trade-off between error rate performance and computational complexity.

cs.IT↗

arXiv2017

Universal Lattice Codes for MIMO Channels

We propose a coding scheme that achieves the capacity of the compound MIMO channel with algebraic lattices. Our lattice construction exploits the multiplicative structure of number fields and their group of units to absorb ill-conditioned channel realizations. To shape the constellation, a discrete Gaussian distribution over the lattice points is applied. These techniques, along with algebraic properties of the proposed lattices, are then used to construct a sub-optimal de-coupled coding schemes that achieves a gap to compound capacity by decoding in a lattice that does not depend of the channel realization. The gap is characterized in terms of algebraic invariants of the codes, and shown to be significantly smaller than previous schemes in the literature. We also exhibit alternative algebraic constructions that achieve the capacity of ergodic fading channels.

cs.IT↗

arXiv2016

Structured Compute-and-Forward with phase precoding scheme: Beyond integers

-In this work, we focus on the K--user Gaussian Symmetric Complex-valued Interference Channels (GS-CIC). The Compute-and-Forward (CoF) protocol in wireless networks have been employed for Gaussian Symmetric Real-valued Interference Channels (GS-RIC) to achieve approximately the capacity of such channels and making Interference Alignment (IA) by solving a system of linear equations at destinations. We aim to adapt this protocol to GS-CIC. At high Signal-to-Noise Ratio (SNR), this protocol shows a fading behavior of the achievable sum-rate for strong and very strong interference regimes. In complex field this random-like behavior is related to the phase of complex cross channel coefficients. To avoid this fading-like behavior, we consider Z[i]--lattice codes and we propose a phase precoding scheme for CoF protocol with partial feedback. In this method the phase of channel coefficients will then be fed back to the transmitters in order to choose the best precoder factors to control this random behavior. We further simulate the achievable sum-rate given by this scheme and showing significant gain improvement can be obtained in terms of limiting the random behavior of achievable sum-rate.

cs.IT↗

arXiv2016

The Two-Modular Fourier Transform of Binary Functions

In this paper, we provide a solution to the open problem of computing the Fourier transform of a binary function defined over $n$-bit vectors taking $m$-bit vector values. In particular, we introduce the two-modular Fourier transform (TMFT) of a binary function $f:G\rightarrow {\cal R}$, where $G = (\mathbb{F}_2^n,+)$ is the group of $n$ bit vectors with bitwise modulo two addition $+$, and ${\cal R}$ is a finite commutative ring of characteristic $2$. Using the specific group structure of $G$ and a sequence of nested subgroups of $G$, we define the fast TMFT and its inverse. Since the image ${\cal R}$ of the binary functions is a ring, we can define the convolution between two functions $f:G\rightarrow {\cal R}$. We then provide the TMFT properties, including the convolution theorem, which can be used to efficiently compute convolutions. Finally, we derive the complexity of the fast TMFT and the inverse fast TMFT.

cs.IT↗

arXiv2016

Quasi-cyclic subcodes of cyclic codes

We completely characterize possible indices of quasi-cyclic subcodes in a cyclic code for a very broad class of cyclic codes. We present enumeration results for quasi-cyclic subcodes of a fixed index and show that the problem of enumeration is equivalent to enumeration of certain vector subspaces in finite fields. In particular, we present enumeration results for quasi-cyclic subcodes of the simplex code and duals of certain BCH codes. Our results are based on the trace representation of cyclic codes.

cs.IT↗

arXiv2014

Compute-and-Forward for the Interference Channel: Diversity Precoding

Interference Alignment is a new solution to over- come the problem of interference in multiuser wireless com- munication systems. Recently, the Compute-and-Forward (CF) transform has been proposed to approximate the capacity of K- user Gaussian Symmetric Interference Channel and practically perform Interference Alignment in wireless networks. However, this technique shows a random behavior in the achievable sum- rate, especially at high SNR. In this work, the origin of this random behavior is analyzed and a novel precoding technique based on the Golden Ratio is proposed to scale down the fadings experiences by the achievable sum-rate at high SNR.

cs.IT↗

arXiv2014

Shifted inverse determinant sums and new bounds for the DMT of space-time lattice codes

This paper considers shifted inverse determinant sums arising from the union bound of the pairwise error probability for space-time codes in multiple-antenna fading channels. Previous work by Vehkalahti et al. focused on the approximation of these sums for low multiplexing gains, providing a complete classification of the inverse determinant sums as a function of constellation size for the most well-known algebraic space-time codes. This work aims at building a general framework for the study of the shifted sums for all multiplexing gains. New bounds obtained using dyadic summing techniques suggest that the behavior of the shifted sums does characterize many properties of a lattice code such as the diversity-multiplexing gain trade-off, both under maximum-likelihood decoding and infinite lattice naive decoding. Moreover, these bounds allow to characterize the signal-to-noise ratio thresholds corresponding to different diversity gains.

cs.IT↗

arXiv2013

Semantically Secure Lattice Codes for the Gaussian Wiretap Channel

We propose a new scheme of wiretap lattice coding that achieves semantic security and strong secrecy over the Gaussian wiretap channel. The key tool in our security proof is the flatness factor which characterizes the convergence of the conditional output distributions corresponding to different messages and leads to an upper bound on the information leakage. We not only introduce the notion of secrecy-good lattices, but also propose the {flatness factor} as a design criterion of such lattices. Both the modulo-lattice Gaussian channel and the genuine Gaussian channel are considered. In the latter case, we propose a novel secrecy coding scheme based on the discrete Gaussian distribution over a lattice, which achieves the secrecy capacity to within a half nat under mild conditions. No \textit{a priori} distribution of the message is assumed, and no dither is used in our proposed schemes.

cs.IT↗

arXiv2013

An Error Probability Approach to MIMO Wiretap Channels

We consider MIMO (Multiple Input Multiple Output) wiretap channels, where a legitimate transmitter Alice is communicating with a legitimate receiver Bob in the presence of an eavesdropper Eve, and communication is done via MIMO channels. We suppose that Alice's strategy is to use a codebook which has a lattice structure, which then allows her to perform coset encoding. We analyze Eve's probability of correctly decoding the message Alice meant to Bob, and from minimizing this probability, we derive a code design criterion for MIMO lattice wiretap codes. The case of block fading channels is treated similarly, and fast fading channels are derived as a particular case. The Alamouti code is carefully studied as an illustration of the analysis provided.

cs.IT↗