Searcharxiv⌕ Search

SEARCH · Searcharxiv

Results for “math.IT”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 recordsLinked to original sources
arXiv2025

Optimal generalisation and learning transition in extensive-width shallow neural networks near interpolation

We consider a teacher-student model of supervised learning with a fully-trained two-layer neural network whose width $k$ and input dimension $d$ are large and proportional. We provide an effective theory for approximating the Bayes-optimal generalisation error of the network for any activation function in the regime of sample size $n$ scaling quadratically with the input dimension, i.e., around the interpolation threshold where the number of trainable parameters $kd+k$ and of data $n$ are comparable. Our analysis tackles generic weight distributions. We uncover a discontinuous phase transition separating a "universal" phase from a "specialisation" phase. In the first, the generalisation error is independent of the weight distribution and decays slowly with the sampling rate $n/d^2$, with the student learning only some non-linear combinations of the teacher weights. In the latter, the error is weight distribution-dependent and decays faster due to the alignment of the student towards the teacher network. We thus unveil the existence of a highly predictive solution near interpolation, which is however potentially hard to find by practical algorithms.

stat.ML↗
arXiv2009

An extension of the order bound for AG codes

The most successful method to obtain lower bounds for the minimum distance of an algebraic geometric code is the order bound, which generalizes the Feng-Rao bound. We provide a significant extension of the bound that improves the order bounds by Beelen and by Duursma and Park. We include an exhaustive numerical comparison of the different bounds for 10168 two-point codes on the Suzuki curve of genus g=124 over the field of 32 elements. Keywords: algebraic geometric code, order bound, Suzuki curve.

math.NT↗
arXiv2010

Restricted Isometries for Partial Random Circulant Matrices

In the theory of compressed sensing, restricted isometry analysis has become a standard tool for studying how efficiently a measurement matrix acquires information about sparse and compressible signals. Many recovery algorithms are known to succeed when the restricted isometry constants of the sampling matrix are small. Many potential applications of compressed sensing involve a data-acquisition process that proceeds by convolution with a random pulse followed by (nonrandom) subsampling. At present, the theoretical analysis of this measurement technique is lacking. This paper demonstrates that the $s$th order restricted isometry constant is small when the number $m$ of samples satisfies $m \gtrsim (s \log n)^{3/2}$, where $n$ is the length of the pulse. This bound improves on previous estimates, which exhibit quadratic scaling.

cs.IT↗
arXiv2014

Extension theorems for self-dual codes over rings and new binary self-dual codes

In this work, extension theorems are generalized to self-dual codes over rings and as applications many new binary self-dual extremal codes are found from self-dual codes over F_2^m+uF_2^m for m = 1, 2. The duality and distance preserving Gray maps from F4 +uF4 to (F_2 +uF_2)^2 and (F_4)^2 are used to obtain self-dual codes whose binary Gray images are [64,32,12]-extremal self-dual. An F_2+uF_2-extension is used and as binary images, 178 extremal binary self-dual codes of length 68 with new weight enumerators are obtained. Especially the first examples of codes with gamma=3 and many codes with the rare gamma= 4, 6 parameters are obtained. In addition to these, two hundred fifty doubly even self dual [96,48,16]-codes with new weight enumerators are obtained from four-circulant codes over F_4 + uF_4. New extremal doubly even binary codes of lengths 80 and 88 are also found by the F_2+uF_2-lifts of binary four circulant codes and a corresponding result about 3-designs is stated.

cs.IT↗
arXiv2011

On Rényi and Tsallis entropies and divergences for exponential families

Many common probability distributions in statistics like the Gaussian, multinomial, Beta or Gamma distributions can be studied under the unified framework of exponential families. In this paper, we prove that both Rényi and Tsallis divergences of distributions belonging to the same exponential family admit a generic closed form expression. Furthermore, we show that Rényi and Tsallis entropies can also be calculated in closed-form for sub-families including the Gaussian or exponential distributions, among others.

cs.IT↗
arXiv2016

Continuity bounds for information characteristics of quantum channels depending on input dimension

We show how to use properties of the quantum conditional mutual information to obtain continuity bounds for information characteristics of quantum channels depending on their input dimension. First we prove tight estimates for variation of the output Holevo quantity with respect to simultaneous variations of a channel and of an input ensemble. Then we obtain tight continuity bounds for output conditional mutual information for a single channel and for $n$ copies of a channel. As a result tight and close-to-tight continuity bounds for basic capacities of quantum channels depending on the input dimension are obtained. They complement the Leung-Smith continuity bounds depending on the output dimension.

quant-ph↗
arXiv2012

On the strong converses for the quantum channel capacity theorems

A unified approach to prove the converses for the quantum channel capacity theorems is presented. These converses include the strong converse theorems for classical or quantum information transfer with error exponents and novel explicit upper bounds on the fidelity measures reminiscent of the Wolfowitz strong converse for the classical channel capacity theorems. We provide a new proof for the error exponents for the classical information transfer. A long standing problem in quantum information theory has been to find out the strong converse for the channel capacity theorem when quantum information is sent across the channel. We give the quantum error exponent thereby giving a one-shot exponential upper bound on the fidelity. We then apply our results to show that the strong converse holds for the quantum information transfer across an erasure channel for maximally entangled channel inputs.

quant-ph↗
arXiv2016

Capacity-Achieving Iterative LMMSE Detection for MIMO-NOMA Systems

This paper considers a iterative Linear Minimum Mean Square Error (LMMSE) detection for the uplink Multiuser Multiple-Input and Multiple-Output (MU-MIMO) systems with Non-Orthogonal Multiple Access (NOMA). The iterative LMMSE detection greatly reduces the system computational complexity by departing the overall processing into many low-complexity distributed calculations. However, it is generally considered to be sub-optimal and achieves relatively poor performance. In this paper, we firstly present the matching conditions and area theorems for the iterative detection of the MIMO-NOMA systems. Based on the proposed matching conditions and area theorems, the achievable rate region of the iterative LMMSE detection is analysed. We prove that by properly design the iterative LMMSE detection, it can achieve (i) the optimal sum capacity of MU-MIMO systems, (ii) all the maximal extreme points in the capacity region of MU-MIMO system, and (iii) the whole capacity region of two-user MIMO systems.

cs.IT↗
arXiv2009

On the Sensitivity of Noncoherent Capacity to the Channel Model

The noncoherent capacity of stationary discrete-time fading channels is known to be very sensitive to the fine details of the channel model. More specifically, the measure of the set of harmonics where the power spectral density of the fading process is nonzero determines if capacity grows logarithmically in SNR or slower than logarithmically. An engineering-relevant problem is to characterize the SNR value at which this sensitivity starts to matter. In this paper, we consider the general class of continuous-time Rayleigh-fading channels that satisfy the wide-sense stationary uncorrelated-scattering (WSSUS) assumption and are, in addition, underspread. For this class of channels, we show that the noncoherent capacity is close to the AWGN capacity for all SNR values of practical interest, independently of whether the scattering function is compactly supported or not. As a byproduct of our analysis, we obtain an information-theoretic pulse-design criterion for orthogonal frequency-division multiplexing systems.

cs.IT↗
arXiv2015

One-Shot Mutual Covering Lemma and Marton's Inner Bound with a Common Message

By developing one-shot mutual covering lemmas, we derive a one-shot achievability bound for broadcast with a common message which recovers Marton's inner bound (with three auxiliary random variables) in the i.i.d.~case. The encoder employed is deterministic. Relationship between the mutual covering lemma and a new type of channel resolvability problem is discussed.

cs.IT↗
arXiv2020

Undirected Unicast Network Capacity: A Partition Bound

In this paper, we present a new technique to obtain upper bounds on undirected unicast network information capacity. Using this technique, we characterize an upper bound, called partition bound, on the symmetric rate of information flow in undirected unicast networks and give an algorithm to compute it. Two classes of networks are presented for which the bound is tight and the capacity is achievable by routing thus confirming the undirected unicast conjecture for these classes of networks. We also show that the bound can be loose in general and present an approach to tighten it.

cs.IT↗
arXiv2013

Sub-Stream Fairness and Numerical Correctness in MIMO Interference Channels

Signal-to-interference plus noise ratio (SINR) and rate fairness in a system are substantial quality-of-service (QoS) metrics. The acclaimed SINR maximization (max-SINR) algorithm does not achieve fairness between user's streams, i.e., sub-stream fairness is not achieved. To this end, we propose a distributed power control algorithm to render sub-stream fairness in the system. Sub-stream fairness is a less restrictive design metric than stream fairness (i.e., fairness between all streams) thus sum-rate degradation is milder. Algorithmic parameters can significantly differentiate the results of numerical algorithms. A complete picture for comparison of algorithms can only be depicted by varying these parameters. For example, a predetermined iteration number or a negligible increment in the sum-rate can be the stopping criteria of an algorithm. While the distributed interference alignment (DIA) can reasonably achieve sub-stream fairness for the later, the imbalance between sub-streams increases as the preset iteration number decreases. Thus comparison of max-SINR and DIA with a low preset iteration number can only depict a part of the picture. We analyze such important parameters and their effects on SINR and rate metrics to exhibit numerical correctness in executing the benchmarks. Finally, we propose group filtering schemes that jointly design the streams of a user in contrast to max-SINR scheme that designs each stream of a user separately.

cs.IT↗
arXiv2014

On the Duality of Erasures and Defects

In this paper, the duality of erasures and defects will be investigated by comparing the binary erasure channel (BEC) and the binary defect channel (BDC). The duality holds for channel capacities, capacity achieving schemes, minimum distances, and upper bounds on the probability of failure to retrieve the original message. Also, the binary defect and erasure channel (BDEC) will be introduced by combining the properties of the BEC and the BDC. It will be shown that the capacity of the BDEC can be achieved by the coding scheme that combines the encoding for the defects and the decoding for the erasures. This coding scheme for the BDEC has two separate redundancy parts for correcting erasures and masking defects. Thus, we will investigate the problem of redundancy allocation between these two parts.

cs.IT↗
arXiv2011

A Maximum Entropy solution of the Covariance Extension Problem for Reciprocal Processes

Stationary reciprocal processes defined on a finite interval of the integer line can be seen as a special class of Markov random fields restricted to one dimension. Non stationary reciprocal processes have been extensively studied in the past especially by Jamison, Krener, Levy and co-workers. The specialization of the non-stationary theory to the stationary case, however, does not seem to have been pursued in sufficient depth in the literature. Stationary reciprocal processes (and reciprocal stochastic models) are potentially useful for describing signals which naturally live in a finite region of the time (or space) line. Estimation or identification of these models starting from observed data seems still to be an open problem which can lead to many interesting applications in signal and image processing. In this paper, we discuss a class of reciprocal processes which is the acausal analog of auto-regressive (AR) processes, familiar in control and signal processing. We show that maximum likelihood identification of these processes leads to a covariance extension problem for block-circulant covariance matrices. This generalizes the famous covariance band extension problem for stationary processes on the integer line. As in the usual stationary setting on the integer line, the covariance extension problem turns out to be a basic conceptual and practical step in solving the identification problem. We show that the maximum entropy principle leads to a complete solution of the problem.

math.OC↗
arXiv2012

The Channel Capacity Increases with Power

It is proved that for memoryless vector channels, maximizing the mutual information over all source distributions with a certain average power or over the larger set of source distributions with upperbounded average power yields the same channel capacity in both cases. Hence, the channel capacity cannot decrease with increasing average transmitted power, not even for channels with severe nonlinear distortion.

cs.IT↗
arXiv2016

Achieving the Uniform Rate Region of General Multiple Access Channels by Polar Coding

We consider the problem of polar coding for transmission over $m$-user multiple access channels. In the proposed scheme, all users encode their messages using a polar encoder, while a multi-user successive cancellation decoder is deployed at the receiver. The encoding is done separately across the users and is independent of the target achievable rate. For the code construction, the positions of information bits and frozen bits for each of the users are decided jointly. This is done by treating the polar transformations across all the $m$ users as a single polar transformation with a certain \emph{polarization base}. We characterize the resolution of achievable rates on the dominant face of the uniform rate region in terms of the number of users $m$ and the length of the polarization base $L$. In particular, we prove that for any target rate on the dominant face, there exists an achievable rate, also on the dominant face, within the distance at most $\frac{(m-1)\sqrt{m}}{L}$ from the target rate. We then prove that the proposed MAC polar coding scheme achieves the whole uniform rate region with fine enough resolution by changing the decoding order in the multi-user successive cancellation decoder, as $L$ and the code block length $N$ grow large. The encoding and decoding complexities are $O(N \log N)$ and the asymptotic block error probability of $O(2^{-N^{0.5 - ε}})$ is guaranteed. Examples of achievable rates for the $3$-user multiple access channel are provided.

cs.IT↗
arXiv2007

Throughput Scaling Laws for Wireless Networks with Fading Channels

A network of n communication links, operating over a shared wireless channel, is considered. Fading is assumed to be the dominant factor affecting the strength of the channels between transmitter and receiver terminals. It is assumed that each link can be active and transmit with a constant power P or remain silent. The objective is to maximize the throughput over the selection of active links. By deriving an upper bound and a lower bound, it is shown that in the case of Rayleigh fading (i) the maximum throughput scales like $\log n$ (ii) the maximum throughput is achievable in a distributed fashion. The upper bound is obtained using probabilistic methods, where the key point is to upper bound the throughput of any random set of active links by a chi-squared random variable. To obtain the lower bound, a decentralized link activation strategy is proposed and analyzed.

cs.IT↗