arXiv subjects

Robert Calderbank

Publications and source records attributed to Robert Calderbank.

At least 91 records · Page 5Linked to original sources

arXiv2020

Managing Device Lifecycle: Reconfigurable Constrained Codes for M/T/Q/P-LC Flash Memories

Flash memory devices are winning the competition for storage density against magnetic recording devices. This outcome results from advances in physics that allow storage of more than one bit per cell, coupled with advances in signal processing that reduce the effect of physical instabilities. Constrained codes are used in storage to avoid problematic patterns. Recently, we introduced binary symmetric lexicographically-ordered constrained codes (LOCO codes) for data storage and data transmission. This paper introduces simple constrained codes that support non-binary physical substrates; multi, triple, quad, and the currently-in-development penta-level cell (M/T/Q/P-LC) Flash memories. The new codes can be easily modified if problematic patterns change with time. These codes are designed to mitigate inter-cell interference, which is a critical source of error in Flash devices. The occurrence of errors is a consequence of parasitic capacitances in and across floating gate transistors, resulting in charge propagation from cells being programmed to the highest charge level to neighboring cells being programmed to lower levels. The new codes are called $q$-ary asymmetric LOCO codes (QA-LOCO codes), and the construction subsumes codes previously designed for single-level cell (SLC) Flash devices (A-LOCO codes). QA-LOCO codes work for a Flash device with any number, $q$, of levels per cell. For $q \geq 4$, we show that QA-LOCO codes can achieve rates greater than $0.95 \log_2 q$ information bits per coded symbol. The complexity of encoding and decoding is modest, and reconfiguring a code is as easy as reprogramming an adder. Capacity-achieving rates, affordable encoding-decoding complexity, and ease of reconfigurability support the growing development of M/T/Q/P-LC Flash memory devices, as well as lifecycle management as the characteristics of these devices change with time.

cs.IT↗

arXiv2019

CHIRRUP: a practical algorithm for unsourced multiple access

Unsourced multiple access abstracts grantless simultaneous communication of a large number of devices (messages) each of which transmits (is transmitted) infrequently. It provides a model for machine-to-machine communication in the Internet of Things (IoT), including the special case of radio-frequency identification (RFID), as well as neighbor discovery in ad hoc wireless networks. This paper presents a fast algorithm for unsourced multiple access that scales to $2^{100}$ devices (arbitrary $100$ bit messages). The primary building block is multiuser detection of binary chirps which are simply codewords in the second order Reed Muller code. The chirp detection algorithm originally presented by Howard et al. is enhanced and integrated into a peeling decoder designed for a patching and slotting framework. In terms of both energy per bit and number of transmitted messages, the proposed algorithm is within a factor of $2$ of state of the art approaches. A significant advantage of our algorithm is its computational efficiency. We prove that the worst-case complexity of the basic chirp reconstruction algorithm is $\mathcal{O}[nK(\log_2 n + K)]$, where $n$ is the codeword length and $K$ is the number of active users, and we report computing times for our algorithm. Our performance and computing time results represent a benchmark against which other practical algorithms can be measured.

eess.SP↗

arXiv2019

LOCO Codes: Lexicographically-Ordered Constrained Codes

Line codes make it possible to mitigate interference, to prevent short pulses, and to generate streams of bipolar signals with no direct-current (DC) power content through balancing. They find application in magnetic recording (MR) devices, in Flash devices, in optical recording devices, and in some computer standards. This paper introduces a new family of fixed-length, binary constrained codes, named lexicographically-ordered constrained codes (LOCO codes), for bipolar non-return-to-zero signaling. LOCO codes are capacity-achieving, the lexicographic indexing enables simple, practical encoding and decoding, and this simplicity is demonstrated through analysis of circuit complexity. LOCO codes are easy to balance, and their inherent symmetry minimizes the rate loss with respect to unbalanced codes having the same constraints. Furthermore, LOCO codes that forbid certain patterns can be used to alleviate inter-symbol interference in MR systems and inter-cell interference in Flash systems. Numerical results demonstrate a gain of up to 10% in rate achieved by LOCO codes with respect to other practical constrained codes, including run-length-limited codes, designed for the same purpose. Simulation results suggest that it is possible to achieve a channel density gain of about 20% in MR systems by using a LOCO code to encode only the parity bits, limiting the rate loss, of a low-density parity-check code before writing.

cs.IT↗

arXiv2019

Geometric Matrix Completion with Deep Conditional Random Fields

The problem of completing high-dimensional matrices from a limited set of observations arises in many big data applications, especially, recommender systems. Existing matrix completion models generally follow either a memory- or a model-based approach, whereas, geometric matrix completion models combine the best from both approaches. Existing deep-learning-based geometric models yield good performance, but, in order to operate, they require a fixed structure graph capturing the relationships among the users and items. This graph is typically constructed by evaluating a pre-defined similarity metric on the available observations or by using side information, e.g., user profiles. In contrast, Markov-random-fields-based models do not require a fixed structure graph but rely on handcrafted features to make predictions. When no side information is available and the number of available observations becomes very low, existing solutions are pushed to their limits. In this paper, we propose a geometric matrix completion approach that addresses these challenges. We consider matrix completion as a structured prediction problem in a conditional random field (CRF), which is characterized by a maximum a posterior (MAP) inference, and we propose a deep model that predicts the missing entries by solving the MAP inference problem. The proposed model simultaneously learns the similarities among matrix entries, computes the CRF potentials, and solves the inference problem. Its training is performed in an end-to-end manner, with a method to supervise the learning of entry similarities. Comprehensive experiments demonstrate the superior performance of the proposed model compared to various state-of-the-art models on popular benchmark datasets and underline its superior capacity to deal with highly incomplete matrices.

cs.LG↗

arXiv2019

Unifying the Clifford Hierarchy via Symmetric Matrices over Rings

The Clifford hierarchy is a foundational concept for universal quantum computation (UQC). It was introduced to show that UQC can be realized via quantum teleportation, given access to certain standard resources. While the full structure of the hierarchy is still not understood, Cui et al. (arXiv:1608.06596) recently described the structure of diagonal unitaries in the hierarchy. They considered diagonal gates whose action on a computational basis qudit state is described by a $2^k$-th root of unity raised to a polynomial function of the state, and they established the level of such unitaries in the hierarchy. For qubit systems, we consider $k$-th level diagonal gates that can be described just by quadratic forms of the state over the ring $\mathbb{Z}_{2^k}$ of integers mod $2^k$. These involve symmetric matrices over $\mathbb{Z}_{2^k}$ that can be used to efficiently describe all $2$-local and certain higher locality diagonal gates in the hierarchy. We also provide explicit algebraic descriptions of their action on Pauli matrices, which establishes a natural recursion to diagonal gates from lower levels. This involves symplectic matrices over $\mathbb{Z}_{2^k}$ and hence our perspective unifies these gates with the binary symplectic framework for Clifford gates. We augment our description with simple examples for certain standard gates. In addition to demonstrating structure, these formulas might prove useful in applications such as (i) classical simulation of quantum circuits, especially via the stabilizer rank approach, (ii) synthesis of logical non-Clifford unitaries, specifically alternatives to magic state distillation, and (iii) decomposition of arbitrary unitaries beyond the Clifford+$T$ set of gates, perhaps leading to shorter depth circuits. Our results suggest that some non-diagonal gates might be understood by generalizing other binary symplectic matrices to integer rings.

quant-ph↗

arXiv2019

Minimizing the Number of Detrimental Objects in Multi-Dimensional Graph-Based Codes

In order to meet the demands of data-hungry applications, data storage devices are required to be increasingly denser. Various sources of error appear with this increase in density. Multi-dimensional (MD) graph-based codes are capable of mitigating error sources like interference and channel non-uniformity in dense storage devices. Recently, a technique was proposed to enhance the performance of MD spatially-coupled codes that are based on circulants. The technique carefully relocates circulants to minimize the number of short cycles. However, cycles become more detrimental when they combine together to form more advanced objects, e.g., absorbing sets, including low-weight codewords. In this paper, we show how MD relocations can be exploited to minimize the number of detrimental objects in the graph of an MD code. Moreover, we demonstrate the savings in the number of relocation arrangements earned by focusing on objects rather than cycles. Our technique is applicable to a wide variety of one-dimensional (OD) codes. Simulation results reveal significant lifetime gains in practical Flash systems achieved by MD codes designed using our technique compared with OD codes having similar parameters.

cs.IT↗

arXiv2019

Hierarchical Coding to Enable Scalability and Flexibility in Heterogeneous Cloud Storage

In order to accommodate the ever-growing data from various, possibly independent, sources and the dynamic nature of data usage rates in practical applications, modern cloud data storage systems are required to be scalable, flexible, and heterogeneous. Codes with hierarchical locality have been intensively studied due to their effectiveness in reducing the average reading time in cloud storage. In this paper, we present the first codes with hierarchical locality that achieve scalability and flexibility in heterogeneous cloud storage using small field size. We propose a double-level construction utilizing so-called Cauchy Reed-Solomon codes. We then develop a triple-level construction based on this double-level code; this construction can be easily generalized into any hierarchical structure with a greater number of layers since it naturally achieves scalability in the cloud storage systems.

cs.IT↗

arXiv2018

Multi-Scale Spectrum Sensing in Dense Multi-Cell Cognitive Networks

Multi-scale spectrum sensing is proposed to overcome the cost of full network state information on the spectrum occupancy of primary users (PUs) in dense multi-cell cognitive networks. Secondary users (SUs) estimate the local spectrum occupancies and aggregate them hierarchically to estimate spectrum occupancy at multiple spatial scales. Thus, SUs obtain fine-grained estimates of spectrum occupancies of nearby cells, more relevant to scheduling tasks, and coarse-grained estimates of those of distant cells. An agglomerative clustering algorithm is proposed to design a cost-effective aggregation tree, matched to the structure of interference, robust to local estimation errors and delays. Given these multi-scale estimates, the SU traffic is adapted in a decentralized fashion in each cell, to optimize the trade-off among SU cell throughput, interference caused to PUs, and mutual SU interference. Numerical evaluations demonstrate a small degradation in SU cell throughput (up to 15% for a 0dB interference-to-noise ratio experienced at PUs) compared to a scheme with full network state information, using only one-third of the cost incurred in the exchange of spectrum estimates. The proposed interference-matched design is shown to significantly outperform a random tree design, by providing more relevant information for network control, and a state-of-the-art consensus-based algorithm, which does not leverage the spatio-temporal structure of interference across the network.

cs.IT↗

arXiv2018

A Characterization of Guesswork on Swiftly Tilting Curves

Given a collection of strings, each with an associated probability of occurrence, the guesswork of each of them is their position in a list ordered from most likely to least likely, breaking ties arbitrarily. Guesswork is central to several applications in information theory: Average guesswork provides a lower bound on the expected computational cost of a sequential decoder to decode successfully the transmitted message; the complementary cumulative distribution function of guesswork gives the error probability in list decoding; the logarithm of guesswork is the number of bits needed in optimal lossless one-to-one source coding; and guesswork is the number of trials required of an adversary to breach a password protected system in a brute-force attack. In this paper, we consider memoryless string-sources that generate strings consisting of i.i.d. characters drawn from a finite alphabet, and characterize their corresponding guesswork. Our main tool is the tilt operation. We show that the tilt operation on a memoryless string-source parametrizes an exponential family of memoryless string-sources, which we refer to as the tilted family. We provide an operational meaning to the tilted families by proving that two memoryless string-sources result in the same guesswork on all strings of all lengths if and only if their respective categorical distributions belong to the same tilted family. Establishing some general properties of the tilt operation, we generalize the notions of weakly typical set and asymptotic equipartition property to tilted weakly typical sets of different orders. We use this new definition to characterize the large deviations for all atypical strings and characterize the volume of weakly typical sets of different orders. We subsequently build on this characterization to prove large deviation bounds on guesswork and provide an accurate approximation of its PMF.

cs.IT↗

arXiv2018

Orthogonal Time Frequency Space Modulation

This paper introduces a new two-dimensional modulation technique called Orthogonal Time Frequency Space (OTFS) modulation. OTFS has the novel and important feature of being designed in the delay-Doppler domain. When coupled with a suitable equalizer, OTFS modulation is able to exploit the full channel diversity over both time and frequency. Moreover, it converts the fading, time-varying wireless channel experienced by modulated signals such as OFDM into a time-independent channel with a complex channel gain that is essentially constant for all symbols. This design obviates the need for transmitter adaptation, and greatly simplifies system operation. The paper describes the basic operating principles of OTFS as well as a possible implementation as an overlay to current or anticipated standardized systems. OTFS is shown to provide significant performance improvement in systems with high Doppler, short packets, and/or large antenna array. In particular, simulation results indicate at least several dB of block error rate performance improvement for OTFS over OFDM in all of these settings.

cs.IT↗

arXiv2018

DCFNet: Deep Neural Network with Decomposed Convolutional Filters

Filters in a Convolutional Neural Network (CNN) contain model parameters learned from enormous amounts of data. In this paper, we suggest to decompose convolutional filters in CNN as a truncated expansion with pre-fixed bases, namely the Decomposed Convolutional Filters network (DCFNet), where the expansion coefficients remain learned from data. Such a structure not only reduces the number of trainable parameters and computation, but also imposes filter regularity by bases truncation. Through extensive experiments, we consistently observe that DCFNet maintains accuracy for image classification tasks with a significant reduction of model parameters, particularly with Fourier-Bessel (FB) bases, and even with random bases. Theoretically, we analyze the representation stability of DCFNet with respect to input variations, and prove representation stability under generic assumptions on the expansion coefficients. The analysis is consistent with the empirical observations.

stat.ML↗

arXiv2018

Throughput Region of Spatially Correlated Interference Packet Networks

In multi-user wireless packet networks interference, typically modeled as packet collision, is the throughput bottleneck. Users become aware of the interference pattern via feedback and use this information for contention resolution and for packet retransmission. Conventional random access protocols interrupt communication to resolve contention which reduces network throughput and increases latency and power consumption. In this work we take a different approach and we develop opportunistic random access protocols rather than pursuing conventional methods. We allow wireless nodes to communicate without interruption and to observe the interference pattern. We then use this interference pattern knowledge and channel statistics to counter the negative impact of interference. We prove the optimality of our protocols using an extremal rank-ratio inequality. An important part of our contributions is the integration of spatial correlation in our assumptions and results. We identify spatial correlation regimes in which inherently outdated feedback becomes as good as idealized instantaneous feedback, and correlation regimes in which feedback does not provide any throughput gain. To better illustrate the results, and as an intermediate step, we characterize the capacity region of finite-field spatially correlated interference channels with delayed channel state information at the transmitters.

cs.IT↗

arXiv2018

Regularizing Autoencoder-Based Matrix Completion Models via Manifold Learning

Autoencoders are popular among neural-network-based matrix completion models due to their ability to retrieve potential latent factors from the partially observed matrices. Nevertheless, when training data is scarce their performance is significantly degraded due to overfitting. In this paper, we mit- igate overfitting with a data-dependent regularization technique that relies on the principles of multi-task learning. Specifically, we propose an autoencoder-based matrix completion model that performs prediction of the unknown matrix values as a main task, and manifold learning as an auxiliary task. The latter acts as an inductive bias, leading to solutions that generalize better. The proposed model outperforms the existing autoencoder-based models designed for matrix completion, achieving high reconstruction accuracy in well-known datasets.

cs.LG↗

arXiv2018

RotDCF: Decomposition of Convolutional Filters for Rotation-Equivariant Deep Networks

Explicit encoding of group actions in deep features makes it possible for convolutional neural networks (CNNs) to handle global deformations of images, which is critical to success in many vision tasks. This paper proposes to decompose the convolutional filters over joint steerable bases across the space and the group geometry simultaneously, namely a rotation-equivariant CNN with decomposed convolutional filters (RotDCF). This decomposition facilitates computing the joint convolution, which is proved to be necessary for the group equivariance. It significantly reduces the model size and computational complexity while preserving performance, and truncation of the bases expansion serves implicitly to regularize the filters. On datasets involving in-plane and out-of-plane object rotations, RotDCF deep features demonstrate greater robustness and interpretability than regular CNNs. The stability of the equivariant representation to input variations is also proved theoretically under generic assumptions on the filters in the decomposed form. The RotDCF framework can be extended to groups other than rotations, providing a general approach which achieves both group equivariance and representation stability at a reduced model size.

cs.CV↗

arXiv2018

Synthesis of Logical Clifford Operators via Symplectic Geometry

Quantum error-correcting codes can be used to protect qubits involved in quantum computation. This requires that logical operators acting on protected qubits be translated to physical operators (circuits) acting on physical quantum states. We propose a mathematical framework for synthesizing physical circuits that implement logical Clifford operators for stabilizer codes. Circuit synthesis is enabled by representing the desired physical Clifford operator in $\mathbb{C}^{N \times N}$ as a partial $2m \times 2m$ binary symplectic matrix, where $N = 2^m$. We state and prove two theorems that use symplectic transvections to efficiently enumerate all symplectic matrices that satisfy a system of linear equations. As an important corollary of these results, we prove that for an $[\![ m,m-k ]\!]$ stabilizer code every logical Clifford operator has $2^{k(k+1)/2}$ symplectic solutions. The desired physical circuits are then obtained by decomposing each solution as a product of elementary symplectic matrices. Our assembly of the possible physical realizations enables optimization over them with respect to a suitable metric. Furthermore, we show that any circuit that normalizes the stabilizer of the code can be transformed into a circuit that centralizes the stabilizer, while realizing the same logical operation. Our method of circuit synthesis can be applied to any stabilizer code, and this paper provides a proof of concept synthesis of universal Clifford gates for the $[\![ 6,4,2 ]\!]$ CSS code. We conclude with a classical coding-theoretic perspective for constructing logical Pauli operators for CSS codes. Since our circuit synthesis algorithm builds on the logical Pauli operators for the code, this paper provides a complete framework for constructing all logical Clifford operators for CSS codes. Programs implementing our algorithms can be found at https://github.com/nrenga/symplectic-arxiv18a

cs.IT↗

arXiv2018

Compressed Neighbour Discovery using Sparse Kerdock Matrices

We study the network-wide neighbour discovery problem in wireless networks in which each node in a network must discovery the network interface addresses (NIAs) of its neighbours. We work within the rapid on-off division duplex framework proposed by Guo and Zhang (2010) in which all nodes are assigned different on-off signatures which allow them listen to the transmissions of neighbouring nodes during their off slots, leading to a compressed sensing problem at each node with a collapsed codebook determined by a given node's transmission signature. We propose sparse Kerdock matrices as codebooks for the neighbour discovery problem. These matrices share the same row space as certain Delsarte-Goethals frames based upon Reed Muller codes, whilst at the same time being extremely sparse. We present numerical experiments using two different compressed sensing recovery algorithms, One Step Thresholding (OST) and Normalised Iterative Hard Thresholding (NIHT). For both algorithms, a higher proportion of neighbours are successfully identified using sparse Kerdock matrices compared to codebooks based on Reed Muller codes with random erasures as proposed by Zhang and Guo (2011). We argue that the improvement is due to the better interference cancellation properties of sparse Kerdock matrices when collapsed according to a given node's transmission signature. We show by explicit calculation that the coherence of the collapsed codebooks resulting from sparse Kerdock matrices remains near-optimal.

cs.IT↗

arXiv2017

Rate-Distortion Bounds on Bayes Risk in Supervised Learning

We present an information-theoretic framework for bounding the number of labeled samples needed to train a classifier in a parametric Bayesian setting. We derive bounds on the average $L_p$ distance between the learned classifier and the true maximum a posteriori classifier, which are well-established surrogates for the excess classification error due to imperfect learning. We provide lower and upper bounds on the rate-distortion function, using $L_p$ loss as the distortion measure, of a maximum a priori classifier in terms of the differential entropy of the posterior distribution and a quantity called the interpolation dimension, which characterizes the complexity of the parametric distribution family. In addition to expressing the information content of a classifier in terms of lossy compression, the rate-distortion function also expresses the minimum number of bits a learning machine needs to extract from training data to learn a classifier to within a specified $L_p$ tolerance. We use results from universal source coding to express the information content in the training data in terms of the Fisher information of the parametric family and the number of training samples available. The result is a framework for computing lower bounds on the Bayes $L_p$ risk. This framework complements the well-known probably approximately correct (PAC) framework, which provides minimax risk bounds involving the Vapnik-Chervonenkis dimension or Rademacher complexity. Whereas the PAC framework provides upper bounds the risk for the worst-case data distribution, the proposed rate-distortion framework lower bounds the risk averaged over the data distribution. We evaluate the bounds for a variety of data models, including categorical, multinomial, and Gaussian models. In each case the bounds are provably tight orderwise, and in two cases we prove that the bounds are tight up to multiplicative constants.

cs.IT↗

arXiv2017

LDMNet: Low Dimensional Manifold Regularized Neural Networks

Deep neural networks have proved very successful on archetypal tasks for which large training sets are available, but when the training data are scarce, their performance suffers from overfitting. Many existing methods of reducing overfitting are data-independent, and their efficacy is often limited when the training set is very small. Data-dependent regularizations are mostly motivated by the observation that data of interest lie close to a manifold, which is typically hard to parametrize explicitly and often requires human input of tangent vectors. These methods typically only focus on the geometry of the input data, and do not necessarily encourage the networks to produce geometrically meaningful features. To resolve this, we propose a new framework, the Low-Dimensional-Manifold-regularized neural Network (LDMNet), which incorporates a feature regularization method that focuses on the geometry of both the input data and the output features. In LDMNet, we regularize the network by encouraging the combination of the input data and the output features to sample a collection of low dimensional manifolds, which are searched efficiently without explicit parametrization. To achieve this, we directly use the manifold dimension as a regularization term in a variational functional. The resulting Euler-Lagrange equation is a Laplace-Beltrami equation over a point cloud, which is solved by the point integral method without increasing the computational complexity. We demonstrate two benefits of LDMNet in the experiments. First, we show that LDMNet significantly outperforms widely-used network regularizers such as weight decay and DropOut. Second, we show that LDMNet can be designed to extract common features of an object imaged via different modalities, which proves to be very useful in real-world applications such as cross-spectral face recognition.

cs.CV↗