Searcharxiv⌕ Search

arXiv subjects

Sanjoy Mitter

Publications and source records attributed to Sanjoy Mitter.

12 recordsLinked to original sources
arXiv2018

Layered black-box, behavioral interconnection perspective and applications to the problem of communication with fidelity criteria, Part I: i.i.d. sources

In this paper, the problem of communication over an essentially unknown channel, which is known to be able to communicate a source to a destination to within a certain distortion level, is considered from a behavioral, interconnection view-point. Rates of reliable communication are derived and source-channel separation for communication with fidelity criteria is proved. The results are then generalized to the multi-user setting under certain assumptions. Other applications of this problem problem which follow from this perspective are discussed.

cs.IT↗
arXiv2013

Testing the Manifold Hypothesis

The hypothesis that high dimensional data tend to lie in the vicinity of a low dimensional manifold is the basis of manifold learning. The goal of this paper is to develop an algorithm (with accompanying complexity guarantees) for fitting a manifold to an unknown probability distribution supported in a separable Hilbert space, only using i.i.d samples from that distribution. More precisely, our setting is the following. Suppose that data are drawn independently at random from a probability distribution $P$ supported on the unit ball of a separable Hilbert space $H$. Let $G(d, V, τ)$ be the set of submanifolds of the unit ball of $H$ whose volume is at most $V$ and reach (which is the supremum of all $r$ such that any point at a distance less than $r$ has a unique nearest point on the manifold) is at least $τ$. Let $L(M, P)$ denote mean-squared distance of a random point from the probability distribution $P$ to $M$. We obtain an algorithm that tests the manifold hypothesis in the following sense. The algorithm takes i.i.d random samples from $P$ as input, and determines which of the following two is true (at least one must be): (a) There exists $M \in G(d, CV, \fracτ{C})$ such that $L(M, P) \leq C ε.$ (b) There exists no $M \in G(d, V/C, Cτ)$ such that $L(M, P) \leq \fracε{C}.$ The answer is correct with probability at least $1-δ$.

math.ST↗
arXiv2013

A universal, operational theory of unicast multi-user communication with fidelity criteria

This is a three part paper. Optimality of source-channel separation for communication with a fidelity criterion when the channel is compound as defined by Csiszar and Korner in their book and general as defined by Verdu and Han, is proved in Part I. It is assumed that random codes are permitted. The word "universal" in the title of this paper refers to the fact that the channel model is compound. The proof uses a layered black-box or a layered input-output view-point. In particular, only the end-to-end description of the channel as being capable of communicating a source to within a certain distortion level is used when proving separation. This implies that the channel model does not play any role for separation to hold as long as there is a source model. Further implications of the layered black-box view-point are discussed. Optimality of source-medium separation for multi-user communication with fidelity criteria over a general, compound medium in the unicast setting is proved in Part II, thus generalizing Part I to the unicast, multi-user setting. Part III gets to an understanding of the question, "Why is a channel which is capable of communicating a source to within a certain distortion level, also capable of communicating bits at any rate less than the infimum of the rates needed to code the source to within the distortion level": this lies at the heart of why optimality of separation for communication with a fidelity criterion holds. The perspective taken to get to this understanding is a randomized covering-packing perspective, and the proof is operational.

cs.IT↗
arXiv2012

Optimal Control and Estimation for Partially Nested Interconnected Systems

In this paper, we study distributed estimation and control problems over graphs under partially nested information patterns. We show a duality result that is very similar to the classical duality result between state estimation and state feedback control with a classical information pattern, under the condition that the disturbances entering different systems on the graph are uncorrelated. The distributed estimation problem decomposes into $N$ separate estimation problems, where $N$ is the number of interconnected subsystems over the graph, and the solution to each subproblem is simply the optimal Kalman filter. This also gives the solution to the distributed control problem due to the duality of distributed estimation and control under partially nested information pattern. We then consider a weighted distributed estimation problem, where we get coupling between the estimators, and separation between the estimators is not possible. We propose a solution based on linear quadratic team decision theory, which provides a generalized Riccati equation for teams. We show that the weighted estimation problem is the dual to a distributed state feedback problem, where the disturbances entering the interconnected systems are correlated.

math.OC↗
arXiv2011

Architecture for communication with a fidelity criterion in unknown networks

We prove that in order to communicate independent sources (this is the unicast problem) between various users over an unknown medium to within various distortion levels, it is sufficient to consider source-channel separation based architectures: architectures which first compress the sources to within the corresponding distortion levels followed by reliable communication over the unknown medium. We are reducing the problem of universal rate-distortion communication of independent sources over a network to the universal reliable communication problem over networks. This is a reductionist view. We are not solving the reliable communication problem in networks.

cs.IT↗
arXiv2009

Equivalence perspectives in communication, source-channel connections and universal source-channel separation

An operational perspective is used to understand the relationship between source and channel coding. This is based on a direct reduction of one problem to another that uses random coding (and hence common randomness) but unlike all prior work, does not involve any functional computations, in particular, no mutual-information computations. This result is then used to prove a universal source-channel separation theorem in the rate-distortion context where universality is in the sense of a compound ``general channel.''

cs.IT↗
arXiv2007

Source coding and channel requirements for unstable processes

Our understanding of information in systems has been based on the foundation of memoryless processes. Extensions to stable Markov and auto-regressive processes are classical. Berger proved a source coding theorem for the marginally unstable Wiener process, but the infinite-horizon exponentially unstable case has been open since Gray's 1970 paper. There were also no theorems showing what is needed to communicate such processes across noisy channels. In this work, we give a fixed-rate source-coding theorem for the infinite-horizon problem of coding an exponentially unstable Markov process. The encoding naturally results in two distinct bitstreams that have qualitatively different QoS requirements for communicating over a noisy medium. The first stream captures the information that is accumulating within the nonstationary process and requires sufficient anytime reliability from the channel used to communicate the process. The second stream captures the historical information that dissipates within the process and is essentially classical. This historical information can also be identified with a natural stable counterpart to the unstable process. A converse demonstrating the fundamentally layered nature of unstable sources is given by means of information-embedding ideas.

cs.IT↗
arXiv2007

The necessity and sufficiency of anytime capacity for stabilization of a linear system over a noisy communication link, Part II: vector systems

In part I, we reviewed how Shannon's classical notion of capacity is not sufficient to characterize a noisy communication channel if the channel is intended to be used as part of a feedback loop to stabilize an unstable scalar linear system. While classical capacity is not enough, a sense of capacity (parametrized by reliability) called "anytime capacity" is both necessary and sufficient for channel evaluation in this context. The rate required is the log of the open-loop system gain and the required reliability comes from the desired sense of stability. Sufficiency is maintained even in cases with noisy observations and without any explicit feedback between the observer and the controller. This established the asymptotic equivalence between scalar stabilization problems and delay-universal communication problems with feedback. Here in part II, the vector-state generalizations are established and it is the magnitudes of the unstable eigenvalues that play an essential role. To deal with such systems, the concept of the anytime rate-region is introduced. This is the region of rates that the channel can support while still meeting potentially different anytime reliability targets for parallel message streams. All the scalar results generalize on an eigenvalue by eigenvalue basis. When there is no explicit feedback of the noisy channel outputs, the intrinsic delay of the unstable system tells us what the feedback delay needs to be while evaluating the anytime-rate-region for the channel. An example involving a binary erasure channel is used to illustrate how differentiated service is required in any separation-based control architecture.

cs.IT↗
arXiv2006

Coding into a source: a direct inverse Rate-Distortion theorem

Shannon proved that if we can transmit bits reliably at rates larger than the rate distortion function $R(D)$, then we can transmit this source to within a distortion $D$. We answer the converse question ``If we can transmit a source to within a distortion $D$, can we transmit bits reliably at rates less than the rate distortion function?'' in the affirmative. This can be viewed as a direct converse of the rate distortion theorem.

cs.IT↗
arXiv2006

The Capacity of Channels with Feedback

We introduce a general framework for treating channels with memory and feedback. First, we generalize Massey's concept of directed information and use it to characterize the feedback capacity of general channels. Second, we present coding results for Markov channels. This requires determining appropriate sufficient statistics at the encoder and decoder. Third, a dynamic programming framework for computing the capacity of Markov channels is presented. Fourth, it is shown that the average cost optimality equation (ACOE) can be viewed as an implicit single-letter characterization of the capacity. Fifth, scenarios with simple sufficient statistics are described.

cs.IT↗
arXiv2006

The necessity and sufficiency of anytime capacity for stabilization of a linear system over a noisy communication link Part I: scalar systems

We review how Shannon's classical notion of capacity is not enough to characterize a noisy communication channel if the channel is intended to be used as part of a feedback loop to stabilize an unstable scalar linear system. While classical capacity is not enough, another sense of capacity (parametrized by reliability) called ``anytime capacity'' is shown to be necessary for the stabilization of an unstable process. The required rate is given by the log of the unstable system gain and the required reliability comes from the sense of stability desired. A consequence of this necessity result is a sequential generalization of the Schalkwijk/Kailath scheme for communication over the AWGN channel with feedback. In cases of sufficiently rich information patterns between the encoder and decoder, adequate anytime capacity is also shown to be sufficient for there to exist a stabilizing controller. These sufficiency results are then generalized to cases with noisy observations, delayed control actions, and without any explicit feedback between the observer and the controller. Both necessary and sufficient conditions are extended to continuous time systems as well. We close with comments discussing a hierarchy of difficulty for communication problems and how these results establish where stabilization problems sit in that hierarchy.

cs.IT↗