arXiv subjects

C. Sinan Güntürk

Publications and source records attributed to C. Sinan Güntürk.

9 recordsLinked to original sources

arXiv2023

Uniform Approximation by Polynomials with Integer Coefficients via the Bernstein Lattice

Let $\mathscr{C}_\mathbb{Z}([0,1])$ be the metric space of real-valued continuous functions on $[0,1]$ with integer values at $0$ and $1$, equipped with the uniform (supremum) metric $d_\infty$. It is a classical theorem in approximation theory that the ring $\mathbb{Z}[X]$ of polynomials with integer coefficients, when considered as a set of functions on $[0,1]$, is dense in $\mathscr{C}_\mathbb{Z}([0,1])$. In this paper, we offer a strengthening of this result by identifying a substantially small subset $\bigcup_n \mathscr{B}_n$ of $\mathbb{Z}[X]$ which is still dense in $\mathscr{C}_\mathbb{Z}([0,1])$. Here $\mathscr{B}_n$, which we call the ``Bernstein lattice,'' is the lattice generated by the polynomials $$p_{n,k}(x) := \binom{n}{k} x^k(1-x)^{n-k}, ~~k=0,\dots,n.$$ Quantitatively, we show that for any $f \in \mathscr{C}_\mathbb{Z}([0,1])$, $$d_\infty(f, \mathscr{B}_n) \leq \frac{9}{4} ω_f(n^{-1/3}) + 2 n^{-1/3}, ~~n \geq 1,$$ where $ω_f$ stands for the modulus of continuity of $f$. We also offer a more general bound which can be optimized to yield better decay of approximation error for specific classes of continuous functions.

math.NT↗

arXiv2023

Approximation of functions with one-bit neural networks

The celebrated universal approximation theorems for neural networks roughly state that any reasonable function can be arbitrarily well-approximated by a network whose parameters are appropriately chosen real numbers. This paper examines the approximation capabilities of one-bit neural networks -- those whose nonzero parameters are $\pm a$ for some fixed $a\not=0$. One of our main theorems shows that for any $f\in C^s([0,1]^d)$ with $\|f\|_\infty<1$ and error $\varepsilon$, there is a $f_{NN}$ such that $|f(\boldsymbol{x})-f_{NN}(\boldsymbol{x})|\leq \varepsilon$ for all $\boldsymbol{x}$ away from the boundary of $[0,1]^d$, and $f_{NN}$ is either implementable by a $\{\pm 1\}$ quadratic network with $O(\varepsilon^{-2d/s})$ parameters or a $\{\pm \frac 1 2 \}$ ReLU network with $O(\varepsilon^{-2d/s}\log (1/\varepsilon))$ parameters, as $\varepsilon\to0$. We establish new approximation results for iterated multivariate Bernstein operators, error estimates for noise-shaping quantization on the Bernstein basis, and novel implementation of the Bernstein polynomials by one-bit quadratic and ReLU neural networks.

cs.LG↗

arXiv2022

Approximation with one-bit polynomials in Bernstein form

We prove various theorems on approximation using polynomials with integer coefficients in the Bernstein basis of any given order. In the extreme, we draw the coefficients from $\{ \pm 1\}$ only. A basic case of our results states that for any Lipschitz function $f:[0,1] \to [-1,1]$ and for any positive integer $n$, there are signs $σ_0,\dots,σ_n \in \{\pm 1\}$ such that $$\left |f(x) - \sum_{k=0}^n σ_k \, \binom{n}{k} x^k (1-x)^{n-k} \right | \leq \frac{C (1+|f|_{\mathrm{Lip}})}{1+\sqrt{nx(1-x)}} ~\mbox{ for all } x \in [0,1].$$ More generally, we show that higher accuracy is achievable for smoother functions: For any integer $s\geq 1$, if $f$ has a Lipschitz $(s{-}1)$st derivative, then approximation accuracy of order $O(n^{-s/2})$ is achievable with coefficients in $\{\pm 1\}$ provided $\|f \|_\infty < 1$, and of order $O(n^{-s})$ with unrestricted integer coefficients, both uniformly on closed subintervals of $(0,1)$ as above. Hence these polynomial approximations are not constrained by the saturation of classical Bernstein polynomials. Our approximations are constructive and can be implemented using feedforward neural networks whose weights are chosen from $\{\pm 1\}$ only.

cs.IT↗

arXiv2022

Quantization for spectral super-resolution

We show that the method of distributed noise-shaping beta-quantization offers superior performance for the problem of spectral super-resolution with quantization whenever there is redundancy in the number of measurements. More precisely, we define the oversampling ratio $λ$ as the largest integer such that $\lfloor M/λ\rfloor - 1\geq 4/Δ$, where $M$ denotes the number of Fourier measurements and $Δ$ is the minimum separation distance associated with the atomic measure to be resolved. We prove that for any number $K\geq 2$ of quantization levels available for the real and imaginary parts of the measurements, our quantization method combined with either TV-min/BLASSO or ESPRIT guarantees reconstruction accuracy of order $O(M^{1/4}λ^{5/4} K^{- λ/2})$ and $O(M^{3/2} λ^{1/2} K^{- λ})$ respectively, where the implicit constants are independent of $M$, $K$ and $λ$. In contrast, naive rounding or memoryless scalar quantization for the same alphabet offers a guarantee of order $O(M^{-1}K^{-1})$ only, regardless of the reconstruction algorithm.

cs.IT↗

arXiv2019

High-performance quantization for spectral super-resolution

We show that the method of distributed noise-shaping beta-quantization offers superior performance for the problem of spectral super-resolution with quantization whenever there is redundancy in the number of measurements. More precisely, if the (integer) oversampling ratio $λ$ is such that $\lfloor M/λ\rfloor - 1\geq 4/Δ$, where $M$ denotes the number of Fourier measurements and $Δ$ is the minimum separation distance associated with the atomic measure to be resolved, then for any number $K\geq 2$ of quantization levels available for the real and imaginary parts of the measurements, our quantization method guarantees reconstruction accuracy of order $O(λK^{- λ/2})$, up to constants which are independent of $K$ and $λ$. In contrast, memoryless scalar quantization offers a guarantee of order $O(K^{-1})$ only.

cs.IT↗

arXiv2019

Unrestricted iterations of relaxed projections in Hilbert space: Regularity, absolute convergence, and statistics of displacements

Given a finite collection $\mathbf{V}:=(V_1,\dots,V_N)$ of closed linear subspaces of a real Hilbert space $H$, let $P_i$ denote the orthogonal projection operator onto $V_i$ and $P_{i,λ}:= (1-λ)I + λP_i$ denote its relaxation with parameter $λ\in [0,2]$, $i=1,\dots,N$. Under a mild regularity assumption on $\mathbf{V}$ known as `innate regularity' (which, for example, is always satisfied if each $V_i$ has finite dimension or codimension), we show that all trajectories $(x_n)_{0}^\infty$ resulting from the iteration $x_{n+1} := P_{i_n,λ_n}(x_n)$, where the $i_n$ and the $λ_n$ are unrestricted other than the assumption that $\{λ_n : n \in \mathbb{N}\} \subset [η,2{-}η]$ for some $η\in (0,1]$, possess uniformly bounded displacement moments of arbitrarily small orders. In particular, we show that $$ \sum_{n=0}^\infty \|x_{n+1} - x_n \|^γ\leq C \|x_0\|^γ~\mbox{ for all }~ γ> 0,$$ where $C:=C(\mathbf{V},η,γ)<\infty$. This result strengthens prior results on norm convergence of these trajectories, known to hold under the same regularity assumption. For example, with $γ=1$, it follows that the displacements series $\sum (x_{n+1}-x_n)$ converges absolutely in $H$. Quantifying the constant $C(\mathbf{V},η,γ)$, we also derive an effective bound on the distribution function of the norms of the displacements (normalized by the norm of the initial condition) which yields a root-exponential type decay bound on their decreasing rearrangement, again uniformly for all trajectories.

math.FA↗

arXiv2017

Convergence of the randomized Kaczmarz method for phase retrieval

The classical Kaczmarz iteration and its randomized variants are popular tools for fast inversion of linear overdetermined systems. This method extends naturally to the setting of the phase retrieval problem via substituting at each iteration the phase of any measurement of the available approximate solution for the unknown phase of the measurement of the true solution. Despite the simplicity of the method, rigorous convergence guarantees that are available for the classical linear setting have not been established so far for the phase retrieval setting. In this short note, we provide a convergence result for the randomized Kaczmarz method for phase retrieval in $\mathbb{R}^d$. We show that with high probability a random measurement system of size $m \asymp d$ will be admissible for this method in the sense that convergence in the mean square sense is guaranteed with any prescribed probability. The convergence is exponential and comparable to the linear setting.

math.NA↗

arXiv2015

Noise-shaping Quantization Methods for Frame-based and Compressive Sampling Systems

Noise shaping refers to an analog-to-digital conversion methodology in which quantization error is arranged to lie mostly outside the signal spectrum by means of oversampling and feedback. Recently it has been successfully applied to more general redundant linear sampling and reconstruction systems associated with frames as well as non-linear systems associated with compressive sampling. This chapter reviews some of the recent progress in this subject.

cs.IT↗

arXiv2010

An Optimal Family of Exponentially Accurate One-Bit Sigma-Delta Quantization Schemes

Sigma-Delta modulation is a popular method for analog-to-digital conversion of bandlimited signals that employs coarse quantization coupled with oversampling. The standard mathematical model for the error analysis of the method measures the performance of a given scheme by the rate at which the associated reconstruction error decays as a function of the oversampling ratio $λ$. It was recently shown that exponential accuracy of the form $O(2^{-rλ})$ can be achieved by appropriate one-bit Sigma-Delta modulation schemes. By general information-entropy arguments $r$ must be less than 1. The current best known value for $r$ is approximately 0.088. The schemes that were designed to achieve this accuracy employ the "greedy" quantization rule coupled with feedback filters that fall into a class we call "minimally supported". In this paper, we study the minimization problem that corresponds to optimizing the error decay rate for this class of feedback filters. We solve a relaxed version of this problem exactly and provide explicit asymptotics of the solutions. From these relaxed solutions, we find asymptotically optimal solutions of the original problem, which improve the best known exponential error decay rate to $r \approx 0.102$. Our method draws from the theory of orthogonal polynomials; in particular, it relates the optimal filters to the zero sets of Chebyshev polynomials of the second kind.

cs.IT↗