arXiv subjects

Shahar Stein Ioushua

Publications and source records attributed to Shahar Stein Ioushua.

4 recordsLinked to original sources

arXiv2024

Batches Stabilize the Minimum Norm Risk in High Dimensional Overparameterized Linear Regression

Learning algorithms that divide the data into batches are prevalent in many machine-learning applications, typically offering useful trade-offs between computational efficiency and performance. In this paper, we examine the benefits of batch-partitioning through the lens of a minimum-norm overparametrized linear regression model with isotropic Gaussian features. We suggest a natural small-batch version of the minimum-norm estimator and derive bounds on its quadratic risk. We then characterize the optimal batch size and show it is inversely proportional to the noise level, as well as to the overparametrization ratio. In contrast to minimum-norm, our estimator admits a stable risk behavior that is monotonically increasing in the overparametrization ratio, eliminating both the blowup at the interpolation point and the double-descent phenomenon. We further show that shrinking the batch minimum-norm estimator by a factor equal to the Weiner coefficient further stabilizes it and results in lower quadratic risk in all settings. Interestingly, we observe that the implicit regularization offered by the batch partition is partially explained by feature overlap between the batches. Our bound is derived via a novel combination of techniques, in particular normal approximation in the Wasserstein metric of noisy projections over random subspaces.

cs.LG↗

arXiv2023

On the Number of Graphs with a Given Histogram

Let $G$ be a large (simple, unlabeled) dense graph on $n$ vertices. Suppose that we only know, or can estimate, the empirical distribution of the number of subgraphs $F$ that each vertex in $G$ participates in, for some fixed small graph $F$. How many other graphs would look essentially the same to us, i.e., would have a similar local structure? In this paper, we derive upper and lower bounds on the number of graphs whose empirical distribution lies close (in the Kolmogorov-Smirnov distance) to that of $G$. Our bounds are given as solutions to a maximum entropy problem on random graphs of a fixed size $k$ that does not depend on $n$, under $d$ global density constraints. The bounds are asymptotically close, with a gap that vanishes with $d$ at a rate that depends on the concentration function of the center of the Kolmogorov-Smirnov ball.

cs.IT↗

arXiv2020

RF Chain Reduction for MIMO Systems: A Hardware Prototype

Radio frequency (RF) chain circuits play a major role in digital receiver architectures, allowing passband communication signals to be processed in baseband. When operating at high frequencies, these circuits tend to be costly. This increased cost imposes a major limitation on future multiple-input multiple-output (MIMO) communication technologies. A common approach to mitigate the increased cost is to utilize hybrid architectures, in which the received signal is combined in analog into a lower dimension, thus reducing the number of RF chains. In this work we study the design and hardware implementation of hybrid architectures via minimizing channel estimation error. We first derive the optimal solution for complex-gain combiners and propose an alternating optimization algorithm for phase-shifter combiners. We then present a hardware prototype implementing analog combining for RF chain reduction. The prototype consists of a specially designed configurable combining board as well as a dedicated experimental setup. Our hardware prototype allows evaluating the effect of analog combining in MIMO systems using actual communication signals. The experimental study, which focuses on channel estimation accuracy in MIMO channels, demonstrates that using the proposed prototype, the achievable channel estimation performance is within a small gap in a statistical sense from that obtained using a costly receiver in which each antenna is connected to a dedicated RF chain.

eess.SP↗

arXiv2017

Hybrid Analog-Digital Beamforming for Massive MIMO Systems

In massive MIMO systems, hybrid beamforming is an essential technique for exploiting the potential array gain without using a dedicated RF chain for each antenna. In this work, we consider the data phase in a massive MIMO communication process, where the transmitter and receiver use fewer RF chains than antennas. We examine several different fully- and partially connected schemes and consider the design of hybrid beamformers that minimize the estimation error in the data. For the hybrid precoder, we introduce a framework for approximating the optimal fully-digital precoder with a feasible hybrid one. We exploit the fact that the fully-digital precoder is unique only up to a unitary matrix and optimize over this matrix and the hybrid precoder alternately. Our alternating minimization of approximation gap (Alt-MaG) framework improves the performance over state-of-the-art methods with no substantial increase in complexity. In addition, we present a special case of Alt-MaG, minimal gap iterative quantization (MaGiQ), that results in low complexity and lower mean squared error (MSE) than other common methods, in the case of very few RF chains. MaGiQ is shown to coincide with the optimal fully-digital solution in some scenarios. For combiner design, we exploit the structure of the MSE objective and develop a greedy ratio trace maximization technique, that achieves low MSE under various settings. All of our algorithms can be used with multiple hardware architectures.

cs.IT↗