arXiv subjects

Sayandev Mukherjee

Publications and source records attributed to Sayandev Mukherjee.

16 recordsLinked to original sources

arXiv2025

SkyMemory: A LEO Edge Cache for Transformer Inference Optimization and Scale Out

We expand the scope of cache memory to include LEO constellations, which are highly distributed systems with thousands of satellites connected with free-space optics inter-satellite links (ISL) always only one hop from any point on earth. We show how to increase the number of cache hits and improve the speed of inference for the important use case of LLMs. These benefits apply not only to LLMs, both terrestrially hosted and on satellites, but also generalize to any cache distributed over multiple locations that needs to be accessed in a timely manner. We show the benefit of our key value cache (KVC) protocol in simulations and present a proof-of-concept implementation of the protocol for KVCs on a testbed comprising 5 Intel NUC Linux mini PCs hosting a 19x5 constellation, with an NVIDIA Jetson Nano 8GB GPU hosting the LLM.

cs.DC↗

arXiv2024

Semantic Navigation for AI-assisted Ideation

We present a novel AI-based ideation assistant and evaluate it in a user study with a group of innovators. The key contribution of our work is twofold: we propose a method of idea exploration in a constrained domain by means of LLM-supported semantic navigation of problem and solution spaces, and employ novel automated data input filtering to improve generations. We found that semantic exploration is preferred to the traditional prompt-output interactions, measured both in explicit survey rankings, and in terms of innovation assistant engagement, where 2.1x more generations were performed using semantic exploration. We also show that filtering input data with metrics such as relevancy, coherence and human alignment leads to improved generations in the same metrics as well as enhanced quality of experience among innovators.

cs.HC↗

arXiv2024

A Cloud in the Sky: Geo-Aware On-board Data Services for LEO Satellites

We propose an architecture with accompanying protocol for on-board satellite data infrastructure designed for Low Earth Orbit (LEO) constellations offering communication services, such as direct-to-cell connectivity. Our design leverages the unused or under-used computing and communication resources of LEO satellites that are orbiting over uninhabited parts of the earth, like the oceans. We show how blockchain-backed distributed transactions can be run efficiently on this architecture to offer smart contract services. A key aspect of the proposed architecture that sets it apart from other blockchain systems is that migration of the ledger is not done solely to recover from failures. Rather, migration is also performed periodically and continuously as the satellites circle around in their orbits and enter and leave the blockchain service area. We show in simulations how message and blockchain processing overhead can be contained using different sizes of dynamic geo-aware service areas.

cs.DC↗

arXiv2024

Randomness Is All You Need: Semantic Traversal of Problem-Solution Spaces with Large Language Models

We present a novel approach to exploring innovation problem and solution domains using LLM fine-tuning with a custom idea database. By semantically traversing the bi-directional problem and solution tree at different temperature levels we achieve high diversity in solution edit distance while still remaining close to the original problem statement semantically. In addition to finding a variety of solutions to a given problem, this method can also be used to refine and clarify the original problem statement. As further validation of the approach, we implemented a proof-of-concept Slack bot to serve as an innovation assistant.

cs.HC↗

arXiv2023

WHO-IS: Wireless Hetnet Optimization using Impact Selection

We propose a method to first identify users who have the most negative impact on the overall network performance, and then offload them to an orthogonal channel. The feasibility of such an approach is verified using real-world traces, network simulations, and a lab experiment that employs multi-homed wireless stations. In our experiment, as offload target, we employ LiFi IR transceivers, and as the primary network we consider a typical Enterprise Wi-Fi setup. We found that a limited number of users can impact the overall experience of the Wi-Fi network negatively, hence motivating targeted offloading. In our simulations and experiments we saw that the proposed solution can improve the collision probability with 82% and achieve a 61 percentage point air utilization improvement compared to random offloading, respectively.

cs.NI↗

arXiv2022

Why Neural Networks Work

We argue that many properties of fully-connected feedforward neural networks (FCNNs), also called multi-layer perceptrons (MLPs), are explainable from the analysis of a single pair of operations, namely a random projection into a higher-dimensional space than the input, followed by a sparsification operation. For convenience, we call this pair of successive operations expand-and-sparsify following the terminology of Dasgupta. We show how expand-and-sparsify can explain the observed phenomena that have been discussed in the literature, such as the so-called Lottery Ticket Hypothesis, the surprisingly good performance of randomly-initialized untrained neural networks, the efficacy of Dropout in training and most importantly, the mysterious generalization ability of overparameterized models, first highlighted by Zhang et al. and subsequently identified even in non-neural network models by Belkin et al.

cs.LG↗

arXiv2021

MASS: Mobile Autonomous Station Simulation

We propose a set of tools to replay wireless network traffic traces, while preserving the privacy of the original traces. Traces are generated by a user- and context-aware trained generative adversarial network (GAN). The replay allows for realistic traces from any number of users and of any trace duration to be produced given contextual parameters like the type of application and the real-time signal strength. We demonstrate the usefulness of the tools in three replay scenarios: Linux- and Android-station experiments and NS3 simulations. We also evaluate the ability of the GAN model to generate traces that retain key statistical properties of the original traces such as feature correlation, statistical moments, and novelty. Our results show that we beat both traditional statistical distribution fitting approaches as well as a state-of-the-art GAN time series generator across these metrics. The ability of our GAN model to generate any number of user traces regardless of the number of users in the original trace also makes our tools more practically applicable compared to previous GAN approaches. Furthermore, we present a use case where our tools were employed in a Wi-Fi research experiment.

cs.NI↗

arXiv2021

Reinforcement Learning for Standards Design

Communications standards are designed via committees of humans holding repeated meetings over months or even years until consensus is achieved. This includes decisions regarding the modulation and coding schemes to be supported over an air interface. We propose a way to "automate" the selection of the set of modulation and coding schemes to be supported over a given air interface and thereby streamline both the standards design process and the ease of extending the standard to support new modulation schemes applicable to new higher-level applications and services. Our scheme involves machine learning, whereby a constructor entity submits proposals to an evaluator entity, which returns a score for the proposal. The constructor employs reinforcement learning to iterate on its submitted proposals until a score is achieved that was previously agreed upon by both constructor and evaluator to be indicative of satisfying the required design criteria (including performance metrics for transmissions over the interface).

stat.ML↗

arXiv2021

SAFE: Secure Aggregation with Failover and Encryption

We propose and experimentally evaluate a novel secure aggregation algorithm targeted at cross-organizational federated learning applications with a fixed set of participating learners. Our solution organizes learners in a chain and encrypts all traffic to reduce the controller of the aggregation to a mere message broker. We show that our algorithm scales better and is less resource demanding than existing solutions, while being easy to implement on constrained platforms. With 36 nodes our method outperforms state-of-the-art secure aggregation by 70x, and 56x with and without failover, respectively.

cs.DC↗

arXiv2021

Bandcoin: Using Smart Contracts to Automate Mobile Network Bandwidth Roaming Agreements

We propose a new way to share licensed spectrum bandwidth capacity in mobile networks between operators, service providers and consumers using blockchain-based smart contracts. We discuss the foundational building blocks in the contract as well as various extensions to support more advanced features such as bulk purchases, future reservations, and various auction mechanisms. Furthermore, we demonstrate how the system can be implemented with an open-source, permissioned Enterprise blockchain, Hyperledger Sawtooth. We show that our smart contract implementation can improve blockchain transaction performance, by approximately four orders of magnitude compared to serial transactions and one order of magnitude compared to parallell transactions, using PKI-driven bulk purchases of mobile access grants, paving the way for fully automated, efficient, and fine-grained roaming agreements.

cs.NI↗

arXiv2021

A Multi-Armed Bandit-based Approach to Mobile Network Provider Selection

We argue for giving users the ability to lease bandwidth temporarily from any mobile network operator. We propose, prototype, and evaluate a spectrum market for mobile network access, where multiple network operators offer blocks of bandwidth at specified prices for short-term leases to users, with autonomous agents on user devices making purchase decisions by trading off price, performance, and budget constraints. We show that the problem of provider selection can be formulated as a so-called Bandit problem. For the case where providers change prices synchronously, we approach the problem through contextual multi-armed bandits and Reinforcement Learning methods like Q-learning either applied directly to the bandit maximization problem or indirectly to approximate the Gittins indices that are known to yield the optimal provider selection policy. For a simulated scenario corresponding to a practical use case, our agent shows a $20-41\%$ QoE improvement over random provider selection under various demand, price and mobility conditions. We implemented a prototype spectrum market using LTE networks and eSIM techology and deployed it on a testbed, using a blockchain to implement the ledger where bandwidth purchase transactions are recorded. Experiments showed that we can learn both user behavior and network performance efficiently, and recorded $25-74\%$ improvements in QoE under various competing agent scenarios.

cs.NI↗

arXiv2020

Market-based Short-Term Allocations in Small Cell Wireless Networks

Mobile users (or UEs, to use 3GPP terminology) served by small cells in dense urban settings may abruptly experience a significant deterioration in their channel to their serving base stations (BSs) in several scenarios, such as after turning a corner around a tall building, or a sudden knot of traffic blocking the direct path between the UE and its serving BS. In this work, we propose a scheme to temporarily increase the data rate to/from this UE with additional bandwidth from the nearest Coordinated Multi-Point (CoMP) cluster of BSs, while the slower process of handover of the UE to a new serving BS is ongoing. We emphasize that this additional bandwidth is additional to the data rates the UE is getting over its primary connection to the current serving BS and, after the handover, to the new serving BS. The key novelty of the present work is the proposal of a decentralized market-based resource allocation method to perform resource allocation to support Coordinated Beamforming (CB) CoMP. It is scalable to large numbers of UEs and BSs, and it is fast because resource allocations are made bilaterally, between BSs and UEs. Once the resource allocation to the UE has been made, the coordinated of transmissions occurs as per the usual CB methods. Thus the proposed method has the benefit of giving the UE access to its desired amount of resources fast, without waiting for handover to complete, or reporting channel state information before it knows the resources it will be allocated for receiving transmissions from the serving BS.

cs.IT↗

arXiv2020

PPD: Permutation Phase Defense Against Adversarial Examples in Deep Learning

Deep neural networks have demonstrated cutting edge performance on various tasks including classification. However, it is well known that adversarially designed imperceptible perturbation of the input can mislead advanced classifiers. In this paper, Permutation Phase Defense (PPD), is proposed as a novel method to resist adversarial attacks. PPD combines random permutation of the image with phase component of its Fourier transform. The basic idea behind this approach is to turn adversarial defense problems analogously into symmetric cryptography, which relies solely on safekeeping of the keys for security. In PPD, safe keeping of the selected permutation ensures effectiveness against adversarial attacks. Testing PPD on MNIST and CIFAR-10 datasets yielded state-of-the-art robustness against the most powerful adversarial attacks currently available.

cs.LG↗

arXiv2019

General Information Bottleneck Objectives and their Applications to Machine Learning

We view the Information Bottleneck Principle (IBP: Tishby et al., 1999; Schwartz-Ziv and Tishby, 2017) and Predictive Information Bottleneck Principle (PIBP: Still et al., 2007; Alemi, 2019) as special cases of a family of general information bottleneck objectives (IBOs). Each IBO corresponds to a particular constrained optimization problem where the constraints apply to: (a) the mutual information between the training data and the learned model parameters or extracted representation of the data, and (b) the mutual information between the learned model parameters or extracted representation of the data and the test data (if any). The heuristics behind the IBP and PIBP are shown to yield different constraints in the corresponding constrained optimization problem formulations. We show how other heuristics lead to a new IBO, different from both the IBP and PIBP, and use the techniques from (Alemi, 2019) to derive and optimize a variational upper bound on the new IBO. We then apply the theory of general IBOs to resolve the seeming contradiction between, on the one hand, the recommendations of IBP and PIBP to maximize the mutual information between the model parameters and test data, and on the other, recent information-theoretic results (see Xu and Raginsky, 2017) suggesting that this mutual information should be minimized. The key insight is that the heuristics (and thus the constraints in the constrained optimization problems) of IBP and PIBP are not applicable to the scenario analyzed by (Xu and Raginsky, 2017) because the latter makes the additional assumption that the parameters of the trained model have been selected to minimize the empirical loss function. Aided by this insight, we formulate a new IBO that accounts for this property of the parameters of the trained model, and derive and optimize a variational bound on this IBO.

cs.LG↗

arXiv2019

Machine Learning using the Variational Predictive Information Bottleneck with a Validation Set

Zellner (1988) modeled statistical inference in terms of information processing and postulated the Information Conservation Principle (ICP) between the input and output of the information processing block, showing that this yielded Bayesian inference as the optimum information processing rule. Recently, Alemi (2019) reviewed Zellner's work in the context of machine learning and showed that the ICP could be seen as a special case of a more general optimum information processing criterion, namely the Predictive Information Bottleneck Objective. However, Alemi modeled the model training step in machine learning as using training and test data sets only, and did not account for the use of a validation data set during training. The present note is an attempt to extend Alemi's information processing formulation of machine learning, and the predictive information bottleneck objective for model training, to the widely-used scenario where training utilizes not only a training but also a validation data set.

cs.LG↗

arXiv2014

Capacity Analysis of LTE-Advanced HetNets with Reduced Power Subframes and Range Expansion

The time domain inter-cell interference coordination techniques specified in LTE Rel. 10 standard improves the throughput of picocell-edge users by protecting them from macrocell interference. On the other hand, it also degrades the aggregate capacity in macrocell because the macro base station (MBS) does not transmit data during certain subframes known as almost blank subframes. The MBS data transmission using reduced power subframes was standardized in LTE Rel. 11, which can improve the capacity in macrocell while not causing high interference to the nearby picocells. In order to get maximum benefit from the reduced power subframes, setting the key system parameters, such as the amount of power reduction, carries critical importance. Using stochastic geometry, this paper lays down a theoretical foundation for the performance evaluation of heterogeneous networks with reduced power subframes and range expansion bias. The analytic expressions for average capacity and 5th percentile throughput are derived as a function of transmit powers, node densities, and interference coordination parameters in a heterogeneous network scenario, and are validated through Monte Carlo simulations. Joint optimization of range expansion bias, power reduction factor, scheduling thresholds, and duty cycle of reduced power subframes are performed to study the trade-offs between aggregate capacity of a cell and fairness among the users. To validate our analysis, we also compare the stochastic geometry based theoretical results with the real MBS deployment (in the city of London) and the hexagonal-grid model. Our analysis shows that with optimum parameter settings, the LTE Rel. 11 with reduced power subframes can provide substantially better performance than the LTE Rel. 10 with almost blank subframes, in terms of both aggregate capacity and fairness.

cs.IT↗