arXiv subjects

Lav R. Varshney

Publications and source records attributed to Lav R. Varshney.

At least 109 records · Page 6Linked to original sources

arXiv2020

Boosting Classifiers with Noisy Inference

We present a principled framework to address resource allocation for realizing boosting algorithms on substrates with communication or computation noise. Boosting classifiers (e.g., AdaBoost) make a final decision via a weighted vote from the outputs of many base classifiers (weak classifiers). Suppose that the base classifiers' outputs are noisy or communicated over noisy channels; these noisy outputs will degrade the final classification accuracy. We show that this degradation can be effectively reduced by allocating more system resources for more important base classifiers. We formulate resource optimization problems in terms of importance metrics for boosting. Moreover, we show that the optimized noisy boosting classifiers can be more robust than bagging for the noise during inference (test stage). We provide numerical evidence to demonstrate the benefits of our approach.

cs.LG↗

arXiv2020

Social Bubbles and Superspreaders: Source Identification for Contagion Processes on Hypertrees

Previous work has shown that for contagion processes on extended star networks (trees with exactly one node of degree > 2), there is a simple, closed-form expression for a highly accurate approximation to the maximum likelihood infection source. Here, we generalize that result to a class of hypertrees which, although somewhat structurally analogous, provides a much richer representation space. In particular, this approach can be used to estimate patient zero sources, even when the infection has been propagated via large group gatherings rather than person-to-person spread, and when it is spreading through interrelated social bubbles with varying degrees of overlap. In contact tracing contexts, this estimator may be used to identify the source of a local outbreak, which can then be used for forward tracing or for further backward tracing (by similar or other means) to an upstream source.

eess.SP↗

arXiv2020

Explaining Creative Artifacts

Human creativity is often described as the mental process of combining associative elements into a new form, but emerging computational creativity algorithms may not operate in this manner. Here we develop an inverse problem formulation to deconstruct the products of combinatorial and compositional creativity into associative chains as a form of post-hoc interpretation that matches the human creative process. In particular, our formulation is structured as solving a traveling salesman problem through a knowledge graph of associative elements. We demonstrate our approach using an example in explaining culinary computational creativity where there is an explicit semantic structure, and two examples in language generation where we either extract explicit concepts that map to a knowledge graph or we consider distances in a word embedding space. We close by casting the length of an optimal traveling salesman path as a measure of novelty in creativity.

cs.AI↗

arXiv2020

Respect for Human Autonomy in Recommender Systems

Recommender systems can influence human behavior in significant ways, in some cases making people more machine-like. In this sense, recommender systems may be deleterious to notions of human autonomy. Many ethical systems point to respect for human autonomy as a key principle arising from human rights considerations, and several emerging frameworks for AI include this principle. Yet, no specific formalization has been defined. Separately, self-determination theory shows that autonomy is an innate psychological need for people, and moreover has a significant body of experimental work that formalizes and measures level of human autonomy. In this position paper, we argue that there is a need to specifically operationalize respect for human autonomy in the context of recommender systems. Moreover, that such an operational definition can be developed based on well-established approaches from experimental psychology, which can then be used to design future recommender systems that respect human autonomy.

cs.CY↗

arXiv2020

On Multiple-Access in Queue-Length Sensitive Systems

We consider transmission of packets over queue-length sensitive unreliable links, where packets are randomly corrupted through a noisy channel whose transition probabilities are modulated by the queue-length. The goal is to characterize the capacity of this channel. We particularly consider multiple-access systems, where transmitters dispatch encoded symbols over a system that is a superposition of continuous-time $GI_k/GI/1$ queues. A server receives and processes symbols in order of arrivals with queue-length dependent noise. We first determine the capacity of single-user queue-length dependent channels. Further, we characterize the best and worst dispatch processes for $GI/M/1$ queues and the best and worst service processes for $M/GI/1$ queues. Then, the multiple-access channel capacity is obtained using point processes. When the number of transmitters is large and each arrival process is sparse, the superposition of arrivals approaches a Poisson point process. In characterizing the Poisson approximation, we show that the capacity of the multiple-access system converges to that of a single-user $M/GI/1$ queue-length dependent system, and an upper bound on the convergence rate is obtained. This implies that the best and worst server behaviors of single-user $M/GI/1$ queues are preserved in the sparse multiple-access case.

cs.IT↗

arXiv2020

Quantization Games on Social Networks and Language Evolution

We consider a strategic network quantizer design setting where agents must balance fidelity in representing their local source distributions against their ability to successfully communicate with other connected agents. We study the problem as a network game and show existence of Nash equilibrium quantizers. For any agent, under Nash equilibrium, the word representing a given partition region is the conditional expectation of the mixture of local and social source probability distributions within the region. Since having knowledge of the original source of information in the network may not be realistic, we show that under certain conditions, the agents need not know the source origin and yet still settle on a Nash equilibrium using only the observed sources. Further, the network may converge to equilibrium through a distributed version of the Lloyd-Max algorithm. In contrast to traditional results in the evolution of language, we find several vocabularies may coexist in the Nash equilibrium, with each individual having exactly one of these vocabularies. The overlap between vocabularies is high for individuals that communicate frequently and have similar local sources. Finally, we argue that error in translation along a chain of communication does not grow if and only if the chain consists of agents with shared vocabulary. Numerical results are given.

cs.IT↗

arXiv2020

A Game-Theoretic Approach to a Task Delegation Problem

We study a setting in which a principal selects an agent to execute a collection of tasks according to a specified priority sequence. Agents, however, have their own individual priority sequences according to which they wish to execute the tasks. There is information asymmetry since each priority sequence is private knowledge for the individual agent. We design a mechanism for selecting the agent and incentivizing the selected agent to realize a priority sequence for executing the tasks that achieves socially optimal performance. Our proposed mechanism consists of two parts. First, the principal runs an auction to select an agent to allocate tasks to with minimum declared priority sequence misalignment. Then, the principal rewards the agent according to the realized priority sequence with which the tasks were performed. We show that the proposed mechanism is individually rational and incentive compatible. Further, it is also socially optimal for the case of linear cost of priority sequence modification for the agents.

cs.GT↗

arXiv2020

A Difficulty in Controlling Blockchain Mining Costs via Cryptopuzzle Difficulty

Blockchain systems often employ proof-of-work consensus protocols to validate and add transactions into hashchains. These protocols stimulate competition among miners in solving cryptopuzzles (e.g. SHA-256 hash computation in Bitcoin) in exchange for a monetary reward. Here, we model mining as an all-pay auction, where miners' computational efforts are interpreted as bids, and the allocation function is the probability of solving the cryptopuzzle in a single attempt with unit (normalized) computational capability. Such an allocation function captures how blockchain systems control the difficulty of the cryptopuzzle as a function of miners' computational abilities (bids). In an attempt to reduce mining costs, we investigate designing a mining auction mechanism which induces a logit equilibrium amongst the miners with choice distributions that are unilaterally decreasing with costs at each miner. We show it is impossible to design a lenient allocation function that does this. Specifically, we show that there exists no allocation function that discourages miners to bid higher costs at logit equilibrium, if the rate of change of difficulty with respect to each miner's cost is bounded by the inverse of the sum of costs at all the miners.

cs.GT↗

arXiv2020

Human Evaluation of Interpretability: The Case of AI-Generated Music Knowledge

Interpretability of machine learning models has gained more and more attention among researchers in the artificial intelligence (AI) and human-computer interaction (HCI) communities. Most existing work focuses on decision making, whereas we consider knowledge discovery. In particular, we focus on evaluating AI-discovered knowledge/rules in the arts and humanities. From a specific scenario, we present an experimental procedure to collect and assess human-generated verbal interpretations of AI-generated music theory/rules rendered as sophisticated symbolic/numeric objects. Our goal is to reveal both the possibilities and the challenges in such a process of decoding expressive messages from AI sources. We treat this as a first step towards 1) better design of AI representations that are human interpretable and 2) a general methodology to evaluate interpretability of AI-discovered knowledge representations.

cs.HC↗

arXiv2020

Energy-Reliability Limits in Nanoscale Feedforward Neural Networks and Formulas

Due to energy-efficiency requirements, computational systems are now being implemented using noisy nanoscale semiconductor devices whose reliability depends on energy consumed. We study circuit-level energy-reliability limits for deep feedforward neural networks (multilayer perceptrons) built using such devices, and en route also establish the same limits for formulas (boolean tree-structured circuits). To obtain energy lower bounds, we extend Pippenger's mutual information propagation technique for characterizing the complexity of noisy circuits, since small circuit complexity need not imply low energy. Many device technologies require all gates to have the same electrical operating point; in circuits of such uniform gates, we show that the minimum energy required to achieve any non-trivial reliability scales superlinearly with the number of inputs. Circuits implemented in emerging device technologies like spin electronics can, however, have gates operate at different electrical points; in circuits of such heterogeneous gates, we show energy scaling can be linear in the number of inputs. Building on our extended mutual information propagation technique and using crucial insights from convex optimization theory, we develop an algorithm to compute energy lower bounds for any given boolean tree under heterogeneous gates. This algorithm runs in linear time in number of gates, and is therefore practical for modern circuit design. As part of our development we find a simple procedure for energy allocation across circuit gates with different operating points and neural networks with differently-operating layers.

cs.IT↗

arXiv2020

Classes of Full-Duplex Channels with Capacity Achieved Without Adaptation

Full-duplex communication allows a terminal to transmit and receive signals simultaneously, and hence, it is helpful in general to adapt transmissions to received signals. However, this often requires unaffordable complexity. This work focuses on simple non-adaptive transmission, and provides two classes of channels for which Shannon's information capacity regions are achieved without adaptation. The first is the injective semi-deterministic two-way channel that includes additive channels with various types of noises modeling wireless, coaxial cable, and other settings. The other is the Poisson two-way channel, for which we show that non-adaptive transmission is asymptotically optimal in the high dark current regime.

cs.IT↗

arXiv2020

Orbit Computation for Atomically Generated Subgroups of Isometries of $\mathbb{Z}^n$

Isometries are ubiquitous in nature; isometries of discrete (quantized) objects---abstracted as the group of isometries of $\mathbb{Z}^n$ denoted by $\mathsf{ISO}(\mathbb{Z}^n)$---are important concepts in the computational world. In this paper, we compute various isometric invariances which mathematically are orbit-computation problems under various isometry-subgroup actions $H \curvearrowright \mathbb{Z}^n, H \leq \mathsf{ISO}(\mathbb{Z}^n)$. One computational challenge here is about the \emph{infinite}: in general, we can have an infinite subgroup acting on $\mathbb{Z}^n$, resulting in possibly an infinite number of orbits of possibly infinite size. In practice, we restrict the set of orbits (a partition of $\mathbb{Z}^n$) to a finite subset $Z \subseteq \mathbb{Z}^n$ (a partition of $Z$), where $Z$ is specified a priori by an application domain or a data set. Our main contribution is an efficient algorithm to solve this \emph{restricted} orbit-computation problem in the special case of \emph{atomically generated subgroups}---a new notion partially motivated from interpretable AI. The atomic property is key to preserving the \emph{semidirect-product structure}---the core structure we leverage to make our algorithm outperform generic approaches. Besides algorithmic merit, our approach enables \emph{parallel-computing} implementations in many subroutines, which can further benefit from hardware boosts. Moreover, our algorithm works efficiently for \emph{any} finite subset ($Z$) regardless of the shape (continuous/discrete, (non)convex) or location; so it is application-independent.

math.GR↗

arXiv2020

Universal and Succinct Source Coding of Deep Neural Networks

Deep neural networks have shown incredible performance for inference tasks in a variety of domains. Unfortunately, most current deep networks are enormous cloud-based structures that require significant storage space, which limits scaling of deep learning as a service (DLaaS) and use for on-device intelligence. This paper is concerned with finding universal lossless compressed representations of deep feedforward networks with synaptic weights drawn from discrete sets, and directly performing inference without full decompression. The basic insight that allows less rate than naive approaches is recognizing that the bipartite graph layers of feedforward networks have a kind of permutation invariance to the labeling of nodes, in terms of inferential operation. We provide efficient algorithms to dissipate this irrelevant uncertainty and then use arithmetic coding to nearly achieve the entropy bound in a universal manner. We also provide experimental results of our approach on several standard datasets.

cs.IT↗

arXiv2020

RecipeGPT: Generative Pre-training Based Cooking Recipe Generation and Evaluation System

Interests in the automatic generation of cooking recipes have been growing steadily over the past few years thanks to a large amount of online cooking recipes. We present RecipeGPT, a novel online recipe generation and evaluation system. The system provides two modes of text generations: (1) instruction generation from given recipe title and ingredients; and (2) ingredient generation from recipe title and cooking instructions. Its back-end text generation module comprises a generative pre-trained language model GPT-2 fine-tuned on a large cooking recipe dataset. Moreover, the recipe evaluation module allows the users to conveniently inspect the quality of the generated recipe contents and store the results for future reference. RecipeGPT can be accessed online at https://recipegpt.org/.

cs.CL↗

arXiv2020

A Near-Optimal Change-Detection Based Algorithm for Piecewise-Stationary Combinatorial Semi-Bandits

We investigate the piecewise-stationary combinatorial semi-bandit problem. Compared to the original combinatorial semi-bandit problem, our setting assumes the reward distributions of base arms may change in a piecewise-stationary manner at unknown time steps. We propose an algorithm, \texttt{GLR-CUCB}, which incorporates an efficient combinatorial semi-bandit algorithm, \texttt{CUCB}, with an almost parameter-free change-point detector, the \emph{Generalized Likelihood Ratio Test} (GLRT). Our analysis shows that the regret of \texttt{GLR-CUCB} is upper bounded by $\mathcal{O}(\sqrt{NKT\log{T}})$, where $N$ is the number of piecewise-stationary segments, $K$ is the number of base arms, and $T$ is the number of time steps. As a complement, we also derive a nearly matching regret lower bound on the order of $Ω(\sqrt{NKT}$), for both piecewise-stationary multi-armed bandits and combinatorial semi-bandits, using information-theoretic techniques and judiciously constructed piecewise-stationary bandit instances. Our lower bound is tighter than the best available regret lower bound, which is $Ω(\sqrt{T})$. Numerical experiments on both synthetic and real-world datasets demonstrate the superiority of \texttt{GLR-CUCB} compared to other state-of-the-art algorithms.

cs.LG↗

arXiv2020

Nearly Optimal Algorithms for Piecewise-Stationary Cascading Bandits

Cascading bandit (CB) is a popular model for web search and online advertising, where an agent aims to learn the $K$ most attractive items out of a ground set of size $L$ during the interaction with a user. However, the stationary CB model may be too simple to apply to real-world problems, where user preferences may change over time. Considering piecewise-stationary environments, two efficient algorithms, \texttt{GLRT-CascadeUCB} and \texttt{GLRT-CascadeKL-UCB}, are developed and shown to ensure regret upper bounds on the order of $\mathcal{O}(\sqrt{NLT\log{T}})$, where $N$ is the number of piecewise-stationary segments, and $T$ is the number of time slots. At the crux of the proposed algorithms is an almost parameter-free change-point detector, the generalized likelihood ratio test (GLRT). Comparing with existing works, the GLRT-based algorithms: i) are free of change-point-dependent information for choosing parameters; ii) have fewer tuning parameters; iii) improve at least the $L$ dependence in regret upper bounds. In addition, we show that the proposed algorithms are optimal (up to a logarithm factor) in terms of regret by deriving a minimax lower bound on the order of $Ω(\sqrt{NLT})$ for piecewise-stationary CB. The efficiency of the proposed algorithms relative to state-of-the-art approaches is validated through numerical experiments on both synthetic and real-world datasets.

cs.LG↗

arXiv2020

Limits of Detecting Text Generated by Large-Scale Language Models

Some consider large-scale language models that can generate long and coherent pieces of text as dangerous, since they may be used in misinformation campaigns. Here we formulate large-scale language model output detection as a hypothesis testing problem to classify text as genuine or generated. We show that error exponents for particular language models are bounded in terms of their perplexity, a standard measure of language generation performance. Under the assumption that human language is stationary and ergodic, the formulation is extended from considering specific language models to considering maximum likelihood language models, among the class of k-order Markov approximations; error probabilities are characterized. Some discussion of incorporating semantic side information is also given.

cs.CL↗

arXiv2020

Finite-Sample Analysis of Image Registration

We study the problem of image registration in the finite-resolution regime and characterize the error probability of algorithms as a function of properties of the transformation and the image capture noise. Specifically, we define a channel-aware Feinstein decoder to obtain upper bounds on the minimum achievable error probability under finite resolution. We specifically focus on the higher-order terms and use Berry-Esseen type CLTs to obtain a stronger characterization of the achievability condition for the problem. Then, we derive a strong type-counting result to characterize the performance of the MMI decoder in terms of the maximum likelihood decoder, in a simplified setting of the problem. We then describe how this analysis, when related to the results from the channel-aware context provide stronger characterization of the finite-sample performance of universal image registration.

cs.IT↗