Searcharxiv⌕ Search

arXiv subjects

Vikram Krishnamurthy

Publications and source records attributed to Vikram Krishnamurthy.

At least 109 records · Page 6Linked to original sources
arXiv2018

Syntactic Enhancement to VSIMM for Roadmap Based Anomalous Trajectory Detection: A Natural Language Processing Approach

The aim of syntactic tracking is to classify spatio-temporal patterns of a target's motion using natural language processing models. In this paper, we generalize earlier work by considering a constrained stochastic context free grammar (CSCFG) for modeling patterns confined to a roadmap. The constrained grammar facilitates modeling specific directions and road names in a roadmap. We present a novel particle filtering algorithm that exploits the CSCFG model for estimating the target's patterns. This meta-level algorithm operates in conjunction with a base-level tracking algorithm. Extensive numerical results using simulated ground moving target indicator (GMTI) radar measurements show substantial improvement in target tracking accuracy.

cs.CL↗
arXiv2017

Dependence Structure Analysis Of Meta-level Metrics in YouTube Videos: A Vine Copula Approach

This paper uses vine copula to analyze the multivariate statistical dependence in a massive YouTube dataset consisting of 6 million videos over 25 thousand channels. Specifically we study the statistical dependency of 7 YouTube meta-level metrics: view count, number of likes, number of comments, length of video title, number of subscribers, click rates, and average percentage watching. Dependency parameters such as the Kendall's tau and tail dependence coefficients are computed to evaluate the pair-wise dependence of these meta-level metrics. The vine copula model yields several interesting dependency structures. We show that view count and number of likes' are in the central position of the dependence structure. Conditioned on these two metrics, the other five meta-level metrics are virtually independent of each other. Also, Sports, Gaming, Fashion, Comedy videos have similar dependence structure to each other, while the News category exhibits a strong tail dependence. We also study Granger causality effects and upload dynamics and their impact on view count. Our findings provide a useful understanding of user engagement in YouTube.

cs.SI↗
arXiv2017

Controlled Information Fusion with Risk-Averse CVaR Social Sensors

Consider a multi-agent network comprised of risk averse social sensors and a controller that jointly seek to estimate an unknown state of nature, given noisy measurements. The network of social sensors perform Bayesian social learning - each sensor fuses the information revealed by previous social sensors along with its private valuation using Bayes' rule - to optimize a local cost function. The controller sequentially modifies the cost function of the sensors by discriminatory pricing (control inputs) to realize long term global objectives. We formulate the stochastic control problem faced by the controller as a Partially Observed Markov Decision Process (POMDP) and derive structural results for the optimal control policy as a function of the risk-aversion factor in the Conditional Value-at-Risk (CVaR) cost function of the sensors. We show that the optimal price sequence when the sensors are risk- averse is a super-martingale; i.e, it decreases on average over time.

math.OC↗
arXiv2017

Multiple Stopping Time POMDPs: Structural Results & Application in Interactive Advertising in Social Media

This paper considers a multiple stopping time problem for a Markov chain observed in noise, where a decision maker chooses at most L stopping times to maximize a cumulative objective. We formulate the problem as a Partially Observed Markov Decision Process (POMDP) and derive structural results for the optimal multiple stopping policy. The main results are as follows: i) The optimal multiple stopping policy is shown to be characterized by threshold curves in the unit simplex of Bayesian Posteriors. ii) The stopping setsl (defined by the threshold curves) are shown to exhibit a nested structure. iii) The optimal cumulative reward is shown to be monotone with respect to the copositive ordering of the transition matrix. iv) A stochastic gradient algorithm is provided for estimating linear threshold policies by exploiting the structural results. These linear threshold policies approximate the threshold curves, and share the monotone structure of the optimal multiple stopping policy. As an illustrative example, we apply the multiple stopping framework to interactively schedule advertisements in live online social media. It is shown that advertisement scheduling using multiple stopping performs significantly better than currently used methods.

eess.SY↗
arXiv2017

Influence Maximization over Markovian Graphs: A Stochastic Optimization Approach

This paper considers the problem of randomized influence maximization over a Markovian graph process: given a fixed set of nodes whose connectivity graph is evolving as a Markov chain, estimate the probability distribution (over this fixed set of nodes) that samples a node which will initiate the largest information cascade (in expectation). Further, it is assumed that the sampling process affects the evolution of the graph i.e. the sampling distribution and the transition probability matrix are functionally dependent. In this setup, recursive stochastic optimization algorithms are presented to estimate the optimal sampling distribution for two cases: 1) transition probabilities of the graph are unknown but, the graph can be observed perfectly 2) transition probabilities of the graph are known but, the graph is observed in noise. These algorithms consist of a neighborhood size estimation algorithm combined with a variance reduction method, a Bayesian filter and a stochastic gradient algorithm. Convergence of the algorithms are established theoretically and, numerical results are provided to illustrate how the algorithms work.

cs.SI↗
arXiv2017

Computing monotone policies for Markov decision processes: a nearly-isotonic penalty approach

This paper discusses algorithms for solving Markov decision processes (MDPs) that have monotone optimal policies. We propose a two-stage alternating convex optimization scheme that can accelerate the search for an optimal policy by exploiting the monotone property. The first stage is a linear program formulated in terms of the joint state-action probabilities. The second stage is a regularized problem formulated in terms of the conditional probabilities of actions given states. The regularization uses techniques from nearly-isotonic regression. While a variety of iterative method can be used in the first formulation of the problem, we show in numerical simulations that, in particular, the alternating method of multipliers (ADMM) can be significantly accelerated using the regularization step.

eess.SY↗
arXiv2017

Asymptotically Efficient Identification of Known-Sensor Hidden Markov Models

We consider estimating the transition probability matrix of a finite-state finite-observation alphabet hidden Markov model with known observation probabilities. The main contribution is a two-step algorithm; a method of moments estimator (formulated as a convex optimization problem) followed by a single iteration of a Newton-Raphson maximum likelihood estimator. The two-fold contribution of this letter is, firstly, to theoretically show that the proposed estimator is consistent and asymptotically efficient, and secondly, to numerically show that the method is computationally less demanding than conventional methods - in particular for large data sets.

eess.SY↗
arXiv2017

POMDP Structural Results for Controlled Sensing

This article provides a short review of some structural results in controlled sensing when the problem is formulated as a partially observed Markov decision process. In particular, monotone value functions, Blackwell dominance and quickest detection are described.

eess.SY↗
arXiv2016

Utility Change Point Detection in Online Social Media: A Revealed Preference Framework

This paper deals with change detection of utility maximization behaviour in online social media. Such changes occur due to the effect of marketing, advertising, or changes in ground truth. First, we use the revealed preference framework to detect the unknown time point (change point) at which the utility function changed. We derive necessary and sufficient conditions for detecting the change point. Second, in the presence of noisy measurements, we propose a method to detect the change point and construct a decision test. Also, an optimization criteria is provided to recover the linear perturbation coefficients. Finally, to reduce the computational cost, a dimensionality reduction algorithm using Johnson-Lindenstrauss transform is presented. The results developed are illustrated on two real datasets: Yahoo! Tech Buzz dataset and Youstatanalyzer dataset. By using the results developed in the paper, several useful insights can be gleaned from these data sets. First, the changes in ground truth affecting the utility of the agent can be detected by utility maximization behaviour in online search. Second, the recovered utility functions satisfy the single crossing property indicating strategic substitute behaviour in online search. Third, due to the large number of videos in YouTube, the utility maximization behaviour was verified through the dimensionality reduction algorithm. Finally, using the utility function recovered in the lower dimension, we devise an algorithm to predict total traffic in YouTube.

math.ST↗
arXiv2016

Engagement dynamics and sensitivity analysis of YouTube videos

YouTube, with millions of content creators, has become the preferred destination for watching videos online. Through the Partner program, YouTube allows content creators to monetize their popular videos. Of significant importance for content creators is which meta-level features (e.g. title, tag, thumbnail) are most sensitive for promoting video popularity. The popularity of videos also depends on the social dynamics, i.e. the interaction of the content creators (or channels) with YouTube users. Using real-world data consisting of about 6 million videos spread over 25 thousand channels, we empirically examine the sensitivity of YouTube meta-level features and social dynamics. The key meta-level features that impact the view counts of a video include: first day view count , number of subscribers, contrast of the video thumbnail, Google hits, number of keywords, video category, title length, and number of upper-case letters in the title respectively and illustrate that these meta-level features can be used to estimate the popularity of a video. In addition, optimizing the meta-level features after a video is posted increases the popularity of videos. In the context of social dynamics, we discover that there is a causal relationship between views to a channel and the associated number of subscribers. Additionally, insights into the effects of scheduling and video playthrough in a channel are also provided. Our findings provide a useful understanding of user engagement in YouTube.

cs.SI↗
arXiv2016

Opportunistic Advertisement Scheduling in Live Social Media: A Multiple Stopping Time POMDP Approach

Live online social broadcasting services like YouTube Live and Twitch have steadily gained popularity due to improved bandwidth, ease of generating content and the ability to earn revenue on the generated content. In contrast to traditional cable television, revenue in online services is generated solely through advertisements, and depends on the number of clicks generated. Channel owners aim to opportunistically schedule advertisements so as to generate maximum revenue. This paper considers the problem of optimal scheduling of advertisements in live online social media. The problem is formulated as a multiple stopping problem and is addressed in a partially observed Markov decision process (POMDP) framework. Structural results are provided on the optimal advertisement scheduling policy. By exploiting the structure of the optimal policy, best linear thresholds are computed using stochastic approximation. The proposed model and framework are validated on real datasets, and the following observations are made: (i) The policy obtained by the multiple stopping problem can be used to detect changes in ground truth from online search data (ii) Numerical results show a significant improvement in the expected revenue by opportunistically scheduling the advertisements. The revenue can be improved by $20-30\%$ in comparison to currently employed periodic scheduling.

cs.SI↗
arXiv2016

Tracking Infection Diffusion in Social Networks: Filtering Algorithms and Threshold Bounds

This paper deals with the statistical signal pro- cessing over graphs for tracking infection diffusion in social networks. Infection (or Information) diffusion is modeled using the Susceptible-Infected-Susceptible (SIS) model. Mean field approximation is employed to approximate the discrete valued infected degree distribution evolution by a deterministic ordinary differential equation for obtaining a generative model for the infection diffusion. The infected degree distribution is shown to follow polynomial dynamics and is estimated using an exact non- linear Bayesian filter. We compute posterior Cramer-Rao bounds to obtain the fundamental limits of the filter which depend on the structure of the network. Considering the time-varying nature of the real world networks, the relationship between the diffusion thresholds and the degree distribution is investigated using generative models for real world networks. In addition, we validate the efficacy of our method with the diffusion data from a real-world online social system, Twitter. We find that SIS model is a good fit for the information diffusion and the non-linear filter effectively tracks the information diffusion.

cs.SI↗
arXiv2016

A Comprehensive Methodology for Volt-VAR Optimization in Active Smart Grids

This paper considers the problem of Volt-VAR Optimization (VVO) in active smart grids. Active smart grids are equipped with distributed generators, distributed storage systems, and tie-line switches that allow for topological reconfiguration. In this paper, the joint operation of the remotely controllable switches, storage units, under load tap changers, and shunt capacitors is formulated for a day-ahead operation scheme. The proposed VVO problem aims at minimizing the expected active power loss in the system and is formulated as a mixed-integer quadratic program. The stochasticity of the wind power generation is addressed through a first-order Markov chain model. Numerical results on a 33-node, 12.66 kV active smart grid using real data from smart meters and wind turbines are presented and compared for various test cases.

math.OC↗
arXiv2016

Partially Observed Markov Decision Processes. Problem Sets and Internet Supplement

This document is an internet supplement to my book "Partially Observed Markov Decision Processes - From Filtering to Controlled Sensing" published by Cambridge University Press in 2016. This internet supplement contains exercises, examples and case studies. The material appears in this internet supplement (instead of the book) so that it can be updated. This document will evolve over time and further discussion and examples will be added. This internet supplement document is work in progress and will be updated periodically. I welcome constructive comments from readers of the book and this internet supplement.

math.OC↗
arXiv2016

Filterbased Stochastic Volatility in Continuous-Time Hidden Markov Models

Regime-switching models, in particular Hidden Markov Models (HMMs) where the switching is driven by an unobservable Markov chain, are widely-used in financial applications, due to their tractability and good econometric properties. In this work we consider HMMs in continuous time with both constant and switching volatility. In the continuous-time model with switching volatility the underlying Markov chain could be observed due to this stochastic volatility, and no estimation (filtering) of it is needed (in theory), while in the discretized model or the model with constant volatility one has to filter for the underlying Markov chain. The motivations for continuous-time models are explicit computations in finance. To have a realistic model with unobservable Markov chain in continuous time and good econometric properties we introduce a regime-switching model where the volatility depends on the filter for the underlying chain and state the filtering equations. We prove an approximation result for a fixed information filtration and further motivate the model by considering social learning arguments. We analyze its relation to the switching volatility model and present a convergence result for the discretized model. We then illustrate its econometric properties by considering numerical simulations.

q-fin.ST↗
arXiv2015

Structural Results for Partially Observed Markov Decision Processes

This article provides an introductory tutorial on structural results in partially observed Markov decision processes (POMDPs). Typically, computing the optimal policy of a POMDP is computationally intractable. We use lattice program- ming methods to characterize the structure of the optimal policy of a POMDP without brute force computations.

math.OC↗
arXiv2015

Myopic Bounds for Optimal Policy of POMDPs: An extension of Lovejoy's structural results

This paper provides a relaxation of the sufficient conditions, and also an extension of the structural results for Partially Observed Markov Decision Processes (POMDPs) given in Lovejoy (1987). Sufficient conditions are provided so that the optimal policy can be upper and lower bounded by judiciously chosen myopic policies. These myopic policy bounds are constructed to maximize the volume of belief states where they coincide with the optimal policy. Numerical examples illustrate these myopic bounds for both continuous and discrete observation sets.

math.OC↗