arXiv subjects

Yury Maximov

Publications and source records attributed to Yury Maximov.

At least 37 records · Page 2Linked to original sources

arXiv2021

Power Grid Reliability Estimation via Adaptive Importance Sampling

Electricity production currently generates approximately 25% of greenhouse gas emissions in the USA. Thus, increasing the amount of renewable energy is a key step to carbon neutrality. However, integrating a large amount of fluctuating renewable generation is a significant challenge for power grid operating and planning. Grid reliability, i.e., an ability to meet operational constraints under power fluctuations, is probably the most important of them. In this paper, we propose computationally efficient and accurate methods to estimate the probability of failure, i.e. reliability constraints violation, under a known distribution of renewable energy generation. To this end, we investigate an importance sampling approach, a flexible extension of Monte-Carlo methods, which adaptively changes the sampling distribution to generate more samples near the reliability boundary. The approach allows to estimate failure probability in real-time based only on a few dozens of random samples, compared to thousands required by the plain Monte-Carlo. Our study focuses on high voltage direct current power transmission grids with linear reliability constraints on power injections and line currents. We propose a novel theoretically justified physics-informed adaptive importance sampling algorithm and compare its performance to state-of-the-art methods on multiple IEEE power grid test cases.

math.OC↗

arXiv2020

Efficient Numerical Methods to Solve Sparse Linear Equations with Application to PageRank

In this paper, we propose three methods to solve the PageRank problem for the transition matrices with both row and column sparsity. Our methods reduce the PageRank problem to the convex optimization problem over the simplex. The first algorithm is based on the gradient descent in L1 norm instead of the Euclidean one. The second algorithm extends the Frank-Wolfe to support sparse gradient updates. The third algorithm stands for the mirror descent algorithm with a randomized projection. We proof converges rates for these methods for sparse problems as well as numerical experiments support their effectiveness.

math.OC↗

arXiv2020

Learning over no-Preferred and Preferred Sequence of items for Robust Recommendation

In this paper, we propose a theoretically founded sequential strategy for training large-scale Recommender Systems (RS) over implicit feedback, mainly in the form of clicks. The proposed approach consists in minimizing pairwise ranking loss over blocks of consecutive items constituted by a sequence of non-clicked items followed by a clicked one for each user. We present two variants of this strategy where model parameters are updated using either the momentum method or a gradient-based approach. To prevent from updating the parameters for an abnormally high number of clicks over some targeted items (mainly due to bots), we introduce an upper and a lower threshold on the number of updates for each user. These thresholds are estimated over the distribution of the number of blocks in the training set. The thresholds affect the decision of RS and imply a shift over the distribution of items that are shown to the users. Furthermore, we provide a convergence analysis of both algorithms and demonstrate their practical efficiency over six large-scale collections, both regarding different ranking measures and computational time.

cs.IR↗

arXiv2020

Gauges, Loops, and Polynomials for Partition Functions of Graphical Models

Graphical models represent multivariate and generally not normalized probability distributions. Computing the normalization factor, called the partition function, is the main inference challenge relevant to multiple statistical and optimization applications. The problem is of an exponential complexity with respect to the number of variables. In this manuscript, aimed at approximating the PF, we consider Multi-Graph Models where binary variables and multivariable factors are associated with edges and nodes, respectively, of an undirected multi-graph. We suggest a new methodology for analysis and computations that combines the Gauge Function technique with the technique from the field of real stable polynomials. We show that the Gauge Function has a natural polynomial representation in terms of gauges/variables associated with edges of the multi-graph. Moreover, it can be used to recover the Partition Function through a sequence of transformations allowing appealing algebraic and graphical interpretations. Algebraically, one step in the sequence consists in application of a differential operator over gauges associated with an edge. Graphically, the sequence is interpreted as a repetitive elimination of edges resulting in a sequence of models on decreasing in size graphs with the same Partition Function. Even though complexity of computing factors in the sequence models grow exponentially with the number of eliminated edges, polynomials associated with the new factors remain bi-stable if the original factors have this property. Moreover, we show that Belief Propagation estimations in the sequence do not decrease, each low-bounding the Partition Function.

cs.LG↗

arXiv2020

A Bayesian Framework for Power System Components Identification

Having actual models for power system components (such as generators and loads or auxiliary equipment) is vital to correctly assess the power system operating state and to establish stability margins. However, power system operators often have limited information about the actual values for power system component parameters. Even when a model is available, its operating parameters and control settings are time-dependent and subject to real-time identification. Ideally, these parameters should be identified from measurement data, such as phasor measurement unit (PMU) signals. However, it is challenging to do this from the ambient measurements in the absence of transient dynamics since the signal-to-noise ratio (SNR) for such signals is not necessarily large. In this paper, we design a Bayesian framework for on-line identification of power system component parameters based on ambient PMU data, which has reliable performance for SNR as low as five and for certain parameters can give good estimations even for unit SNR. We support the framework with a robust and time-efficient numerical method. We illustrate the approach efficiency on a synchronous generator example.

eess.SP↗

arXiv2019

Tractable Minor-free Generalization of Planar Zero-field Ising Models

We present a new family of zero-field Ising models over $N$ binary variables/spins obtained by consecutive "gluing" of planar and $O(1)$-sized components and subsets of at most three vertices into a tree. The polynomial-time algorithm of the dynamic programming type for solving exact inference (computing partition function) and exact sampling (generating i.i.d. samples) consists in a sequential application of an efficient (for planar) or brute-force (for $O(1)$-sized) inference and sampling to the components as a black box. To illustrate the utility of the new family of tractable graphical models, we first build a polynomial algorithm for inference and sampling of zero-field Ising models over $K_{3,3}$-minor-free topologies and over $K_{5}$-minor-free topologies -- both are extensions of the planar zero-field Ising models -- which are neither genus - nor treewidth-bounded. Second, we demonstrate empirically an improvement in the approximation quality of the NP-hard problem of inference over the square-grid Ising model in a node-dependent non-zero "magnetic" field.

cs.DS↗

arXiv2019

Entropy Penalized Semidefinite Programming

Low-rank methods for semidefinite programming (SDP) have gained a lot of interest recently, especially in machine learning applications. Their analysis often involves determinant-based or Schatten-norm penalties, which are hard to implement in practice due to high computational efforts. In this paper, we propose Entropy Penalized Semi-definite programming (EP-SDP) which provides a unified framework for a wide class of penalty functions used in practice to promote a low-rank solution. We show that EP-SDP problems admit efficient numerical algorithm having (almost) linear time complexity of the gradient iteration which makes it useful for many machine learning and optimization problems. We illustrate the practical efficiency of our approach on several combinatorial optimization and machine learning problems.

math.OC↗

arXiv2019

A New Family of Tractable Ising Models

We present a new family of zero-field Ising models over N binary variables/spins obtained by consecutive "gluing" of planar and $O(1)$-sized components along with subsets of at most three vertices into a tree. The polynomial time algorithm of the dynamic programming type for solving exact inference (partition function computation) and sampling consists of a sequential application of an efficient (for planar) or brute-force (for $O(1)$-sized) inference and sampling to the components as a black box. To illustrate the utility of the new family of tractable graphical models, we first build an $O(N^{3/2})$ algorithm for inference and sampling of the K5-minor-free zero-field Ising models - an extension of the planar zero-field Ising models - which is neither genus- nor treewidth-bounded. Second, we demonstrate empirically an improvement in the approximation quality of the NP-hard problem of the square-grid Ising model (with non-zero field) inference.

cs.DS↗

arXiv2019

Inference and Sampling of $K_{33}$-free Ising Models

We call an Ising model tractable when it is possible to compute its partition function value (statistical inference) in polynomial time. The tractability also implies an ability to sample configurations of this model in polynomial time. The notion of tractability extends the basic case of planar zero-field Ising models. Our starting point is to describe algorithms for the basic case computing partition function and sampling efficiently. To derive the algorithms, we use an equivalent linear transition to perfect matching counting and sampling on an expanded dual graph. Then, we extend our tractable inference and sampling algorithms to models, whose triconnected components are either planar or graphs of $O(1)$ size. In particular, it results in a polynomial-time inference and sampling algorithms for $K_{33}$ (minor) free topologies of zero-field Ising models - a generalization of planar graphs with a potentially unbounded genus.

stat.CO↗

arXiv2019

Sequential Learning over Implicit Feedback for Robust Large-Scale Recommender Systems

In this paper, we propose a robust sequential learning strategy for training large-scale Recommender Systems (RS) over implicit feedback mainly in the form of clicks. Our approach relies on the minimization of a pairwise ranking loss over blocks of consecutive items constituted by a sequence of non-clicked items followed by a clicked one for each user. Parameter updates are discarded if for a given user the number of sequential blocks is below or above some given thresholds estimated over the distribution of the number of blocks in the training set. This is to prevent from an abnormal number of clicks over some targeted items, mainly due to bots; or very few user interactions. Both scenarios affect the decision of RS and imply a shift over the distribution of items that are shown to the users. We provide a theoretical analysis showing that in the case where the ranking loss is convex, the deviation between the loss with respect to the sequence of weights found by the proposed algorithm and its minimum is bounded. Furthermore, experimental results on five large-scale collections demonstrate the efficiency of the proposed algorithm with respect to the state-of-the-art approaches, both regarding different ranking measures and computation time.

cs.IR↗

arXiv2019

Learning a Generator Model from Terminal Bus Data

In this work we investigate approaches to reconstruct generator models from measurements available at the generator terminal bus using machine learning (ML) techniques. The goal is to develop an emulator which is trained online and is capable of fast predictive computations. The training is illustrated on synthetic data generated based on available open-source dynamical generator model. Two ML techniques were developed and tested: (a) standard vector auto-regressive (VAR) model; and (b) novel customized long short-term memory (LSTM) deep learning model. Trade-offs in reconstruction ability between computationally light but linear AR model and powerful but computationally demanding LSTM model are established and analyzed.

stat.ML↗

arXiv2018

Importance sampling the union of rare events with an application to power systems analysis

We consider importance sampling to estimate the probability $μ$ of a union of $J$ rare events $H_j$ defined by a random variable $\boldsymbol{x}$. The sampler we study has been used in spatial statistics, genomics and combinatorics going back at least to Karp and Luby (1983). It works by sampling one event at random, then sampling $\boldsymbol{x}$ conditionally on that event happening and it constructs an unbiased estimate of $μ$ by multiplying an inverse moment of the number of occuring events by the union bound. We prove some variance bounds for this sampler. For a sample size of $n$, it has a variance no larger than $μ(\barμ-μ)/n$ where $\barμ$ is the union bound. It also has a coefficient of variation no larger than $\sqrt{(J+J^{-1}-2)/(4n)}$ regardless of the overlap pattern among the $J$ events. Our motivating problem comes from power system reliability, where the phase differences between connected nodes have a joint Gaussian distribution and the $J$ rare events arise from unacceptably large phase differences. In the grid reliability problems even some events defined by $5772$ constraints in $326$ dimensions, with probability below $10^{-22}$, are estimated with a coefficient of variation of about $0.0024$ with only $n=10{,}000$ sample values.

stat.CO↗

arXiv2018

Representation Learning and Pairwise Ranking for Implicit Feedback in Recommendation Systems

In this paper, we propose a novel ranking framework for collaborative filtering with the overall aim of learning user preferences over items by minimizing a pairwise ranking loss. We show the minimization problem involves dependent random variables and provide a theoretical analysis by proving the consistency of the empirical risk minimization in the worst case where all users choose a minimal number of positive and negative items. We further derive a Neural-Network model that jointly learns a new representation of users and items in an embedded space as well as the preference relation of users over the pairs of items. The learning objective is based on three scenarios of ranking losses that control the ability of the model to maintain the ordering over the items induced from the users' preferences, as well as, the capacity of the dot-product defined in the learned embedded space to produce the ordering. The proposed model is by nature suitable for implicit feedback and involves the estimation of only very few parameters. Through extensive experiments on several real-world benchmarks on implicit data, we show the interest of learning the preference and the embedding simultaneously when compared to learning those separately. We also demonstrate that our approach is very competitive with the best state-of-the-art collaborative filtering techniques proposed for implicit feedback.

stat.ML↗

arXiv2018

Belief Propagation Min-Sum Algorithm for Generalized Min-Cost Network Flow

Belief Propagation algorithms are instruments used broadly to solve graphical model optimization and statistical inference problems. In the general case of a loopy Graphical Model, Belief Propagation is a heuristic which is quite successful in practice, even though its empirical success, typically, lacks theoretical guarantees. This paper extends the short list of special cases where correctness and/or convergence of a Belief Propagation algorithm is proven. We generalize formulation of Min-Sum Network Flow problem by relaxing the flow conservation (balance) constraints and then proving that the Belief Propagation algorithm converges to the exact result.

stat.ML↗

arXiv2018

Rademacher Complexity Bounds for a Penalized Multiclass Semi-Supervised Algorithm

We propose Rademacher complexity bounds for multiclass classifiers trained with a two-step semi-supervised model. In the first step, the algorithm partitions the partially labeled data and then identifies dense clusters containing $κ$ predominant classes using the labeled training examples such that the proportion of their non-predominant classes is below a fixed threshold. In the second step, a classifier is trained by minimizing a margin empirical loss over the labeled training set and a penalization term measuring the disability of the learner to predict the $κ$ predominant classes of the identified clusters. The resulting data-dependent generalization error bound involves the margin distribution of the classifier, the stability of the clustering technique used in the first step and Rademacher complexity terms corresponding to partially labeled training data. Our theoretical result exhibit convergence rates extending those proposed in the literature for the binary case, and experimental results on different multiclass classification problems show empirical evidence that supports the theory.

stat.ML↗

arXiv2017

Aggressive Sampling for Multi-class to Binary Reduction with Applications to Text Classification

We address the problem of multi-class classification in the case where the number of classes is very large. We propose a double sampling strategy on top of a multi-class to binary reduction strategy, which transforms the original multi-class problem into a binary classification problem over pairs of examples. The aim of the sampling strategy is to overcome the curse of long-tailed class distributions exhibited in majority of large-scale multi-class classification problems and to reduce the number of pairs of examples in the expanded data. We show that this strategy does not alter the consistency of the empirical risk minimization principle defined over the double sample reduction. Experiments are carried out on DMOZ and Wikipedia collections with 10,000 to 100,000 classes where we show the efficiency of the proposed approach in terms of training and prediction time, memory consumption, and predictive performance with respect to state-of-the-art approaches.

stat.ML↗

arXiv2017

Efficient Rank Minimization to Tighten Semidefinite Programming for Unconstrained Binary Quadratic Optimization

We propose a method for low-rank semidefinite programming in application to the semidefinite relaxation of unconstrained binary quadratic problems. The method improves an existing solution of the semidefinite programming relaxation to achieve a lower rank solution. This procedure is computationally efficient as it does not require projecting on the cone of positive-semidefinite matrices. Its performance in terms of objective improvement and rank reduction is tested over multiple graphs of large-scale Gset graph collection and over binary optimization problems from the Biq Mac collection.

math.OC↗

arXiv2016

Searching equillibriums in Beckmann's and Nesterov--de Palma's models

In this paper we propose and develop classical Frank--Wolf algorithm for Beckmann's type models. This is not new, but we investigate details that allows us to speed up. We also consider stable dynamic like models. First model of this type was proposed 15 years ago by Yu. Nesterov and A. DePalma. We propose randomized dual averaging method with special (sum-type) randomization. For both of the problems we obtain the rates of convergences. It seems that this estimations to be unimprovable without additional assumption about problem formulation.

math.OC↗