Searcharxiv⌕ Search

arXiv subjects

Vladimir Kolmogorov

Publications and source records attributed to Vladimir Kolmogorov.

54 records · Page 3Linked to original sources
arXiv2014

A Multi-Plane Block-Coordinate Frank-Wolfe Algorithm for Training Structural SVMs with a Costly max-Oracle

Structural support vector machines (SSVMs) are amongst the best performing models for structured computer vision tasks, such as semantic image segmentation or human pose estimation. Training SSVMs, however, is computationally costly, because it requires repeated calls to a structured prediction subroutine (called \emph{max-oracle}), which has to solve an optimization problem itself, e.g. a graph cut. In this work, we introduce a new algorithm for SSVM training that is more efficient than earlier techniques when the max-oracle is computationally expensive, as it is frequently the case in computer vision tasks. The main idea is to (i) combine the recent stochastic Block-Coordinate Frank-Wolfe algorithm with efficient hyperplane caching, and (ii) use an automatic selection rule for deciding whether to call the exact max-oracle or to rely on an approximate one based on the cached hyperplanes. We show experimentally that this strategy leads to faster convergence to the optimum with respect to the number of requires oracle calls, and that this translates into faster convergence with respect to the total runtime when the max-oracle is slow compared to the other steps of the algorithm. A publicly available C++ implementation is provided at http://pub.ist.ac.at/~vnk/papers/SVM.html .

cs.LG↗
arXiv2014

Combining pattern-based CRFs and weighted context-free grammars

We consider two models for the sequence labeling (tagging) problem. The first one is a {\em Pattern-Based Conditional Random Field }(\PB), in which the energy of a string (chain labeling) $x=x_1\ldots x_n\in D^n$ is a sum of terms over intervals $[i,j]$ where each term is non-zero only if the substring $x_i\ldots x_j$ equals a prespecified word $w\in Λ$. The second model is a {\em Weighted Context-Free Grammar }(\WCFG) frequently used for natural language processing. \PB and \WCFG encode local and non-local interactions respectively, and thus can be viewed as complementary. We propose a {\em Grammatical Pattern-Based CRF model }(\GPB) that combines the two in a natural way. We argue that it has certain advantages over existing approaches such as the {\em Hybrid model} of Bened{í} and Sanchez that combines {\em $\mbox{$N$-grams}$} and \WCFGs. The focus of this paper is to analyze the complexity of inference tasks in a \GPB such as computing MAP. We present a polynomial-time algorithm for general \GPBs and a faster version for a special case that we call {\em Interaction Grammars}.

cs.FL↗
arXiv2013

Simplifying Energy Optimization using Partial Enumeration

Energies with high-order non-submodular interactions have been shown to be very useful in vision due to their high modeling power. Optimization of such energies, however, is generally NP-hard. A naive approach that works for small problem instances is exhaustive search, that is, enumeration of all possible labelings of the underlying graph. We propose a general minimization approach for large graphs based on enumeration of labelings of certain small patches. This partial enumeration technique reduces complex high-order energy formulations to pairwise Constraint Satisfaction Problems with unary costs (uCSP), which can be efficiently solved using standard methods like TRW-S. Our approach outperforms a number of existing state-of-the-art algorithms on well known difficult problems (e.g. curvature regularization, stereo, deconvolution); it gives near global minimum and better speed. Our main application of interest is curvature regularization. In the context of segmentation, our partial enumeration technique allows to evaluate curvature directly on small patches using a novel integral geometry approach.

cs.CV↗
arXiv2013

Potts model, parametric maxflow and k-submodular functions

The problem of minimizing the Potts energy function frequently occurs in computer vision applications. One way to tackle this NP-hard problem was proposed by Kovtun [19,20]. It identifies a part of an optimal solution by running $k$ maxflow computations, where $k$ is the number of labels. The number of "labeled" pixels can be significant in some applications, e.g. 50-93% in our tests for stereo. We show how to reduce the runtime to $O(\log k)$ maxflow computations (or one {\em parametric maxflow} computation). Furthermore, the output of our algorithm allows to speed-up the subsequent alpha expansion for the unlabeled part, or can be used as it is for time-critical applications. To derive our technique, we generalize the algorithm of Felzenszwalb et al. [7] for {\em Tree Metrics}. We also show a connection to {\em $k$-submodular functions} from combinatorial optimization, and discuss {\em $k$-submodular relaxations} for general energy functions.

cs.CV↗
arXiv2013

Towards Minimizing k-Submodular Functions

In this paper we investigate k-submodular functions. This natural family of discrete functions includes submodular and bisubmodular functions as the special cases k = 1 and k = 2 respectively. In particular we generalize the known Min-Max-Theorem for submodular and bisubmodular functions. This theorem asserts that the minimum of the (bi)submodular function can be found by solving a maximization problem over a (bi)submodular polyhedron. We define and investigate a k-submodular polyhedron and prove a Min-Max-Theorem for k-submodular functions.

cs.DM↗
arXiv2013

Optimal Coalition Structures in Cooperative Graph Games

Representation languages for coalitional games are a key research area in algorithmic game theory. There is an inherent tradeoff between how general a language is, allowing it to capture more elaborate games, and how hard it is computationally to optimize and solve such games. One prominent such language is the simple yet expressive Weighted Graph Games (WGGs) representation [14], which maintains knowledge about synergies between agents in the form of an edge weighted graph. We consider the problem of finding the optimal coalition structure in WGGs. The agents in such games are vertices in a graph, and the value of a coalition is the sum of the weights of the edges present between coalition members. The optimal coalition structure is a partition of the agents to coalitions, that maximizes the sum of utilities obtained by the coalitions. We show that finding the optimal coalition structure is not only hard for general graphs, but is also intractable for restricted families such as planar graphs which are amenable for many other combinatorial problems. We then provide algorithms with constant factor approximations for planar, minor-free and bounded degree graphs.

cs.GT↗
arXiv2012

The power of linear programming for valued CSPs: a constructive characterization

A class of valued constraint satisfaction problems (VCSPs) is characterised by a valued constraint language, a fixed set of cost functions on a finite domain. An instance of the problem is specified by a sum of cost functions from the language with the goal to minimise the sum. We study which classes of finite-valued languages can be solved exactly by the basic linear programming relaxation (BLP). Thapper and Zivny showed [20] that if BLP solves the language then the language admits a binary commutative fractional polymorphism. We prove that the converse is also true. This leads to a necessary and a sufficient condition which can be checked in polynomial time for a given language. In contrast, the previous necessary and sufficient condition due to [20] involved infinitely many inequalities. More recently, Thapper and Zivny [21] showed (using, in particular, a technique introduced in this paper) that core languages that do not satisfy our condition are NP-hard. Taken together, these results imply that a finite-valued language can either be solved using Linear Programming or is NP-hard.

cs.CC↗
arXiv2012

Generalized sequential tree-reweighted message passing

This paper addresses the problem of approximate MAP-MRF inference in general graphical models. Following [36], we consider a family of linear programming relaxations of the problem where each relaxation is specified by a set of nested pairs of factors for which the marginalization constraint needs to be enforced. We develop a generalization of the TRW-S algorithm [9] for this problem, where we use a decomposition into junction chains, monotonic w.r.t. some ordering on the nodes. This generalizes the monotonic chains in [9] in a natural way. We also show how to deal with nested factors in an efficient way. Experiments show an improvement over min-sum diffusion, MPLP and subgradient ascent algorithms on a number of computer vision and natural language processing problems.

cs.CV↗
arXiv2012

On the optimality of tree-reweighted max-product message-passing

Tree-reweighted max-product (TRW) message passing is a modified form of the ordinary max-product algorithm for attempting to find minimal energy configurations in Markov random field with cycles. For a TRW fixed point satisfying the strong tree agreement condition, the algorithm outputs a configuration that is provably optimal. In this paper, we focus on the case of binary variables with pairwise couplings, and establish stronger properties of TRW fixed points that satisfy only the milder condition of weak tree agreement (WTA). First, we demonstrate how it is possible to identify part of the optimal solution|i.e., a provably optimal solution for a subset of nodes| without knowing a complete solution. Second, we show that for submodular functions, a WTA fixed point always yields a globally optimal solution. We establish that for binary variables, any WTA fixed point always achieves the global maximum of the linear programming relaxation underlying the TRW method.

cs.AI↗
arXiv2012

Minimizing a sum of submodular functions

We consider the problem of minimizing a function represented as a sum of submodular terms. We assume each term allows an efficient computation of {\em exchange capacities}. This holds, for example, for terms depending on a small number of variables, or for certain cardinality-dependent terms. A naive application of submodular minimization algorithms would not exploit the existence of specialized exchange capacity subroutines for individual terms. To overcome this, we cast the problem as a {\em submodular flow} (SF) problem in an auxiliary graph, and show that applying most existing SF algorithms would rely only on these subroutines. We then explore in more detail Iwata's capacity scaling approach for submodular flows (Math. Programming, 76(2):299--308, 1997). In particular, we show how to improve its complexity in the case when the function contains cardinality-dependent terms.

cs.DS↗
arXiv2011

The complexity of conservative valued CSPs

We study the complexity of valued constraint satisfaction problems (VCSP). A problem from VCSP is characterised by a \emph{constraint language}, a fixed set of cost functions over a finite domain. An instance of the problem is specified by a sum of cost functions from the language and the goal is to minimise the sum. We consider the case of languages containing all possible unary cost functions. In the case of languages consisting of only $\{0,\infty\}$-valued cost functions (i.e. relations), such languages have been called \emph{conservative} and studied by Bulatov [LICS'03] and recently by Barto [LICS'11]. Since we study valued languages, we call a language conservative if it contains all finite-valued unary cost functions. The complexity of conservative valued languages has been studied by Cohen et al. [AIJ'06] for languages over Boolean domains, by Deineko et al. [JACM'08] for $\{0,1\}$-valued languages (a.k.a Max-CSP), and by Takhanov [STACS'10] for $\{0,\infty\}$-valued languages containing all finite-valued unary cost functions (a.k.a. Min-Cost-Hom). We prove a Schaefer-like dichotomy theorem for conservative valued languages: if all cost functions in the language satisfy a certain condition (specified by a complementary combination of \emph{STP and MJN multimorphisms}), then any instance can be solved in polytime (via a new algorithm developed in this paper), otherwise the language is NP-hard. This is the \emph{first} complete complexity classification of \emph{general-valued constraint languages} over non-Boolean domains. This generalises previous results by Takhanov [STACS'10] and (a subset of results) by Cohen et al. [AIJ'06] and Deineko et al. [JACM'08]. Moreover, our results do not rely on any computer-assisted search as in Deineko et al. [JACM'08], and provide a powerful tool for proving hardness of finite- and general-valued languages.

cs.CC↗
arXiv2011

Submodularity on a tree: Unifying $L^\natural$-convex and bisubmodular functions

We introduce a new class of functions that can be minimized in polynomial time in the value oracle model. These are functions $f$ satisfying $f(x)+f(y)\ge f(x \sqcap y)+f(x \sqcup y)$ where the domain of each variable $x_i$ corresponds to nodes of a rooted binary tree, and operations $\sqcap,\sqcup$ are defined with respect to this tree. Special cases include previously studied $L^\natural$-convex and bisubmodular functions, which can be obtained with particular choices of trees. We present a polynomial-time algorithm for minimizing functions in the new class. It combines Murota's steepest descent algorithm for $L^\natural$-convex functions with bisubmodular minimization algorithms.

cs.DM↗
arXiv2011

Submodular Decomposition Framework for Inference in Associative Markov Networks with Global Constraints

In the paper we address the problem of finding the most probable state of discrete Markov random field (MRF) with associative pairwise terms. Although of practical importance, this problem is known to be NP-hard in general. We propose a new type of MRF decomposition, submodular decomposition (SMD). Unlike existing decomposition approaches SMD decomposes the initial problem into subproblems corresponding to a specific class label while preserving the graph structure of each subproblem. Such decomposition enables us to take into account several types of global constraints in an efficient manner. We study theoretical properties of the proposed approach and demonstrate its applicability on a number of problems.

cs.CV↗
arXiv2010

Generalized roof duality and bisubmodular functions

Consider a convex relaxation $\hat f$ of a pseudo-boolean function $f$. We say that the relaxation is {\em totally half-integral} if $\hat f(x)$ is a polyhedral function with half-integral extreme points $x$, and this property is preserved after adding an arbitrary combination of constraints of the form $x_i=x_j$, $x_i=1-x_j$, and $x_i=γ$ where $γ\in\{0, 1, 1/2}$ is a constant. A well-known example is the {\em roof duality} relaxation for quadratic pseudo-boolean functions $f$. We argue that total half-integrality is a natural requirement for generalizations of roof duality to arbitrary pseudo-boolean functions. Our contributions are as follows. First, we provide a complete characterization of totally half-integral relaxations $\hat f$ by establishing a one-to-one correspondence with {\em bisubmodular functions}. Second, we give a new characterization of bisubmodular functions. Finally, we show some relationships between general totally half-integral relaxations and relaxations based on the roof duality.

cs.DM↗
arXiv2010

A dichotomy theorem for conservative general-valued CSPs

We study the complexity of valued constraint satisfaction problems (VCSP). A problem from VCSP is characterised by a \emph{constraint language}, a fixed set of cost functions over a finite domain. An instance of the problem is specified by a sum of cost functions from the language and the goal is to minimise the sum. We consider the case of so-called \emph{conservative} languages; that is, languages containing all unary cost functions, thus allowing arbitrary restrictions on the domains of the variables. We prove a Schaefer-like dichotomy theorem for this case: if all cost functions in the language satisfy a certain condition (specified by a complementary combination of \emph{STP and MJN multimorphisms}) then any instance can be solved in polynomial time by the algorithm of Kolmogorov and Zivny (arXiv:1008.3104v1), otherwise the language is NP-hard. This generalises recent results of Takhanov (STACS'10) who considered $\{0,\infty\}$-valued languages containing additionally all finite-valued unary cost functions, and Kolmogorov and Zivny (arXiv:1008.1555v1) who considered \emph{finite-valued} conservative languages.

cs.CC↗
arXiv2010

Generalising tractable VCSPs defined by symmetric tournament pair multimorphisms

We study optimisation problems that can be formulated as valued constraint satisfaction problems (VCSP). A problem from VCSP is characterised by a \emph{constraint language}, a fixed set of cost functions taking finite and infinite costs over a finite domain. An instance of the problem is specified by a sum of cost functions from the language and the goal is to minimise the sum. We are interested in \emph{tractable} constraint languages; that is, languages that give rise to VCSP instances solvable in polynomial time. Cohen et al. (AIJ'06) have shown that constraint languages that admit the MJN multimorphism are tractable. Moreover, using a minimisation algorithm for submodular functions, Cohen et al. (TCS'08) have shown that constraint languages that admit an STP (symmetric tournament pair) multimorphism are tractable. We generalise these results by showing that languages admitting the MJN multimorphism on a subdomain and an STP multimorphisms on the complement of the subdomain are tractable. The algorithm is a reduction to the algorithm for languages admitting an STP multimorphism.

cs.CC↗
arXiv2010

The complexity of conservative finite-valued CSPs

We study the complexity of valued constraint satisfaction problems (VCSP). A problem from VCSP is characterised by a \emph{constraint language}, a fixed set of cost functions over a finite domain. An instance of the problem is specified by a sum of cost functions from the language and the goal is to minimise the sum. We consider the case of so-called \emph{conservative} languages; that is, languages containing all unary cost functions, thus allowing arbitrary restrictions on the domains of the variables. This problem has been studied by Bulatov [LICS'03] for $\{0,\infty\}$-valued languages (i.e. CSP), by Cohen~\etal\ (AIJ'06) for Boolean domains, by Deineko et al. (JACM'08) for $\{0,1\}$-valued cost functions (i.e. Max-CSP), and by Takhanov (STACS'10) for $\{0,\infty\}$-valued languages containing all finite-valued unary cost functions (i.e. Min-Cost-Hom). We give an elementary proof of a complete complexity classification of conservative finite-valued languages: we show that every conservative finite-valued language is either tractable or NP-hard. This is the \emph{first} dichotomy result for finite-valued VCSPs over non-Boolean domains.

cs.CC↗