arXiv subjects

Sebastian Riedel

Publications and source records attributed to Sebastian Riedel.

139 records · Page 8Linked to original sources

arXiv2015

Extraction of evidence tables from abstracts of randomized clinical trials using a maximum entropy classifier and global constraints

Systematic use of the published results of randomized clinical trials is increasingly important in evidence-based medicine. In order to collate and analyze the results from potentially numerous trials, evidence tables are used to represent trials concerning a set of interventions of interest. An evidence table has columns for the patient group, for each of the interventions being compared, for the criterion for the comparison (e.g. proportion who survived after 5 years from treatment), and for each of the results. Currently, it is a labour-intensive activity to read each published paper and extract the information for each field in an evidence table. There have been some NLP studies investigating how some of the features from papers can be extracted, or at least the relevant sentences identified. However, there is a lack of an NLP system for the systematic extraction of each item of information required for an evidence table. We address this need by a combination of a maximum entropy classifier, and integer linear programming. We use the later to handle constraints on what is an acceptable classification of the features to be extracted. With experimental results, we demonstrate substantial advantages in using global constraints (such as the features describing the patient group, and the interventions, must occur before the features describing the results of the comparison).

cs.CL↗

arXiv2014

A Levy-area between Brownian motion and rough paths with applications to robust non-linear filtering and RPDEs

We give meaning to differential equations with a rough path term and a Brownian noise term as driving signals. Such differential equations as well as the question of regularity of the solution map arise naturally and we discuss two applications: one revisits Clark's robustness problem in nonlinear filtering, the other is a Feynman--Kac type representation of linear RPDEs. En passant, we give a short and direct argument that implies integrability estimates for rough differential equations with Gaussian driving signals which is of independent interest.

math.PR↗

arXiv2013

Anytime Belief Propagation Using Sparse Domains

Belief Propagation has been widely used for marginal inference, however it is slow on problems with large-domain variables and high-order factors. Previous work provides useful approximations to facilitate inference on such models, but lacks important anytime properties such as: 1) providing accurate and consistent marginals when stopped early, 2) improving the approximation when run longer, and 3) converging to the fixed point of BP. To this end, we propose a message passing algorithm that works on sparse (partially instantiated) domains, and converges to consistent marginals using dynamic message scheduling. The algorithm grows the sparse domains incrementally, selecting the next value to add using prioritization schemes based on the gradients of the marginal inference objective. Our experiments demonstrate local anytime consistency and fast convergence, providing significant speedups over BP to obtain low-error marginals: up to 25 times on grid models, and up to 6 times on a real-world natural language processing task.

stat.ML↗

arXiv2013

Spatial rough path lifts of stochastic convolutions

We present sufficient conditions for finite controlled rho-variation of the covariance of Gaussian processes with stationary increments, based on concavity or convexity of their variance function. The motivation for this type of conditions comes from recent work of Hairer [CPAM,2011]. Our results allow to construct rough paths lifts of solutions to a class of fractional stochastic heat equations with additive, possibly colored Wiener noise with respect to their space variable.

math.PR↗

arXiv2013

On the Variational Regularity of Cameron-Martin paths

It is a well-known fact that finite rho-variation of the covariance (in 2D sense) of a general Gaussian process implies finite rho-variation of Cameron-Martin paths. In the special case of fractional Brownian motion (think: 2H=1/rho), in the rougher than Brownian regime, a sharper result holds thanks to a Besov-type embedding [Friz-Victoir, JFA, 2006]. In the present note we give a general result which closes this gap. We comment on the importance of this result for various applications.

math.PR↗

arXiv2013

Automorphism Groups of Graphical Models and Lifted Variational Inference

Using the theory of group action, we first introduce the concept of the automorphism group of an exponential family or a graphical model, thus formalizing the general notion of symmetry of a probabilistic model. This automorphism group provides a precise mathematical framework for lifted inference in the general exponential family. Its group action partitions the set of random variables and feature functions into equivalent classes (called orbits) having identical marginals and expectations. Then the inference problem is effectively reduced to that of computing marginals or expectations for each class, thus avoiding the need to deal with each individual variable or feature. We demonstrate the usefulness of this general framework in lifting two classes of variational approximation for maximum a posteriori (MAP) inference: local linear programming (LP) relaxation and local LP relaxation with cycle constraints; the latter yields the first lifted variational inference algorithm that operates on a bound tighter than the local constraints.

cs.AI↗

arXiv2013

Latent Relation Representations for Universal Schemas

Traditional relation extraction predicts relations within some fixed and finite target schema. Machine learning approaches to this task require either manual annotation or, in the case of distant supervision, existing structured sources of the same schema. The need for existing datasets can be avoided by using a universal schema: the union of all involved schemas (surface form predicates as in OpenIE, and relations in the schemas of pre-existing databases). This schema has an almost unlimited set of relations (due to surface forms), and supports integration with existing structured data (through the relation types of existing databases). To populate a database of such schema we present a family of matrix factorization models that predict affinity between database tuples and relations. We show that this achieves substantially higher accuracy than the traditional classification approach. More importantly, by operating simultaneously on relations observed in text and in pre-existing structured DBs such as Freebase, we are able to reason about unstructured and structured data in mutually-supporting ways. By doing so our approach outperforms state-of-the-art distant supervision systems.

cs.LG↗

arXiv2012

A simple proof of distance bounds for Gaussian rough paths

We derive explicit distance bounds for Stratonovich iterated integrals along two Gaussian processes (also known as signatures of Gaussian rough paths) based on the regularity assumption of their covariance functions. Similar estimates have been obtained recently in [Friz-Riedel, AIHP, to appear]. One advantage of our argument is that we obtain the bound for the third level iterated integrals merely based on the first two levels, and this reflects the intrinsic nature of rough paths. Our estimates are sharp when both covariance functions have finite 1-variation, which includes a large class of Gaussian processes. Two applications of our estimates are discussed. The first one gives the a.s. convergence rates for approximated solutions to rough differential equations driven by Gaussian processes. In the second example, we show how to recover the optimal time regularity for solutions of some rough SPDEs.

math.PR↗

arXiv2012

Automorphism Groups of Graphical Models and Lifted Variational Inference

Using the theory of group action, we first introduce the concept of the automorphism group of an exponential family or a graphical model, thus formalizing the general notion of symmetry of a probabilistic model. This automorphism group provides a precise mathematical framework for lifted inference in the general exponential family. Its group action partitions the set of random variables and feature functions into equivalent classes (called orbits) having identical marginals and expectations. Then the inference problem is effectively reduced to that of computing marginals or expectations for each class, thus avoiding the need to deal with each individual variable or feature. We demonstrate the usefulness of this general framework in lifting two classes of variational approximation for MAP inference: local LP relaxation and local LP relaxation with cycle constraints; the latter yields the first lifted inference that operate on a bound tighter than local constraints. Initial experimental results demonstrate that lifted MAP inference with cycle constraints achieved the state of the art performance, obtaining much better objective function values than local approximation while remaining relatively efficient.

cs.AI↗

arXiv2012

Improving the Accuracy and Efficiency of MAP Inference for Markov Logic

In this work we present Cutting Plane Inference (CPI), a Maximum A Posteriori (MAP) inference method for Statistical Relational Learning. Framed in terms of Markov Logic and inspired by the Cutting Plane Method, it can be seen as a meta algorithm that instantiates small parts of a large and complex Markov Network and then solves these using a conventional MAP method. We evaluate CPI on two tasks, Semantic Role Labelling and Joint Entity Resolution, while plugging in two different MAP inference methods: the current method of choice for MAP inference in Markov Logic, MaxWalkSAT, and Integer Linear Programming. We observe that when used with CPI both methods are significantly faster than when used alone. In addition, CPI improves the accuracy of MaxWalkSAT and maintains the exactness of Integer Linear Programming.

cs.AI↗

arXiv2012

Convergence rates for the full Gaussian rough paths

Under the key assumption of finite ρ-variation, ρ\in[1,2), of the covariance of the underlying Gaussian process, sharp a.s. convergence rates for approximations of Gaussian rough paths are established. When applied to Brownian resp. fractional Brownian motion (fBM), ρ=1 resp. ρ=1/(2H), we recover and extend the respective results of [Hu--Nualart; Rough path analysis via fractional calculus; TAMS 361 (2009) 2689-2718] and [Deya--Neuenkirch--Tindel; A Milstein-type scheme without Lévy area terms for SDEs driven by fractional Brownian motion; AIHP (2011)]. In particular, we establish an a.s. rate k^{-(1/ρ-1/2-ε)}, any ε>0, for Wong-Zakai and Milstein-type approximations with mesh-size 1/k. When applied to fBM this answers a conjecture in the afore-mentioned references.

math.PR↗

arXiv2012

Inference by Minimizing Size, Divergence, or their Sum

We speed up marginal inference by ignoring factors that do not significantly contribute to overall accuracy. In order to pick a suitable subset of factors to ignore, we propose three schemes: minimizing the number of model factors under a bound on the KL divergence between pruned and full models; minimizing the KL divergence under a bound on factor count; and minimizing the weighted sum of KL divergence and factor count. All three problems are solved using an approximation of the KL divergence than can be calculated in terms of marginals computed on a simple seed graph. Applied to synthetic image denoising and to three different types of NLP parsing models, this technique performs marginal inference up to 11 times faster than loopy BP, with graph sizes reduced up to 98%-at comparable error in marginals and parsing accuracy. We also show that minimizing the weighted sum of divergence and size is substantially faster than minimizing either of the other objectives based on the approximation to divergence presented here.

cs.LG↗

arXiv2012

Integrability of (non-)linear rough differential equations and integrals

Integrability properties of (classical, linear, linear growth) rough differential equations (RDEs) are considered, the Jacobian of the RDE flow driven by Gaussian signals being a motivating example. We revisit and extend some recent ground-breaking work of Cass-Litterer-Lyons in this regard; as by-product, we obtain a user-friendly "transitivity property" of such integrability estimates. We also consider rough integrals; as a novel application, uniform Weibull tail estimates for a class of (random) rough integrals are obtained. A concrete example arises from the stochastic heat-equation, spatially mollified by hyper-viscosity, and we can recover (in fact: sharpen) a technical key result of [Hairer, Comm.PureAppl.Math.64,no.11,(2011),1547-1585].

math.PR↗