Phylogenetic flexibility via Hall-type inequalities and submodularity

Abstract

Given a collection $\tau$ of subsets of a finite set $X$, we say that $\tau$ is {\em phylogenetically flexible} if, for any collection $R$ of rooted phylogenetic trees whose leaf sets comprise the collection $\tau$, $R$ is compatible (i.e. there is a rooted phylogenetic $X$--tree that displays each tree in $R$). We show that $\tau$ is phylogenetically flexible if and only if it satisfies a Hall-type inequality condition of being `slim'. Using submodularity arguments, we show that there is a polynomial-time algorithm for determining whether or not $\tau$ is slim. This `slim' condition reduces to a simpler inequality in the case where all of the sets in $\tau$ have size 3, a property we call `thin'. Thin sets were recently shown to be equivalent to the existence of an (unrooted) tree for which the median function provides an injective mapping to its vertex set; we show here that the unrooted tree in this representation can always be chosen to be a caterpillar tree. We also characterise when a collection $\tau$ of subsets of size 2 is thin (in terms of the flexibility of total orders rather than phylogenies) and show that this holds if and only if an associated bipartite graph is a forest. The significance of our results for phylogenetics is in providing precise and efficiently verifiable conditions under which supertree methods that require consistent inputs of trees, can be applied to any input trees on given subsets of species.