arXiv subjects

Eduardo Laber

Publications and source records attributed to Eduardo Laber.

12 recordsLinked to original sources

arXiv2024

New bounds on the cohesion of complete-link and other linkage methods for agglomeration clustering

Linkage methods are among the most popular algorithms for hierarchical clustering. Despite their relevance the current knowledge regarding the quality of the clustering produced by these methods is limited. Here, we improve the currently available bounds on the maximum diameter of the clustering obtained by complete-link for metric spaces. One of our new bounds, in contrast to the existing ones, allows us to separate complete-link from single-link in terms of approximation for the diameter, which corroborates the common perception that the former is more suitable than the latter when the goal is producing compact clusters. We also show that our techniques can be employed to derive upper bounds on the cohesion of a class of linkage methods that includes the quite popular average-link.

cs.LG↗

arXiv2022

Shallow decision trees for explainable $k$-means clustering

A number of recent works have employed decision trees for the construction of explainable partitions that aim to minimize the $k$-means cost function. These works, however, largely ignore metrics related to the depths of the leaves in the resulting tree, which is perhaps surprising considering how the explainability of a decision tree depends on these depths. To fill this gap in the literature, we propose an efficient algorithm that takes into account these metrics. In experiments on 16 datasets, our algorithm yields better results than decision-tree clustering algorithms such as the ones presented in \cite{dasgupta2020explainable}, \cite{frost2020exkmc}, \cite{laber2021price} and \cite{DBLP:conf/icml/MakarychevS21}, typically achieving lower or equivalent costs with considerably shallower trees. We also show, through a simple adaptation of existing techniques, that the problem of building explainable partitions induced by binary trees for the $k$-means cost function does not admit an $(1+ε)$-approximation in polynomial time unless $P=NP$, which justifies the quest for approximation algorithms and/or heuristics.

cs.LG↗

arXiv2022

Time-Constrained Learning

Consider a scenario in which we have a huge labeled dataset ${\cal D}$ and a limited time to train some given learner using ${\cal D}$. Since we may not be able to use the whole dataset, how should we proceed? Questions of this nature motivate the definition of the Time-Constrained Learning Task (TCL): Given a dataset ${\cal D}$ sampled from an unknown distribution $μ$, a learner ${\cal L}$ and a time limit $T$, the goal is to obtain in at most $T$ units of time the classification model with highest possible accuracy w.r.t. to $μ$, among those that can be built by ${\cal L}$ using the dataset ${\cal D}$. We propose TCT, an algorithm for the TCL task designed based that on principles from Machine Teaching. We present an experimental study involving 5 different Learners and 20 datasets where we show that TCT consistently outperforms two other algorithms: the first is a Teacher for black-box learners proposed in [Dasgupta et al., ICML 19] and the second is a natural adaptation of random sampling for the TCL setting. We also compare TCT with Stochastic Gradient Descent training -- our method is again consistently better. While our work is primarily practical, we also show that a stripped-down version of TCT has provable guarantees. Under reasonable assumptions, the time our algorithm takes to achieve a certain accuracy is never much bigger than the time it takes the batch teacher (which sends a single batch of examples) to achieve similar accuracy, and in some case it is almost exponentially better.

cs.LG↗

arXiv2021

On the price of explainability for some clustering problems

The price of explainability for a clustering task can be defined as the unavoidable loss,in terms of the objective function, if we force the final partition to be explainable. Here, we study this price for the following clustering problems: $k$-means, $k$-medians, $k$-centers and maximum-spacing. We provide upper and lower bounds for a natural model where explainability is achieved via decision trees. For the $k$-means and $k$-medians problems our upper bounds improve those obtained by [Moshkovitz et. al, ICML 20] for low dimensions. Another contribution is a simple and efficient algorithm for building explainable clusterings for the $k$-means problem. We provide empirical evidence that its performance is better than the current state of the art for decision-tree based explainable clustering.

cs.LG↗

arXiv2020

Speeding up Word Mover's Distance and its variants via properties of distances between embeddings

The Word Mover's Distance (WMD) proposed by Kusner et al. is a distance between documents that takes advantage of semantic relations among words that are captured by their embeddings. This distance proved to be quite effective, obtaining state-of-art error rates for classification tasks, but is also impracticable for large collections/documents due to its computational complexity. For circumventing this problem, variants of WMD have been proposed. Among them, Relaxed Word Mover's Distance (RWMD) is one of the most successful due to its simplicity, effectiveness, and also because of its fast implementations. Relying on assumptions that are supported by empirical properties of the distances between embeddings, we propose an approach to speed up both WMD and RWMD. Experiments over 10 datasets suggest that our approach leads to a significant speed-up in document classification tasks while maintaining the same error rates.

cs.CL↗

arXiv2019

Information theoretical clustering is hard to approximate

An impurity measures $I: \mathbb{R}^d \mapsto \mathbb{R}^+$ is a function that assigns a $d$-dimensional vector ${\bf v}$ to a non-negative value $I({\bf v})$ so that the more homogeneous ${\bf v}$, with respect to the values of its coordinates, the larger its impurity. A well known example of impurity measures is the Entropy impurity. We study the problem of clustering based on impurity measures. Let $V$ be a collection of $n$ many $d$-dimensional vectors with non-negative components. Given $V$ and an impurity measure $I$, the goal is to find a partition ${\mathcal P}$ of $V$ into $k$ groups $V_1,\ldots,V_k$ so as to minimize the sum of the impurities of the groups in ${\cal P}$, i.e., $I({\cal P})= \sum_{i=1}^{k} I\bigg(\sum_{ {\bf v} \in V_i} {\bf v} \bigg).$ Impurity minimization has been widely used as quality assessment measure in probability distribution clustering (KL-divergence) as well as in categorical clustering. However, in contrast to the case of metric based clustering, the current knowledge of impurity measure based clustering in terms of approximation and inapproximability results is very limited. Here, we contribute to change this scenario by proving that for the Entropy impurity measure the problem does not admit a PTAS even when all vectors have the same $\ell_1$ norm. This result solves a question that remained open in previous work on this topic [Chaudhuri and McGregor COLT 08; Ackermann et. al. ECCC 11].

cs.DS↗

arXiv2018

Approximation Algorithms for Clustering via Weighted Impurity Measures

An impurity measures $I:{R}^k \to {R}^+$ maps a $k$-dimensional vector ${\bf v}$ to a non-negative value $I({\bf v})$ so that the more homogeneous ${\bf v}$, the larger its impurity. We study clustering based on impurity measures: given a collection $V$ of $n$ many $k$-dimensional vectors and an impurity measure $I$, the goal is to find a partition ${\cal P}$ of $V$ into $L$ groups $V_1,\ldots,V_L$ that minimizes the total impurities of the groups in ${\cal P}$, i.e., $I({\cal P})= \sum_{m=1}^{L} I(\sum_{{\bf v} \in V_m}{\bf v}).$ Impurity minimization is widely used as quality assessment measure in probability distribution clustering and in categorical clustering where it is not possible to rely on geometric properties of the data set. However, in contrast to the case of metric based clustering, the current knowledge of impurity measure based clustering in terms of approximation and inapproximability results is very limited. Our research contributes to fill this gap. We first present a simple linear time algorithm that simultaneously achieves $3$-approximation for the Gini impurity measure and $O(\log(\sum_{{\bf v} \in V} \| {\bf v} \|_1))$-approximation for the Entropy impurity measure. Then, for the Entropy impurity measure---where we also show that finding the optimal clustering is strongly NP-hard---we are able to design a polynomial time $O(\log^2(\min\{k,L\}))$-approximation algorithm. Our algorithm relies on a nontrivial characterization of a class of clusterings that necessarily includes a partition achieving $O(\log^2(\min\{k,L\}))$--approximation of the impurity of the optimal partition. Remarkably, this is the first polynomial time algorithm with approximation guarantee independent of the number of points/vector and not relying on any restriction on the components of the vectors for producing clusterings with minimum entropy.

cs.DS↗

arXiv2015

Searching for a superlinear lower bounds for the Maximum Consecutive Subsums Problem and the (min,+)-convolution

Given a sequence of n numbers, the Maximum Consecutive Subsums Problem (MCSP) asks for the maximum consecutive sum of lengths l for each l = 1,...,n. No algorithm is known for this problem which is significantly better than the naive quadratic solution. Nor a super linear lower bound is known. The best known bound for the MCSP is based on the the computation of the (min,+)-convolution, another problem for which neither an O(n^{2-ε}) upper bound is known nor a super linear lower bound. We show that the two problems are in fact computationally equivalent by providing linear reductions between them. Then, we concentrate on the problem of finding super linear lower bounds and provide empirical evidence for an Ω(nlogn) lower bounds for both problems in the decision tree model.

cs.DS↗

arXiv2014

Decision Trees for Function Evaluation - Simultaneous Optimization of Worst and Expected Cost

In several applications of automatic diagnosis and active learning a central problem is the evaluation of a discrete function by adaptively querying the values of its variables until the values read uniquely determine the value of the function. In general, the process of reading the value of a variable might involve some cost, computational or even a fee to be paid for the experiment required for obtaining the value. This cost should be taken into account when deciding the next variable to read. The goal is to design a strategy for evaluating the function incurring little cost (in the worst case or in expectation according to a prior distribution on the possible variables' assignments). Our algorithm builds a strategy (decision tree) which attains a logarithmic approxima- tion simultaneously for the expected and worst cost spent. This is best possible under the assumption that $P \neq NP.$

cs.DS↗

arXiv2014

Trading off Worst and Expected Cost in Decision Tree Problems and a Value Dependent Model

We study the problem of evaluating a discrete function by adaptively querying the values of its variables until the values read uniquely determine the value of the function. Reading the value of a variable is done at the expense of some cost, and the goal is to design a strategy (decision tree) for evaluating the function incurring as little cost as possible in the worst case or in expectation (according to a prior distribution on the possible variables assignments). Except for particular cases of the problem, in general, only the minimization of one of these two measures is addressed in the literature. However, there are instances of the problem for which the minimization of one measure leads to a strategy with a high cost with respect to the other measure (even exponentially bigger than the optimal). We provide a new construction which can guarantee a trade-off between the two criteria. More precisely, given a decision tree guaranteeing expected cost $E$ and a decision tree guaranteeing worst cost $W$ our method can guarantee for any chosen trade-off value $ρ$ to produce a decision tree whose worst cost is $(1 + ρ)W$ and whose expected cost is $(1 + \frac{1}ρ)E.$ These bounds are improved for the relevant case of uniform testing costs. Motivated by applications, we also study a variant of the problem where the cost of reading a variable depends on the variable's value. We provide an $O(\log n)$ approximation algorithm for the minimization of the worst cost measure, which is best possible under the assumption $P \neq NP$.

cs.DS↗

arXiv2010

Competitive Boolean Function Evaluation: Beyond Monotonicity, and the Symmetric Case

We study the extremal competitive ratio of Boolean function evaluation. We provide the first non-trivial lower and upper bounds for classes of Boolean functions which are not included in the class of monotone Boolean functions. For the particular case of symmetric functions our bounds are matching and we exactly characterize the best possible competitiveness achievable by a deterministic algorithm. Our upper bound is obtained by a simple polynomial time algorithm.

cs.DS↗

arXiv2009

On the Complexity of Searching in Trees: Average-case Minimization

We focus on the average-case analysis: A function w : V -> Z+ is given which defines the likelihood for a node to be the one marked, and we want the strategy that minimizes the expected number of queries. Prior to this paper, very little was known about this natural question and the complexity of the problem had remained so far an open question. We close this question and prove that the above tree search problem is NP-complete even for the class of trees with diameter at most 4. This results in a complete characterization of the complexity of the problem with respect to the diameter size. In fact, for diameter not larger than 3 the problem can be shown to be polynomially solvable using a dynamic programming approach. In addition we prove that the problem is NP-complete even for the class of trees of maximum degree at most 16. To the best of our knowledge, the only known result in this direction is that the tree search problem is solvable in O(|V| log|V|) time for trees with degree at most 2 (paths). We match the above complexity results with a tight algorithmic analysis. We first show that a natural greedy algorithm attains a 2-approximation. Furthermore, for the bounded degree instances, we show that any optimal strategy (i.e., one that minimizes the expected number of queries) performs at most O(Δ(T) (log |V| + log w(T))) queries in the worst case, where w(T) is the sum of the likelihoods of the nodes of T and Δ(T) is the maximum degree of T. We combine this result with a non-trivial exponential time algorithm to provide an FPTAS for trees with bounded degree.

cs.DS↗