arXiv subjects

Bahman Kalantari

Publications and source records attributed to Bahman Kalantari.

At least 19 recordsLinked to original sources

arXiv2023

Approximating Nash Equilibrium Via Multilinear Minimax

Nash equilibrium} (NE) can be stated as a formal theorem on a multilinear form, free of game theory terminology. On the other hand, inspired by this formalism, we state and prove a {\it multilinear minimax theorem}, a generalization of von Neumann's bilinear minimax theorem. As in the bilinear case, the proof is based on relating the underlying optimizations to a primal-dual pair of linear programming problems, albeit more complicated LPs. The theorem together with its proof is of independent interest. Next, we use the theorem to associate to a multilinear form in NE a {\it multilinear minimax relaxation} (MMR), where the primal-dual pair of solutions induce an approximate equilibrium point that provides a nontrivial upper bound on a convex combination of {\it expected payoffs} in any NE solution. In fact we show any positive probability vector associated to the players induces a corresponding {\it diagonally-scaled} MMR approximate equilibrium with its associated upper bound. By virtue of the proof of the multilinear minimax theorem, MMR solution can be computed in polynomial-time. On the other hand, it is known that even in bimatrix games NE is {\it PPAD-complete}, a complexity class in NP not known to be in P. The quality of MMR solution and the efficiency of solving the underlying LPs are the subject of further investigation. However, as shown in a separate article, for a large set of test problems in bimatrix games, not only the MMR payoffs for both players are better than any NE payoffs, so is the computing time of MMR in contrast with that of Lemke-Howsen algorithm. In large size problems the latter algorithm even fails to produce a Nash equilibrium. In summary, solving MMR provides a worthy approximation even if Nash equilibrium is shown to be computable in polynomial-time.

cs.GT↗

arXiv2023

Approximating Bimatrix Nash Equilibrium Via Trilinear Minimax

The Bimatrix Nash Equilibrium (NE) for $m \times n$ real matrices $R$ and $C$, denoted as the {\it Row} and {\it Column} players, is characterized as follows: Let $Δ=S_m \times S_n$, where $S_k$ denotes the unit simplex in $\mathbb{R}^k$. For a given point $p=(x,y) \in Δ$, define $R[p]=x^TRy$ and $C[p]=x^TCy$. Consequently, there exists a subset $Δ_* \subset Δ$ such that for any $p_*=(x_*,y_*) \in Δ_*$, $\max_{p \in Δ, y=y_*}R[p]=R[p_*]$ and $\max_{p \in Δ, x=x_* } C[p]=C[p_*]$. The computational complexity of bimatrix NE falls within the class of {\it PPAD-complete}. Although the von Neumann Minimax Theorem is a special case of bimatrix NE, we introduce a novel extension termed {\it Trilinear Minimax Relaxation} (TMR) with the following implications: Let $λ^*=\min_{α\in S_{2}} \max_{p \in Δ} (α_1 R[p]+ α_2C[p])$ and $λ_*=\max_{p \in Δ} \min_{α\in S_{2}} (α_1 R[p]+ α_2C[p])$. $λ^* \geq λ_*$. $λ^*$ is computable as a linear programming in $O(mn)$ time, ensuring $\max_{p_* \in Δ_*}\min \{R[p_*], C[p_*]\} \leq λ^*$, meaning that in any Nash Equilibrium it is not possible to have both players' payoffs to exceed $λ^*$. $λ^*=λ_*$ if and only if there exists $p^* \in Δ$ such that $λ^*= \min\{R[p^*], C[p^*]\}$. Such a $p^*$ serves as an approximate Nash Equilibrium. We analyze the cases where such $p^*$ exists and is computable. Even when $λ^* > λ_*$, we derive approximate Nash Equilibria. In summary, the aforementioned properties of TMR and its efficient computational aspects underscore its significance and relevance for Nash Equilibrium, irrespective of the computational complexity associated with bimatrix Nash Equilibrium. Finally, we extend TMR to scenarios involving three or more players.

cs.GT↗

arXiv2023

A Family of Iteration Functions for General Linear Systems

We introduce innovative algorithms for computing exact or approximate (minimum-norm) solutions to $Ax=b$ or the {\it normal equation} $A^TAx=A^Tb$, where $A$ is an $m \times n$ real matrix of arbitrary rank. We present more efficient algorithms when $A$ is symmetric PSD. First, we introduce the {\it Triangle Algorithm} (TA), a {\it convex-hull membership} algorithm that given $b_k=Ax_k$ in the ellipsoid $E_{A,ρ}=\{Ax: \Vert x \Vert \leq ρ\}$, it either computes an improved approximation $b_{k+1}=Ax_{k+1}$ or proves $b \not \in E_{A,ρ}$. We then give a dynamic variant of TA, the {\it Centering Triangle Algorithm} (CTA), generating residual, $r_k=b -Ax_k$ via the iteration of $F_1(r)=r-(r^THr/r^TH^2r)Hr$, where $H=AA^T$. If $A$ is symmetric PSD, $H$ can be taken as $A$. Next, for each $t=1, \dots, m$, we derive $F_t(r)=r- \sum_{i=1}^t α_{t,i}(r) H^i r$ whose iterations correspond to a Krylov subspace method with restart. If $κ^+(H)$ is the ratio of the largest to smallest positive eigenvalues of $H$, when $Ax=b$ is consistent, in $k=O({κ^+(H)}{t^{-1}} \ln \varepsilon^{-1})$ iterations of $F_t$, $\Vert r_k \Vert \leq \varepsilon$. Each iteration takes $O(tN+t^3)$ operations, $N$ the number of nonzero entries in $A$. By directly applying $F_t$ to the normal equation, we get $\Vert A^TAx_k - A^Tb \Vert \leq \varepsilon$ in $O({κ^+(AA^T)}{t}^{-1} \ln \varepsilon^{-1})$ iterations. On the other hand, given any residual $r$, we compute $s$, the degree of its minimal polynomial with respect to $H$ in $O(sN+s^3)$ operations. Then $F_s(r)$ gives the minimum-norm solution of $Ax=b$ or an exact solution of $A^TAx=A^Tb$. The proposed algorithms are simple to implementation and theoretically robust. We present sample computational results, comparing the performance of CTA with CG and GMRES methods. The results support CTA as a highly competitive option.

math.NA↗

arXiv2023

On the Performance of a Novel Class of Linear System Solvers and Comparison with State-of-The-Art Algorithms

We present a comprehensive computational study of a class of linear system solvers, called {\it Triangle Algorithm} (TA) and {\it Centering Triangle Algorithm} (CTA), developed by Kalantari \cite{kalantari23}. The algorithms compute an approximate solution or minimum-norm solution to $Ax=b$ or $A^TAx=A^Tb$, where $A$ is an $m \times n$ real matrix of arbitrary rank. The algorithms specialize when $A$ is symmetric positive semi-definite. Based on the description and theoretical properties of TA and CTA from \cite{kalantari23}, we give an implementation of the algorithms that is easy-to-use for practitioners, versatile for a wide range of problems, and robust in that our implementation does not necessitate any constraints on $A$. Next, we make computational comparisons of our implementation with the Matlab implementations of two state-of-the-art algorithms, GMRES and ``lsqminnorm". We consider square and rectangular matrices, for $m$ up to $10000$ and $n$ up to $1000000$, encompassing a variety of applications. These results indicate that our implementation outperforms GMRES and ``lsqminnorm" both in runtime and quality of residuals. Moreover, the relative residuals of CTA decrease considerably faster and more consistently than GMRES, and our implementation provides high precision approximation, faster than GMRES reports lack of convergence. With respect to ``lsqminnorm", our implementation runs faster, producing better solutions. Additionally, we present a theoretical study in the dynamics of iterations of residuals in CTA and complement it with revealing visualizations. Lastly, we extend TA for LP feasibility problems, handling non-negativity constraints. Computational results show that our implementation for this extension is on par with those of TA and CTA, suggesting applicability in linear programming and related problems.

math.NA↗

arXiv2023

Solution of Real Cubic Equations without Cardano's Formula

Building on a classification of zeros of cubic equations due to the $12$-th century Persian mathematician Sharaf al-Din Tusi, together with Smale's theory of {\it point estimation}, we derive an efficient recipe for computing high-precision approximation to a real root of an arbitrary real cubic equation. First, via reversible transformations we reduce any real cubic equation into one of four canonical forms with $0$, $\pm 1$ coefficients, except for the constant term as $\pm q$, $q \geq 0$. Next, given any form, if $ρ_q$ is an approximation to $\sqrt[3]{q}$ to within a relative error of five percent, we prove a {\it seed} $x_0$ in $\{ ρ_q, \pm .95 ρ_q, -\frac{1}{3}, 1 \}$ can be selected such that in $t$ Newton iterations $|x_t - θ_q| \leq \sqrt[3]{q}\cdot 2^{-2^{t}}$ for some real root $θ_q$. While computing a good seed, even for approximation of $\sqrt[3]{q}$, is considered to be ``somewhat of black art'' (see Wikipedia), as we justify, $ρ_q$ is readily computable from {\it mantissa} and {\it exponent} of $q$. It follows that the above approach gives a simple recipe for numerical approximation of solutions of real cubic equations independent of Cardano's formula.

math.NA↗

arXiv2022

On Tusi's Classification of Cubic Equations and its Connections to Cardano's Formula and Khayyam's Geometric Solution

Omar Khayyam's studies on cubic equations inspired the 12th century Persian mathematician Sharaf al-Din Tusi to investigate the number of positive roots. According to the noted mathematical historian Rashed, Tusi analyzed the problem for five different types of equations. In fact all cubic equations are reducible to a form {\it Tusi form} $x^2-x^3=c$. Tusi determined that the maximum of $x^2-x^3$ on $(0,1)$ occurs at $\frac{2}{3}$ and concluded when $c=\frac{4}{27} δ$, $δ\in (0,1)$, there are roots in $(0, \frac{2}{3})$ and $(\frac{2}{3},1)$, ignoring the root in $(-\frac{1}{3},0)$. Given a {\it reduced form} $x^3+px+q=0$, when $p <0$, we show it is reducible to a Tusi form with $δ= \frac{1}{2} + {3\sqrt{3} q}/{4\sqrt{-p^3}}$. It follows there are three real roots if and only if $Δ=-(\frac{q^2}{4}+\frac{p^3}{27})$ is positive. This gives an explicit connection between $δ$ in Tusi form and $Δ$ in Cardano's formula. Thus when $δ\in (0,1)$, rather than using Cardano's formula in complex numbers one can approximate the roots iteratively. On the other hand, for a reduced form with $p >0$ we give a novel proof of Cardono's formula. While Rashed attributes Tusi's computation of the maximum to the use of derivatives, according to Hogendijk, Tusi was probably influenced by Euclid. Here we show the maximizer in Tusi form is computable via elementary algebraic manipulations. Indeed for a {\it quadratic Tusi form}, $x-x^2=δ/4$, Tusi's approach results in a simple derivation of the quadratic formula, comparable with the pedagogical approach of Po-Shen Loh. Moreover, we derive analogous results for the {\it general Tusi form}. Finally, we present a novel derivation of Khayyam's geometric solution. The results complement previous findings on Tusi's work and reveal further facts on history, mathematics and pedagogy in solving cubic equations.

math.GM↗

arXiv2021

Spherical Triangle Algorithm: A Fast Oracle for Convex Hull Membership Queries

The it Convex Hull Membership(CHM) problem is: Given a point $p$ and a subset $S$ of $n$ points in $\mathbb{R}^m$, is $p \in conv(S)$? CHM is not only a fundamental problem in Linear Programming, Computational Geometry, Machine Learning and Statistics, it also serves as a query problem in many applications e.g. Topic Modeling, LP Feasibility, Data Reduction. The {\it Triangle Algorithm} (TA) \cite{kalantari2015characterization} either computes an approximate solution in the convex hull, or a separating hyperplane. The {\it Spherical}-CHM is a CHM, where $p=0$ and each point in $S$ has unit norm. First, we prove the equivalence of exact and approximate versions of CHM and Spherical-CHM. On the one hand, this makes it possible to state a simple version of the original TA. On the other hand, we prove that under the satisfiability of a simple condition in each iteration, the complexity improves to $O(1/\varepsilon)$. The analysis also suggests a strategy for when the property does not hold at an iterate. This suggests the \textit{Spherical-TA} which first converts a given CHM into a Spherical-CHM before applying the algorithm. Next we introduce a series of applications of Spherical-TA. In particular, Spherical-TA serves as a fast version of vanilla TA to boost its efficiency. As an example, this results in a fast version of \emph{AVTA} \cite{awasthi2018robust}, called \emph{AVTA$^+$} for solving exact or approximate irredundancy problem. Computationally, we have considered CHM, LP and Strict LP Feasibility and the Irredundancy problem. Based on substantial amount of computing, Spherical-TA achieves better efficiency than state of the art algorithms. Leveraging on the efficiency of Spherical-TA, we propose AVTA$^+$ as a pre-processing step for data reduction which arises in such applications as in computing the Minimum Volume Enclosing Ellipsoid \cite{moshtagh2005minimum}.

cs.CG↗

arXiv2020

On the Equivalence of SDP Feasibility and a Convex Hull Relaxation for System of Quadratic Equations

We show {\it semidefinite programming} (SDP) feasibility problem is equivalent to solving a {\it convex hull relaxation} (CHR) for a finite system of quadratic equations. On the one hand, this offers a simple description of SDP. On the other hand, this equivalence makes it possible to describe a version of the {\it Triangle Algorithm} for SDP feasibility based on solving CHR. Specifically, the Triangle Algorithm either computes an approximation to the least-norm feasible solution of SDP, or using its {\it distance duality}, provides a separation when no solution within a prescribed norm exists. The worst-case complexity of each iteration is computing the largest eigenvalue of a symmetric matrix arising in that iteration. Alternate complexity bounds on the total number of iterations can be derived. The Triangle Algorithm thus provides an alternative to the existing interior-point algorithms for SDP feasibility and SDP optimization. In particular, based on a preliminary computational result, we can efficiently solve SDP relaxation of {\it binary quadratic} feasibility via the Triangle Algorithm. This finds application in solving SDP relaxation of MAX-CUT. We also show in the case of testing the feasibility of a system of convex quadratic inequalities, the problem is reducible to a corresponding CHR, where the worst-case complexity of each iteration via the Triangle Algorithm is solving a {\it trust region subproblem}. Gaining from these results, we discuss potential extension of CHR and the Triangle Algorithm to solving general system of polynomial equations.

math.OC↗

arXiv2020

A Geometric Algorithm for Solving Linear Systems

Based on the geometric {\it Triangle Algorithm} for testing membership of a point in a convex set, we present a novel iterative algorithm for testing the solvability of a real linear system $Ax=b$, where $A$ is an $m \times n$ matrix of arbitrary rank. Let $C_{A,r}$ be the ellipsoid determined as the image of the Euclidean ball of radius $r$ under the linear map $A$. The basic procedure in our algorithm computes a point in $C_{A,r}$ that is either within $\varepsilon$ distance to $b$, or acts as a certificate proving $b \not \in C_{A,r}$. Each iteration takes $O(mn)$ operations and when $b$ is well-situated in $C_{A,r}$, the number of iterations is proportional to $\log{(1/\varepsilon)}$. If $Ax=b$ is solvable the algorithm computes an approximate solution or the minimum-norm solution. Otherwise, it computes a certificate to unsolvability, or the minimum-norm least-squares solution. It is also applicable to complex input. In a computational comparison with the state-of-the-art algorithm BiCGSTAB ({\it Bi-conjugate gradient method stabilized}), the Triangle Algorithm is very competitive. In fact, when the iterates of BiCGSTAB do not converge, our algorithm can verify $Ax=b$ is unsolvable and approximate the minimum-norm least-squares solution. The Triangle Algorithm is robust, simple to implement, and requires no preconditioner, making it attractive to practitioners, as well as researchers and educators.

math.NA↗

arXiv2020

A Globally Convergent Newton Method for Polynomials

Newton's method for polynomial root finding is one of mathematics' most well-known algorithms. The method also has its shortcomings: it is undefined at critical points, it could exhibit chaotic behavior and is only guaranteed to converge locally. Based on the {\it Geometric Modulus Principle} for a complex polynomial $p(z)$, together with a {\it Modulus Reduction Theorem} proved here, we develop the {\it Robust Newton's method} (RNM), defined everywhere with a step-size that guarantees an {\it a priori} reduction in polynomial modulus in each iteration. Furthermore, we prove RNM iterates converge globally, either to a root or a critical point. Specifically, given $\varepsilon $ and any seed $z_0$, in $t=O(1/\varepsilon^{2})$ iterations of RNM, independent of degree of $p(z)$, either $|p(z_t)| \leq \varepsilon$ or $|p(z_t) p'(z_t)| \leq \varepsilon$. By adjusting the iterates at {\it near-critical points}, we describe a {\it modified} RNM that necessarily convergence to a root. In combination with Smale's point estimation, RNM results in a globally convergent Newton's method having a locally quadratic rate. We present sample polynomiographs that demonstrate how in contrast with Newton's method RNM smooths out the fractal boundaries of basins of attraction of roots. RNM also finds potentials in computing all roots of arbitrary degree polynomials. A particular consequence of RNM is a simple algorithm for solving cubic equations.

math.NA↗

arXiv2020

Collatz polynomials: an introduction with bounds on their zeros

The Collatz Conjecture (also known as the 3x+1 Problem) proposes that the following algorithm will, after a certain number of iterations, always yield the number 1: given a natural number, multiply by three and add one if the number is odd, halve the resulting number, then repeat. In this article, for each $N$ for which the Collatz Conjecture holds we define the $N^{th}$ Collatz polynomial to be the monic polynomial with constant term $N$ and $k^{th}$ term (for $k > 1$) the $k^{th}$ iterate of $N$ under the Collatz function. In particular, we bound the moduli of the roots of these polynomials, prove theorems on when they have rational integer roots, and suggest further applications and avenues of research.

math.NT↗

arXiv2019

A Spectral Generalization of Von Neumann Minimax Theorem

Given $n \times n$ real symmetric matrices $A_1, \dots, A_m$, the following {\it spectral minimax} property holds: $$\min_{X \in \mathbfΔ_n} \max_{y \in S_m} \sum_{i=1}^m y_iA_i \bullet X=\max_{y \in S_m} \min_{X \in \mathbfΔ_n} \sum_{i=1}^m y_iA_i \bullet X,$$ where $S_m$ is the simplex and $\mathbfΔ_n$ the spectraplex. For diagonal $A_i$'s this reduces to the classic minimax.

math.OC↗

arXiv2019

A Triangle Algorithm for Semidefinite Version of Convex Hull Membership Problem

Given a subset $\mathbf{S}=\{A_1, \dots, A_m\}$ of $\mathbb{S}^n$, the set of $n \times n$ real symmetric matrices, we define its {\it spectrahull} as the set $SH(\mathbf{S}) = \{p(X) \equiv (Tr(A_1 X), \dots, Tr(A_m X))^T : X \in \mathbfΔ_n\}$, where ${\bf Δ}_n$ is the {\it spectraplex}, $\{ X \in \mathbb{S}^n : Tr(X)=1, X \succeq 0 \}$. We let {\it spectrahull membership} (SHM) to be the problem of testing if a given $b \in \mathbb{R}^m$ lies in $SH(\mathbf{S})$. On the one hand when $A_i$'s are diagonal matrices, SHM reduces to the {\it convex hull membership} (CHM), a fundamental problem in LP. On the other hand, a bounded SDP feasibility is reducible to SHM. By building on the {\it Triangle Algorithm} (TA) \cite{kalchar,kalsep}, developed for CHM and its generalization, we design a TA for SHM, where given $\varepsilon$, in $O(1/\varepsilon^2)$ iterations it either computes a hyperplane separating $b$ from $SH(\mathbf{S})$, or $X_\varepsilon \in \mathbfΔ_n$ such that $\Vert p(X_\varepsilon) - b \Vert \leq \varepsilon R$, $R$ maximum error over $\mathbfΔ_n$. Under certain conditions iteration complexity improves to $O(1/\varepsilon)$ or even $O(\ln 1/\varepsilon)$. The worst-case complexity of each iteration is $O(mn^2)$, plus testing the existence of a pivot, shown to be equivalent to estimating the least eigenvalue of a symmetric matrix. This together with a semidefinite version of Carathéodory theorem allow implementing TA as if solving a CHM, resorting to the {\it power method} only as needed, thereby improving the complexity of iterations. The proposed Triangle Algorithm for SHM is simple, practical and applicable to general SDP feasibility and optimization. Also, it extends to a spectral analogue of SVM for separation of two spectrahulls.

math.OC↗

arXiv2018

Robust Vertex Enumeration for Convex Hulls in High Dimensions

Computation of the vertices of the convex hull of a set $S$ of $n$ points in $\mathbb{R} ^m$ is a fundamental problem in computational geometry, optimization, machine learning and more. We present "All Vertex Triangle Algorithm" (AVTA), a robust and efficient algorithm for computing the subset $\overline S$ of all $K$ vertices of $conv(S)$, the convex hull of $S$. If $Γ_*$ is the minimum of the distances from each vertex to the convex hull of the remaining vertices, given any $γ\leq γ_* = Γ_*/R$, $R$ the diameter of $S$, $AVTA$ computes $\overline S$ in $O(nK(m+ γ^{-2}))$ operations. If $γ_*$ is unknown but $K$ is known, AVTA computes $\overline S$ in $O(nK(m+ γ_*^{-2})) \log(γ_*^{-1})$ operations. More generally, given $t \in (0,1)$, AVTA computes a subset $\overline S^t$ of $\overline S$ in $O(n |\overline S^t|(m+ t^{-2}))$ operations, where the distance between any $p \in conv(S)$ to $conv(\overline S^t)$ is at most $t R$. Next we consider AVTA where input is $S_\varepsilon$, an $\varepsilon$ perturbation of $S$. Assuming a bound on $\varepsilon$ in terms of the minimum of the distances of vertices of $conv(S)$ to the convex hull of the remaining point of $S$, we derive analogous complexity bounds for computing $\overline S_\varepsilon$. We also analyze AVTA under random projections of $S$ or $S_\varepsilon$. Finally, via AVTA we design new practical algorithms for two popular machine learning problems: topic modeling and non-negative matrix factorization. For topic models AVTA leads to significantly better reconstruction of the topic-word matrix than state of the art approaches~\cite{arora2013practical, bansal2014provable}. For non-negative matrix AVTA is competitive with existing methods~\cite{arora2012computing}. Empirically AVTA is robust and can handle larger amounts of noise than existing methods.

cs.CG↗

arXiv2018

The edge-vertex inequality in a planar graph and a bipartition for the class of all planar graphs

For a planar graph with a given f-vector $(f_{0}, f_{1}, f_{2}),$ we introduce a cubic polynomial whose coefficients depend on the f-vector. The planar graph is said to be real if all the roots of the corresponding polynomial are real. Thus we have a bipartition of all planar graphs into two disjoint class of graphs, real and complex ones. As a contribution toward a full recognition of planar graphs in this bipartition, we study and recognize completely a subclass of planar graphs that includes all the connected grid subgraphs. Finally, all the 2-connected triangle-free complex planar graphs of 7 vertices are listed.

math.CO↗

arXiv2017

An Invitation to Polynomiography via Exponential Series

The subject of Polynomiography deals with algorithmic visualization of polynomial equations, having many applications in STEM and art, see [Kal04]. Here we consider the polynomiography of the partial sums of the exponential series. While the exponential function is taught in standard calculus courses, it is unlikely that properties of zeros of its partial sums are considered in such courses, let alone their visualization as science or art. The Monthly article Zemyan discusses some mathematical properties of these zeros. Here we exhibit some fractal and non-fractal polynomiographs of the partial sums while also presenting a brief introduction of the underlying concepts. Polynomiography establishes a different kind of appreciation of the significance of polynomials in STEM, as well as in art. It helps in the teaching of various topics at diverse levels. It also leads to new discoveries on polynomials and inspires new applications. We also present a link for the educator to get access to a demo polynomiography software together with a module that helps teach basic topics to middle and high school students, as well as undergraduates.

math.HO↗

arXiv2016

An Algorithmic Separating Hyperplane Theorem and Its Applications

We first prove a new separating hyperplane theorem characterizing when a pair of compact convex subsets $K, K'$ of the Euclidean space intersect, and when they are disjoint. The theorem is distinct from classical separation theorems. It generalizes the {\it distance duality} proved in our earlier work for testing the membership of a distinguished point in the convex hull of a finite point set. Next by utilizing the theorem, we develop a substantially generalized and stronger version of the {\it Triangle Algorithm} introduced in the previous work to perform any of the following three tasks: (1) To compute a pair $(p,p') \in K \times K'$, where either the Euclidean distance $d(p,p')$ is to within a prescribed tolerance, or the orthogonal bisecting hyperplane of the line segment $pp'$ separates the two sets; (2) When $K$ and $K'$ are disjoint, to compute $(p,p') \in K \times K'$ so that $d(p,p')$ approximates $d(K,K')$ to within a prescribed tolerance; (3) When $K$ and $K'$ are disjoint, to compute a pair of parallel supporting hyperplanes $H,H'$ so that $d(H,H')$ is to within a prescribed tolerance of the optimal margin. The worst-case complexity of each iteration is solving a linear objective over $K$ or $K'$. The resulting algorithm is a fully polynomial-time approximation scheme for such important special cases as when $K$ and $K'$ are convex hulls of finite points sets, or the intersection of a finite number of halfspaces. The results find many theoretical and practical applications, such as in machine learning, statistics, linear, quadratic and convex programming. In particular, in a separate article we report on a comparison of the Triangle Algorithm and SMO for solving the hard margin problem. In future work we extend the applications to combinatorial and NP-complete problems.

cs.CC↗

arXiv2016

A Comparison of the Triangle Algorithm and SMO for Solving the Hard Margin Problem

In this article we consider the problem of testing, for two finite sets of points in the Euclidean space, if their convex hulls are disjoint and computing an optimal supporting hyperplane if so. This is a fundamental problem of classification in machine learning known as the hard-margin SVM. The problem can be formulated as a quadratic programming problem. The SMO algorithm is the current state of art algorithm for solving it, but it does not answer the question of separability. An alternative to solving both problems is the Triangle Algorithm, a geometrically inspired algorithm, initially described for the convex hull membership problem, a fundamental problem in linear programming. First, we describe the experimental performance of the Triangle Algorithm for testing the intersection of two convex hulls. Next, we compare the performance of Triangle Algorithm with SMO for finding the optimal supporting hyperplane. Based on experimental results ranging up to 5000 points in each set in dimensions up to 10000, the Triangle Algorithm outperforms SMO.

cs.CG↗