arXiv subjects

Radoslav Harman

Publications and source records attributed to Radoslav Harman.

At least 19 recordsLinked to original sources

arXiv2025

Laplacian Network Optimization via Information Functions

Designing networks to optimize robustness and other performance metrics is a well-established problem with applications ranging from electrical engineering to communication networks. Many such performance measures rely on the Laplacian spectrum; notable examples include total effective resistance, the number of spanning trees, and algebraic connectivity. This paper advances the study of Laplacian-based network optimization by drawing on ideas from experimental design in statistics. We present a theoretical framework for analyzing performance measures by introducing the notion of information functions, which captures a set of their desirable properties. Then, we formulate a new parametric family of information functions, Kiefer's measures, which encompasses the three most common spectral objectives. We provide a regular reformulation of the Laplacian optimization problem, and we use this reformulation to compute directional derivatives of Kiefer's measures. The directional derivatives provide a unified treatment of quantities recurring in Laplacian optimization, such as gradients and subgradients, and we show that they are connected to Laplacian-based measures of node distance, which we call node dissimilarities. We apply the node dissimilarities to derive efficient rank-one update formulas for Kiefer's criteria, and to devise a new edge-exchange method for network optimization. These update formulas enable greedy and exchange algorithms with reduced asymptotic time complexity.

cs.SI↗

arXiv2025

Removal of Redundant Candidate Points for the Exact D-Optimal Design Problem

One of the most common problems in statistical experimentation is computing D-optimal designs on large finite candidate sets. While optimal approximate (i.e., infinite-sample) designs can be efficiently computed using convex methods, constructing optimal exact (i.e., finite-sample) designs is a substantially more difficult integer-optimization problem. In this paper, we propose necessary conditions, based on approximate designs, that must be satisfied by any support point of a D-optimal exact design. These conditions enable rapid elimination of redundant candidate points without loss of optimality, thereby reducing memory requirements and runtime of subsequent exact design algorithms. In addition, we prove that for sufficiently large sample sizes, the supports of D-optimal exact designs are contained in a typically small maximum-variance set. We demonstrate the approach on randomly generated benchmark models with candidate sets up to 100 million points, and on commonly used constrained mixture models with up to one million points. The proposed approach reduces the initial candidate sets by several orders of magnitude, thereby making it possible to compute exact D-optimal designs for these problems via mixed-integer second-order cone programming, which provides optimality guarantees.

stat.CO↗

arXiv2025

Optimal Exact Designs of Multiresponse Experiments under Linear and Sparsity Constraints

We propose a computational approach to constructing exact designs on finite design spaces that are optimal for multiresponse regression experiments under a combination of the standard linear and specific 'sparsity' constraints. The linear constraints address, for example, limits on multiple resource consumption and the problem of optimal design augmentation, while the sparsity constraints control the set of distinct trial conditions utilized by the design. The key idea is to construct an artificial optimal design problem that can be solved using any existing mathematical programming technique for univariate-response optimal designs under pure linear constraints. The solution to this artificial problem can then be directly converted into an optimal design for the primary multivariate-response setting with combined linear and sparsity constraints. We demonstrate the utility and flexibility of the approach through dose-response experiments with constraints on safety, efficacy, and cost, where cost also depends on the number of distinct doses used.

stat.ME↗

arXiv2025

Green LIME: Improving AI Explainability through Design of Experiments

In artificial intelligence (AI), the complexity of many models and processes surpasses human understanding, making it challenging to determine why a specific prediction is made. This lack of transparency is particularly problematic in critical fields like healthcare, where trust in a model's predictions is paramount. As a result, the explainability of machine learning (ML) and other complex models has become a key area of focus. Efforts to improve model explainability often involve experimenting with AI systems and approximating their behavior through interpretable surrogate mechanisms. However, these procedures can be resource-intensive. Optimal design of experiments, which seeks to maximize the information obtained from a limited number of observations, offers promising methods for improving the efficiency of these explainability techniques. To demonstrate this potential, we explore Local Interpretable Model-agnostic Explanations (LIME), a widely used method introduced by Ribeiro et al. (2016). LIME provides explanations by generating new data points near the instance of interest and passing them through the model. While effective, this process can be computationally expensive, especially when predictions are costly or require many samples. LIME is highly versatile and can be applied to a wide range of models and datasets. In this work, we focus on models involving tabular data, regression tasks, and linear models as interpretable local approximations. By utilizing optimal design of experiments' techniques, we reduce the number of function evaluations of the complex model, thereby reducing the computational effort of LIME by a significant amount. We consider this modified version of LIME to be energy-efficient or "green".

stat.ML↗

arXiv2024

A Randomized Exchange Algorithm for Optimal Design of Multi-Response Experiments

Despite the increasing prevalence of vector observations, computation of optimal experimental design for multi-response models has received limited attention. To address this problem within the framework of approximate designs, we introduce mREX, an algorithm that generalizes the randomized exchange algorithm REX (J Am Stat Assoc 115:529, 2020), originally specialized for single-response models. The mREX algorithm incorporates several improvements: a novel method for computing efficient sparse initial designs, an extension to all differentiable Kiefer's optimality criteria, and an efficient method for performing optimal exchanges of weights. For the most commonly used D-optimality criterion, we propose a technique for optimal weight exchanges based on the characteristic matrix polynomial. The mREX algorithm is applicable to linear, nonlinear, and generalized linear models, and scales well to large problems. It typically converges to optimal designs faster than available alternative methods, although it does not require advanced mathematical programming solvers. We demonstrate the usefulness of mREX to bivariate dose-response Emax models for clinical trials, both without and with the inclusion of covariates.

stat.CO↗

arXiv2023

The Polytope of Optimal Approximate Designs: Extending the Selection of Informative Experiments

Consider the problem of constructing an experimental design, optimal for estimating parameters of a given statistical model with respect to a chosen criterion. To address this problem, the literature usually provides a single solution. Often, however, there exists a rich set of optimal designs, and the knowledge of this set can lead to substantially greater freedom to select an appropriate experiment. In this paper, we demonstrate that the set of all optimal approximate designs generally corresponds to a polytope. Particularly important elements of the polytope are its vertices, which we call vertex optimal designs. We prove that the vertex optimal designs possess unique properties, such as small supports, and outline strategies for how they can facilitate the construction of suitable experiments. Moreover, we show that for a variety of situations it is possible to construct the vertex optimal designs with the assistance of a computer, by employing error-free rational-arithmetic calculations. In such cases the vertex optimal designs are exact, often closely related to known combinatorial designs. Using this approach, we were able to determine the polytope of optimal designs for some of the most common multifactor regression models, thereby extending the choice of informative experiments for a large variety of applications.

stat.CO↗

arXiv2023

Mixed-integer linear programming for computing optimal experimental designs

The problem of computing an exact experimental design that is optimal for the least-squares estimation of the parameters of a regression model is considered. We show that this problem can be solved via mixed-integer linear programming (MILP) for a wide class of optimality criteria, including the criteria of A-, I-, G- and MV-optimality. This approach improves upon the current state-of-the-art mathematical programming formulation, which uses mixed-integer second-order cone programming. The key idea underlying the MILP formulation is McCormick relaxation, which critically depends on finite interval bounds for the elements of the covariance matrix of the least-squares estimator corresponding to an optimal exact design. We provide both analytic and algorithmic methods for constructing these bounds. We also demonstrate the unique advantages of the MILP approach, such as the possibility of incorporating multiple design constraints into the optimization problem, including constraints on the variances and covariances of the least-squares estimator.

stat.CO↗

arXiv2021

Optimal Design of Multifactor Experiments via Grid Exploration

For computing efficient approximate designs of multifactor experiments, we propose a simple algorithm based on adaptive exploration of the grid of all combinations of factor levels. We demonstrate that the algorithm significantly outperforms several state-of-the-art competitors for problems with discrete, continuous, as well as mixed factors. Importantly, we provide a free R code that permits direct verification of the numerical results and allows the researchers to easily compute optimal or nearly-optimal experimental designs for their own statistical models.

stat.CO↗

arXiv2019

On greedy heuristics for computing D-efficient saturated subsets

Let $\mathcal{F}$ be a set consisting of $n$ real vectors of dimension $m \leq n$. For any saturated, i.e., $m$-element, subset $\mathcal{S}$ of $\mathcal{F}$, let $\mathrm{vol}(\mathcal{S})$ be the volume of the parallelotope formed by the vectors of $\mathcal{S}$. A set $\mathcal{S}^*$ is called a $D$-optimal saturated subset of $\mathcal{F}$, if it maximizes $\mathrm{vol}(\mathcal{S})$ among all saturated subsets of $\mathcal{F}$. In this paper, we propose two greedy heuristics for the construction of saturated subsets performing well with respect to the criterion of $D$-optimality: an improvement of the method suggested by Galil and Kiefer for the initiation of $D$-optimal experimental design algorithms, and a modification of the Kumar-Yildirim method, the original version of which was proposed for the initiation of the minimum-volume enclosing ellipsoid algorithms. We provide geometric and analytic insights into the two methods, and compare them to the commonly used random and regularized greedy heuristics. We also suggest variants of the greedy methods for a large set $\mathcal{F}$, for the construction of $D$-efficient non-saturated subsets, and for alternative optimality criteria.

stat.CO↗

arXiv2019

Ascent with Quadratic Assistance for the Construction of Exact Experimental Designs

In the area of statistical planning, there is a large body of theoretical knowledge and computational experience concerning so-called optimal approximate designs of experiments. However, for an approximate design to be executed in practice, it must be converted into an exact, i.e., integer, design, which is usually done via rounding procedures. Although rapid, rounding procedures have many drawbacks; in particular, they often yield worse exact designs than heuristics that do not require approximate designs at all. In this paper, we build on an alternative principle of utilizing optimal approximate designs for the computation of optimal, or nearly-optimal, exact designs. The principle, which we call ascent with quadratic assistance (AQuA), is an integer programming method based on the quadratic approximation of the design criterion in the neighborhood of the optimal approximate information matrix. To this end, we present quadratic approximations of all Kiefer's criteria with an integer parameter, including D- and A-optimality and, by a model transformation, I-optimality. Importantly, we prove a low-rank property of the associated quadratic forms, which enables us to apply AQuA to large design spaces, for example via mixed integer conic quadratic solvers. We numerically demonstrate the robustness and superior performance of the proposed method for models under various types of constraints. More precisely, we compute optimal size-constrained exact designs for the model of spring-balance weighing, and optimal symmetric marginally restricted exact designs for the Scheffe mixture model. We also show how can iterative application of AQuA be used for a stratified information-based subsampling of large datasets under a lower bound on the quality and an upper bound on the cost of the subsample.

stat.CO↗

arXiv2019

Iterated partial summations applied to finite-support discrete distributions

The problem of iterated partial summations is solved for some discrete distributions defined on discrete supports. The power method, usually used as a computational approach to finding matrix eigenvalues and eigenvectors, is in some cases an effective tool to prove the existence of the limit distribution, which is then expressed as a solution of a system of linear equations. Some examples are presented.

math.PR↗

arXiv2018

A design criterion for symmetric model discrimination based on nominal confidence sets

Experimental design applications for discriminating between models have been hampered by the assumption to know beforehand which model is the true one, which is counter to the very aim of the experiment. Previous approaches to alleviate this requirement were either symmetrizations of asymmetric techniques, or Bayesian, minimax and sequential approaches. Here we present a genuinely symmetric criterion based on a linearized distance between mean-value surfaces and the newly introduced tool of flexible nominal confidence sets. We demonstrate the computational efficiency of the approach using the proposed criterion and provide a Monte-Carlo evaluation of its discrimination performance on the basis of the likelihood ratio. An application for a pair of competing models in enzyme kinetics is given.

stat.ME↗

arXiv2018

Removal of the points that do not support an E-optimal experimental design

We propose a method of removal of design points that cannot support any E-optimal experimental design of a linear regression model with uncorrelated observations. The proposed method can be used to reduce the size of some large E-optimal design problems such that they can be efficiently solved by semidefinite programming. This paper complements the results of Pronzato [Pronzato, L., 2013. A delimitation of the support of optimal designs for Kiefer's $ϕ_p$-class of criteria. Statistics & Probability Letters 83, 2721--2728], who studied the same problem for analytically simpler criteria of design optimality.

math.ST↗

arXiv2018

A Randomized Exchange Algorithm for Computing Optimal Approximate Designs of Experiments

We propose a class of subspace ascent methods for computing optimal approximate designs that covers both existing as well as new and more efficient algorithms. Within this class of methods, we construct a simple, randomized exchange algorithm (REX). Numerical comparisons suggest that the performance of REX is comparable or superior to the performance of state-of-the-art methods across a broad range of problem structures and sizes. We focus on the most commonly used criterion of D-optimality that also has applications beyond experimental design, such as the construction of the minimum volume ellipsoid containing a given set of data-points. For D-optimality, we prove that the proposed algorithm converges to the optimum. We also provide formulas for the optimal exchange of weights in the case of the criterion of A-optimality. These formulas enable one to use REX for computing A-optimal and I-optimal designs.

stat.CO↗

arXiv2017

Computing optimal experimental designs with respect to a compound Bayes risk criterion

We consider the problem of computing optimal experimental design on a finite design space with respect to a compound Bayes risk criterion, which includes the linear criterion for prediction in a random coefficient regression model. We show that the problem can be restated as constrained A-optimality in an artificial model. This permits using recently developed computational tools, for instance the algorithms based on the second-order cone programming for optimal approximate design, and mixed-integer second-order cone programming for optimal exact designs. We demonstrate the use of the proposed method for the problem of computing optimal designs of a random coefficient regression model with respect to an integrated mean squared error criterion.

stat.CO↗

arXiv2016

Optimal Approximate Designs for Comparison with Control in Dose-Escalation Studies

Consider an experiment, where a new drug is tested for the first time on human subjects - healthy volunteers. Such experiments are often performed as dose-escalation studies: a set of increasing doses is pre-selected, individuals are grouped into cohorts, and in each cohort, the dose number $i$ can be administered only if the dose number $i-1$ has already been tested in the previous cohort. If an adverse effect of a dose is observed, the experiment stops and thus no subjects are exposed to higher doses. In this paper, we assume that the response is affected both by the dose or placebo effects as well as by the cohort effects. We provide optimal approximate designs for selected optimality criteria ($E$-, $MV$- and $LV$-optimality) for estimating the effects of the drug doses compared with the placebo. In particular, we obtain the optimality of Senn designs and extended Senn designs with respect to multiple criteria.

math.ST↗

arXiv2015

Privacy sets for constrained space-filling

The paper provides typology for space filling into what we call "soft" and "hard" methods along with introducing the central notion of privacy sets for dealing with the latter. A heuristic algorithm based on this notion is presented and we compare its performance on some well-known examples.

stat.ME↗

arXiv2015

Computing exact $D$-optimal designs by mixed integer second-order cone programming

Let the design of an experiment be represented by an $s$-dimensional vector $\mathbf {w}$ of weights with nonnegative components. Let the quality of $\mathbf {w}$ for the estimation of the parameters of the statistical model be measured by the criterion of $D$-optimality, defined as the $m$th root of the determinant of the information matrix $M(\mathbf {w})=\sum_{i=1}^sw_iA_iA_i^T$, where $A_i,i=1,\ldots,s$ are known matrices with $m$ rows. In this paper, we show that the criterion of $D$-optimality is second-order cone representable. As a result, the method of second-order cone programming can be used to compute an approximate $D$-optimal design with any system of linear constraints on the vector of weights. More importantly, the proposed characterization allows us to compute an exact $D$-optimal design, which is possible thanks to high-quality branch-and-cut solvers specialized to solve mixed integer second-order cone programming problems. Our results extend to the case of the criterion of $D_K$-optimality, which measures the quality of $\mathbf {w}$ for the estimation of a linear parameter subsystem defined by a full-rank coefficient matrix $K$. We prove that some other widely used criteria are also second-order cone representable, for instance, the criteria of $A$-, $A_K$-, $G$- and $I$-optimality. We present several numerical examples demonstrating the efficiency and general applicability of the proposed method. We show that in many cases the mixed integer second-order cone programming approach allows us to find a provably optimal exact design, while the standard heuristics systematically miss the optimum.

math.ST↗