arXiv subjects

Calvin Tsay

Publications and source records attributed to Calvin Tsay.

46 records · Page 3Linked to original sources

arXiv2023

SnAKe: Bayesian Optimization with Pathwise Exploration

Bayesian Optimization is a very effective tool for optimizing expensive black-box functions. Inspired by applications developing and characterizing reaction chemistry using droplet microfluidic reactors, we consider a novel setting where the expense of evaluating the function can increase significantly when making large input changes between iterations. We further assume we are working asynchronously, meaning we have to select new queries before evaluating previous experiments. This paper investigates the problem and introduces 'Sequential Bayesian Optimization via Adaptive Connecting Samples' (SnAKe), which provides a solution by considering large batches of queries and preemptively building optimization paths that minimize input costs. We investigate some convergence properties and empirically show that the algorithm is able to achieve regret similar to classical Bayesian Optimization algorithms in both synchronous and asynchronous settings, while reducing input costs significantly. We show the method is robust to the choice of its single hyper-parameter and provide a parameter-free alternative.

cs.LG↗

arXiv2022

Tree ensemble kernels for Bayesian optimization with known constraints over mixed-feature spaces

Tree ensembles can be well-suited for black-box optimization tasks such as algorithm tuning and neural architecture search, as they achieve good predictive performance with little or no manual tuning, naturally handle discrete feature spaces, and are relatively insensitive to outliers in the training data. Two well-known challenges in using tree ensembles for black-box optimization are (i) effectively quantifying model uncertainty for exploration and (ii) optimizing over the piece-wise constant acquisition function. To address both points simultaneously, we propose using the kernel interpretation of tree ensembles as a Gaussian Process prior to obtain model variance estimates, and we develop a compatible optimization formulation for the acquisition function. The latter further allows us to seamlessly integrate known constraints to improve sampling efficiency by considering domain-knowledge in engineering settings and modeling search space symmetries, e.g., hierarchical relationships in neural architecture search. Our framework performs as well as state-of-the-art methods for unconstrained black-box optimization over continuous/discrete features and outperforms competing methods for problems combining mixed-variable feature spaces and known input constraints.

stat.ML↗

arXiv2022

OMLT: Optimization & Machine Learning Toolkit

The optimization and machine learning toolkit (OMLT) is an open-source software package incorporating neural network and gradient-boosted tree surrogate models, which have been trained using machine learning, into larger optimization problems. We discuss the advances in optimization technology that made OMLT possible and show how OMLT seamlessly integrates with the algebraic modeling language Pyomo. We demonstrate how to use OMLT for solving decision-making problems in both computer science and engineering.

stat.ML↗

arXiv2022

Maximizing information from chemical engineering data sets: Applications to machine learning

It is well-documented how artificial intelligence can have (and already is having) a big impact on chemical engineering. But classical machine learning approaches may be weak for many chemical engineering applications. This review discusses how challenging data characteristics arise in chemical engineering applications. We identify four characteristics of data arising in chemical engineering applications that make applying classical artificial intelligence approaches difficult: (1) high variance, low volume data, (2) low variance, high volume data, (3) noisy/corrupt/missing data, and (4) restricted data with physics-based limitations. For each of these four data characteristics, we discuss applications where these data characteristics arise and show how current chemical engineering research is extending the fields of data science and machine learning to incorporate these challenges. Finally, we identify several challenges for future research.

stat.ML↗

arXiv2021

Multi-Objective Constrained Optimization for Energy Applications via Tree Ensembles

Energy systems optimization problems are complex due to strongly non-linear system behavior and multiple competing objectives, e.g. economic gain vs. environmental impact. Moreover, a large number of input variables and different variable types, e.g. continuous and categorical, are challenges commonly present in real-world applications. In some cases, proposed optimal solutions need to obey explicit input constraints related to physical properties or safety-critical operating conditions. This paper proposes a novel data-driven strategy using tree ensembles for constrained multi-objective optimization of black-box problems with heterogeneous variable spaces for which underlying system dynamics are either too complex to model or unknown. In an extensive case study comprised of synthetic benchmarks and relevant energy applications we demonstrate the competitive performance and sampling efficiency of the proposed algorithm compared to other state-of-the-art tools, making it a useful all-in-one solution for real-world applications with limited evaluation budgets.

stat.ML↗

arXiv2021

Partition-based formulations for mixed-integer optimization of trained ReLU neural networks

This paper introduces a class of mixed-integer formulations for trained ReLU neural networks. The approach balances model size and tightness by partitioning node inputs into a number of groups and forming the convex hull over the partitions via disjunctive programming. At one extreme, one partition per input recovers the convex hull of a node, i.e., the tightest possible formulation for each node. For fewer partitions, we develop smaller relaxations that approximate the convex hull, and show that they outperform existing formulations. Specifically, we propose strategies for partitioning variables based on theoretical motivations and validate these strategies using extensive computational experiments. Furthermore, the proposed scheme complements known algorithmic approaches, e.g., optimization-based bound tightening captures dependencies within a partition.

math.OC↗

arXiv2021

Between steps: Intermediate relaxations between big-M and convex hull formulations

This work develops a class of relaxations in between the big-M and convex hull formulations of disjunctions, drawing advantages from both. The proposed "P-split" formulations split convex additively separable constraints into P partitions and form the convex hull of the partitioned disjuncts. Parameter P represents the trade-off of model size vs. relaxation strength. We examine the novel formulations and prove that, under certain assumptions, the relaxations form a hierarchy starting from a big-M equivalent and converging to the convex hull. We computationally compare the proposed formulations to big-M and convex hull formulations on a test set including: K-means clustering, P_ball problems, and ReLU neural networks. The computational results show that the intermediate P-split formulations can form strong outer approximations of the convex hull with fewer variables and constraints than the extended convex hull formulations, giving significant computational advantages over both the big-M and convex hull.

math.OC↗

arXiv2020

Identification and online updating of dynamic models for demand response of an industrial air separation unit

Demand-response operation of air separation units requires frequent changes in production rate(s), and scheduling calculations must explicitly consider process dynamics to ensure feasibility of the solutions. To this end, scale-bridging models (SBMs) approximate the scheduling-relevant dynamics of a process and its controller in a low-order representation. In contrast to previous works that have employed nonlinear SBMs, this paper proposes linear SBMs, developed using time-series analysis, to facilitate online scheduling computations. Using a year-long industrial dataset, we find that compact linear SBMs are suitable approximations over typical scheduling horizons, but that their accuracies are unpredictable over time. We introduce a strategy for online updating of the SBMs, based on Kalman filtering schemes for online parameter estimation. The approach greatly improves the accuracy of SBM predictions and will enable the use of linear SBM-based demand-response scheduling in the future.

math.OC↗

arXiv2020

Integrating production scheduling and process control using latent variable dynamic models

Given their increasing participation in fast-changing markets, the integration of scheduling and control is an important consideration in chemical process operations. This generally involves computing optimal production schedules using dynamic models, which is challenging due to the nonlinearity and high-dimensionality of the models of chemical processes. In this paper, we begin by observing that the intrinsic dimensionality of process dynamics (as relevant to scheduling) is often much lower than the number of model state and/or algebraic variables. We introduce a data mining approach to "learn" closed-loop process dynamics on a low-dimensional, latent manifold. The manifold dimensionality is selected based on a tradeoff between model accuracy and complexity. After projecting process data, system identification and optimal scheduling calculations can be performed in the low-dimensional, latent-variable space. We apply these concepts to schedule an air separation unit under time-varying electricity prices. We show that our approach reduces the computational effort, while offering more detailed dynamic information compared to previous related works.

math.OC↗

arXiv2020

Modeling, state estimation, and optimal control for the US COVID-19 outbreak

The novel coronavirus SARS-CoV-2 and resulting COVID-19 disease have had an unprecedented spread and continue to cause an increasing number of fatalities worldwide. While vaccines are still under development, social distancing, extensive testing, and quarantining of confirmed infected subjects remain the most effective measures to contain the pandemic. These measures carry a significant socioeconomic cost. In this work, we introduce a novel optimization-based decision-making framework for managing the COVID-19 outbreak in the US. This includes modeling the dynamics of affected populations, estimating the model parameters and hidden states from data, and an optimal control strategy for sequencing social distancing and testing events such that the number of infections is minimized. The analysis of our extensive computational efforts reveals that social distancing and quarantining are most effective when implemented early, with quarantining of confirmed infected subjects having a much higher impact. Further, we find that "on-off" policies alternating between strict social distancing and relaxing such restrictions can be effective at "flattening" the curve while likely minimizing social and economic cost.

q-bio.PE↗