arXiv subjects

Xiaodong Luo

Publications and source records attributed to Xiaodong Luo.

At least 37 records · Page 2Linked to original sources

arXiv2024

An Efficient Unsupervised Framework for Convex Quadratic Programs via Deep Unrolling

Quadratic programs (QPs) arise in various domains such as machine learning, finance, and control. Recently, learning-enhanced primal-dual hybrid gradient (PDHG) methods have shown great potential in addressing large-scale linear programs; however, this approach has not been extended to QPs. In this work, we focus on unrolling "PDQP", a PDHG algorithm specialized for convex QPs. Specifically, we propose a neural network model called "PDQP-net" to learn optimal QP solutions. Theoretically, we demonstrate that a PDQP-net of polynomial size can align with the PDQP algorithm, returning optimal primal-dual solution pairs. We propose an unsupervised method that incorporates KKT conditions into the loss function. Unlike the standard learning-to-optimize framework that requires optimization solutions generated by solvers, our unsupervised method adjusts the network weights directly from the evaluation of the primal-dual gap. This method has two benefits over supervised learning: first, it helps generate better primal-dual gap since the primal-dual gap is in the objective function; second, it does not require solvers. We show that PDQP-net trained in this unsupervised manner can effectively approximate optimal QP solutions. Extensive numerical experiments confirm our findings, indicating that using PDQP-net predictions to warm-start PDQP can achieve up to 45% acceleration on QP instances. Moreover, it achieves 14% to 31% acceleration on out-of-distribution instances.

math.OC↗

arXiv2024

A Unified Three-State Model Framework for Analysis of Treatment Crossover in Survival Trials

We present a unified three-state model (TSM) framework for evaluating treatment effects in clinical trials in the presence of treatment crossover. Researchers have proposed diverse methodologies to estimate the treatment effect that would have hypothetically been observed if treatment crossover had not occurred. However, there is little work on understanding the connections between these different approaches from a statistical point of view. The proposed TSM framework unifies existing methods, effectively identifying potential biases, model assumptions, and inherent limitations for each method. This can guide researchers in understanding when these methods are appropriate and choosing a suitable approach for their data. The TSM framework also facilitates the creation of new methods to adjust for confounding effects from treatment crossover. To illustrate this capability, we introduce a new imputation method that falls under its scope. Through simulation experiments, we demonstrate the performance of different approaches for estimating the treatment effects. Codes for implementing the methods within the TSM framework are available at https://github.com/JasonZhao111/TSM.

stat.ME↗

arXiv2024

Equilibria for Time-inconsistent Singular Control Problems

We study a time-inconsistent singular control problem originating from irreversible reinsurance decisions with non-exponential discount. A novel definition of equilibrium for time-inconsistent singular control problems is introduced. For the problem with non-exponential discount, both sufficient and necessary conditions are derived, providing a thorough mathematical characterization of the equilibrium. Specifically, the equilibrium can be characterized by an extended HJB system, which is a coupled system of non-local parabolic equations with free boundaries. Finally, by showing the existence of classical solutions to the extended HJB system, the existence of equilibria is established under some additional assumptions.

math.OC↗

arXiv2024

Stackelberg reinsurance and premium decisions with MV criterion and irreversibility

We study a reinsurance Stackelberg game in which both the insurer and the reinsurer adopt the mean-variance (abbr. MV) criterion in their decision-making and the reinsurance is irreversible. We apply a unified singular control framework where irreversible reinsurance contracts can be signed in both discrete and continuous times. The results theoretically illustrate that, rather than continuous-time contracts or a bunch of discrete-time contracts, a single once-for-all reinsurance contract is preferred. Moreover, the Stackelberg game turns out to be centering on the signing time of the single contract. The insurer signs the contract if the premium rate is lower than a time-dependent threshold and the reinsurer designs a premium that triggers the signing of the contract at his preferred time. Further, we find that reinsurance preference, discount and reversion have a decreasing dominance in the reinsurer's decision-making, which is not seen for the insurer.

q-fin.MF↗

arXiv2023

Combining direct and indirect sparse data for learning generalizable turbulence models

Learning turbulence models from observation data is of significant interest in discovering a unified model for a broad range of practical flow applications. Either the direct observation of Reynolds stress or the indirect observation of velocity has been used to improve the predictive capacity of turbulence models. In this work, we propose combining the direct and indirect sparse data to train neural network-based turbulence models. The backpropagation technique and the observation augmentation approach are used to train turbulence models with different observation data in a unified ensemble-based framework. These two types of observation data can explore synergy to constrain the model training in different observation spaces, which enables learning generalizable models from very sparse data. The present method is tested in secondary flows in a square duct and separated flows over periodic hills. Both cases demonstrate that combining direct and indirect observations is able to improve the generalizability of the learned model in similar flow configurations, compared to using only indirect data. The ensemble-based method can serve as a practical tool for model learning from different types of observations due to its non-intrusive and derivative-free nature.

physics.flu-dyn↗

arXiv2023

A GNN-Guided Predict-and-Search Framework for Mixed-Integer Linear Programming

Mixed-integer linear programming (MILP) is widely employed for modeling combinatorial optimization problems. In practice, similar MILP instances with only coefficient variations are routinely solved, and machine learning (ML) algorithms are capable of capturing common patterns across these MILP instances. In this work, we combine ML with optimization and propose a novel predict-and-search framework for efficiently identifying high-quality feasible solutions. Specifically, we first utilize graph neural networks to predict the marginal probability of each variable, and then search for the best feasible solution within a properly defined ball around the predicted solution. We conduct extensive experiments on public datasets, and computational results demonstrate that our proposed framework achieves 51.1% and 9.9% performance improvements to MILP solvers SCIP and Gurobi on primal gaps, respectively.

math.OC↗

arXiv2023

A Simulation-free Group Sequential Design with Max-combo Tests in the Presence of Non-proportional Hazards

Non-proportional hazards (NPH) have been observed recently in many immuno-oncology clinical trials. Weighted log-rank tests (WLRT) with suitably chosen weights can be used to improve the power of detecting the difference of the two survival curves in the presence of NPH. However, it is not easy to choose a proper WLRT in practice when both robustness and efficiency are considered. A versatile maxcombo test was proposed to achieve the balance of robustness and efficiency and has received increasing attentions in both methodology development and application. However, survival trials often warrant interim analyses due to its high cost and long duration. The integration and application of maxcombo tests in interim analyses often require extensive simulation studies. In this paper, we propose a simulation-free approach for group sequential design with maxcombo test in survival trials. The simulation results support that the proposed approaches successfully control both the type I error rate and offer great accuracy and flexibility in estimating sample sizes, at the expense of light computation burden. Notably, our methods display a strong robustness towards various model misspecifications, and have been implemented in an R package for free access online.

stat.ME↗

arXiv2022

Consumption-investment decisions with endogenous reference point and drawdown constraint

We propose a consumption-investment decision model where past consumption peak $h$ plays a crucial role. There are two important consumption levels: the lowest constrained level and a reference level, at which the risk aversion in terms of consumption rate is changed. We solve this stochastic control problem and derive the value function, optimal consumption plan, and optimal investment strategy in semi-explicit forms. We find five important thresholds of wealth, all as functions of $h$, and most of them are nonlinear functions. As can be seen from numerical results and theoretical analysis, this intuitive and simple model has significant economic implications, and there are at least three important predictions: the marginal propensity to consume out of wealth is generally decreasing but can be increasing for intermediate wealth levels, and it jumps inversely proportional to the risk aversion at the reference point; the implied relative risk aversion is roughly a smile in wealth; the welfare of the poor is more vulnerable to wealth shocks than the wealthy. Moreover, locally changing the risk aversion influences the optimal strategies globally, revealing some risk allocation behaviors.

q-fin.PM↗

arXiv2022

Ensemble Kalman method for learning turbulence models from indirect observation data

In this work, we propose using an ensemble Kalman method to learn a nonlinear eddy viscosity model, represented as a tensor basis neural network, from velocity data. Data-driven turbulence models have emerged as a promising alternative to traditional models for providing closure mapping from the mean velocities to Reynolds stresses. Most data-driven models in this category need full-field Reynolds stress data for training, which not only places stringent demand on the data generation but also makes the trained model ill-conditioned and lacks robustness. This difficulty can be alleviated by incorporating the Reynolds-averaged Navier-Stokes (RANS) solver in the training process. However, this would necessitate developing adjoint solvers of the RANS model, which requires extra effort in code development and maintenance. Given this difficulty, we present an ensemble Kalman method with an adaptive step size to train a neural network-based turbulence model by using indirect observation data. To our knowledge, this is the first such attempt in turbulence modelling. The ensemble method is first verified on the flow in a square duct, where it correctly learns the underlying turbulence models from velocity data. Then, the generalizability of the learned model is evaluated on a family of separated flows over periodic hills. It is demonstrated that the turbulence model learned in one flow can predict flows in similar configurations with varying slopes.

physics.flu-dyn↗

arXiv2022

Continuous Hyper-parameter OPtimization (CHOP) in an ensemble Kalman filter

Practical data assimilation algorithms often contain hyper-parameters, which may arise due to, for instance, the use of certain auxiliary techniques like covariance inflation and localization in an ensemble Kalman filter, the re-parameterization of certain quantities such as model and/or observation error covariance matrices, and so on. Given the richness of the established assimilation algorithms, and the abundance of the approaches through which hyper-parameters are introduced to the assimilation algorithms, one may ask whether it is possible to develop a sound and generic method to efficiently choose various types of (sometimes high-dimensional) hyper-parameters. This work aims to explore a feasible, although likely partial, answer to this question. Our main idea is built upon the notion that a data assimilation algorithm with hyper-parameters can be considered as a parametric mapping that links a set of quantities of interest (e.g., model state variables and/or parameters) to a corresponding set of predicted observations in the observation space. As such, the choice of hyper-parameters can be recast as a parameter estimation problem, in which our objective is to tune the hyper-parameters in such a way that the resulted predicted observations can match the real observations to a good extent. From this perspective, we propose a hyper-parameter estimation workflow and investigate the performance of this workflow in an ensemble Kalman filter. In a series of experiments, we observe that the proposed workflow works efficiently even in the presence of a relatively large amount (up to $10^3$) of hyper-parameters, and exhibits reasonably good and consistent performance under various conditions.

stat.CO↗

arXiv2022

The Machine Learning for Combinatorial Optimization Competition (ML4CO): Results and Insights

Combinatorial optimization is a well-established area in operations research and computer science. Until recently, its methods have focused on solving problem instances in isolation, ignoring that they often stem from related data distributions in practice. However, recent years have seen a surge of interest in using machine learning as a new approach for solving combinatorial problems, either directly as solvers or by enhancing exact solvers. Based on this context, the ML4CO aims at improving state-of-the-art combinatorial optimization solvers by replacing key heuristic components. The competition featured three challenging tasks: finding the best feasible solution, producing the tightest optimality certificate, and giving an appropriate solver configuration. Three realistic datasets were considered: balanced item placement, workload apportionment, and maritime inventory routing. This last dataset was kept anonymous for the contestants.

cs.LG↗

arXiv2022

Efficient primal heuristics for mixed-integer linear programs

This paper is a short report about our work for the primal task in the Machine Learning for Combinatorial Optimization NeurIPS 2021 Competition. For each dataset of our interest in the competition, we propose customized primal heuristic methods to efficiently identify high-quality feasible solutions. The computational studies demonstrate the superiority of our proposed approaches over the competitors'.

math.OC↗

arXiv2020

Demand Estimation from Sales Transaction Data -- Practical Extensions

In this paper we discuss practical limitations of the standard choice-based demand models used in the literature to estimate demand from sales transaction data. We present modifications and extensions of the models and discuss data preprocessing and solution techniques which are useful for practitioners dealing with sales transaction data. Among these, we present an algorithm to split sales transaction data observed under partial availability, we extend a popular Expectation Maximization (EM) algorithm for non-homogeneous product sets, and we develop two iterative optimization algorithms which can handle much of the extensions discussed in the paper.

math.OC↗

arXiv2020

Bayesian Methods for the Analysis of Early-Phase Oncology Basket Trials with Information Borrowing across Cancer Types

Research in oncology has changed the focus from histological properties of tumors in a specific organ to a specific genomic aberration potentially shared by multiple cancer types. This motivates the basket trial, which assesses the efficacy of treatment simultaneously on multiple cancer types that have a common aberration. Although the assumption of homogeneous treatment effects seems reasonable given the shared aberration, in reality, the treatment effect may vary by cancer type, and potentially only a subgroup of the cancer types respond to the treatment. Various approaches have been proposed to increase the trial power by borrowing information across cancer types, which, however, tend to inflate the type I error rate. In this paper, we review some representative Bayesian information borrowing methods for the analysis of early-phase basket trials. We then propose a novel method called the Bayesian hierarchical model with a correlated prior (CBHM), which conducts more flexible borrowing across cancer types according to sample similarity. We did simulation studies to compare CBHM with independent analysis and three information borrowing approaches: the conventional Bayesian hierarchical model, the EXNEX approach and Liu's two-stage approach. Simulation results show that all information borrowing approaches substantially improve the power of independent analysis if a large proportion of the cancer types truly respond to the treatment. Our proposed CBHM approach shows an advantage over the existing information borrowing approaches, with a power similar to that of EXNEX or Liu's approach, but the potential to provide substantially better control of type I error rate.

stat.AP↗

arXiv2019

A Decision Support System for Multi-target Geosteering

Geosteering is a sequential decision process under uncertainty. The goal of geosteering is to maximize the expected value of the well, which should be defined by an objective value-function for each operation. In this paper we present a real-time decision support system (DSS) for geosteering that aims to approximate the uncertainty in the geological interpretation with an ensemble of geomodel realizations. As the drilling operation progresses, the ensemble Kalman filter is used to sequentially update the realizations using the measurements from real-time logging while drilling. At every decision point a discrete dynamic programming algorithm computes all potential well trajectories for the entire drilling operation and the corresponding value of the well for each realization. Then, the DSS considers all immediate alternatives (continue/steer/stop) and chooses the one that gives the best predicted value across the realizations. This approach works for a variety of objectives and constraints and suggests reproducible decisions under uncertainty. Moreover, it has real-time performance. The system is tested on synthetic cases in a layer-cake geological environment where the target layer should be selected dynamically based on the prior (pre-drill) model and the electromagnetic observations received while drilling. The numerical closed-loop simulation experiments demonstrate the ability of the DSS to perform successful geosteering and landing of a well for different geological configurations of drilling targets. Furthermore, the DSS allows to adjust and reweight the objectives, making the DSS useful before fully automated geosteering becomes reality.

cs.CE↗

arXiv2019

Alternative Analysis Methods for Time to Event Endpoints under Non-proportional Hazards: A Comparative Analysis

The log-rank test is most powerful under proportional hazards (PH). In practice, non-PH patterns are often observed in clinical trials, such as in immuno-oncology; therefore, alternative methods are needed to restore the efficiency of statistical testing. Three categories of testing methods were evaluated, including weighted log-rank tests, Kaplan-Meier curve-based tests (including weighted Kaplan-Meier and Restricted Mean Survival Time, RMST), and combination tests (including Breslow test, Lee's combo test, and MaxCombo test). Nine scenarios representing the PH and various non-PH patterns were simulated. The power, type I error, and effect estimates of each method were compared. In general, all tests control type I error well. There is not a single most powerful test across all scenarios. In the absence of prior knowledge regarding the PH or non-PH patterns, the MaxCombo test is relatively robust across patterns. Since the treatment effect changes overtime under non-PH, the overall profile of the treatment effect may not be represented comprehensively based on a single measure. Thus, multiple measures of the treatment effect should be pre-specified as sensitivity analyses to evaluate the totality of the data.

stat.AP↗

arXiv2019

FTGAN: A Fully-trained Generative Adversarial Networks for Text to Face Generation

As a sub-domain of text-to-image synthesis, text-to-face generation has huge potentials in public safety domain. With lack of dataset, there are almost no related research focusing on text-to-face synthesis. In this paper, we propose a fully-trained Generative Adversarial Network (FTGAN) that trains the text encoder and image decoder at the same time for fine-grained text-to-face generation. With a novel fully-trained generative network, FTGAN can synthesize higher-quality images and urge the outputs of the FTGAN are more relevant to the input sentences. In addition, we build a dataset called SCU-Text2face for text-to-face synthesis. Through extensive experiments, the FTGAN shows its superiority in boosting both generated images' quality and similarity to the input descriptions. The proposed FTGAN outperforms the previous state of the art, boosting the best reported Inception Score to 4.63 on the CUB dataset. On SCU-text2face, the face images generated by our proposed FTGAN just based on the input descriptions is of average 59% similarity to the ground-truth, which set a baseline for text-to-face synthesis.

cs.CV↗

arXiv2019

Ensemble-based kernel learning for a class of data assimilation problems with imperfect forward simulators

Simulator imperfection, often known as model error, is ubiquitous in practical data assimilation problems. Despite the enormous efforts dedicated to addressing this problem, properly handling simulator imperfection in data assimilation remains to be a challenging task. In this work, we propose an approach to dealing with simulator imperfection from a point of view of functional approximation that can be implemented through a certain machine learning method, such as kernel-based learning adopted in the current work. To this end, we start from considering a class of supervised learning problems, and then identify similarities between supervised learning and variational data assimilation. These similarities found the basis for us to develop an ensemble-based learning framework to tackle supervised learning problems, while achieving various advantages of ensemble-based methods over the variational ones. After establishing the ensemble-based learning framework, we proceed to investigate the integration of ensemble-based learning into an ensemble-based data assimilation framework to handle simulator imperfection. In the course of our investigations, we also develop a strategy to tackle the issue of multi-modality in supervised-learning problems, and transfer this strategy to data assimilation problems to help improve assimilation performance. For demonstration, we apply the ensemble-based learning framework and the integrated, ensemble-based data assimilation framework to a supervised learning problem and a data assimilation problem with an imperfect forward simulator, respectively. The experiment results indicate that both frameworks achieve good performance in relevant case studies, and that functional approximation through machine learning may serve as a viable way to account for simulator imperfection in data assimilation problems.

math.OC↗