arXiv subjects

Xinwei Shen

Publications and source records attributed to Xinwei Shen.

At least 37 records · Page 2Linked to original sources

arXiv2025

Towards Better Generalization via Distributional Input Projection Network

As overparameterized models become increasingly prevalent, training loss alone offers limited insight into generalization performance. While smoothness has been linked to improved generalization across various settings, directly enforcing smoothness in neural networks remains challenging. To address this, we introduce Distributional Input Projection Networks (DIPNet), a novel framework that projects inputs into learnable distributions at each layer. This distributional representation induces a smoother loss landscape with respect to the input, promoting better generalization. We provide theoretical analysis showing that DIPNet reduces both local smoothness measures and the Lipschitz constant of the network, contributing to improved generalization performance. Empirically, we validate DIPNet across a wide range of architectures and tasks, including Vision Transformers (ViTs), Large Language Models (LLMs), ResNet and MLPs. Our method consistently enhances test performance under standard settings, adversarial attacks, out-of-distribution inputs, and reasoning benchmarks. We demonstrate that the proposed input projection strategy can be seamlessly integrated into existing models, providing a general and effective approach for boosting generalization performance in modern deep learning.

cs.LG↗

arXiv2025

Reverse Markov Learning: Multi-Step Generative Models for Complex Distributions

Learning complex distributions is a fundamental challenge in contemporary applications. Shen and Meinshausen (2024) introduced engression, a generative approach based on scoring rules that maps noise (and covariates, if available) directly to data. While effective, engression can struggle with highly complex distributions, such as those encountered in image data. In this work, we propose reverse Markov learning (RML), a framework that defines a general forward process transitioning from the target distribution to a known distribution (e.g., Gaussian) and then learns a reverse Markov process using multiple engression models. This reverse process reconstructs the target distribution step by step. This framework accommodates general forward processes, allows for dimension reduction, and naturally discretizes the generative process. In the special case of diffusion-based forward processes, RML provides an efficient discretization strategy for both training and inference in diffusion models. We further introduce an alternating sampling scheme to enhance post-training performance. Our statistical analysis establishes error bounds for RML and elucidates its advantages in estimation efficiency and flexibility in forward process design. Empirical results on simulated and climate data corroborate the theoretical findings, demonstrating the effectiveness of RML in capturing complex distributions.

cs.LG↗

arXiv2025

Causality-Inspired Robustness for Nonlinear Models via Representation Learning

Distributional robustness is a central goal of prediction algorithms due to the prevalent distribution shifts in real-world data. The prediction model aims to minimize the worst-case risk among a class of distributions, a.k.a., an uncertainty set. Causality provides a modeling framework with a rigorous robustness guarantee in the above sense, where the uncertainty set is data-driven rather than pre-specified as in traditional distributional robustness optimization. However, current causality-inspired robustness methods possess finite-radius robustness guarantees only in the linear settings, where the causal relationships among the covariates and the response are linear. In this work, we propose a nonlinear method under a causal framework by incorporating recent developments in identifiable representation learning and establish a distributional robustness guarantee. To our best knowledge, this is the first causality-inspired robustness method with such a finite-radius robustness guarantee in nonlinear settings. Empirical validation of the theoretical findings is conducted on both synthetic data and real-world single-cell data, also illustrating that finite-radius robustness is crucial.

stat.ML↗

arXiv2025

Causality-oriented robustness: exploiting general noise interventions

Since distribution shifts are common in real-world applications, there is a pressing need to develop prediction models that are robust against such shifts. Existing frameworks, such as empirical risk minimization or distributionally robust optimization, either lack generalizability for unseen distributions or rely on postulated distance measures. Alternatively, causality offers a data-driven and structural perspective to robust predictions. However, the assumptions necessary for causal inference can be overly stringent, and the robustness offered by such causal models often lacks flexibility. In this paper, we focus on causality-oriented robustness and propose Distributional Robustness via Invariant Gradients (DRIG), a method that exploits general noise interventions in training data for robust predictions against unseen interventions, and naturally interpolates between in-distribution prediction and causality. In a linear setting, we prove that DRIG yields predictions that are robust among a data-dependent class of distribution shifts. Furthermore, we show that our framework includes anchor regression as a special case, and that it yields prediction models that protect against more diverse perturbations. We establish finite-sample results and extend our approach to semi-supervised domain adaptation to further improve prediction performance. Finally, we empirically validate our methods on synthetic simulations and on single-cell and intensive health care datasets.

stat.ME↗

arXiv2024

Two-Stage Robust Planning Model for Park-Level Integrated Energy System Considering Uncertain Equipment Contingency

To enhance the reliability of Integrated Energy Systems (IESs) and address the research gap in reliability-based planning methods, this paper proposes a two-stage robust planning model specifically for park-level IESs. The proposed planning model considers uncertainties like load demand fluctuations and equipment contingencies, and provides a reliable scheme of equipment selection and sizing for IES investors. Inspired by the unit commitment problem, we formulate an equipment contingency uncertainty set to accurately describe the potential equipment contingencies which happen and can be repaired within a day. Then, a modified nested column-and-constraint generation algorithm is applied to solve this two-stage robust planning model with integer recourse efficiently. In the case study, the role of energy storage system for IES reliability enhancement is analyzed in detail. Computational results demonstrate the advantage of the proposed model over other planning models in terms of improving reliability.

eess.SY↗

arXiv2024

Reliability-Based Planning of Cable Layout for Offshore Wind Farm Electrical Collector System Considering Post-Fault Network Reconfiguration

The electrical collector system (ECS) plays a crucial role in determining the performance of offshore wind farms (OWFs). Existing research has predominantly restricted ECS cable layouts to conventional radial or ring structures and employed graph theory heuristics for solutions. However, both economic efficiency and reliability of the OWFs heavily depend on their ECS structure, and the optimal ECS cable layout often deviates from typical configurations. In this context, this paper introduces a novel reliability-based ECS cable layout planning method for large-scale OWFs, employing a two-stage stochastic programming approach to address uncertainties of wind power and contingencies. To enhance reliability, the model incorporates optimal post-fault network reconfiguration strategies by adjusting wind turbine power supply paths through link cables. To tackle computation challenges arising from numerous contingency scenarios, a customized progressive contingency incorporation (CPCI) framework is developed to solve the model with higher efficiency by iteratively identifying non-trivial scenarios and solving the simplified problems. The convergence and optimality are theoretically proven. Numerical tests on several real-world OWFs validate the necessity of fully optimizing ECS structures and demonstrate the efficiency of the CPCI algorithm.

eess.SY↗

arXiv2024

Engression: Extrapolation through the Lens of Distributional Regression

Distributional regression aims to estimate the full conditional distribution of a target variable, given covariates. Popular methods include linear and tree-ensemble based quantile regression. We propose a neural network-based distributional regression methodology called `engression'. An engression model is generative in the sense that we can sample from the fitted conditional distribution and is also suitable for high-dimensional outcomes. Furthermore, we find that modelling the conditional distribution on training data can constrain the fitted function outside of the training support, which offers a new perspective to the challenging extrapolation problem in nonlinear regression. In particular, for `pre-additive noise' models, where noise is added to the covariates before applying a nonlinear transformation, we show that engression can successfully perform extrapolation under some assumptions such as monotonicity, whereas traditional regression approaches such as least-squares or quantile regression fall short under the same assumptions. Our empirical results, from both simulated and real data, validate the effectiveness of the engression method and indicate that the pre-additive noise model is typically suitable for many real-world scenarios. The software implementations of engression are available in both R and Python.

stat.ME↗

arXiv2024

Invariant Probabilistic Prediction

In recent years, there has been a growing interest in statistical methods that exhibit robust performance under distribution changes between training and test data. While most of the related research focuses on point predictions with the squared error loss, this article turns the focus towards probabilistic predictions, which aim to comprehensively quantify the uncertainty of an outcome variable given covariates. Within a causality-inspired framework, we investigate the invariance and robustness of probabilistic predictions with respect to proper scoring rules. We show that arbitrary distribution shifts do not, in general, admit invariant and robust probabilistic predictions, in contrast to the setting of point prediction. We illustrate how to choose evaluation metrics and restrict the class of distribution shifts to allow for identifiability and invariance in the prototypical Gaussian heteroscedastic linear model. Motivated by these findings, we propose a method to yield invariant probabilistic predictions, called IPP, and study the consistency of the underlying parameters. Finally, we demonstrate the empirical performance of our proposed procedure on simulated as well as on single-cell data.

stat.ME↗

arXiv2024

Dynamic Microgrid Formation Considering Time-dependent Contingency: A Distributionally Robust Approach

The increasing frequency of extreme weather events has posed significant risks to the operation of power grids. During long-duration extreme weather events, microgrid formation (MF) is an essential solution to enhance the resilience of the distribution systems by proactively partitioning the distribution system into several microgrids to mitigate the impact of contingencies. This paper proposes a distributionally robust dynamic microgrid formation (DR-DMF) approach to fully consider the temporal characteristics of line failure probability during long-duration extreme weather events like typhoons. The boundaries of each microgrid are dynamically adjusted to enhance the resilience of the system. Furthermore, the expected load shedding is minimized by a distributionally robust optimization model considering the uncertainty of line failure probability regarding the worst-case distribution of contingencies. The effectiveness of the proposed model is verified by numerical simulations on a modified IEEE 37-node system.

eess.SY↗

arXiv2023

Joint Planning of Active Distribution Network and EV Charging Stations Considering Vehicle-to-Grid Functionality and Reactive Power Support

This paper proposes a collaborative planning model for the active distribution network (ADN) and electric vehicle (EV) charging stations that fully considers the vehicle-to-grid (V2G) function and reactive power support of EVs in different regions. This paper employs a sequential decomposition method based on the physical characteristics of the problem, breaking down the holistic problem into two sub-problems for solution. Subproblem I optimizes the charging and discharging behavior of the autopilot electric vehicles (AEVs) using a mixed-integer linear programming (MILP) model. Subproblem II uses a mixed-integer second-order cone programming (MISOCP) model to plan the ADN and retrofit or construct V2G charging stations (V2GCS), as well as multiple distributed generation resources (DGRs). The paper also analyzes the impact of the bi-directional active-reactive power interaction of V2GCS on ADN planning. The presented model was tested in the 47 nodes ADN in Longgang District, Shenzhen, China, and the IEEE 33 nodes ADN, demonstrating that decomposition can significantly improve the speed of solving large-scale problems while maintaining accuracy with low AEV penetration.

eess.SY↗

arXiv2023

CausalVAE: Structured Causal Disentanglement in Variational Autoencoder

Learning disentanglement aims at finding a low dimensional representation which consists of multiple explanatory and generative factors of the observational data. The framework of variational autoencoder (VAE) is commonly used to disentangle independent factors from observations. However, in real scenarios, factors with semantics are not necessarily independent. Instead, there might be an underlying causal structure which renders these factors dependent. We thus propose a new VAE based framework named CausalVAE, which includes a Causal Layer to transform independent exogenous factors into causal endogenous ones that correspond to causally related concepts in data. We further analyze the model identifiabitily, showing that the proposed model learned from observations recovers the true one up to a certain degree by providing supervision signals (e.g. feature labels). Experiments are conducted on various datasets, including synthetic and real word benchmark CelebA. Results show that the causal representations learned by CausalVAE are semantically interpretable, and their causal relationship as a Directed Acyclic Graph (DAG) is identified with good accuracy. Furthermore, we demonstrate that the proposed CausalVAE model is able to generate counterfactual data through "do-operation" to the causal factors.

cs.LG↗

arXiv2023

A Robust Planning Model for Offshore Microgrid Considering Tidal Power and Desalination

Increasing attention has been paid to resources on islands, thus microgrids on islands need to be invested. Different from onshore microgrids, offshore microgrids (OM) are usually abundant in ocean renewable energy (ORE), such as offshore wind, tidal power generation (TPG), etc. Moreover, some special loads such as seawater desalination unit (SDU) should be included. In this sense, this paper proposes a planning method for OM to minimize the investment cost while the ORE's fluctuation could be accommodated with robustness. First, a deterministic planning model (DPM) is formulated for the OM with TPG and SDU. A robust planning model (RPM) is then developed considering the uncertainties from both TPG and load demand. The Column-and-constraint generation (C&CG) algorithm is then employed to solve the RPM, producing planning results for the OM that is robust against the worst scenario. Results of the case studies show that the investment and operation decisions of the proposed model are robust, and TPG shows good complementarity with the other RESs.

eess.SY↗

arXiv2023

A Smart Switch Configuration and Reliability Assessment Method for Large-Scale Offshore Wind Farm Electrical Collector System

With the development of offshore wind farms (OWFs) in far-offshore and deep-sea areas, each OWF could contain more and more wind turbines and cables, making it imperative to study high-reliability electrical collector system (ECS) for OWF. Enlightened by active distribution network, for OWF, we propose an ECS switch configuration that enables post-fault network recovery, along with a reliability assessment (RA) method based on optimization models. It can also determine the optimal normal state and network reconfiguration strategies to maximize ECS reliability. Case studies on several OWFs demonstrate that the proposed RA method is more computationally efficient and accurate than the traditional sequential Monte-Carlo simulation method. Moreover, the proposed switch configuration, in conjunction with the network reconfiguration strategy and proper topology, provides significant benefits to ECS reliability.

eess.SY↗

arXiv2023

Optimal Planning for Electrical Collector System of Offshore Wind Farm with Double-sided Ring Topology

We propose a planning method for offshore wind farm electrical collector system (OWF-ECS) with double-sided ring topology meeting the "N-1" criterion on cable faults, in which the submarine cables layout of OWF is optimized considering cable length and power losses. The proposed mixed-integer quadratic programming (MIQP) model is based on the Capacitated Vehicle Routing Problem (CVRP) formulation and power network expansion planning, which could approximate the power losses in OWF-ECS. In addition, cross-avoidance constraints are proposed to avoid crossing cables, and the minimum k-degree center tree model is included to improve the convergence. Case studies on OWFs with 30 and 62 WTs demonstrate the effectiveness of the proposed method. Considering the potential outage cost in the radial topology, the total cost of the planning result is reduced by up to 25.9% with reliability improvement. The cable investment is reduced by 4%~8% with the proposed method compared with conventional heuristic methods and Google OR-tools. The proposed method/model can also achieve acceptable computation efficiency and OWF-ECS planning results with good optimality. Moreover, it could be solved by modern commercial solvers/optimization software, thus it's easy to use even for large-scale OWF.

eess.SY↗

arXiv2022

Asymptotic Statistical Analysis of $f$-divergence GAN

Generative Adversarial Networks (GANs) have achieved great success in data generation. However, its statistical properties are not fully understood. In this paper, we consider the statistical behavior of the general $f$-divergence formulation of GAN, which includes the Kullback--Leibler divergence that is closely related to the maximum likelihood principle. We show that for parametric generative models that are correctly specified, all $f$-divergence GANs with the same discriminator classes are asymptotically equivalent under suitable regularity conditions. Moreover, with an appropriately chosen local discriminator, they become equivalent to the maximum likelihood estimate asymptotically. For generative models that are misspecified, GANs with different $f$-divergences {converge to different estimators}, and thus cannot be directly compared. However, it is shown that for some commonly used $f$-divergences, the original $f$-GAN is not optimal in that one can achieve a smaller asymptotic variance when the discriminator training in the original $f$-GAN formulation is replaced by logistic regression. The resulting estimation method is referred to as Adversarial Gradient Estimation (AGE). Empirical studies are provided to support the theory and to demonstrate the advantage of AGE over the original $f$-GANs under model misspecification.

math.ST↗

arXiv2022

Weakly Supervised Disentangled Generative Causal Representation Learning

This paper proposes a Disentangled gEnerative cAusal Representation (DEAR) learning method under appropriate supervised information. Unlike existing disentanglement methods that enforce independence of the latent variables, we consider the general case where the underlying factors of interests can be causally related. We show that previous methods with independent priors fail to disentangle causally related factors even under supervision. Motivated by this finding, we propose a new disentangled learning method called DEAR that enables causal controllable generation and causal representation learning. The key ingredient of this new formulation is to use a structural causal model (SCM) as the prior distribution for a bidirectional generative model. The prior is then trained jointly with a generator and an encoder using a suitable GAN algorithm incorporated with supervised information on the ground-truth factors and their underlying causal structure. We provide theoretical justification on the identifiability and asymptotic convergence of the proposed method. We conduct extensive experiments on both synthesized and real data sets to demonstrate the effectiveness of DEAR in causal controllable generation, and the benefits of the learned representations for downstream tasks in terms of sample efficiency and distributional robustness.

cs.LG↗

arXiv2022

Reframed GES with a Neural Conditional Dependence Measure

In a nonparametric setting, the causal structure is often identifiable only up to Markov equivalence, and for the purpose of causal inference, it is useful to learn a graphical representation of the Markov equivalence class (MEC). In this paper, we revisit the Greedy Equivalence Search (GES) algorithm, which is widely cited as a score-based algorithm for learning the MEC of the underlying causal structure. We observe that in order to make the GES algorithm consistent in a nonparametric setting, it is not necessary to design a scoring metric that evaluates graphs. Instead, it suffices to plug in a consistent estimator of a measure of conditional dependence to guide the search. We therefore present a reframing of the GES algorithm, which is more flexible than the standard score-based version and readily lends itself to the nonparametric setting with a general measure of conditional dependence. In addition, we propose a neural conditional dependence (NCD) measure, which utilizes the expressive power of deep neural networks to characterize conditional independence in a nonparametric manner. We establish the optimality of the reframed GES algorithm under standard assumptions and the consistency of using our NCD estimator to decide conditional independence. Together these results justify the proposed approach. Experimental results demonstrate the effectiveness of our method in causal discovery, as well as the advantages of using our NCD measure over kernel-based measures.

stat.ML↗

arXiv2022

A Stochastic Planning Method for Low-carbon Building-level Integrated Energy System Considering Electric-Heat-V2G Coupling

The concept of low-carbon building is proposed to ameliorate the climate change caused by environmental problems and realize carbon neutrality at the building level in urban areas. In addition, renewable energy curtailment in the power distribution system, as well as low efficiency due to independent operation of traditional energy systems, has been addressed by the application of integrated energy system (IES) to some extent. In this paper, we propose a planning method for low-carbon building-level IES, in which electric vehicles (EV) and the mode of Vehicle to Grid (V2G) are considered and further increase the flexibility of low-carbon buildings. The proposed planning model optimize the investment, operation costs and CO2 emission for building-level IES, so as to achieve the maximum benefit of the construction of the low-carbon building and help the realization of carbon neutrality. Moreover, we consider the uncertainty of distributed renewable energy, multi-energy load fluctuation and the random behavior of EV users, then formulating a two-stage stochastic programming model with chance constraints, in which heuristic moment matching scenario generation (HMMSG) and sample average approximation (SAA) method are applied. In case study, a real IES commercial building in Shanghai, where photovoltaic (PV), energy storage system (ESS), fuel cell (FC), EV, etc. are included as planning options, is used as numerical example to verify the effectiveness of the proposed planning method, with functions of ESS and EV in IES are analyzed in detail in different operation scenarios.

eess.SY↗