arXiv subjects

Nan Chen

Publications and source records attributed to Nan Chen.

At least 109 records · Page 6Linked to original sources

arXiv2022

LASSO-Based Multiple-Line Outage Identification In Partially Observable Power Systems

Phasor measurement units (PMUs) create ample real-time monitoring opportunities for modern power systems. Among them, line outage detection and identification remains a crucial but challenging task. Current works on outage identification succeed in full PMU deployment and single-line outages. Performance however degrades for multiple-line outage with partial system observability. We propose a novel framework of multiple-line outage identification using partial nodal voltage measurements. Using alternating current (AC) power flow model, phase angle signatures of outages are extracted and used to group lines into minimal diagnosable clusters. Identification is then formulated into an underdetermined sparse regression problem solved by lasso. Tested on IEEE 39-bus system with 25% and 50% PMU coverage, the proposed identification method is 93% and 80% accurate for single- and double-line outages. Our study suggests that the AC power flow is better at capturing outage patterns and sacrificing some precision could yield substantial improvement in identification accuracy. These findings could contribute to the development of future control schemes that help power systems resist and recover from outage disruptions in real time.

eess.SY↗

arXiv2022

Bridging Gaps in the Climate Observation Network: A Physics-based Nonlinear Dynamical Interpolation of Lagrangian Ice Floe Measurements via Data-Driven Stochastic Models

Modeling and understanding sea ice dynamics in marginal ice zones relies on acquiring Lagrangian ice floe measurements. However, optical satellite images are susceptible to atmospheric noise, leading to gaps in the retrieved time series of floe positions. This paper presents an efficient and statistically accurate nonlinear dynamical interpolation framework for recovering missing floe observations. It exploits a balanced physics-based and data-driven construction to address the challenges posed by the high-dimensional and nonlinear nature of the coupled atmosphere-ice-ocean system, where effective reduced-order stochastic models, nonlinear data assimilation, and simultaneous parameter estimation are systematically integrated. The new method succeeds in recovering the locations, curvatures, angular displacements, and the associated strong non-Gaussian distributions of the missing floes in the Beaufort Sea. It also accurately estimates floe thickness and recovers the unobserved underlying ocean field with an appropriate uncertainty quantification, advancing our understanding of Arctic climate.

physics.ao-ph↗

arXiv2022

A Physics-Informed Data-Driven Algorithm for Ensemble Forecast of Complex Turbulent Systems

A new ensemble forecast algorithm, named as the physics-informed data-driven algorithm with conditional Gaussian statistics (PIDD-CG), is developed to predict the time evolution of the probability density functions (PDFs) of complex turbulent systems with partial observations. The PIDD-CG algorithm integrates a unique multiscale statistical closure model with an extremely efficient nonlinear data assimilation scheme to represent the PDF as a mixture of conditional statistics, which overcomes the curse of dimensionality for high-dimensional systems. The multiscale features in the time evolution of each conditional statistics ensemble member effectively captured by an appropriate combination of physics-informed analytic formulae and recurrent neural networks. An information metric is adopted as the loss function for the latter to more accurately calibrate the key turbulent signals with strong fluctuations. The proposed algorithm succeeds in forecasting both the transient and statistical equilibrium non-Gaussian PDFs of strongly turbulent systems with intermittency, regime switching and extreme events.

physics.flu-dyn↗

arXiv2022

MC-UNet Multi-module Concatenation based on U-shape Network for Retinal Blood Vessels Segmentation

Accurate segmentation of the blood vessels of the retina is an important step in clinical diagnosis of ophthalmic diseases. Many deep learning frameworks have come up for retinal blood vessels segmentation tasks. However, the complex vascular structure and uncertain pathological features make the blood vessel segmentation still very challenging. A novel U-shaped network named Multi-module Concatenation which is based on Atrous convolution and multi-kernel pooling is put forward to retinal vessels segmentation in this paper. The proposed network structure retains three layers the essential structure of U-Net, in which the atrous convolution combining the multi-kernel pooling blocks are designed to obtain more contextual information. The spatial attention module is concatenated with dense atrous convolution module and multi-kernel pooling module to form a multi-module concatenation. And different dilation rates are selected by cascading to acquire a larger receptive field in atrous convolution. Adequate comparative experiments are conducted on these public retinal datasets: DRIVE, STARE and CHASE_DB1. The results show that the proposed method is effective, especially for microvessels. The code will be put out at https://github.com/Rebeccala/MC-UNet

eess.IV↗

arXiv2022

An Efficient Data-Driven Multiscale Stochastic Reduced Order Modeling Framework for Complex Systems

Suitable reduced order models (ROMs) are computationally efficient tools in characterizing key dynamical and statistical features of nature. In this paper, a systematic multiscale stochastic ROM framework is developed for complex systems with strong chaotic or turbulent behavior. The new ROMs are fundamentally different from the traditional Galerkin ROM (G-ROM) or those deterministic ROMs that aim at minimizing the path-wise errors and applying mainly to laminar systems. Here, the new ROM focuses on recovering the large-scale dynamics to the maximum extent while it also exploits cheap but effective conditional linear functions as the closure terms to capture the statistical features of the medium-scale variables and its feedback to the large scales. In addition, physics constraints are incorporated into the new ROM. One unique feature of the resulting ROM is that it facilitates an efficient and accurate scheme for nonlinear data assimilation, the solution of which is provided by closed analytic formulae. Such an analytic solvable data assimilation solution significantly accelerates the computational efficiency and allows the new ROM to avoid many potential numerical and sampling issues in recovering the unobserved states from partial observations. The overall model calibration is efficient and systematic via explicit mathematical formulae. The new ROM framework is applied to complex nonlinear systems, in which the intrinsic turbulent behavior is either triggered by external random forcing or deterministic nonlinearity. It is shown that the new ROM significantly outperforms the G-ROM in both scenarios in terms of reproducing the dynamical and statistical features as well as recovering unobserved states via the associated efficient data assimilation scheme.

physics.comp-ph↗

arXiv2022

Superfloe Parameterization with Physics Constraints for Uncertainty Quantification of Sea Ice Floes

The discrete element method (DEM) is providing a new modeling approach for describing sea ice dynamics. It exploits particle-based methods to characterize the physical quantities of each sea ice floe along its trajectory under Lagrangian coordinates. One major challenge in applying the DEM models is the heavy computational cost when the number of floes becomes large. In this paper, an efficient Lagrangian parameterization algorithm is developed, which aims at reducing the computational cost of simulating the DEM models while preserving the key features of the sea ice. The new parameterization takes advantage of a small number of artificial ice floes, named the superfloes, to effectively approximate a considerable number of the floes, where the parameterization scheme satisfies several important physics constraints. The physics constraints guarantee the superfloe parameterized system will have similar short-term dynamical behavior as the full system. These constraints also allow the superfloe parameterized system to accurately quantify the long-range uncertainty, especially the non-Gaussian statistical features, of the full system. In addition, the superfloe parameterization facilitates a systematic noise inflation strategy that significantly advances an ensemble-based data assimilation algorithm for recovering the unobserved ocean field underneath the sea ice. Such a new noise inflation method avoids ad hoc tunings as in many traditional algorithms and is computationally extremely efficient. Numerical experiments based on an idealized DEM model with multiscale features illustrate the success of the superfloe parameterization in quantifying the uncertainty and assimilating both the sea ice and the associated ocean field.

math.NA↗

arXiv2021

Shock trace prediction by reduced models for a viscous stochastic Burgers equation

Viscous shocks are a particular type of extreme events in nonlinear multiscale systems, and their representation requires small scales. Model reduction can thus play an important role in reducing the computational cost for an efficient prediction of shocks. Yet, reduced models typically aim to approximate large-scale dominating dynamics, which do not resolve the small scales by design. To resolve this representation barrier, we introduce a new qualitative characterization of the space-time locations of shocks, named as the ``shock trace'', via a space-time indicator function based on an empirical resolution-adaptive threshold. Different from the exact shocks, the shock traces can be captured within the representation capacity of the large scales, which facilitates the forecast of the timing and locations of the shocks utilizing reduced models. Within the context of a viscous stochastic Burgers equation, we show that a data-driven reduced model, in the form of nonlinear autoregression (NAR) time series models, can accurately predict the random shock traces, with relatively low rates of false predictions. The NAR model significantly outperforms the corresponding Galerkin truncated model in the scenario of either noiseless or noisy observations. The results illustrate the importance of the data-driven closure terms in the NAR model, which account for the effects of the unresolved small scale dynamics on the resolved ones due to nonlinear interactions.

math.NA↗

arXiv2021

Conditional Gaussian Nonlinear System: a Fast Preconditioner and a Cheap Surrogate Model For Complex Nonlinear Systems

Developing suitable approximate models for analyzing and simulating complex nonlinear systems is practically important. This paper aims at exploring the skill of a rich class of nonlinear stochastic models, known as the conditional Gaussian nonlinear system (CGNS), as both a cheap surrogate model and a fast preconditioner for facilitating many computationally challenging tasks. The CGNS preserves the underlying physics to a large extent and can reproduce intermittency, extreme events and other non-Gaussian features in many complex systems arising from practical applications. Three interrelated topics are studied. First, the closed analytic formulae of solving the conditional statistics provide an efficient and accurate data assimilation scheme. It is shown that the data assimilation skill of a suitable CGNS approximate forecast model outweighs that by applying an ensemble method even to the perfect model with strong nonlinearity, where the latter suffers from filter divergence. Second, the CGNS allows the development of a fast algorithm for simultaneously estimating the parameters and the unobserved variables with uncertainty quantification in the presence of only partial observations. Utilizing an appropriate CGNS as a preconditioner significantly reduces the computational cost in accurately estimating the parameters in the original complex system. Finally, the CGNS advances rapid and statistically accurate algorithms for computing the probability density function and sampling the trajectories of the unobserved state variables. These fast algorithms facilitate the development of an efficient and accurate data-driven method for predicting the linear response of the original system with respect to parameter perturbations based on a suitable CGNS preconditioner.

math.NA↗

arXiv2021

Dynamic Power Systems Line Outage Detection Using Particle Filter and Partially Observed States

Real-time transmission line outage detection is difficult because of partial phasor measurement unit (PMU) deployment and varying outage signal strength. Existing detection approaches focus on monitoring PMU-measured nodal algebraic states, i.e., voltage phase angle and magnitude. The success of such approaches, however, is largely predicated on strong outage signals and the presence of PMUs in the outage location's vicinity. To overcome these limitations, a unified framework is proposed in this work by utilizing both nodal voltage information and generator dynamic states, e.g., rotor angular position. The proposed scheme is shown to be faster and more robust to unknown outage locations through the incorporation of generator dynamics. Using the IEEE 39-bus system simulation data, the proposed scheme's properties and performances compared to existing approaches are presented. The new approach could help improve operators' real-time situational awareness by detecting outages faster and providing a breakdown of outage signals for diagnostic purposes, making future power systems more resilient.

eess.SY↗

arXiv2021

BAMCAFE: A Bayesian Machine Learning Advanced Forecast Ensemble Method for Complex Turbulent Systems with Partial Observations

Ensemble forecast based on physics-informed models is one of the most widely used forecast algorithms for complex turbulent systems. A major difficulty in such a method is the model error that is ubiquitous in practice. Data-driven machine learning (ML) forecasts can reduce the model error but they often suffer from the partial and noisy observations. In this paper, a simple but effective Bayesian machine learning advanced forecast ensemble (BAMCAFE) method is developed, which combines an available imperfect physics-informed model with data assimilation (DA) to facilitate the ML ensemble forecast. In the BAMCAFE framework, a Bayesian ensemble DA is applied to create the training data of the ML model, which reduces the intrinsic error in the imperfect physics-informed model simulations and provides the training data of the unobserved variables. Then a generalized DA is employed for the initialization of the ML ensemble forecast. In addition to forecasting the optimal point-wise value, the BAMCAFE also provides an effective approach of quantifying the forecast uncertainty utilizing a non-Gaussian probability density function that characterizes the intermittency and extreme events. It is shown using a two-layer Lorenz 96 model that the BAMCAFE method can significant improve the forecasting skill compared to the typical reduced-order imperfect models with bare truncation or stochastic parameterization for both the observed and unobserved large-scale variables. It is also shown via a nonlinear conceptual model that the BAMCAFE leads to a comparable non-Gaussian forecast uncertainty as the perfect model while the associated imperfect physics-informed model suffers from large forecast biases.

physics.ao-ph↗

arXiv2021

An Efficient and Statistically Accurate Lagrangian Data Assimilation Algorithm with Applications to Discrete Element Sea Ice Models

Lagrangian data assimilation of complex nonlinear turbulent flows is an important but computationally challenging topic. In this article, an efficient data-driven statistically accurate reduced-order modeling algorithm is developed that significantly accelerates the computational efficiency of Lagrangian data assimilation. The algorithm starts with a Fourier transform of the high-dimensional flow field, which is followed by an effective model reduction that retains only a small subset of the Fourier coefficients corresponding to the energetic modes. Then a linear stochastic model is developed to approximate the nonlinear dynamics of each Fourier coefficient. Effective additive and multiplicative noise processes are incorporated to characterize the modes that exhibit Gaussian and non-Gaussian statistics, respectively. All the parameters in the reduced order system, including the multiplicative noise coefficients, are determined systematically via closed analytic formulae. These linear stochastic models succeed in forecasting the uncertainty and facilitate an extremely rapid data assimilation scheme. The new Lagrangian data assimilation is then applied to observations of sea ice floe trajectories that are driven by atmospheric winds and turbulent ocean currents. It is shown that observing only about $30$ non-interacting floes in a $200$km$\times200$km domain is sufficient to recover the key multi-scale features of the ocean currents. The additional observations of the floe angular displacements are found to be suitable supplements to the center-of-mass positions for improving the data assimilation skill. In addition, the observed large and small floes are more useful in recovering the large- and small-scale features of the ocean, respectively. The Fourier domain data assimilation also succeeds in recovering the ocean features in the areas where cloud cover obscures the observations.

physics.ao-ph↗

arXiv2021

A Bayesian Machine Learning Algorithm for Predicting ENSO Using Short Observational Time Series

A simple and efficient Bayesian machine learning (BML) training and forecasting algorithm, which exploits only a 20-year short observational time series and an approximate prior model, is developed to predict the Niño 3 sea surface temperature (SST) index. The BML forecast significantly outperforms model-based ensemble predictions and standard machine learning forecasts. Even with a simple feedforward neural network, the BML forecast is skillful for 9.5 months. Remarkably, the BML forecast overcomes the spring predictability barrier to a large extent: the forecast starting from spring remains skillful for nearly 10 months. The BML algorithm can also effectively utilize multiscale features: the BML forecast of SST using SST, thermocline, and wind burst improves on the BML forecast using just SST by at least 2 months. Finally, the BML algorithm also reduces the forecast uncertainty of neural networks and is robust to input perturbations.

physics.ao-ph↗

arXiv2021

Rotation Based Slice Error Correction Protocol for Continuous-variable Quantum Key Distribution and its Implementation with Polar Codes

Reconciliation is an essential procedure for continuous-variable quantum key distribution (CV-QKD). As the most commonly used reconciliation protocol in short-distance CV-QKD, the slice error correction (SEC) allows a system to distill more than 1 bit from each pulse. However, its quantization efficiency is greatly affected by the noisy channel with a low signal-to-noise ratio (SNR), which usually limits the secure distance to about 30 km. In this paper, an improved SEC protocol, named Rotation-based SEC (RSEC), is proposed through performing a random orthogonal rotation on the raw data before quantization, and deducing a new estimator for quantized sequences. Moreover, the RSEC protocol is implemented with polar codes. Experimental results show that the proposed protocol can reach up to a quantization efficiency of about 99\%, and maintains at around 96\% even at the relatively low SNRs $(0.5,1)$, which theoretically extends the secure distance to about 45 km. When implemented with the polar codes with block length of 16 Mb, the RSEC can achieve a reconciliation efficiency of above 95\%, which outperforms all previous SEC schemes. In terms of finite-size effects, we achieve a secret key rate of $7.83\times10^{-3}$ bits/pulse at a distance of 33.93 km (the corresponding SNR value is 1). These results indicate that the proposed protocol significantly improves the performance of SEC and is a competitive reconciliation scheme for the CV-QKD system.

quant-ph↗

arXiv2021

An Efficient Continuous Data Assimilation Algorithm for the Sabra Shell Model of Turbulence

Complex nonlinear turbulent dynamical systems are ubiquitous in many areas. Recovering unobserved state variables is an important topic for the data assimilation of turbulent systems. In this article, an efficient continuous in time data assimilation scheme is developed, which exploits closed analytic formulae for updating the unobserved state variables. Therefore, it is computationally efficient and accurate. The new data assimilation scheme is combined with a simple reduced order modeling technique that involves a cheap closure approximation and a noise inflation. In such a way, many complicated turbulent dynamical systems can satisfy the requirements of the mathematical structures for the proposed efficient data assimilation scheme. The new data assimilation scheme is then applied to the Sabra shell model, which is a conceptual model for nonlinear turbulence. The goal is to recover the unobserved shell velocities across different spatial scales. It has been shown that the new data assimilation scheme is skillful in capturing the nonlinear features of turbulence including the intermittency and extreme events in both the chaotic and the turbulent dynamical regimes. It has also been shown that the new data assimilation scheme is more accurate and computationally cheaper than the standard ensemble Kalman filter and nudging data assimilation schemes for assimilating the Sabra shell model.

physics.flu-dyn↗

arXiv2021

A Multiscale Model for El Niño Complexity

El Niño-Southern Oscillation (ENSO) exhibits diverse characteristics in spatial pattern, peak intensity, and temporal evolution. Here we develop a three-region multiscale stochastic model to show that the observed ENSO complexity can be explained by combining intraseasonal, interannual, and decadal processes. The model starts with a deterministic three-region system for the interannual variabilities. Then two stochastic processes of the intraseasonal and decadal variation are incorporated. The model can reproduce not only the general properties of the observed ENSO events, but also the complexity in patterns (e.g., Central Pacific vs. Eastern Pacific events), intensity (e.g., 10-20 year reoccurrence of extreme El Niños), and temporal evolution (e.g., more multi-year La Niñas than multi-year El Niños). While conventional conceptual models were typically used to understand the dynamics behind the common properties of ENSO, this model offers a powerful tool to understand and predict ENSO complexity that challenges our understanding of the 21st-century ENSO.

physics.ao-ph↗

arXiv2020

Dynamic RAN Slicing for Service-Oriented Vehicular Networks via Constrained Learning

In this paper, we investigate a radio access network (RAN) slicing problem for Internet of vehicles (IoV) services with different quality of service (QoS) requirements, in which multiple logically-isolated slices are constructed on a common roadside network infrastructure. A dynamic RAN slicing framework is presented to dynamically allocate radio spectrum and computing resource, and distribute computation workloads for the slices. To obtain an optimal RAN slicing policy for accommodating the spatial-temporal dynamics of vehicle traffic density, we first formulate a constrained RAN slicing problem with the objective to minimize long-term system cost. This problem cannot be directly solved by traditional reinforcement learning (RL) algorithms due to complicated coupled constraints among decisions. Therefore, we decouple the problem into a resource allocation subproblem and a workload distribution subproblem, and propose a two-layer constrained RL algorithm, named Resource Allocation and Workload diStribution (RAWS) to solve them. Specifically, an outer layer first makes the resource allocation decision via an RL algorithm, and then an inner layer makes the workload distribution decision via an optimization subroutine. Extensive trace-driven simulations show that the RAWS effectively reduces the system cost while satisfying QoS requirements with a high probability, as compared with benchmarks.

cs.LG↗

arXiv2020

Visual Causality Analysis of Event Sequence Data

Causality is crucial to understanding the mechanisms behind complex systems and making decisions that lead to intended outcomes. Event sequence data is widely collected from many real-world processes, such as electronic health records, web clickstreams, and financial transactions, which transmit a great deal of information reflecting the causal relations among event types. Unfortunately, recovering causalities from observational event sequences is challenging, as the heterogeneous and high-dimensional event variables are often connected to rather complex underlying event excitation mechanisms that are hard to infer from limited observations. Many existing automated causal analysis techniques suffer from poor explainability and fail to include an adequate amount of human knowledge. In this paper, we introduce a visual analytics method for recovering causalities in event sequence data. We extend the Granger causality analysis algorithm on Hawkes processes to incorporate user feedback into causal model refinement. The visualization system includes an interactive causal analysis framework that supports bottom-up causal exploration, iterative causal verification and refinement, and causal comparison through a set of novel visualizations and interactions. We report two forms of evaluation: a quantitative evaluation of the model improvements resulting from the user-feedback mechanism, and a qualitative evaluation through case studies in different application domains to demonstrate the usefulness of the system.

cs.AI↗

arXiv2020

Information Relaxation and A Duality-Driven Algorithm for Stochastic Dynamic Programs

We use the technique of information relaxation to develop a duality-driven iterative approach to obtaining and improving confidence interval estimates for the true value of finite-horizon stochastic dynamic programming problems. We show that the sequence of dual value estimates yielded from the proposed approach in principle monotonically converges to the true value function in a finite number of dual iterations. Aiming to overcome the curse of dimensionality in various applications, we also introduce a regression-based Monte Carlo algorithm for implementation. The new approach can be used not only to assess the quality of heuristic policies, but also to improve them if we find that their duality gap is large. We obtain the convergence rate of our Monte Carlo method in terms of the amounts of both basis functions and the sampled states. Finally, we demonstrate the effectiveness of our method in an optimal order execution problem with market friction and in an inventory management problem in the presence of lost sale and lead time. Both examples are well known in the literature to be difficult to solve for optimality. The experiments show that our method can significantly improve the heuristics suggested in the literature and obtain new policies with a satisfactory performance guarantee.

math.OC↗