arXiv subjects

Weiqi Ji

Publications and source records attributed to Weiqi Ji.

13 recordsLinked to original sources

arXiv2026

Performance Monitoring of Proton Exchange Membrane Water Electrolyzer by Transformers-Based Machine Learning Model

Green hydrogen plays an essential role in decarbonization, with capacity projected to scale to 560 GW by 2030 (vs. 1.39 GW in 2023) in net-zero settings. Proton exchange membrane (PEM) electrolysis is one of the most promising technology routes to green hydrogen production, and real-time system health monitoring of PEM electrolyzers is essential for their scalable deployment. In lab settings, performance degradation can be characterized through electrochemical testing protocols by periodic pauses of normal operation. Such interruption is not practical for full-scale stack deployments, limiting system operators' ability to make real-time assessments of state-of-health (SoH). We present a machine learning (ML) framework that performs virtual electrochemical characterization during normal operation. The method uses an encoder-decoder transformer, conditioned on operational data, to reconstruct characterization outputs, focusing here on polarization curves. Inspired by patch-based sequence tokenization, we segment the inputs into patches and encode them to form meaningful tokens, which substantially improves learning efficiency. Across four longitudinal runs, lasting up to 478 hours on different test cells and loading cycles, the model accurately reconstructed polarization curves and achieved 10x reduction in mean squared error (MSE) compared to a vanilla transformer. This proof-of-concept demonstrates that ML models can enable continuous performance monitoring for PEM electrolyzers and that the encoder captures meaningful latent representations of SoH, opening up opportunities to derive interpretable indicators in future work.

cs.LG↗

arXiv2022

Kinetics Parameter Optimization via Neural Ordinary Differential Equations

Chemical kinetics mechanisms are essential for understanding, analyzing, and simulating complex combustion phenomena. In this study, a Neural Ordinary Differential Equation (Neural ODE) framework is employed to optimize kinetics parameters of reaction mechanisms. Given experimental or high-cost simulated observations as training data, the proposed algorithm can optimally recover the hidden characteristics in the data. Different datasets of various sizes, types, and noise levels are tested. A classic toy problem of stiff Robertson ODE is first used to demonstrate the learning capability, efficiency, and robustness of the Neural ODE approach. A 41-species, 232-reactions JP-10 skeletal mechanism and a 34-species, 121-reactions n-heptane skeletal mechanism are then optimized with species' temporal profiles and ignition delay times, respectively. Results show that the proposed algorithm can optimize stiff chemical models with sufficient accuracy and efficiency. It is noted that the trained mechanism not only fits the data perfectly but also retains its physical interpretability, which can be further integrated and validated in practical turbulent combustion simulations.

physics.chem-ph↗

arXiv2022

Autonomous Kinetic Modeling of Biomass Pyrolysis using Chemical Reaction Neural Networks

Modeling the burning processes of biomass such as wood, grass, and crops is crucial for the modeling and prediction of wildland and urban fire behavior. Despite its importance, the burning of solid fuels remains poorly understood, which can be partly attributed to the unknown chemical kinetics of most solid fuels. Most available kinetic models were built upon expert knowledge, which requires chemical insights and years of experience. This work presents a framework for autonomously discovering biomass pyrolysis kinetic models from thermogravimetric analyzer (TGA) experimental data using the recently developed chemical reaction neural networks (CRNN). The approach incorporated the CRNN model into the framework of neural ordinary differential equations to predict the residual mass in TGA data. In addition to the flexibility of neural-network-based models, the learned CRNN model is interpretable, by incorporating the fundamental physics laws, such as the law of mass action and Arrhenius law, into the neural network structure. The learned CRNN model can then be translated into the classical forms of biomass chemical kinetic models, which facilitates the extraction of chemical insights and the integration of the kinetic model into large-scale fire simulations. We demonstrated the effectiveness of the framework in predicting the pyrolysis and oxidation of cellulose. This successful demonstration opens the possibility of rapid and autonomous chemical kinetic modeling of solid fuels, such as wildfire fuels and industrial polymers.

physics.chem-ph↗

arXiv2021

Stiff Neural Ordinary Differential Equations

Neural Ordinary Differential Equations (ODE) are a promising approach to learn dynamic models from time-series data in science and engineering applications. This work aims at learning Neural ODE for stiff systems, which are usually raised from chemical kinetic modeling in chemical and biological systems. We first show the challenges of learning neural ODE in the classical stiff ODE systems of Robertson's problem and propose techniques to mitigate the challenges associated with scale separations in stiff systems. We then present successful demonstrations in stiff systems of Robertson's problem and an air pollution problem. The demonstrations show that the usage of deep networks with rectified activations, proper scaling of the network outputs as well as loss functions, and stabilized gradient calculations are the key techniques enabling the learning of stiff neural ODE. The success of learning stiff neural ODE opens up possibilities of using neural ODEs in applications with widely varying time-scales, like chemical dynamics in energy conversion, environmental engineering, and the life sciences.

math.NA↗

arXiv2021

Stiff-PINN: Physics-Informed Neural Network for Stiff Chemical Kinetics

Recently developed physics-informed neural network (PINN) has achieved success in many science and engineering disciplines by encoding physics laws into the loss functions of the neural network, such that the network not only conforms to the measurements, initial and boundary conditions but also satisfies the governing equations. This work first investigates the performance of PINN in solving stiff chemical kinetic problems with governing equations of stiff ordinary differential equations (ODEs). The results elucidate the challenges of utilizing PINN in stiff ODE systems. Consequently, we employ Quasi-Steady-State-Assumptions (QSSA) to reduce the stiffness of the ODE systems, and the PINN then can be successfully applied to the converted non/mild-stiff systems. Therefore, the results suggest that stiffness could be the major reason for the failure of the regular PINN in the studied stiff chemical kinetic systems. The developed Stiff-PINN approach that utilizes QSSA to enable PINN to solve stiff chemical kinetics shall open the possibility of applying PINN to various reaction-diffusion systems involving stiff dynamics.

math.NA↗

arXiv2021

KiNet: A Deep Neural Network Representation of Chemical Kinetics

Deep learning is a potential approach to automatically develop kinetic models from experimental data. We propose a deep neural network model of KiNet to represent chemical kinetics. KiNet takes the current composition states and predicts the evolution of the states after a fixed time step. The long-period evolution of the states and their gradients to model parameters can be efficiently obtained by recursively applying the KiNet model multiple times. To address the challenges of the high-dimensional composition space and error accumulation in long-period prediction, the architecture of KiNet incorporates the residual network model (ResNet), and the training employs backpropagation through time (BPTT) approach to minimize multi-step prediction error. In addition, an approach for efficiently computing the gradient of the ignition delay time (IDT) to KiNet model parameters is proposed to train the KiNet against the rich database of IDT from literature, which could address the scarcity of time-resolved species measurements. The KiNet is first trained and compared with the simulated species profiles during the auto-ignition of H2/air mixtures. The obtained KiNet model can accurately predict the auto-ignition processes for various initial conditions that cover a wide range of pressures, temperatures, and equivalence ratios. Then, we show that the gradient of IDT to KiNet model parameters is parallel to the gradient of the temperature at the ignition point. This correlation enables efficient computation of the gradient of IDT via backpropagation and is demonstrated as a feasible approach for fine-tuning the KiNet against IDT. These demonstrations shall open up the possibility of building data-driven kinetic models autonomously. Finally, the trained KiNet could be potentially applied to kinetic model reduction and chemistry acceleration in turbulent combustion simulations.

physics.comp-ph↗

arXiv2021

Neural Differential Equations for Inverse Modeling in Model Combustors

Monitoring the dynamics processes in combustors is crucial for safe and efficient operations. However, in practice, only limited data can be obtained due to limitations in the measurable quantities, visualization window, and temporal resolution. This work proposes an approach based on neural differential equations to approximate the unknown quantities from available sparse measurements. The approach tackles the challenges of nonlinearity and the curse of dimensionality in inverse modeling by representing the dynamic signal using neural network models. In addition, we augment physical models for combustion with neural differential equations to enable learning from sparse measurements. We demonstrated the inverse modeling approach in a model combustor system by simulating the oscillation of an industrial combustor with a perfectly stirred reactor. Given the sparse measurements of the temperature inside the combustor, upstream fluctuations in compositions and/or flow rates can be inferred. Various types of fluctuations in the upstream, as well as the responses in the combustor, were synthesized to train and validate the algorithm. The results demonstrated that the approach can efficiently and accurately infer the dynamics of the unknown inlet boundary conditions, even without assuming the types of fluctuations. Those demonstrations shall open a lot of opportunities in utilizing neural differential equations for fault diagnostics and model-based dynamic control of industrial power systems.

physics.flu-dyn↗

arXiv2021

Arrhenius.jl: A Differentiable Combustion SimulationPackage

Combustion kinetic modeling is an integral part of combustion simulation, and extensive studies have been devoted to developing both high fidelity and computationally affordable models. Despite these efforts, modeling combustion kinetics is still challenging due to the demand for expert knowledge and optimization against experiments, as well as the lack of understanding of the associated uncertainties. Therefore, data-driven approaches that enable efficient discovery and calibration of kinetic models have received much attention in recent years, the core of which is the optimization based on big data. Differentiable programming is a promising approach for learning kinetic models from data by efficiently computing the gradient of objective functions to model parameters. However, it is often challenging to implement differentiable programming in practice. Therefore, it is still not available in widely utilized combustion simulation packages such as CHEMKIN and Cantera. Here, we present a differentiable combustion simulation package leveraging the eco-system in Julia, including DifferentialEquations.jl for solving differential equations, ForwardDiff.jl for auto-differentiation, and Flux.jl for incorporating neural network models into combustion simulations and optimizing neural network models using the state-of-the-art deep learning optimizers. We demonstrate the benefits of differentiable programming in efficient and accurate gradient computations, with applications in uncertainty quantification, kinetic model reduction, data assimilation, and model discovery.

physics.chem-ph↗

arXiv2021

Machine Learning Approaches to Learn HyChem Models

The HyChem approach has recently been proposed for modeling high-temperature combustion of real, multi-component fuels. The approach combines lumped reaction steps for fuel thermal and oxidative pyrolysis with detailed chemistry for the oxidation of the resulting pyrolysis products. However, the approach usually shows substantial discrepancies with experimental data within the Negative Temperature Coefficient (NTC) regime, as the low-temperature chemistry is more fuel-specific than high-temperature chemistry. This paper proposes a machine learning approach to learn the HyChem models that can cover both high-temperature and low-temperature regimes. Specifically, we develop a HyChem model using the experimental datasets of ignition delay times covering a wide range of temperatures and equivalence ratios. The chemical kinetic model is treated as a neural network model, and we then employ stochastic gradient descent (SGD), a technique that was developed for deep learning, for the training. We demonstrate the approach in learning the HyChem model for F-24, which is a Jet-A derived fuel, and compare the results with previous work employing genetic algorithms. The results show that the SGD approach can achieve comparable model performance with genetic algorithms but the computational cost is reduced by 1000 times. In addition, with regularization in SGD, the SGD approach changes the kinetic parameters from their original values much less than genetic algorithm and is thus more likely to retrain mechanistic meanings. Finally, our approach is built upon open-source packages and can be applied to the development and optimization of chemical kinetic models for internal combustion engine simulations.

physics.chem-ph↗

arXiv2021

Inference of cell dynamics on perturbation data using adjoint sensitivity

Data-driven dynamic models of cell biology can be used to predict cell response to unseen perturbations. Recent work (CellBox) had demonstrated the derivation of interpretable models with explicit interaction terms, in which the parameters were optimized using machine learning techniques. While the previous work was tested only in a single biological setting, this work aims to extend the range of applicability of this model inference approach to a diversity of biological systems. Here we adapted CellBox in Julia differential programming and augmented the method with adjoint algorithms, which has recently been used in the context of neural ODEs. We trained the models using simulated data from both abstract and biology-inspired networks, which afford the ability to evaluate the recovery of the ground truth network structure. The resulting accuracy of prediction by these models is high both in terms of low error against data and excellent agreement with the network structure used for the simulated training data. While there is no analogous ground truth for real life biological systems, this work demonstrates the ability to construct and parameterize a considerable diversity of network models with high predictive ability. The expectation is that this kind of procedure can be used on real perturbation-response data to derive models applicable to diverse biological systems.

q-bio.MN↗

arXiv2021

Autonomous Discovery of Unknown Reaction Pathways from Data by Chemical Reaction Neural Network

Chemical reactions occur in energy, environmental, biological, and many other natural systems, and the inference of the reaction networks is essential to understand and design the chemical processes in engineering and life sciences. Yet, revealing the reaction pathways for complex systems and processes is still challenging due to the lack of knowledge of the involved species and reactions. Here, we present a neural network approach that autonomously discovers reaction pathways from the time-resolved species concentration data. The proposed Chemical Reaction Neural Network (CRNN), by design, satisfies the fundamental physics laws, including the Law of Mass Action and the Arrhenius Law. Consequently, the CRNN is physically interpretable such that the reaction pathways can be interpreted, and the kinetic parameters can be quantified simultaneously from the weights of the neural network. The inference of the chemical pathways is accomplished by training the CRNN with species concentration data via stochastic gradient descent. We demonstrate the successful implementations and the robustness of the approach in elucidating the chemical reaction pathways of several chemical engineering and biochemical systems. The autonomous inference by the CRNN approach precludes the need for expert knowledge in proposing candidate networks and addresses the curse of dimensionality in complex systems. The physical interpretability also makes the CRNN capable of not only fitting the data for a given system but also developing knowledge of unknown pathways that could be generalized to similar chemical systems.

q-bio.MN↗

arXiv2020

Kinetic Similarity between Extinction Strain Rate and Laminar Flame Speed

Extinction strain rate (ESR) and laminar flame speed (LFS) are fundamental properties of a fuel/air mixture that are often utilized as scaling parameters in turbulent combustion. While LFS at atmospheric and elevated pressures are extensively investigated, experimental measurements of ESR with counterflow premixed flames are very limited for flame instability often occurs near extinction, especially at high pressures. Due to the scarcity of ESR measurements, most combustion kinetic models are mainly validated and optimized against LFS. However, it is questionable whether the controlling reactions are the same for ESR and LFS such that those models are also valid for predicting ESR. This work quantifies the kinetic similarities between ESR and LFS by analyzing their kinetic sensitivity directions. The direction is represented by a unit vector composed of the normalized sensitivity of ESR or LFS to the rate constant for each elemental reaction. Consequently, the similarity between the two directions is measured by the inner product of the corresponding unit vectors. The sensitivity directions of ESR and LFS are found parallel for various fuels, equivalence ratios, and pressures. Furthermore, sensitivity directions at various strain rates are also similar for the maximum temperature, local temperature at various locations in the flame coordinate, and ESR in counterflow premixed flames. These findings suggest that LFS and ESR are similarly effective as the target for constraining and optimizing rate constants in kinetic models. In addition, the independence of the sensitivity directions on the strain rate also enables us to perform uncertainty quantification for turbulent flames with a wide range of strain rates based on the kinetic sensitivity of ESR and LFS.

physics.flu-dyn↗

arXiv2020

Uncertainty Propagation in Deep Neural Network Using Active Subspace

The inputs of deep neural network (DNN) from real-world data usually come with uncertainties. Yet, it is challenging to propagate the uncertainty in the input features to the DNN predictions at a low computational cost. This work employs a gradient-based subspace method and response surface technique to accelerate the uncertainty propagation in DNN. Specifically, the active subspace method is employed to identify the most important subspace in the input features using the gradient of the DNN output to the inputs. Then the response surface within that low-dimensional subspace can be efficiently built, and the uncertainty of the prediction can be acquired by evaluating the computationally cheap response surface instead of the DNN models. In addition, the subspace can help explain the adversarial examples. The approach is demonstrated in MNIST datasets with a convolutional neural network. Code is available at: https://github.com/jiweiqi/nnsubspace.

stat.ML↗