Searcharxiv⌕ Search

arXiv subjects

Shen Wang

Publications and source records attributed to Shen Wang.

At least 109 records · Page 6Linked to original sources
arXiv2022

Model Order Reduction for Water Quality Dynamics

A state-space representation of water quality dynamics describing disinfectant (e.g., chlorine) transport dynamics in drinking water distribution networks has been recently proposed. Such representation is a byproduct of space- and time-discretization of the PDE modeling transport dynamics. This results in a large state-space dimension even for small networks with tens of nodes. Although such a state-space model provides a model-driven approach to predict water quality dynamics, incorporating it into model-based control algorithms or state estimators for large networks is challenging and at times intractable. To that end, this paper investigates model order reduction (MOR) methods for water quality dynamics with the objective of performing post-reduction feedback control. The presented investigation focuses on reducing state-dimension by orders of magnitude, the stability of the MOR methods, and the application of these methods to model predictive control.

eess.SY↗
arXiv2021

Stacked Generative Machine Learning Models for Fast Approximations of Steady-State Navier-Stokes Equations

Computational fluid dynamics (CFD) simulations are broadly applied in engineering and physics. A standard description of fluid dynamics requires solving the Navier-Stokes (N-S) equations in different flow regimes. However, applications of CFD simulations are computationally-limited by the availability, speed, and parallelism of high-performance computing. To improve computational efficiency, machine learning techniques have been used to create accelerated data-driven approximations for CFD. A majority of such approaches rely on large labeled CFD datasets that are expensive to obtain at the scale necessary to build robust data-driven models. We develop a weakly-supervised approach to solve the steady-state N-S equations under various boundary conditions, using a multi-channel input with boundary and geometric conditions. We achieve state-of-the-art results without any labeled simulation data, but using a custom data-driven and physics-informed loss function by using and small-scale solutions to prime the model to solve the N-S equations. To improve the resolution and predictability, we train stacked models of increasing complexity generating the numerical solutions for N-S equations. Without expensive computations, our model achieves high predictability with a variety of obstacles and boundary conditions. Given its high flexibility, the model can generate a solution on a 64 x 64 domain within 5 ms on a regular desktop computer which is 1000 times faster than a regular CFD solver. Translation of interactive CFD simulation on local consumer computing hardware enables new applications in real-time predictions on the internet of things devices where data transfer is prohibitive and can increase the scale, speed, and computational cost of boundary-value fluid problems.

cs.CE↗
arXiv2021

Fine-Grained Control of Artistic Styles in Image Generation

Recent advances in generative models and adversarial training have enabled artificially generating artworks in various artistic styles. It is highly desirable to gain more control over the generated style in practice. However, artistic styles are unlike object categories -- there are a continuous spectrum of styles distinguished by subtle differences. Few works have been explored to capture the continuous spectrum of styles and apply it to a style generation task. In this paper, we propose to achieve this by embedding original artwork examples into a continuous style space. The style vectors are fed to the generator and discriminator to achieve fine-grained control. Our method can be used with common generative adversarial networks (such as StyleGAN). Experiments show that our method not only precisely controls the fine-grained artistic style but also improves image quality over vanilla StyleGAN as measured by FID.

cs.CV↗
arXiv2021

Weak Solution and Invariant Probability Measure for McKean-Vlasov SDEs with Integrable Drifts

In this paper, by utilizing Wang's Harnack inequality with power and the Banach fixed point theorem, the weak well-posedness for McKean-Vlasov SDEs with integrable drift is investigated. In addition, using the decoupled method, some regularity such as relative entropy and Sobolev's estimate of invariant probability measure are proved. Finally, by Banach's fixed theorem, the existence and uniqueness of invariant probability measure for symmetric McKean-Vlasov SDEs and stochastic Hamiltonian system with integrable drifts are obtained.

math.PR↗
arXiv2021

Knowledge Enhanced Pretrained Language Models: A Compreshensive Survey

Pretrained Language Models (PLM) have established a new paradigm through learning informative contextualized representations on large-scale text corpus. This new paradigm has revolutionized the entire field of natural language processing, and set the new state-of-the-art performance for a wide variety of NLP tasks. However, though PLMs could store certain knowledge/facts from training corpus, their knowledge awareness is still far from satisfactory. To address this issue, integrating knowledge into PLMs have recently become a very active research area and a variety of approaches have been developed. In this paper, we provide a comprehensive survey of the literature on this emerging and fast-growing field - Knowledge Enhanced Pretrained Language Models (KE-PLMs). We introduce three taxonomies to categorize existing work. Besides, we also survey the various NLU and NLG applications on which KE-PLM has demonstrated superior performance over vanilla PLMs. Finally, we discuss challenges that face KE-PLMs and also promising directions for future research.

cs.CL↗
arXiv2021

Can CAV Reduce Non-Recurrent Urban Road Congestion?

A well-designed resilient and sustainable urban transportation system can recover quickly from the non-recurrent road congestion (NRC), which is often caused by en-route events (e.g., road closure due to car collisions). Existing solutions, such as on-board navigation systems and temporary rerouting road signs, are not effective due to delayed responses. Connected Autonomous Vehicles (CAV) can be helpful in improving recurrent traffic as they can autonomously adjust their speed according to their real-time surrounding traffic, sensed by vehicular communications. Preliminary simulation results in this short paper show that CAV can also improve traffic when non-recurrent congestion occurs. Other results in fuel consumption, CO2 emission, and traditional traffic safety indicators are open for future discussions.

eess.SY↗
arXiv2021

Delving into Macro Placement with Reinforcement Learning

In physical design, human designers typically place macros via trial and error, which is a Markov decision process. Reinforcement learning (RL) methods have demonstrated superhuman performance on the macro placement. In this paper, we propose an extension to this prior work (Mirhoseini et al., 2020). We first describe the details of the policy and value network architecture. We replace the force-directed method with DREAMPlace for placing standard cells in the RL environment. We also compare our improved method with other academic placers on public benchmarks.

cs.LG↗
arXiv2021

Modeling Sequences as Distributions with Uncertainty for Sequential Recommendation

The sequential patterns within the user interactions are pivotal for representing the user's preference and capturing latent relationships among items. The recent advancements of sequence modeling by Transformers advocate the community to devise more effective encoders for the sequential recommendation. Most existing sequential methods assume users are deterministic. However, item-item transitions might fluctuate significantly in several item aspects and exhibit randomness of user interests. This \textit{stochastic characteristics} brings up a solid demand to include uncertainties in representing sequences and items. Additionally, modeling sequences and items with uncertainties expands users' and items' interaction spaces, thus further alleviating cold-start problems. In this work, we propose a Distribution-based Transformer for Sequential Recommendation (DT4SR), which injects uncertainties into sequential modeling. We use Elliptical Gaussian distributions to describe items and sequences with uncertainty. We describe the uncertainty in items and sequences as Elliptical Gaussian distribution. And we adopt Wasserstein distance to measure the similarity between distributions. We devise two novel Trans-formers for modeling mean and covariance, which guarantees the positive-definite property of distributions. The proposed method significantly outperforms the state-of-the-art methods. The experiments on three benchmark datasets also demonstrate its effectiveness in alleviating cold-start issues. The code is available inhttps://github.com/DyGRec/DT4SR.

cs.IR↗
arXiv2021

Anti-Counterfeiting for Polymer Banknotes Based on Polymer Substrate Fingerprinting

Polymer banknotes are the trend for printed currency and have been adopted by more than fifty countries worldwide. However, over the past years, the quantity of polymer counterfeits has been increasing, so has the quality of counterfeits. This shows that the initial advantage of bringing a new polymer technology to fight against counterfeiting is reducing. To maintain one step ahead of counterfeiters, we propose a novel anti-counterfeiting technique called Polymer Substrate Fingerprinting (PSF). Our technique is built based on the observation that the opacity coating, a critical step during the production of polymer notes, is a stochastic manufacturing process, leaving uneven thickness in the coating layer and the random dispersion of impurities from the ink. The imperfections in the coating layer result in random translucent patterns when a polymer banknote is back-lit by a light source. We show these patterns can be reliably captured by a commodity negative-film scanner and processed into a compact fingerprint to uniquely identify each banknote. Using an extensive dataset of 6,200 sample images collected from 340 UK banknotes, we show that our method can reliably authenticate banknotes, and is robust against rough daily handling of banknotes. Furthermore, we show the extracted fingerprints contain around 900 bits of entropy, which makes it extremely scalable to identify every polymer note circulated globally. As compared with previous or existing anti-counterfeiting mechanisms for banknotes, our method has a distinctive advantage: it ensures that even in the extreme case when counterfeiters have procured the same printing equipment and ink as used by a legitimate government, counterfeiting banknotes remains infeasible because of the difficulty to replicate a stochastic manufacturing process.

cs.CR↗
arXiv2021

Probabilistic State Estimation in Water Networks

State estimation in water distribution networks (WDN), the problem of estimating all unknown network heads and flows given select measurements, is challenging due to the nonconvexity of hydraulic models and significant uncertainty from water demands, network parameters, and measurements. To this end, a probabilistic modeling for state estimation (PSE) in WDNs is proposed. After linearizing the nonlinear hydraulic WDN model, the proposed PSE shows that the covariance matrix of unknown system states (unmeasured heads and flows) can be linearly expressed by the covariance matrix of three uncertainty sources (i.e., measurement noise, network parameters, and water demands). Instead of providing deterministic results for unknown states, the proposed PSE approach (i) regards the system states and uncertainty sources as random variables and yields variances of individual unknown states, (ii) considers thorough modeling of various types of valves and measurement scenarios in WDNs, and (iii) is also useful for uncertainty quantification, extended period simulations, and confidence limit analysis. The effectiveness and scalability of the proposed approach is tested using several WDN case studies.

math.OC↗
arXiv2021

Transferable Graph Optimizers for ML Compilers

Most compilers for machine learning (ML) frameworks need to solve many correlated optimization problems to generate efficient machine code. Current ML compilers rely on heuristics based algorithms to solve these optimization problems one at a time. However, this approach is not only hard to maintain but often leads to sub-optimal solutions especially for newer model architectures. Existing learning based approaches in the literature are sample inefficient, tackle a single optimization problem, and do not generalize to unseen graphs making them infeasible to be deployed in practice. To address these limitations, we propose an end-to-end, transferable deep reinforcement learning method for computational graph optimization (GO), based on a scalable sequential attention mechanism over an inductive graph neural network. GO generates decisions on the entire graph rather than on each individual node autoregressively, drastically speeding up the search compared to prior methods. Moreover, we propose recurrent attention layers to jointly optimize dependent graph optimization tasks and demonstrate 33%-60% speedup on three graph optimization tasks compared to TensorFlow default optimization. On a diverse set of representative graphs consisting of up to 80,000 nodes, including Inception-v3, Transformer-XL, and WaveNet, GO achieves on average 21% improvement over human experts and 18% improvement over the prior state of the art with 15x faster convergence, on a device placement task evaluated in real systems.

cs.LG↗
arXiv2020

How Effective is Model Predictive Control in Real-Time Water Quality Regulation? State-Space Modeling and Scalable Control

Real-time water quality control (WQC) in water distribution networks (WDN), the problem of regulating disinfectant levels, is challenging due to lack of (i) a proper control-oriented modeling considering complicated components (junctions, reservoirs, tanks, pipes, pumps, and valves) for water quality modeling in WDN and (ii) a corresponding scalable control algorithm that performs real-time water quality regulation. In this paper, we solve the WQC problem by (a) proposing a novel state-space representation of the WQC problem that provides an explicit relationship between inputs (chlorine dosage at booster stations) and states/outputs (chlorine concentrations in the entire network) and (b) designing a highly scalable model predictive control (MPC) algorithm that showcases fast response time and resilience against some sources of uncertainty.

math.OC↗
arXiv2020

Revisiting the Water Quality Sensor Placement Problem: Optimizing Network Observability and State Estimation Metrics

Real-time water quality (WQ) sensors in water distribution networks (WDN) have the potential to enable network-wide observability of water quality indicators, contamination event detection, and closed-loop feedback control of WQ dynamics. To that end, prior research has investigated a wide range of methods that guide the geographic placement of WQ sensors. These methods assign a metric for fixed sensor placement (SP) followed by \textit{metric-optimization} to obtain optimal SP. These metrics include minimizing intrusion detection time, minimizing the expected population and amount of contaminated water affected by an intrusion event. In contrast to the literature, the objective of this paper is to provide a computational method that considers the overlooked metric of state estimation and network-wide observability of the WQ dynamics. This metric finds the optimal WQ sensor placement that minimizes the state estimation error via the Kalman filter for noisy WQ dynamics -- a metric that quantifies WDN observability. To that end, the state-space dynamics of WQ states for an entire WDN are given and the observability-driven sensor placement algorithm is presented. The algorithm takes into account the time-varying nature of WQ dynamics due to changes in the hydraulic profile -- a collection of hydraulic states including heads (pressures) at nodes and flow rates in links which are caused by a demand profile over a certain period of time. Thorough case studies are given, highlighting key findings, observations, and recommendations for WDN operators. Github codes are included for reproducibility.

eess.SY↗
arXiv2020

Unveiling the Stable Nature of the Solid Electrolyte Interphase between Lithium Metal and LiPON via Cryogenic Electron Microscopy

The solid electrolyte interphase (SEI) is regarded as the most complex but the least understood constituent in secondary batteries using liquid and solid electrolytes. The nanostructures of SEIs were recently reported to be equally important to the chemistry of SEIs for stabilizing Li metal in liquid electrolyte. However, the dearth of such knowledge in all-solid-state battery (ASSB) has hindered a complete understanding of how certain solid-state electrolytes, such as LiPON, manifest exemplary stability against Li metal. Characterizing such solid-solid interfaces is difficult due to the buried, highly reactive, and beam-sensitive nature of the constituents within. By employing cryogenic electron microscopy (cryo-EM), the interphase between Li metal and LiPON is successfully preserved and probed, revealing a multilayer mosaic SEI structure with concentration gradients of nitrogen and phosphorous, materializing as crystallites within an amorphous matrix. This unique SEI nanostructure is less than 80 nm and is shown stable and free of any organic lithium containing species or lithium fluoride components, in contrast to SEIs often found in state-of-the-art organic liquid electrolytes. Our findings reveal insights on the nanostructures and chemistry of such SEIs as a key component in lithium metal batteries to stabilize Li metal anode.

cond-mat.mtrl-sci↗
arXiv2020

Hyperfine Group Ratio: A Recipe for Deriving Kinetic Temperature from Ammonia Inversion Lines

Although ammonia is a widely used interstellar thermometer, the estimation of its rotational and kinetic temperatures can be affected by the blended Hyperfine Components (HFCs). We developed a new recipe, referred to as the HyperFine Group Ratio (HFGR), which utilizes only direct observables, namely the intensity ratios between the grouped HFCs. As tested on the model spectra, the empirical formulae in HFGR can derive the rotational temperature ($T_{\rm rot}$) from the HFC group ratios in an unambiguous manner. We compared HFGR with two other classical methods, intensity ratio and hyperfine fitting, based on both simulated spectra and real data. HFGR has three major improvements. First, HFGR does not require modeling the HFC or fitting the line profiles, thus is more robust against the effect of HFC blending. Second, the simulation-enabled empirical formulae are much faster than fitting the spectra over the parameter space, so the computer time and human time can be both largely saved. Third, the statistical uncertainty of the temperature $ΔT_{\rm rot}$ as a function of the signal-to-noise ratio (SNR) is a natural product of the HFGR recipe. The internal error of HFGR is $ΔT_{\rm rot}\leq0.5$ K over a broad parameter space of rotational temperature (10 to 60 K), line width (0.3 to 4 km/s), and optical depth (0 to 5). When there is a spectral noise, HFGR can also maintain a reasonable uncertainty level at $ΔT_{\rm rot}\leq 1.0$ K (1 $σ$) when SNR > 4.

astro-ph.GA↗
arXiv2020

Context-Aware Refinement Network Incorporating Structural Connectivity Prior for Brain Midline Delineation

Brain midline delineation can facilitate the clinical evaluation of brain midline shift, which plays an important role in the diagnosis and prognosis of various brain pathology. Nevertheless, there are still great challenges with brain midline delineation, such as the largely deformed midline caused by the mass effect and the possible morphological failure that the predicted midline is not a connected curve. To address these challenges, we propose a context-aware refinement network (CAR-Net) to refine and integrate the feature pyramid representation generated by the UNet. Consequently, the proposed CAR-Net explores more discriminative contextual features and a larger receptive field, which is of great importance to predict largely deformed midline. For keeping the structural connectivity of the brain midline, we introduce a novel connectivity regular loss (CRL) to punish the disconnectivity between adjacent coordinates. Moreover, we address the ignored prerequisite of previous regression-based methods that the brain CT image must be in the standard pose. A simple pose rectification network is presented to align the source input image to the standard pose image. Extensive experimental results on the CQ dataset and one inhouse dataset show that the proposed method requires fewer parameters and outperforms three state-of-the-art methods in terms of four evaluation metrics. Code is available at https://github.com/ShawnBIT/Brain-Midline-Detection.

cs.CV↗
arXiv2020

Filament Intersections and Cold Dense Cores in Orion A North

We studied the filament structures and dense cores in OMC-2,3 region in Orion A North molecular cloud using the high-resolution N2H+ (1-0) spectral cube observed with the Atacama Large Millimeter/Submillimeter Array (ALMA). The filament network over a total length of 2 pc is found to contain 170 intersections and 128 candidate dense cores. The dense cores are all displaced from the infrared point sources (possible young stars), and the major fraction of cores (103) are located around the intersections. Towards the intersections, there is also an increasing trend for the total column density Ntot as well as the the power-law index of the column-density Probability Distribution Function (N-PDF), suggesting that the intersections would in general have more significant gas assembly than the other part of the filament paths. The virial analysis shows that the dense cores mostly have virial mass ratio of alpha_vir=M_vir/M_gas<1.0, suggesting them to be bounded by the self gravity. In the mean time, only about 23 percent of the cores have critical mass ratio of alpha_crit=M_crit/M_gas<1.0, suggesting them to be unstable against core collapse. Combining these results, it shows that the major fraction of the cold starless and possible prestellar cores in OMC-2,3 are being assembled around the intersections, and currently in a gravitationally bound state. But more extensive core collapse and star formation may still require continuous core-mass growth or other perturbatio

astro-ph.GA↗