arXiv subjects

Xinyi Wang

Publications and source records attributed to Xinyi Wang.

At least 217 records · Page 12Linked to original sources

arXiv2022

Asynchronous Decentralized Federated Learning for Collaborative Fault Diagnosis of PV Stations

Due to the different losses caused by various photovoltaic (PV) array faults, accurate diagnosis of fault types is becoming increasingly important. Compared with a single one, multiple PV stations collect sufficient fault samples, but their data is not allowed to be shared directly due to potential conflicts of interest. Therefore, federated learning can be exploited to train a collaborative fault diagnosis model. However, the modeling efficiency is seriously affected by the model update mechanism since each PV station has a different computing capability and amount of data. Moreover, for the safe and stable operation of the PV system, the robustness of collaborative modeling must be guaranteed rather than simply being processed on a central server. To address these challenges, a novel asynchronous decentralized federated learning (ADFL) framework is proposed. Each PV station not only trains its local model but also participates in collaborative fault diagnosis by exchanging model parameters to improve the generalization without losing accuracy. The global model is aggregated distributedly to avoid central node failure. By designing the asynchronous update scheme, the communication overhead and training time are greatly reduced. Both the experiments and numerical simulations are carried out to verify the effectiveness of the proposed method.

cs.LG↗

arXiv2021

Numerical Simulation of a Fundamental Mechanism of Solar Eruption with Different Magnetic Flux Distributions

Solar eruptions are explosive release of coronal magnetic field energy as manifested in solar flares and coronal mass ejection. Observations have shown that the core of eruption-productive regions are often a sheared magnetic arcade, i.e., a single bipolar configuration, and, particularly, the corresponding magnetic polarities at the photosphere are elongated along a strong-gradient polarity inversion line (PIL). It remains unclear what mechanism triggers the eruption in a single bipolar field and why the one with a strong PIL is eruption-productive. Recently, using high accuracy simulations, we have established a fundamental mechanism of solar eruption initiation that a bipolar field as driven by quasi-static shearing motion at the photosphere can form an internal current sheet, and then fast magnetic reconnection triggers and drives the eruption. Here we investigate the behavior of the fundamental mechanism with different photospheric magnetic flux distributions, i.e., magnetograms, by combining theoretical analysis and numerical simulation. Our study shows that the bipolar fields of different magnetograms, as sheared continually, all exhibit similar evolutions from the slow storage to fast release of magnetic energy in accordance with the fundamental mechanism, which demonstrates the robustness of the mechanism. We further found that the magnetograms with stronger PIL produce larger eruptions, and the key reason is that the sheared bipolar fields with stronger PIL can achieve more non-potentiality, and their internal current sheet can form at a lower height and with a larger current density, by which the reconnection can be more efficient. This also provides a viable trigger mechanism for the observed eruptions in active region with strong PIL.

astro-ph.SR↗

arXiv2021

The Initiation of Shear Band Formation in Deformed Metallic Glasses from Soft Localized Domains

It has long been thought that shear band (SB) formation in amorphous solids initiates from relatively 'soft' regions in the material in which large-scale non-affine deformations become localized. The test of this hypothesis requires an effective means of identifying 'soft' regions and their evolution as the material is deformed to varying degrees, where the metric of 'softness' must also account for the effect of temperature on local material stiffness. We show that the mean square atomic displacement on a caging timescale , the 'Debye-Waller factor', provides a useful method for estimating the shear modulus of the entire material and, by extension, the material stiffness at an atomic scale. Based on this 'softness' metrology, we observe that SB formation indeed occurs through the strain-induced formation of localized soft regions in our deformed metallic glass free-standing films. Unexpectedly, the critical strain condition of SB formation occurs when the softness ( ) distribution within the emerging soft regions approaches that of the interfacial region in its undeformed state, initiating an instability with similarities to the transition to turbulence. Correspondingly, no SBs arise when the material is so thin that the entire material can be approximately described as being 'interfacial' in nature. We also quantify relaxation in the glass and the nature and origin of highly non-Gaussian particle displacements in the dynamically heterogeneous SB regions at times longer than the caging time.

cond-mat.mtrl-sci↗

arXiv2021

Counterfactual Maximum Likelihood Estimation for Training Deep Networks

Although deep learning models have driven state-of-the-art performance on a wide array of tasks, they are prone to spurious correlations that should not be learned as predictive clues. To mitigate this problem, we propose a causality-based training framework to reduce the spurious correlations caused by observed confounders. We give theoretical analysis on the underlying general Structural Causal Model (SCM) and propose to perform Maximum Likelihood Estimation (MLE) on the interventional distribution instead of the observational distribution, namely Counterfactual Maximum Likelihood Estimation (CMLE). As the interventional distribution, in general, is hidden from the observational data, we then derive two different upper bounds of the expected negative log-likelihood and propose two general algorithms, Implicit CMLE and Explicit CMLE, for causal predictions of deep learning models using observational data. We conduct experiments on both simulated data and two real-world tasks: Natural Language Inference (NLI) and Image Captioning. The results show that CMLE methods outperform the regular MLE method in terms of out-of-domain generalization performance and reducing spurious correlations, while maintaining comparable performance on the regular evaluations.

cs.LG↗

arXiv2021

A Dataset for Answering Time-Sensitive Questions

Time is an important dimension in our physical world. Lots of facts can evolve with respect to time. For example, the U.S. President might change every four years. Therefore, it is important to consider the time dimension and empower the existing QA models to reason over time. However, the existing QA datasets contain rather few time-sensitive questions, hence not suitable for diagnosing or benchmarking the model's temporal reasoning capability. In order to promote research in this direction, we propose to construct a time-sensitive QA dataset. The dataset is constructed by 1) mining time-evolving facts from WikiData and aligning them to their corresponding Wikipedia page, 2) employing crowd workers to verify and calibrate these noisy facts, 3) generating question-answer pairs based on the annotated time-sensitive facts. Our dataset poses challenges in the aspect of both temporal understanding and temporal reasoning. We evaluate different SoTA long-document QA systems like BigBird and FiD on our dataset. The best-performing model FiD can only achieve 46\% accuracy, still far behind the human performance of 87\%. We demonstrate that these models are still lacking the ability to perform consistent temporal reasoning. Therefore, we believe that our dataset could serve as a benchmark to develop NLP models more sensitive to temporal shifts. The dataset and code are released in~\url{https://github.com/wenhuchen/Time-Sensitive-QA}.

cs.CL↗

arXiv2021

Formation of Magnetic Flux Rope during Solar Eruption. I. Evolution of Toroidal Flux and Reconnection Flux

Magnetic flux ropes (MFRs) constitute the core structure of coronal mass ejections (CMEs), but hot debates remain on whether the MFR forms before or during solar eruptions. Furthermore, how flare reconnection shapes the erupting MFR is still elusive in three dimensions. Here we studied a new MHD simulation of CME initiation by tether-cutting magnetic reconnection in a single magnetic arcade. The simulation follows the whole life, including the birth and subsequent evolution, of an MFR during eruption. In the early phase, the MFR is partially separated from its ambient field by a magnetic quasi-separatrix layer (QSL) that has a double-J shaped footprint on the bottom surface. With the ongoing of the reconnection, the arms of the two J-shaped footprints continually separate from each other, and the hooks of the J shaped footprints expand and eventually become closed almost at the eruption peak time, and thereafter the MFR is fully separated from the un-reconnected field by the QSL. We further studied the evolution of the toroidal flux in the MFR and compared it with that of the reconnected flux. Our simulation reproduced an evolution pattern of increase-to-decrease of the toroidal flux, which is reported recently in observations of variations in flare ribbons and transient coronal dimming. The increase of toroidal flux is owing to the flare reconnection in the early phase that transforms the sheared arcade to twisted field lines, while its decrease is a result of reconnection between field lines in the interior of the MFR in the later phase.

astro-ph.SR↗

arXiv2021

Efficient Test Time Adapter Ensembling for Low-resource Language Varieties

Adapters are light-weight modules that allow parameter-efficient fine-tuning of pretrained models. Specialized language and task adapters have recently been proposed to facilitate cross-lingual transfer of multilingual pretrained models (Pfeiffer et al., 2020b). However, this approach requires training a separate language adapter for every language one wishes to support, which can be impractical for languages with limited data. An intuitive solution is to use a related language adapter for the new language variety, but we observe that this solution can lead to sub-optimal performance. In this paper, we aim to improve the robustness of language adapters to uncovered languages without training new adapters. We find that ensembling multiple existing language adapters makes the fine-tuned model significantly more robust to other language varieties not included in these adapters. Building upon this observation, we propose Entropy Minimized Ensemble of Adapters (EMEA), a method that optimizes the ensemble weights of the pretrained language adapters for each test sentence by minimizing the entropy of its predictions. Experiments on three diverse groups of language varieties show that our method leads to significant improvements on both named entity recognition and part-of-speech tagging across all languages.

cs.CL↗

arXiv2021

Modeling Disclosive Transparency in NLP Application Descriptions

Broader disclosive transparency$-$truth and clarity in communication regarding the function of AI systems$-$is widely considered desirable. Unfortunately, it is a nebulous concept, difficult to both define and quantify. This is problematic, as previous work has demonstrated possible trade-offs and negative consequences to disclosive transparency, such as a confusion effect, where "too much information" clouds a reader's understanding of what a system description means. Disclosive transparency's subjective nature has rendered deep study into these problems and their remedies difficult. To improve this state of affairs, We introduce neural language model-based probabilistic metrics to directly model disclosive transparency, and demonstrate that they correlate with user and expert opinions of system transparency, making them a valid objective proxy. Finally, we demonstrate the use of these metrics in a pilot study quantifying the relationships between transparency, confusion, and user perceptions in a corpus of real NLP system descriptions.

cs.CL↗

arXiv2021

Innovations Autoencoder and its Application in One-class Anomalous Sequence Detection

An innovations sequence of a time series is a sequence of independent and identically distributed random variables with which the original time series has a causal representation. The innovation at a time is statistically independent of the history of the time series. As such, it represents the new information contained at present but not in the past. Because of its simple probability structure, an innovations sequence is the most efficient signature of the original. Unlike the principle or independent component analysis representations, an innovations sequence preserves not only the complete statistical properties but also the temporal order of the original time series. An long-standing open problem is to find a computationally tractable way to extract an innovations sequence of non-Gaussian processes. This paper presents a deep learning approach, referred to as Innovations Autoencoder (IAE), that extracts innovations sequences using a causal convolutional neural network. An application of IAE to the one-class anomalous sequence detection problem with unknown anomaly and anomaly-free models is also presented.

stat.ML↗

arXiv2021

BiX-NAS: Searching Efficient Bi-directional Architecture for Medical Image Segmentation

The recurrent mechanism has recently been introduced into U-Net in various medical image segmentation tasks. Existing studies have focused on promoting network recursion via reusing building blocks. Although network parameters could be greatly saved, computational costs still increase inevitably in accordance with the pre-set iteration time. In this work, we study a multi-scale upgrade of a bi-directional skip connected network and then automatically discover an efficient architecture by a novel two-phase Neural Architecture Search (NAS) algorithm, namely BiX-NAS. Our proposed method reduces the network computational cost by sifting out ineffective multi-scale features at different levels and iterations. We evaluate BiX-NAS on two segmentation tasks using three different medical image datasets, and the experimental results show that our BiX-NAS searched architecture achieves the state-of-the-art performance with significantly lower computational cost.

eess.IV↗

arXiv2021

A Deep Learning Approach to Anomaly Sequence Detection for High-Resolution Monitoring of Power Systems

A deep learning approach is proposed to detect data and system anomalies using high-resolution continuous point-on-wave (CPOW) or phasor measurements. Both the anomaly and anomaly-free measurement models are assumed to have unknown temporal dependencies and probability distributions. Historical training samples are assumed for the anomaly-free model, while no training samples are available for the anomaly measurements. By transforming the anomaly-free observations into uniform independent and identically distributed sequences via a generative adversarial network, the proposed approach deploys a uniformity test for anomaly detection at the sensor level. A distributed detection scheme that combines sensor level detections at the control center is also proposed that combines local detections to form more reliable detections. Numerical results demonstrate significant improvement over the state-of-the-art solutions for various bad-data cases using real and synthetic CPOW and PMU data sets.

eess.SY↗

arXiv2021

Optimizing Data Usage via Differentiable Rewards

To acquire a new skill, humans learn better and faster if a tutor, based on their current knowledge level, informs them of how much attention they should pay to particular content or practice problems. Similarly, a machine learning model could potentially be trained better with a scorer that "adapts" to its current learning state and estimates the importance of each training data instance. Training such an adaptive scorer efficiently is a challenging problem; in order to precisely quantify the effect of a data instance at a given time during the training, it is typically necessary to first complete the entire training process. To efficiently optimize data usage, we propose a reinforcement learning approach called Differentiable Data Selection (DDS). In DDS, we formulate a scorer network as a learnable function of the training data, which can be efficiently updated along with the main model being trained. Specifically, DDS updates the scorer with an intuitive reward signal: it should up-weigh the data that has a similar gradient with a dev set upon which we would finally like to perform well. Without significant computing overhead, DDS delivers strong and consistent improvements over several strong baselines on two very different tasks of machine translation and image classification.

cs.LG↗

arXiv2021

RefBERT: Compressing BERT by Referencing to Pre-computed Representations

Recently developed large pre-trained language models, e.g., BERT, have achieved remarkable performance in many downstream natural language processing applications. These pre-trained language models often contain hundreds of millions of parameters and suffer from high computation and latency in real-world applications. It is desirable to reduce the computation overhead of the models for fast training and inference while keeping the model performance in downstream applications. Several lines of work utilize knowledge distillation to compress the teacher model to a smaller student model. However, they usually discard the teacher's knowledge when in inference. Differently, in this paper, we propose RefBERT to leverage the knowledge learned from the teacher, i.e., facilitating the pre-computed BERT representation on the reference sample and compressing BERT into a smaller student model. To guarantee our proposal, we provide theoretical justification on the loss function and the usage of reference samples. Significantly, the theoretical result shows that including the pre-computed teacher's representations on the reference samples indeed increases the mutual information in learning the student model. Finally, we conduct the empirical evaluation and show that our RefBERT can beat the vanilla TinyBERT over 8.1\% and achieves more than 94\% of the performance of $\BERTBASE$ on the GLUE benchmark. Meanwhile, RefBERT is 7.4x smaller and 9.5x faster on inference than BERT$_{\rm BASE}$.

cs.CL↗

arXiv2021

Identifying latent shared mobility preference segments in low-income communities: ride-hailing, fixed-route bus, and mobility-on-demand transit

Concepts of Mobility-on-Demand (MOD) and Mobility as a Service (MaaS), which feature the integration of various shared-use mobility options, have gained widespread popularity in recent years. While these concepts promise great benefits to travelers, their heavy reliance on technology raises equity concerns as socially disadvantaged population groups can be left out in an era of on-demand mobility. This paper investigates the potential uptake of MOD transit services (integrated fixed-route and on-demand services) among travelers living in low-income communities. Specially, we analyze people's latent attitude towards three shared-use mobility services, including ride-hailing services, fixed-route transit, and MOD transit. We conduct a latent class cluster analysis of 825 survey respondents sampled from low-income neighborhoods in Detroit and Ypsilanti, Michigan. We identified three latent segments: shared-mode enthusiast, shared-mode opponent, and fixed-route transit loyalist. People from the shared-mode enthusiast segment often use ride-hailing services and live in areas with poor transit access, and they are likely to be the early adopters of MOD transit services. The shared-mode opponent segment mainly includes vehicle owners who lack interests in shared mobility options. The fixed-route transit loyalist segment includes a considerable share of low-income individuals who face technological barriers to use the MOD transit. We also find that males, college graduates, car owners, people with a mobile data plan, and people living in poor-transit-access areas have a higher level of preferences for MOD transit services. We conclude with policy recommendations for developing more accessible and equitable MOD transit services.

physics.soc-ph↗

arXiv2021

Applications of Artificial Intelligence to aid detection of dementia: a narrative review on current capabilities and future directions

With populations ageing, the number of people with dementia worldwide is expected to triple to 152 million by 2050. Seventy percent of cases are due to Alzheimer's disease (AD) pathology and there is a 10-20 year 'pre-clinical' period before significant cognitive decline occurs. We urgently need, cost effective, objective methods to detect AD, and other dementias, at an early stage. Risk factor modification could prevent 40% of cases and drug trials would have greater chances of success if participants are recruited at an earlier stage. Currently, detection of dementia is largely by pen and paper cognitive tests but these are time consuming and insensitive to pre-clinical phases. Specialist brain scans and body fluid biomarkers can detect the earliest stages of dementia but are too invasive or expensive for widespread use. With the advancement of technology, Artificial Intelligence (AI) shows promising results in assisting with detection of early-stage dementia. Existing AI-aided methods and potential future research directions are reviewed and discussed.

cs.AI↗

arXiv2021

Multi-view Subword Regularization

Multilingual pretrained representations generally rely on subword segmentation algorithms to create a shared multilingual vocabulary. However, standard heuristic algorithms often lead to sub-optimal segmentation, especially for languages with limited amounts of data. In this paper, we take two major steps towards alleviating this problem. First, we demonstrate empirically that applying existing subword regularization methods(Kudo, 2018; Provilkov et al., 2020) during fine-tuning of pre-trained multilingual representations improves the effectiveness of cross-lingual transfer. Second, to take full advantage of different possible input segmentations, we propose Multi-view Subword Regularization (MVR), a method that enforces the consistency between predictions of using inputs tokenized by the standard and probabilistic segmentations. Results on the XTREME multilingual benchmark(Hu et al., 2020) show that MVR brings consistent improvements of up to 2.5 points over using standard segmentation algorithms.

cs.CL↗

arXiv2021

Adaptive Subband Compression for Streaming of Continuous Point-on-Wave and PMU Data

A data compression system capable of providing real-time streaming of high-resolution continuous point-on-wave (CPOW) and phasor measurement unit (PMU) measurements is proposed. Referred to as adaptive subband compression (ASBC), the proposed technique partitions the signal space into subbands and adaptively compresses subband signals based on each subband's active bandwidth. The proposed technique conforms to existing industry phasor measurement standards, making it suitable for streaming high-resolution CPOW and PMU data either in continuous or burst on-demand/event-triggered modes. Experiments on synthetic and real data show that ASBC reduces the CPOW sampling rates by several orders of magnitude for real-time streaming while maintaining the precision required by industry standards.

eess.SP↗

arXiv2021

Dynamic Heterogeneity, Cooperative Motion, and Johari-Goldstein $β$-Relaxation in a Metallic Glass-Forming Material Exhibiting a Fragile to Strong Transition

We investigate the Johari-Goldstein (JG) $β$-relaxation process in a model metallic glass-forming (GF) material (Al90Sm10), previously studied extensively by both frequency-dependent mechanical measurements and simulation studies devoted to equilibrium properties, by molecular dynamics simulations based on validated and optimized interatomic potentials with the primary aim of better understanding the nature of this universal relaxation process from a dynamic heterogeneity (DH) perspective. The present relatively low temperature and long-time simulations reveal a direct correspondence between the JG $β$-relaxation time $τ_{JG}$ and the lifetime of the mobile particle clusters $τ_M$, defined as in previous DH studies, a relationship dual to the corresponding previously observed relationship between the $α$-relaxation time $τ_α$ and the lifetime of immobile particle clusters $τ_{IM}$. Moreover, we find that the average diffusion coefficient D nearly coincides with D$_{Al}$, of the smaller atomic species (Al), and that the hopping time associated with D coincides with $τ_{JG}$ to within numerical uncertainty, both trends being in accord with experimental studies. This indicates that the JG $α$-relaxation is dominated by the smaller atomic species and the observation of a direct relation between this relaxation process and rate of molecular diffusion in GF materials at low temperatures where the JG $β$-relaxation becomes the prevalent mode of structural relaxation. As an unanticipated aspect of our study, we find that Al90Sm10 exhibits fragile-to-strong (FS) glass-formation, as found in many other metallic GF liquids, but this fact does not greatly alter the geometrical nature of DH in this material and the relation of DH to dynamical properties.

cond-mat.mtrl-sci↗