arXiv subjects

Meng Zhang

Publications and source records attributed to Meng Zhang.

At least 181 records · Page 10Linked to original sources

arXiv2022

Experiments on Anomaly Detection in Autonomous Driving by Forward-Backward Style Transfers

Great progress has been achieved in the community of autonomous driving in the past few years. As a safety-critical problem, however, anomaly detection is a huge hurdle towards a large-scale deployment of autonomous vehicles in the real world. While many approaches, such as uncertainty estimation or segmentation-based image resynthesis, are extremely promising, there is more to be explored. Especially inspired by works on anomaly detection based on image resynthesis, we propose a novel approach for anomaly detection through style transfer. We leverage generative models to map an image from its original style domain of road traffic to an arbitrary one and back to generate pixelwise anomaly scores. However, our experiments have proven our hypothesis wrong, and we were unable to produce significant results. Nevertheless, we want to share our findings, so that others can learn from our experiments.

cs.CV↗

arXiv2022

Universal Conditional Masked Language Pre-training for Neural Machine Translation

Pre-trained sequence-to-sequence models have significantly improved Neural Machine Translation (NMT). Different from prior works where pre-trained models usually adopt an unidirectional decoder, this paper demonstrates that pre-training a sequence-to-sequence model but with a bidirectional decoder can produce notable performance gains for both Autoregressive and Non-autoregressive NMT. Specifically, we propose CeMAT, a conditional masked language model pre-trained on large-scale bilingual and monolingual corpora in many languages. We also introduce two simple but effective methods to enhance the CeMAT, aligned code-switching & masking and dynamic dual-masking. We conduct extensive experiments and show that our CeMAT can achieve significant performance improvement for all scenarios from low- to extremely high-resource languages, i.e., up to +14.4 BLEU on low resource and +7.9 BLEU improvements on average for Autoregressive NMT. For Non-autoregressive NMT, we demonstrate it can also produce consistent performance gains, i.e., up to +5.3 BLEU. To the best of our knowledge, this is the first work to pre-train a unified model for fine-tuning on both NMT tasks. Code, data, and pre-trained models are available at https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/CeMAT.

cs.CL↗

arXiv2022

Triangular Transfer: Freezing the Pivot for Triangular Machine Translation

Triangular machine translation is a special case of low-resource machine translation where the language pair of interest has limited parallel data, but both languages have abundant parallel data with a pivot language. Naturally, the key to triangular machine translation is the successful exploitation of such auxiliary data. In this work, we propose a transfer-learning-based approach that utilizes all types of auxiliary data. As we train auxiliary source-pivot and pivot-target translation models, we initialize some parameters of the pivot side with a pre-trained language model and freeze them to encourage both translation models to work in the same pivot language space, so that they can be smoothly transferred to the source-target translation model. Experiments show that our approach can outperform previous ones.

cs.CL↗

arXiv2022

Dynamic Multi-Branch Layers for On-Device Neural Machine Translation

With the rapid development of artificial intelligence (AI), there is a trend in moving AI applications, such as neural machine translation (NMT), from cloud to mobile devices. Constrained by limited hardware resources and battery, the performance of on-device NMT systems is far from satisfactory. Inspired by conditional computation, we propose to improve the performance of on-device NMT systems with dynamic multi-branch layers. Specifically, we design a layer-wise dynamic multi-branch network with only one branch activated during training and inference. As not all branches are activated during training, we propose shared-private reparameterization to ensure sufficient training for each branch. At almost the same computational cost, our method achieves improvements of up to 1.7 BLEU points on the WMT14 English-German translation task and 1.8 BLEU points on the WMT20 Chinese-English translation task over the Transformer model, respectively. Compared with a strong baseline that also uses multiple branches, the proposed method is up to 1.5 times faster with the same number of parameters.

cs.CL↗

arXiv2021

A review of data-driven short-term voltage stability assessment of power systems: Concept, principle, and challenges

With the rapid growth of power market reform and power demand, the power transmission capacity of a power grid is approaching its limit, and the secure and stable operation of power systems becomes increasingly important. In particular, in modern power grids, the proportion of dynamic loads with fast recovery characteristics such as air conditioners, refrigerators, and industrial motors is increasing. As well as the increasing proportion of different forms of renewable energy in power systems. Therefore, short-term voltage stability (STVS) of power systems cannot be ignored. This article comprehensively sorts out the STVS problems of power systems from the perspective of data-driven methods and discusses existing challenges.

eess.SY↗

arXiv2021

SimulSLT: End-to-End Simultaneous Sign Language Translation

Sign language translation as a kind of technology with profound social significance has attracted growing researchers' interest in recent years. However, the existing sign language translation methods need to read all the videos before starting the translation, which leads to a high inference latency and also limits their application in real-life scenarios. To solve this problem, we propose SimulSLT, the first end-to-end simultaneous sign language translation model, which can translate sign language videos into target text concurrently. SimulSLT is composed of a text decoder, a boundary predictor, and a masked encoder. We 1) use the wait-k strategy for simultaneous translation. 2) design a novel boundary predictor based on the integrate-and-fire module to output the gloss boundary, which is used to model the correspondence between the sign language video and the gloss. 3) propose an innovative re-encode method to help the model obtain more abundant contextual information, which allows the existing video features to interact fully. The experimental results conducted on the RWTH-PHOENIX-Weather 2014T dataset show that SimulSLT achieves BLEU scores that exceed the latest end-to-end non-simultaneous sign language translation model while maintaining low latency, which proves the effectiveness of our method.

cs.CV↗

arXiv2021

Inhibition of Current-spike Formation Based on Longitudinal Phase Space Manipulation for High-Repetition-Rate X-ray FEL

The formation of a double-horn current profile is a challenging issue in the achievement of electron bunch with high peak current, especially for high-repetition-rate X-ray free-electron lasers (XFELs) where more nonlinear and intricate beam dynamics propagation is involved. Here, we propose to correct the nonlinear beam longitudinal phase space distortion in the photoinjector section with a dual-mode buncher. In combination with the evolutionary many-objective beam dynamics optimization, this method is shown to be effective in manipulating the longitudinal phase space at the injector exit. Furthermore, the formation of the current spike is avoided after the multi-stage charge density modulation and electron bunch with a peak current of 1.6 kA is achieved for 100-pC bunch charge.Start-to-end simulations based on the Shanghai high-repetition-rate XFEL and extreme light facility demonstrate that the proposed scheme can increase the FEL pulse energy by more than 3 times in a cascading operation of echo-enabled harmonic generation and high-gain harmonic generation. Moreover, this method can also be used for longitudinal phase space shaping of electron beams operating at a high repetition rate to meet the specific demands of different researches.

physics.acc-ph↗

arXiv2021

A Deep-Learning Intelligent System Incorporating Data Augmentation for Short-Term Voltage Stability Assessment of Power Systems

Facing the difficulty of expensive and trivial data collection and annotation, how to make a deep learning-based short-term voltage stability assessment (STVSA) model work well on a small training dataset is a challenging and urgent problem. Although a big enough dataset can be directly generated by contingency simulation, this data generation process is usually cumbersome and inefficient; while data augmentation provides a low-cost and efficient way to artificially inflate the representative and diversified training datasets with label preserving transformations. In this respect, this paper proposes a novel deep-learning intelligent system incorporating data augmentation for STVSA of power systems. First, due to the unavailability of reliable quantitative criteria to judge the stability status for a specific power system, semi-supervised cluster learning is leveraged to obtain labeled samples in an original small dataset. Second, to make deep learning applicable to the small dataset, conditional least squares generative adversarial networks (LSGAN)-based data augmentation is introduced to expand the original dataset via artificially creating additional valid samples. Third, to extract temporal dependencies from the post-disturbance dynamic trajectories of a system, a bi-directional gated recurrent unit with attention mechanism based assessment model is established, which bi-directionally learns the significant time dependencies and automatically allocates attention weights. The test results demonstrate the presented approach manages to achieve better accuracy and a faster response time with original small datasets. Besides classification accuracy, this work employs statistical measures to comprehensively examine the performance of the proposal.

cs.LG↗

arXiv2021

Faithful Edge Federated Learning: Scalability and Privacy

Federated learning enables machine learning algorithms to be trained over a network of multiple decentralized edge devices without requiring the exchange of local datasets. Successfully deploying federated learning requires ensuring that agents (e.g., mobile devices) faithfully execute the intended algorithm, which has been largely overlooked in the literature. In this study, we first use risk bounds to analyze how the key feature of federated learning, unbalanced and non-i.i.d. data, affects agents' incentives to voluntarily participate and obediently follow traditional federated learning algorithms. To be more specific, our analysis reveals that agents with less typical data distributions and relatively more samples are more likely to opt out of or tamper with federated learning algorithms. To this end, we formulate the first faithful implementation problem of federated learning and design two faithful federated learning mechanisms which satisfy economic properties, scalability, and privacy. Further, the time complexity of computing all agents' payments in the number of agents is $\mathcal{O}(1)$. First, we design a Faithful Federated Learning (FFL) mechanism which approximates the Vickrey-Clarke-Groves (VCG) payments via an incremental computation. We show that it achieves (probably approximate) optimality, faithful implementation, voluntary participation, and some other economic properties (such as budget balance). Second, by partitioning agents into several subsets, we present a scalable VCG mechanism approximation. We further design a scalable and Differentially Private FFL (DP-FFL) mechanism, the first differentially private faithful mechanism, that maintains the economic properties. Our mechanism enables one to make three-way performance tradeoffs among privacy, the iterations needed, and payment accuracy loss.

cs.LG↗

arXiv2021

A Review of Deep Reinforcement Learning for Smart Building Energy Management

Global buildings account for about 30% of the total energy consumption and carbon emission, raising severe energy and environmental concerns. Therefore, it is significant and urgent to develop novel smart building energy management (SBEM) technologies for the advance of energy-efficient and green buildings. However, it is a nontrivial task due to the following challenges. Firstly, it is generally difficult to develop an explicit building thermal dynamics model that is both accurate and efficient enough for building control. Secondly, there are many uncertain system parameters (e.g., renewable generation output, outdoor temperature, and the number of occupants). Thirdly, there are many spatially and temporally coupled operational constraints. Fourthly, building energy optimization problems can not be solved in real-time by traditional methods when they have extremely large solution spaces. Fifthly, traditional building energy management methods have respective applicable premises, which means that they have low versatility when confronted with varying building environments. With the rapid development of Internet of Things technology and computation capability, artificial intelligence technology find its significant competence in control and optimization. As a general artificial intelligence technology, deep reinforcement learning (DRL) is promising to address the above challenges. Notably, the recent years have seen the surge of DRL for SBEM. However, there lacks a systematic overview of different DRL methods for SBEM. To fill the gap, this paper provides a comprehensive review of DRL for SBEM from the perspective of system scale. In particular, we identify the existing unresolved issues and point out possible future research directions.

eess.SY↗

arXiv2021

Uncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Training

Learning multilingual and multi-domain translation model is challenging as the heterogeneous and imbalanced data make the model converge inconsistently over different corpora in real world. One common practice is to adjust the share of each corpus in the training, so that the learning process is balanced and low-resource cases can benefit from the high resource ones. However, automatic balancing methods usually depend on the intra- and inter-dataset characteristics, which is usually agnostic or requires human priors. In this work, we propose an approach, MultiUAT, that dynamically adjusts the training data usage based on the model's uncertainty on a small set of trusted clean data for multi-corpus machine translation. We experiments with two classes of uncertainty measures on multilingual (16 languages with 4 settings) and multi-domain settings (4 for in-domain and 2 for out-of-domain on English-German translation) and demonstrate our approach MultiUAT substantially outperforms its baselines, including both static and dynamic strategies. We analyze the cross-domain transfer and show the deficiency of static and similarity based methods.

cs.CL↗

arXiv2021

Most "young" $α$-rich stars have high masses but are actually old

Recent observations have revealed a population of $α$-element abundances enhanced giant stars with unexpected high masses ($\gtrsim$1 $M_\odot$) from asteroseismic analysis and spectroscopy. Assuming single-star evolution, their masses imply young ages ($τ<6$Gyr) incompatible with the canonical Galactic chemical evolution scenario. Here we study the chemistry and kinematics of a large sample of such $α$-rich, high-mass red giant branch (RGB) stars drawn from the LAMOST spectroscopic surveys. Using LAMOST and Gaia, we found these stars share the same kinematics as the canonical high-$α$ old stellar population in the Galactic thick disk. The stellar abundances show that these high-$α$ massive stars have $α$- and iron-peak element abundances similar to those of the high-$α$ old thick disk stars. However, a portion of them exhibit higher [(N+C)/Fe] and [Ba/Fe] ratios, which implies they have gained C- and Ba-rich materials from extra sources, presumably asymptotic giant branch (AGB) companions. The results support the previous suggestion that these RGB stars are products of binary evolution. Their high masses thus mimic "young" single stars, yet in fact they belong to an intrinsic old stellar population. To fully explain the stellar abundance patterns of our sample stars, a variety of binary evolution channels, such as, main-sequence (MS) + RGB, MS + AGB, RGB + RGB and RGB + AGB, are required, pointing to diverse formation mechanisms of these seemly rejuvenated cannibals. With this larger sample, our results confirm earlier findings that most, if not all, $α$-rich stars in the Galactic disk seem to be old.

astro-ph.GA↗

arXiv2021

SimulLR: Simultaneous Lip Reading Transducer with Attention-Guided Adaptive Memory

Lip reading, aiming to recognize spoken sentences according to the given video of lip movements without relying on the audio stream, has attracted great interest due to its application in many scenarios. Although prior works that explore lip reading have obtained salient achievements, they are all trained in a non-simultaneous manner where the predictions are generated requiring access to the full video. To breakthrough this constraint, we study the task of simultaneous lip reading and devise SimulLR, a simultaneous lip Reading transducer with attention-guided adaptive memory from three aspects: (1) To address the challenge of monotonic alignments while considering the syntactic structure of the generated sentences under simultaneous setting, we build a transducer-based model and design several effective training strategies including CTC pre-training, model warm-up and curriculum learning to promote the training of the lip reading transducer. (2) To learn better spatio-temporal representations for simultaneous encoder, we construct a truncated 3D convolution and time-restricted self-attention layer to perform the frame-to-frame interaction within a video segment containing fixed number of frames. (3) The history information is always limited due to the storage in real-time scenarios, especially for massive video data. Therefore, we devise a novel attention-guided adaptive memory to organize semantic information of history segments and enhance the visual representations with acceptable computation-aware latency. The experiments show that the SimulLR achieves the translation speedup 9.10$\times$ compared with the state-of-the-art non-simultaneous methods, and also obtains competitive results, which indicates the effectiveness of our proposed methods.

cs.CV↗

arXiv2021

Projected entangled pair states study of anisotropic-exchange magnets on triangular lattice

The anisotropic-exchange spin-1/2 model on a triangular lattice has been used to describe the rare-earth chalcogenides, which may have exotic ground states. We investigate the quantum phase diagram of the model by using the projected entangled pair state (PEPS) method, and compare it to the classical phase diagram. Besides two stripe-ordered phase, and the 120$^\circ$ state, there is also a multi-\textbf{Q} phase. We identify the multi-\textbf{Q} phase as a $Z_{2}$ vortex state. No quantum spin liquid state is found in the phase diagram, contrary to the previous DMRG calculations.

cond-mat.str-el↗

arXiv2021

Autoencoder-based Semantic Novelty Detection: Towards Dependable AI-based Systems

Many autonomous systems, such as driverless taxis, perform safety critical functions. Autonomous systems employ artificial intelligence (AI) techniques, specifically for the environment perception. Engineers cannot completely test or formally verify AI-based autonomous systems. The accuracy of AI-based systems depends on the quality of training data. Thus, novelty detection - identifying data that differ in some respect from the data used for training - becomes a safety measure for system development and operation. In this paper, we propose a new architecture for autoencoder-based semantic novelty detection with two innovations: architectural guidelines for a semantic autoencoder topology and a semantic error calculation as novelty criteria. We demonstrate that such a semantic novelty detection outperforms autoencoder-based novelty detection approaches known from literature by minimizing false negatives.

cs.AI↗

arXiv2021

Multi-window SRS imaging using a rapid widely tunable fiber laser

Spectroscopic stimulated Raman scattering (SRS) imaging has become a useful tool finding a broad range of applications. Yet, wider adoption is hindered by the bulky and environmentally-sensitive solid-state optical parametric oscillator (OPO) in current SRS microscope. Moreover, chemically-informative multi-window SRS imaging across C-H, C-D and fingerprint Raman regions is challenging due to the slow wavelength tuning speed of the solid-state OPO. In this work, we present a multi-window SRS imaging system based on a compact and robust fiber laser with rapid and widely tuning capability. To address the relative intensity noise intrinsic to fiber laser, we implemented auto-balanced detection which enhances the signal-to-noise ratio of stimulated Raman loss imaging by 23 times. We demonstrate high-quality SRS metabolic imaging of fungi, cancer cells, and Caenorhabditis elegans across the C-H, C-D and fingerprint Raman windows. Our re-sults showcase the potential of the compact multi-window SRS system for a broad range of applications.

physics.optics↗

arXiv2021

Pricing Fresh Data

We introduce the concept of {\it fresh data trading}, in which a destination user requests, and pays for, fresh data updates from a source provider, and data freshness is captured by the {\it age of information} (AoI) metric. Keeping data fresh relies on frequent data updates by the source, which motivates the source to {\it price fresh data}. In this work, the destination incurs an age-related cost, modeled as a general increasing function of the AoI. The source designs a pricing mechanism to maximize its profit; the destination chooses a data update schedule to trade off its payments to the source and its age-related cost. Depending on different real-time applications and scenarios, we study both a predictable-deadline and an unpredictable-deadline models. The key challenge of designing the optimal pricing scheme lies in the destination's time-interdependent valuations, due to the nature of AoI and the infinite-dimensional and dynamic optimization. To this end, we consider three pricing schemes that exploit and understand the profitability of three different dimensions in designing pricing: a {\it time-dependent} pricing scheme, in which the price for each update depends on when it is requested; a {\it quantity-based} pricing scheme, in which the price of each update depends on how many updates have been previously requested; a {\it subscription-based} pricing scheme, in which the price for each update is flat-rate but the source charges an additional subscription fee. Our analysis reveals that the optimal subscription-based pricing maximizes the source's profit among all possible pricing schemes under both predictable deadline and unpredictable deadline models; the optimal quantity-based pricing scheme is only optimal with a predictable deadline; the time-dependent pricing scheme, under the unpredictable deadline, is asymptotically optimal under significant time discounting.

cs.NI↗

arXiv2021

First-principles study of magnetic structures of triangular antiferromagnets NaYbS$_2$ and NaYbO$_2$

We investigate the magnetic interactions in triangular rare-earth delafossites materials NaYbO$_2$ and NaYbS$_2$ via first-principles calculations. The calculated Curie-Weiss temperatures are in good agreement with experiments. We perform classical Monte Carlo simulations of the two compounds using the extracted exchange parameters. We find that if only the nearest neighbor interactions are considered, the magnetic ground states of NaYbO$_2$ and NaYbS$_2$ are a stripe and a planar 120\degree~ Néel state, respectively. The simulated transition temperatures are much higher than the lowest experimental temperatures, where no magnetic ordering was observed. However, we show by adding suitable second neighbor interactions, the {\it classical} magnetic ground state of NaYbO$_2$ becomes to the $Z_2$ vortex phase, and the simulated specific heat $C_v$ are very similar to the experimental observations, with no obvious phase transition down to the extremely low temperature.

cond-mat.mtrl-sci↗