arXiv subjects

Xiaonan Li

Publications and source records attributed to Xiaonan Li.

33 records · Page 2Linked to original sources

arXiv2022

Towards Implementing ML-Based Failure Detectors

Most existing failure detection algorithms rely on statistical methods, and very few use machine learning (ML). This paper explores the viability of ML in the field of failure detection: is it possible to implement an ML-based detector that achieves a satisfactory quality of service? We implement a prototype that uses a basic long short-term memory neural network algorithm, and study its behavior with real traces. Although ML model has comparatively longer computing time, our prototype performs well in terms of accuracy and detection time.

cs.DC↗

arXiv2022

An Embarrassingly Easy but Strong Baseline for Nested Named Entity Recognition

Named entity recognition (NER) is the task to detect and classify the entity spans in the text. When entity spans overlap between each other, this problem is named as nested NER. Span-based methods have been widely used to tackle the nested NER. Most of these methods will get a score $n \times n$ matrix, where $n$ means the length of sentence, and each entry corresponds to a span. However, previous work ignores spatial relations in the score matrix. In this paper, we propose using Convolutional Neural Network (CNN) to model these spatial relations in the score matrix. Despite being simple, experiments in three commonly used nested NER datasets show that our model surpasses several recently proposed methods with the same pre-trained encoders. Further analysis shows that using CNN can help the model find more nested entities. Besides, we found that different papers used different sentence tokenizations for the three nested NER datasets, which will influence the comparison. Thus, we release a pre-processing script to facilitate future comparison.

cs.CL↗

arXiv2021

Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoning

\textbf{P}re-\textbf{T}rained \textbf{M}odel\textbf{s} have been widely applied and recently proved vulnerable under backdoor attacks: the released pre-trained weights can be maliciously poisoned with certain triggers. When the triggers are activated, even the fine-tuned model will predict pre-defined labels, causing a security threat. These backdoors generated by the poisoning methods can be erased by changing hyper-parameters during fine-tuning or detected by finding the triggers. In this paper, we propose a stronger weight-poisoning attack method that introduces a layerwise weight poisoning strategy to plant deeper backdoors; we also introduce a combinatorial trigger that cannot be easily detected. The experiments on text classification tasks show that previous defense methods cannot resist our weight-poisoning method, which indicates that our method can be widely applied and may provide hints for future model robustness studies.

cs.CR↗

arXiv2021

Accelerating BERT Inference for Sequence Labeling via Early-Exit

Both performance and efficiency are crucial factors for sequence labeling tasks in many real-world scenarios. Although the pre-trained models (PTMs) have significantly improved the performance of various sequence labeling tasks, their computational cost is expensive. To alleviate this problem, we extend the recent successful early-exit mechanism to accelerate the inference of PTMs for sequence labeling tasks. However, existing early-exit mechanisms are specifically designed for sequence-level tasks, rather than sequence labeling. In this paper, we first propose a simple extension of sentence-level early-exit for sequence labeling tasks. To further reduce the computational cost, we also propose a token-level early-exit mechanism that allows partial tokens to exit early at different layers. Considering the local dependency inherent in sequence labeling, we employed a window-based criterion to decide for a token whether or not to exit. The token-level early-exit brings the gap between training and inference, so we introduce an extra self-sampling fine-tuning stage to alleviate it. The extensive experiments on three popular sequence labeling tasks show that our approach can save up to 66%-75% inference cost with minimal performance degradation. Compared with competitive compressed models such as DistilBERT, our approach can achieve better performance under the same speed-up ratios of 2X, 3X, and 4X.

cs.CL↗

arXiv2020

BERT for Monolingual and Cross-Lingual Reverse Dictionary

Reverse dictionary is the task to find the proper target word given the word description. In this paper, we tried to incorporate BERT into this task. However, since BERT is based on the byte-pair-encoding (BPE) subword encoding, it is nontrivial to make BERT generate a word given the description. We propose a simple but effective method to make BERT generate the target word for this specific task. Besides, the cross-lingual reverse dictionary is the task to find the proper target word described in another language. Previous models have to keep two different word embeddings and learn to align these embeddings. Nevertheless, by using the Multilingual BERT (mBERT), we can efficiently conduct the cross-lingual reverse dictionary with one subword embedding, and the alignment between languages is not necessary. More importantly, mBERT can achieve remarkable cross-lingual reverse dictionary performance even without the parallel corpus, which means it can conduct the cross-lingual reverse dictionary with only corresponding monolingual data. Code is publicly available at https://github.com/yhcc/BertForRD.git.

cs.CL↗

arXiv2020

Deterministic Zeckendorf Games

Zeckendorf proved that every positive integer can be written uniquely as the sum of non-adjacent Fibonacci numbers. We further explore a two-player Zeckendorf game introduced in Baird-Smith, Epstein, Flint, and Miller: Given a fixed integer $n$ and an initial decomposition of $n = nF_1$, players alternate using moves related to the recurrence relation $F_{n+1} = F_n + F_{n_1}$, and the last player to move wins. We improve the upper bound on the number of moves possible and show that it is of the same order in $n$ as the lower bound; this is an improvement by a logarithm over previous work. The new upper bound is $3n - 3Z(n) - IZ(n) + 1$, and the existing lower bound is sharp at $n - Z(n)$ moves, where $Z(n)$ is the number of terms in the Zeckendorf decomposition of $n$ and $IZ(n)$ is the sum of indices in the same Zeckendorf decomposition of $n$. We also studied four deterministic variants of the game, where there was a fixed order on which available move one takes: Combine Largest, Split Largest, Combine Smallest and Split Smallest. We prove that Combine Largest and Split Largest realize the lower bound. Split Smallest has the largest number of moves over all possible games, and is close to the new upper bound. For Combine Split games, the number of moves grows linearly with $n$.

math.NT↗

arXiv2020

FLAT: Chinese NER Using Flat-Lattice Transformer

Recently, the character-word lattice structure has been proved to be effective for Chinese named entity recognition (NER) by incorporating the word information. However, since the lattice structure is complex and dynamic, most existing lattice-based models are hard to fully utilize the parallel computation of GPUs and usually have a low inference-speed. In this paper, we propose FLAT: Flat-LAttice Transformer for Chinese NER, which converts the lattice structure into a flat structure consisting of spans. Each span corresponds to a character or latent word and its position in the original lattice. With the power of Transformer and well-designed position encoding, FLAT can fully leverage the lattice information and has an excellent parallelization ability. Experiments on four datasets show FLAT outperforms other lexicon-based models in performance and efficiency.

cs.CL↗

arXiv2020

Does It Make Sense? And Why? A Pilot Study for Sense Making and Explanation

Introducing common sense to natural language understanding systems has received increasing research attention. It remains a fundamental question on how to evaluate whether a system has a sense making capability. Existing benchmarks measures commonsense knowledge indirectly and without explanation. In this paper, we release a benchmark to directly test whether a system can differentiate natural language statements that make sense from those that do not make sense. In addition, a system is asked to identify the most crucial reason why a statement does not make sense. We evaluate models trained over large-scale language modeling tasks as well as human performance, showing that there are different challenges for system sense making.

cs.AI↗

arXiv2019

TENER: Adapting Transformer Encoder for Named Entity Recognition

The Bidirectional long short-term memory networks (BiLSTM) have been widely used as an encoder in models solving the named entity recognition (NER) task. Recently, the Transformer is broadly adopted in various Natural Language Processing (NLP) tasks owing to its parallelism and advantageous performance. Nevertheless, the performance of the Transformer in NER is not as good as it is in other NLP tasks. In this paper, we propose TENER, a NER architecture adopting adapted Transformer Encoder to model the character-level features and word-level features. By incorporating the direction and relative distance aware attention and the un-scaled attention, we prove the Transformer-like encoder is just as effective for NER as other NLP tasks.

cs.CL↗

arXiv2019

Learning Sparse Sharing Architectures for Multiple Tasks

Most existing deep multi-task learning models are based on parameter sharing, such as hard sharing, hierarchical sharing, and soft sharing. How choosing a suitable sharing mechanism depends on the relations among the tasks, which is not easy since it is difficult to understand the underlying shared factors among these tasks. In this paper, we propose a novel parameter sharing mechanism, named \emph{Sparse Sharing}. Given multiple tasks, our approach automatically finds a sparse sharing structure. We start with an over-parameterized base network, from which each task extracts a subnetwork. The subnetworks of multiple tasks are partially overlapped and trained in parallel. We show that both hard sharing and hierarchical sharing can be formulated as particular instances of the sparse sharing framework. We conduct extensive experiments on three sequence labeling tasks. Compared with single-task models and three typical multi-task learning baselines, our proposed approach achieves consistent improvement while requiring fewer parameters.

cs.CL↗

arXiv2019

Study of $\bar{B}_{u,d,s}^* \to D_{u,d,s}^* V\,(V=D_{d,s}^{-}\,,K^{-}\,,ρ^-)$ weak decays

Motivated by the rapid development of heavy flavor physics experiment, we study the tree-dominated nonleptonic $\bar{B}_{u,d,s}^* \to D_{u,d,s}^*V$~($V=D^{*-},D_s^{*-},K^{*-},ρ^-$) decays within the factorization approach. The relevant transition form factors are calculated by employing the covariant light-front quark model. The helicity amplitudes are calculated and analyzed in detail, and a very clear hierarchy structure $ |H_{-0}| \approx 2|H_{00}| > |H_{0-}|\approx|H_{--}|>|H_{0+}|\approx|H_{++}|$ is presented. The branching fractions are computed and discussed. Numerically, the CKM-favored $\bar{B}^*_q\to D^*_q ρ^{-}$ and $D^*_q D_s^{*-}$ decays have relatively large branching fractions, $\gtrsim {\cal O}(10^{-8})$, and are hopeful to be observed by LHC and Belle-II experiments in the future.

hep-ph↗

arXiv2015

Neutrino Physics with JUNO

The Jiangmen Underground Neutrino Observatory (JUNO), a 20 kton multi-purpose underground liquid scintillator detector, was proposed with the determination of the neutrino mass hierarchy as a primary physics goal. It is also capable of observing neutrinos from terrestrial and extra-terrestrial sources, including supernova burst neutrinos, diffuse supernova neutrino background, geoneutrinos, atmospheric neutrinos, solar neutrinos, as well as exotic searches such as nucleon decays, dark matter, sterile neutrinos, etc. We present the physics motivations and the anticipated performance of the JUNO detector for various proposed measurements. By detecting reactor antineutrinos from two power plants at 53-km distance, JUNO will determine the neutrino mass hierarchy at a 3-4 sigma significance with six years of running. The measurement of antineutrino spectrum will also lead to the precise determination of three out of the six oscillation parameters to an accuracy of better than 1\%. Neutrino burst from a typical core-collapse supernova at 10 kpc would lead to ~5000 inverse-beta-decay events and ~2000 all-flavor neutrino-proton elastic scattering events in JUNO. Detection of DSNB would provide valuable information on the cosmic star-formation rate and the average core-collapsed neutrino energy spectrum. Geo-neutrinos can be detected in JUNO with a rate of ~400 events per year, significantly improving the statistics of existing geoneutrino samples. The JUNO detector is sensitive to several exotic searches, e.g. proton decay via the $p\to K^++\barν$ decay channel. The JUNO detector will provide a unique facility to address many outstanding crucial questions in particle and astrophysics. It holds the great potential for further advancing our quest to understanding the fundamental properties of neutrinos, one of the building blocks of our Universe.

physics.ins-det↗

arXiv2014

A Combined Fit on the Annihilation Corrections in $B_{u,d,s}$ $\to$ $PP$ Decays Within QCDF

Motivated by the possible large annihilation contributions implied by recent CDF and LHCb measurements on nonleptonic annihilation B meson decays, and the refined experimental measurements on hadronic B meson decays, we study the strength of annihilation contributions within QCD factorization (QCDF) in this paper. With the available measurements of two-body B_{u,d,s} -> pi pi, pi K, K K decays, a comprehensive fit on the phenomenological parameters X_A^{i,f} (or rho_A^{i,f} and phi_A^{i,f}) which are used to parameterize the endpoint singularity in annihilation amplitudes is performed with the statistical chi^2 approach. It is found that (1) flavor symmetry breaking effects are hardly to be distinguished between X_{A,s}^i and X_{A,d}^i due to the large experimental errors and theoretical uncertainties, where X_{A,s}^i and X_{A,d}^i are related to the nonfactorization annihilation contributions in B_s and B_{u,d} decays, respectively. So X_{A,s}^i = X_{A,d}^i is a good approximation by now. (2) In principle, parameter X_{A}^f which is related to the factorization annihilation contributions and independent of the initial state can be regarded as the same variable for B_{u,d,s} decays. (3) Numerically, two solutions are found, one is (rho_A^i, phi_A^i) = (2.98^+1.12_-0.86,-105^+34_-24) and (rho_A^f, phi_A^f) = (1.18^+0.20_-0.23,-40^+11_-8), the other is (rho_A^i, phi_A^i) = (2.97^+1.19_-0.90,-105^+32_-24) and (rho_A^f, phi_A^f) = (2.80^+0.25_-0.21,165^+4_-3). Obviously, nonfactorization annihilation parameter X_A^i is generally unequal to factorization annihilation parameter X_A^f, which differ from the traditional treatment. With the fitted parameters, all results for observables of B_{u,d,s} ->pi pi, pi K, K K decays are in good agreement with experimental data.

hep-ph↗

arXiv2014

Spectator Scattering and Annihilation Contributions as a Solution to the $πK$ and $ππ$ Puzzles within QCD Factorization Approach

The large branching ratios for pure annihilation $\bar{B}_s^0$ $\to$ $π^+ π^-$ and $\bar{B}_d^0$ $\to$ $K^+ K^-$ decays reported by CDF and LHCb collaborations recently and the so-called $πK$ and $ππ$ puzzles indicate that spectator scattering and annihilation contributions are important to the penguin-dominated, color-suppressed tree dominated, and pure annihilation nonleptonic $B$ decays. Combining the available experimental data for $B_{u,d}$ ${\to}$ $ππ$, $πK$ and $K \bar{K}$ decays, we do a global fit on the spectator scattering and annihilation parameters $X_H(ρ_H$, $ϕ_H)$, $X_A^i(ρ_A^{i},ϕ_A^{i})$ and $X_A^f(ρ_A^{f},ϕ_A^{f})$, which are used to parameterize the endpoint singularity in amplitudes of spectator scattering, nonfactorizable and factorizable annihilation topologies within the QCD factorization framework, in three scenarios for different purpose. Numerically, in scenario II, we get $(ρ_A^{i},ϕ_A^{i}[^{\circ}])=(2.88^{+1.52}_{-1.30},-103^{+33}_{-40})$ and $(ρ_A^{f},ϕ_A^{f}[^{\circ}])=(1.21^{+0.22}_{-0.25},-40^{+12}_{-8})$ at the $68\%$ confidence level, which are mainly demanded by resolving $πK$ puzzle and confirm the presupposition that $X_A^i\neq X_A^f$. In addition, correspondingly, the $B$-meson wave function parameter $λ_B$ is also fitted to be $0.18^{+0.11}_{-0.08}\, MeV$, which plays an important role for resolving both $πK$ and $ππ$ puzzles. With the fitted parameters, the QCDF results for observables of $B_{u,d}$ $\to$ $ππ$, $πK$ and $K \bar{K}$ decays are in good agreement with experimental measurements. Much more experimental and theoretical efforts are expected to understand the underlying QCD dynamics of spectator scattering and annihilation contributions.

hep-ph↗

arXiv2014

The high-speed after pulse measurement system for PMT

A system employing a desktop FADC has been developed to investigate the features of 8 inches Hamamatsu PMT. The system stands out for its high-speed and informative results as a consequence of adopting fast waveform sampling technology. Recording full waveforms allows us to perform digital signal processing, pulse shape analysis, and precision timing extraction. High precision after pulse time and charge distribution characteristics are presented in this manuscript. Other photomultipliers characteristics, such as dark rate and transit time spread, can also be obtained by exploiting waveform analysis using this system.

physics.ins-det↗