arXiv subjects

Xiaohui Hu

Publications and source records attributed to Xiaohui Hu.

26 records · Page 2Linked to original sources

arXiv2021

Cross Modification Attention Based Deliberation Model for Image Captioning

The conventional encoder-decoder framework for image captioning generally adopts a single-pass decoding process, which predicts the target descriptive sentence word by word in temporal order. Despite the great success of this framework, it still suffers from two serious disadvantages. Firstly, it is unable to correct the mistakes in the predicted words, which may mislead the subsequent prediction and result in error accumulation problem. Secondly, such a framework can only leverage the already generated words but not the possible future words, and thus lacks the ability of global planning on linguistic information. To overcome these limitations, we explore a universal two-pass decoding framework, where a single-pass decoding based model serving as the Drafting Model first generates a draft caption according to an input image, and a Deliberation Model then performs the polishing process to refine the draft caption to a better image description. Furthermore, inspired from the complementarity between different modalities, we propose a novel Cross Modification Attention (CMA) module to enhance the semantic expression of the image features and filter out error information from the draft captions. We integrate CMA with the decoder of our Deliberation Model and name it as Cross Modification Attention based Deliberation Model (CMA-DM). We train our proposed framework by jointly optimizing all trainable components from scratch with a trade-off coefficient. Experiments on MS COCO dataset demonstrate that our approach obtains significant improvements over single-pass decoding baselines and achieves competitive performances compared with other state-of-the-art two-pass decoding based methods.

cs.CV↗

arXiv2021

MvSR-NAT: Multi-view Subset Regularization for Non-Autoregressive Machine Translation

Conditional masked language models (CMLM) have shown impressive progress in non-autoregressive machine translation (NAT). They learn the conditional translation model by predicting the random masked subset in the target sentence. Based on the CMLM framework, we introduce Multi-view Subset Regularization (MvSR), a novel regularization method to improve the performance of the NAT model. Specifically, MvSR consists of two parts: (1) \textit{shared mask consistency}: we forward the same target with different mask strategies, and encourage the predictions of shared mask positions to be consistent with each other. (2) \textit{model consistency}, we maintain an exponential moving average of the model weights, and enforce the predictions to be consistent between the average model and the online model. Without changing the CMLM-based architecture, our approach achieves remarkable performance on three public benchmarks with 0.36-1.14 BLEU gains over previous NAT models. Moreover, compared with the stronger Transformer baseline, we reduce the gap to 0.01-0.44 BLEU scores on small datasets (WMT16 RO$\leftrightarrow$EN and IWSLT DE$\rightarrow$EN).

cs.CL↗

arXiv2021

PiSLTRc: Position-informed Sign Language Transformer with Content-aware Convolution

Since the superiority of Transformer in learning long-term dependency, the sign language Transformer model achieves remarkable progress in Sign Language Recognition (SLR) and Translation (SLT). However, there are several issues with the Transformer that prevent it from better sign language understanding. The first issue is that the self-attention mechanism learns sign video representation in a frame-wise manner, neglecting the temporal semantic structure of sign gestures. Secondly, the attention mechanism with absolute position encoding is direction and distance unaware, thus limiting its ability. To address these issues, we propose a new model architecture, namely PiSLTRc, with two distinctive characteristics: (i) content-aware and position-aware convolution layers. Specifically, we explicitly select relevant features using a novel content-aware neighborhood gathering method. Then we aggregate these features with position-informed temporal convolution layers, thus generating robust neighborhood-enhanced sign representation. (ii) injecting the relative position information to the attention mechanism in the encoder, decoder, and even encoder-decoder cross attention. Compared with the vanilla Transformer model, our model performs consistently better on three large-scale sign language benchmarks: PHOENIX-2014, PHOENIX-2014-T and CSL. Furthermore, extensive experiments demonstrate that the proposed method achieves state-of-the-art performance on translation quality with $+1.6$ BLEU improvements.

cs.CV↗

arXiv2021

Multi-Scale Local-Temporal Similarity Fusion for Continuous Sign Language Recognition

Continuous sign language recognition (cSLR) is a public significant task that transcribes a sign language video into an ordered gloss sequence. It is important to capture the fine-grained gloss-level details, since there is no explicit alignment between sign video frames and the corresponding glosses. Among the past works, one promising way is to adopt a one-dimensional convolutional network (1D-CNN) to temporally fuse the sequential frames. However, CNNs are agnostic to similarity or dissimilarity, and thus are unable to capture local consistent semantics within temporally neighboring frames. To address the issue, we propose to adaptively fuse local features via temporal similarity for this task. Specifically, we devise a Multi-scale Local-Temporal Similarity Fusion Network (mLTSF-Net) as follows: 1) In terms of a specific video frame, we firstly select its similar neighbours with multi-scale receptive regions to accommodate different lengths of glosses. 2) To ensure temporal consistency, we then use position-aware convolution to temporally convolve each scale of selected frames. 3) To obtain a local-temporally enhanced frame-wise representation, we finally fuse the results of different scales using a content-dependent aggregator. We train our model in an end-to-end fashion, and the experimental results on RWTH-PHOENIX-Weather 2014 datasets (RWTH) demonstrate that our model achieves competitive performance compared with several state-of-the-art models.

cs.CV↗

arXiv2020

Infusing Sequential Information into Conditional Masked Translation Model with Self-Review Mechanism

Non-autoregressive models generate target words in a parallel way, which achieve a faster decoding speed but at the sacrifice of translation accuracy. To remedy a flawed translation by non-autoregressive models, a promising approach is to train a conditional masked translation model (CMTM), and refine the generated results within several iterations. Unfortunately, such approach hardly considers the \textit{sequential dependency} among target words, which inevitably results in a translation degradation. Hence, instead of solely training a Transformer-based CMTM, we propose a Self-Review Mechanism to infuse sequential information into it. Concretely, we insert a left-to-right mask to the same decoder of CMTM, and then induce it to autoregressively review whether each generated word from CMTM is supposed to be replaced or kept. The experimental results (WMT14 En$\leftrightarrow$De and WMT16 En$\leftrightarrow$Ro) demonstrate that our model uses dramatically less training computations than the typical CMTM, as well as outperforms several state-of-the-art non-autoregressive models by over 1 BLEU. Through knowledge distillation, our model even surpasses a typical left-to-right Transformer model, while significantly speeding up decoding.

cs.CL↗

arXiv2016

Study of nonleptonic $B_{q}^{\ast}$ ${\to}$ $D_{q}V$ and $P_{q} D^*$ weak decays

Motivated by the powerful capability of measurement for the $b$-flavored hadron rare decays at LHC and SuperKEKB/Belle-II, the nonleptonic $\bar{B}^{\ast}$ ${\to}$ $D\bar{D}^{\ast}$, $D{ρ^-}$, $DK^{\ast-}$, $πD^{\ast}$ and $KD^{\ast}$ weak decays are studied in detail. With the amplitudes calculated with factorization approach and the form factors of $B^{\ast}$ transition into pseudoscalar meson evaluated with the BSW model, branching fractions and polarization fractions are firstly presented. Numerically, the CKM-favored $\bar{B}_{q}^{\ast}$ ${\to}$ $D_{q}D_{s}^{{\ast}-}$ and $D_{q}ρ^{-}$ decays have large branching fractions, $\sim$ $10^{-8}$, which should be sought for with priority and firstly observed by LHC and Belle-II experiments. The $\bar{B}^{\ast}_q$ ${\to}$ $D_qK^{\ast}$ and $D_qρ$ decays are dominated by the longitudinal polarization states. While, the parallel polarization fractions of $\bar{B}^{\ast}_q$ ${\to}$ $D_q\bar{D}^{\ast}$ decays are comparable with the longitudinal ones, numerically, $f_{\parallel}$ $+$ $f_{L}$ ${\simeq}$ 95\% and $f_{L}:f_{\parallel}$ $\simeq$ $5:4$. Some comparisons between $\bar{B}^{*0}_q$ $\to$ $D_q V$ and their corresponding $\bar{B}^{0}_q$ $\to$ $D^*_q V$ decays are performed, and the relation $ f_{L,\parallel}(\bar{B}^{\ast 0}\to D V)\simeq f_{L,\parallel}(\bar{B}^0\to D^{\ast +} V^-) $ is presented. Besides, with the implication of $SU(3)$ flavor symmetry, some useful ratios $ R_{\rm du}$ and $ R_{\rm ds}$ are discussed in detail, and suggested to be verified experimentally.

hep-ph↗

arXiv2015

Probing Spectator Scattering and Annihilation Corrections in $B_{s}$ $\to$ $PV$ Decays

Motivated by the recent LHCb measurements on $\bar{B}_{s}$ $\to$ $π^{-}K^{*+}$ and $\bar{B}_{s}$ $\to$ $K^{\pm}K^{*\mp}$ decay modes, we revisit the $B_{s}$ $\to$ $PV$ decays within QCD factorization framework. The effects of hard-spectator scattering and annihilation corrections are studied in detail. After performing a $χ^2$-fit on the end-point parameters $X_A^{i,f}$ ($ρ_A^{i,f}$, $ϕ_A^{i,f}$) and $X_H$ ($ρ_H$, $ϕ_H$) with available data, it is found that although some possible mismatches exist, the universalities of $X_A^{i,f}$ and $X_H$ in $B_s$ and $B_{u,d}$ systems are still allowed within theoretical uncertainties and experimental errors. With the end-point parameters gotten from $B_{u,d}$ $\to$ $PV$ decays, the numerical results and detailed analyses for the observables of $\bar{B}_{s}$ ${\to}$ $πK^{\ast}$, $ρK$, $πρ$, $πϕ$ and $Kϕ$ decay modes are presented. In addition, we have identified a few useful observables, especially the ones of $\bar{B}_{s}$ $\to$ $π^{0}ϕ$ decay for instance, for probing hard-spectator scattering and annihilation contributions.

hep-ph↗

arXiv2015

Constraints on hard spectator scattering and annihilation corrections in $B_{u,d}$ ${\to}$ $PV$ decays within QCD factorization

In this paper, we investigate the contributions of hard spectator scattering and annihilation in $B\to PV$ decays within the QCD factorization framework. With available experimental data on $B\to πK^{\ast}$, $ρK$, $πρ$ and $Kϕ$ decays, comprehensive $χ^2$ analyses of the parameters $X_{A,H}^{i,f}(ρ_{A,H}^{i,f},ϕ_{A,H}^{i,f})$ are performed, where $X_A^f$ ($X_A^i$) and $X_H$ are used to parameterize the endpoint divergences of the (non)factorizable annihilation and hard spectator scattering amplitudes, respectively. Based on $χ^2$ analyses, it is observed that (1) The topology-dependent parameterization scheme is feasible for $B\to PV$ decays; (2) At the current accuracy of experimental measurements and theoretical evaluations, $X_H=X_A^i$ is allowed by $B\to PV$ decays, but $X_{H}\neq X_A^f$ at $68%$ C. L.; (3) With the simplification $X_H=X_A^i$, parameters $X_A^f$ and $X_A^i$ should be treated individually. The above-described findings are very similar to those obtained from $B\to PP$ decays. Numerically, for $B\to PV$ decays, we obtain $(ρ_{A,H}^i,ϕ_{A,H}^i[^{\circ}]) =(2.87^{+0.66}_{-1.95}, -145^{+14}_{-21})$ and $(ρ_A^f,ϕ_A^f[^{\circ}]) = (0.91^{+0.12}_{-0.13}, -37^{+10}_{-9})$ at $68%$ C. L.. With the best-fit values, most of the theoretical results are in good agreement with the experimental data within errors. However, significant corrections to the color-suppressed tree amplitude $α_2$ related to a large $ρ_H$ result in the wrong sign for $A^{dir}_{CP}(B^- \to π^0 K^{{\ast}-})$ compared with the most recent BABAR data, which presents a new obstacle in solving "$ππ$" and "$πK$" puzzles through $α_2$. A crosscheck with measurements at Belle (or Belle II) and LHCb, which offer higher precision, is urgently expected to confirm or refute such possible mismatch.

hep-ph↗