arXiv subjects

Qiushi Huang

Publications and source records attributed to Qiushi Huang.

23 records · Page 2Linked to original sources

arXiv2021

An Encoder-Decoder Based Audio Captioning System With Transfer and Reinforcement Learning

Automated audio captioning aims to use natural language to describe the content of audio data. This paper presents an audio captioning system with an encoder-decoder architecture, where the decoder predicts words based on audio features extracted by the encoder. To improve the proposed system, transfer learning from either an upstream audio-related task or a large in-domain dataset is introduced to mitigate the problem induced by data scarcity. Besides, evaluation metrics are incorporated into the optimization of the model with reinforcement learning, which helps address the problem of ``exposure bias'' induced by ``teacher forcing'' training strategy and the mismatch between the evaluation metrics and the loss function. The resulting system was ranked 3rd in DCASE 2021 Task 6. Ablation studies are carried out to investigate how much each element in the proposed system can contribute to final performance. The results show that the proposed techniques significantly improve the scores of the evaluation metrics, however, reinforcement learning may impact adversely on the quality of the generated captions.

eess.AS↗

arXiv2021

Audio Captioning Transformer

Audio captioning aims to automatically generate a natural language description of an audio clip. Most captioning models follow an encoder-decoder architecture, where the decoder predicts words based on the audio features extracted by the encoder. Convolutional neural networks (CNNs) and recurrent neural networks (RNNs) are often used as the audio encoder. However, CNNs can be limited in modelling temporal relationships among the time frames in an audio signal, while RNNs can be limited in modelling the long-range dependencies among the time frames. In this paper, we propose an Audio Captioning Transformer (ACT), which is a full Transformer network based on an encoder-decoder architecture and is totally convolution-free. The proposed method has a better ability to model the global information within an audio signal as well as capture temporal relationships between audio events. We evaluate our model on AudioCaps, which is the largest audio captioning dataset publicly available. Our model shows competitive performance compared to other state-of-the-art approaches.

eess.AS↗

arXiv2018

Observation of the angular distribution of a x-ray characteristic emission through a periodic multilayer

We present the observation of the angular distribution of a characteristic x-ray emission through a periodic multilayer. The emission coming from the substrate on which the multilayer is deposited is used for this purpose. It is generated upon proton irradiation through the multilayer and detected with an energy sensitive CCD camera. The observed distribution in the low detection angle range presents a clear dip at a position characteristic of the emitting element. Thus, such a device can be envisaged as a spectrometer without mechanical displacement and using various ionizing sources (electrons, x-rays, ions), their incident direction being irrelevant.

physics.ins-det↗

arXiv2018

Observation of the topological edge state in X-ray band

The possibility of obtaining robust edge state of light by mimicking the topological properties of solid state system, have brought a profound impact on optical sciences. With the advent of high-brilliance, accelerator-driven light sources such as storage rings or X-ray lasers, it has become attractive to extend the concept of optical topological manipulation to the X-ray regime. In this paper, we theoretically proposed and experimentally demonstrated the topological edge state at the interface of two photonic crystals having different band-gap topological characteristics for X-ray. Remarkably, this topologically protected edge state is immune to the weak disorder in form of the thickness disorder and strong disorder in form of the positional disorder of layers in the structure, as long as the zero-average-effective-mass condition is satisfied. Our investigation therefore brings the topological characteristics to the X-ray regime, provides new theoretical tools to study X-ray optics and may pave way to exploit some important potential applications, such as the high efficiency band filter in X-ray band.

physics.app-ph↗

arXiv2017

Sequential Prediction of Social Media Popularity with Deep Temporal Context Networks

Prediction of popularity has profound impact for social media, since it offers opportunities to reveal individual preference and public attention from evolutionary social systems. Previous research, although achieves promising results, neglects one distinctive characteristic of social data, i.e., sequentiality. For example, the popularity of online content is generated over time with sequential post streams of social media. To investigate the sequential prediction of popularity, we propose a novel prediction framework called Deep Temporal Context Networks (DTCN) by incorporating both temporal context and temporal attention into account. Our DTCN contains three main components, from embedding, learning to predicting. With a joint embedding network, we obtain a unified deep representation of multi-modal user-post data in a common embedding space. Then, based on the embedded data sequence over time, temporal context learning attempts to recurrently learn two adaptive temporal contexts for sequential popularity. Finally, a novel temporal attention is designed to predict new popularity (the popularity of a new user-post pair) with temporal coherence across multiple time-scales. Experiments on our released image dataset with about 600K Flickr photos demonstrate that DTCN outperforms state-of-the-art deep prediction algorithms, with an average of 21.51% relative performance improvement in the popularity prediction (Spearman Ranking Correlation).

cs.SI↗