arXiv subjects

Sheng Liang

Publications and source records attributed to Sheng Liang.

21 records · Page 2Linked to original sources

arXiv2022

Modular and Parameter-Efficient Multimodal Fusion with Prompting

Recent research has made impressive progress in large-scale multimodal pre-training. In the context of the rapid growth of model size, it is necessary to seek efficient and flexible methods other than finetuning. In this paper, we propose to use prompt vectors to align the modalities. Our method achieves comparable performance to several other multimodal fusion methods in low-resource settings. We further show that our method is modular and parameter-efficient for processing tasks involving two or more data modalities.

cs.CL↗

arXiv2021

Locating Language-Specific Information in Contextualized Embeddings

Multilingual pretrained language models (MPLMs) exhibit multilinguality and are well suited for transfer across languages. Most MPLMs are trained in an unsupervised fashion and the relationship between their objective and multilinguality is unclear. More specifically, the question whether MPLM representations are language-agnostic or they simply interleave well with learned task prediction heads arises. In this work, we locate language-specific information in MPLMs and identify its dimensionality and the layers where this information occurs. We show that language-specific information is scattered across many dimensions, which can be projected into a linear subspace. Our study contributes to a better understanding of MPLM representations, going beyond treating them as unanalyzable blobs of information.

cs.CL↗

arXiv2010

Inverse-Gaussian-Apodized Fiber Bragg Grating for Dual Wavelength Lasing

A fiber Bragg grating (FBG) with an inverse-Gaussian apodization function is proposed and fabricated. It is shown that such a FBG possesses easily controllable dual-wavelength narrow transmission peaks. Incorporating such a FBG filter in a fiber laser with a linear cavity, stable dual-wavelength emission with 0.146 nm wavelength spacing is obtained. It provides a simple and low cost approach of achieving the dual-wavelength fiber laser operation.

physics.atom-ph↗