arXiv subjects

Mohan Li

Publications and source records attributed to Mohan Li.

34 records · Page 2Linked to original sources

arXiv2025

A Survey on Federated Learning in Human Sensing

Human Sensing, a field that leverages technology to monitor human activities, psycho-physiological states, and interactions with the environment, enhances our understanding of human behavior and drives the development of advanced services that improve overall quality of life. However, its reliance on detailed and often privacy-sensitive data as the basis for its machine learning (ML) models raises significant legal and ethical concerns. The recently proposed ML approach of Federated Learning (FL) promises to alleviate many of these concerns, as it is able to create accurate ML models without sending raw user data to a central server. While FL has demonstrated its usefulness across a variety of areas, such as text prediction and cyber security, its benefits in Human Sensing are under-explored, given the particular challenges in this domain. This survey conducts a comprehensive analysis of the current state-of-the-art studies on FL in Human Sensing, and proposes a taxonomy and an eight-dimensional assessment for FL approaches. Through the eight-dimensional assessment, we then evaluate whether the surveyed studies consider a specific FL-in-Human-Sensing challenge or not. Finally, based on the overall analysis, we discuss open challenges and highlight five research aspects related to FL in Human Sensing that require urgent research attention. Our work provides a comprehensive corpus of FL studies and aims to assist FL practitioners in developing and evaluating solutions that effectively address the real-world complexities of Human Sensing.

cs.LG↗

arXiv2024

WHISMA: A Speech-LLM to Perform Zero-shot Spoken Language Understanding

Speech large language models (speech-LLMs) integrate speech and text-based foundation models to provide a unified framework for handling a wide range of downstream tasks. In this paper, we introduce WHISMA, a speech-LLM tailored for spoken language understanding (SLU) that demonstrates robust performance in various zero-shot settings. WHISMA combines the speech encoder from Whisper with the Llama-3 LLM, and is fine-tuned in a parameter-efficient manner on a comprehensive collection of SLU-related datasets. Our experiments show that WHISMA significantly improves the zero-shot slot filling performance on the SLURP benchmark, achieving a relative gain of 26.6% compared to the current state-of-the-art model. Furthermore, to evaluate WHISMA's generalisation capabilities to unseen domains, we develop a new task-agnostic benchmark named SLU-GLUE. The evaluation results indicate that WHISMA outperforms an existing speech-LLM (Qwen-Audio) with a relative gain of 33.0%.

eess.AS↗

arXiv2024

Prompting Whisper for QA-driven Zero-shot End-to-end Spoken Language Understanding

Zero-shot spoken language understanding (SLU) enables systems to comprehend user utterances in new domains without prior exposure to training data. Recent studies often rely on large language models (LLMs), leading to excessive footprints and complexity. This paper proposes the use of Whisper, a standalone speech processing model, for zero-shot end-to-end (E2E) SLU. To handle unseen semantic labels, SLU tasks are integrated into a question-answering (QA) framework, which prompts the Whisper decoder for semantics deduction. The system is efficiently trained with prefix-tuning, optimising a minimal set of parameters rather than the entire Whisper model. We show that the proposed system achieves a 40.7% absolute gain for slot filling (SLU-F1) on SLURP compared to a recently introduced zero-shot benchmark. Furthermore, it performs comparably to a Whisper-GPT-2 modular system under both in-corpus and cross-corpus evaluation settings, but with a relative 34.8% reduction in model parameters.

eess.AS↗

arXiv2024

DiaLoc: An Iterative Approach to Embodied Dialog Localization

Multimodal learning has advanced the performance for many vision-language tasks. However, most existing works in embodied dialog research focus on navigation and leave the localization task understudied. The few existing dialog-based localization approaches assume the availability of entire dialog prior to localizaiton, which is impractical for deployed dialog-based localization. In this paper, we propose DiaLoc, a new dialog-based localization framework which aligns with a real human operator behavior. Specifically, we produce an iterative refinement of location predictions which can visualize current pose believes after each dialog turn. DiaLoc effectively utilizes the multimodal data for multi-shot localization, where a fusion encoder fuses vision and dialog information iteratively. We achieve state-of-the-art results on embodied dialog-based localization task, in single-shot (+7.08% in Acc5@valUnseen) and multi-shot settings (+10.85% in Acc5@valUnseen). DiaLoc narrows the gap between simulation and real-world applications, opening doors for future research on collaborative localization and navigation.

cs.CV↗

arXiv2024

Economic Forces in Stock Returns

When analyzing the components influencing the stock prices, it is commonly believed that economic activities play an important role. More specifically, asset prices are more sensitive to the systematic economic news that impose a pervasive effect on the whole market. Moreover, the investors will not be rewarded for bearing idiosyncratic risks as such risks are diversifiable. In the paper Economic Forces and the Stock Market 1986, the authors introduced an attribution model to identify the specific systematic economic forces influencing the market. They first defined and examined five classic factors from previous research papers: Industrial Production, Unanticipated Inflation, Change in Expected Inflation, Risk Premia, and The Term Structure. By adding in new factors, the Market Indices, Consumptions and Oil Prices, one by one, they examined the significant contribution of each factor to the stock return. The paper concluded that the stock returns are exposed to the systematic economic news, and they are priced with respect to their risk exposure. Also, the significant factors can be identified by simply adopting their model. Driven by such motivation, we conduct an attribution analysis based on the general framework of their model to further prove the importance of the economic factors and identify the specific identity of significant factors.

econ.GN↗

arXiv2023

Self-regularised Minimum Latency Training for Streaming Transformer-based Speech Recognition

This paper proposes a self-regularised minimum latency training (SR-MLT) method for streaming Transformer-based automatic speech recognition (ASR) systems. In previous works, latency was optimised by truncating the online attention weights based on the hard alignments obtained from conventional ASR models, without taking into account the potential loss of ASR accuracy. On the contrary, here we present a strategy to obtain the alignments as a part of the model training without external supervision. The alignments produced by the proposed method are dynamically regularised on the training data, such that the latency reduction does not result in the loss of ASR accuracy. SR-MLT is applied as a fine-tuning step on the pre-trained Transformer models that are based on either monotonic chunkwise attention (MoChA) or cumulative attention (CA) algorithms for online decoding. ASR experiments on the AIShell-1 and Librispeech datasets show that when applied on a decent pre-trained MoChA or CA baseline model, SR-MLT can effectively reduce the latency with the relative gains ranging from 11.8% to 39.5%. Furthermore, we also demonstrate that under certain accuracy levels, the models trained with SR-MLT can achieve lower latency when compared to those supervised using external hard alignments.

eess.AS↗

arXiv2023

Non-autoregressive End-to-end Approaches for Joint Automatic Speech Recognition and Spoken Language Understanding

This paper presents the use of non-autoregressive (NAR) approaches for joint automatic speech recognition (ASR) and spoken language understanding (SLU) tasks. The proposed NAR systems employ a Conformer encoder that applies connectionist temporal classification (CTC) to transcribe the speech utterance into raw ASR hypotheses, which are further refined with a bidirectional encoder representations from Transformers (BERT)-like decoder. In the meantime, the intent and slot labels of the utterance are predicted simultaneously using the same decoder. Both Mask-CTC and self-conditioned CTC (SC-CTC) approaches are explored for this study. Experiments conducted on the SLURP dataset show that the proposed SC-Mask-CTC NAR system achieves 3.7% and 3.2% absolute gains in SLU metrics and a competitive level of ASR accuracy, when compared to a Conformer-Transformer based autoregressive (AR) model. Additionally, the NAR systems achieve 6x faster decoding speed than the AR baseline.

eess.AS↗

arXiv2022

Multiple-hypothesis RNN-T Loss for Unsupervised Fine-tuning and Self-training of Neural Transducer

This paper proposes a new approach to perform unsupervised fine-tuning and self-training using unlabeled speech data for recurrent neural network (RNN)-Transducer (RNN-T) end-to-end (E2E) automatic speech recognition (ASR) systems. Conventional systems perform fine-tuning/self-training using ASR hypothesis as the targets when using unlabeled audio data and are susceptible to the ASR performance of the base model. Here in order to alleviate the influence of ASR errors while using unlabeled data, we propose a multiple-hypothesis RNN-T loss that incorporates multiple ASR 1-best hypotheses into the loss function. For the fine-tuning task, ASR experiments on Librispeech show that the multiple-hypothesis approach achieves a relative reduction of 14.2% word error rate (WER) when compared to the single-hypothesis approach, on the test_other set. For the self-training task, ASR models are trained using supervised data from Wall Street Journal (WSJ), Aurora-4 along with CHiME-4 real noisy data as unlabeled data. The multiple-hypothesis approach yields a relative reduction of 3.3% WER on the CHiME-4's single-channel real noisy evaluation set when compared with the single-hypothesis approach.

cs.CL↗

arXiv2022

Transformer-based Streaming ASR with Cumulative Attention

In this paper, we propose an online attention mechanism, known as cumulative attention (CA), for streaming Transformer-based automatic speech recognition (ASR). Inspired by monotonic chunkwise attention (MoChA) and head-synchronous decoder-end adaptive computation steps (HS-DACS) algorithms, CA triggers the ASR outputs based on the acoustic information accumulated at each encoding timestep, where the decisions are made using a trainable device, referred to as halting selector. In CA, all the attention heads of the same decoder layer are synchronised to have a unified halting position. This feature effectively alleviates the problem caused by the distinct behaviour of individual heads, which may otherwise give rise to severe latency issues as encountered by MoChA. The ASR experiments conducted on AIShell-1 and Librispeech datasets demonstrate that the proposed CA-based Transformer system can achieve on par or better performance with significant reduction in latency during inference, when compared to other streaming Transformer systems in literature.

eess.AS↗

arXiv2021

Head-synchronous Decoding for Transformer-based Streaming ASR

Online Transformer-based automatic speech recognition (ASR) systems have been extensively studied due to the increasing demand for streaming applications. Recently proposed Decoder-end Adaptive Computation Steps (DACS) algorithm for online Transformer ASR was shown to achieve state-of-the-art performance and outperform other existing methods. However, like any other online approach, the DACS-based attention heads in each of the Transformer decoder layers operate independently (or asynchronously) and lead to diverged attending positions. Since DACS employs a truncation threshold to determine the halting position, some of the attention weights are cut off untimely and might impact the stability and precision of decoding. To overcome these issues, here we propose a head-synchronous (HS) version of the DACS algorithm, where the boundary of attention is jointly detected by all the DACS heads in each decoder layer. ASR experiments on Wall Street Journal (WSJ), AIShell-1 and Librispeech show that the proposed method consistently outperforms vanilla DACS and achieves state-of-the-art performance. We will also demonstrate that HS-DACS has reduced decoding cost when compared to vanilla DACS.

eess.AS↗

arXiv2020

Transformer-based Online Speech Recognition with Decoder-end Adaptive Computation Steps

Transformer-based end-to-end (E2E) automatic speech recognition (ASR) systems have recently gained wide popularity, and are shown to outperform E2E models based on recurrent structures on a number of ASR tasks. However, like other E2E models, Transformer ASR also requires the full input sequence for calculating the attentions on both encoder and decoder, leading to increased latency and posing a challenge for online ASR. The paper proposes Decoder-end Adaptive Computation Steps (DACS) algorithm to address the issue of latency and facilitate online ASR. The proposed algorithm streams the decoding of Transformer ASR by triggering an output after the confidence acquired from the encoder states reaches a certain threshold. Unlike other monotonic attention mechanisms that risk visiting the entire encoder states for each output step, the paper introduces a maximum look-ahead step into the DACS algorithm to prevent from reaching the end of speech too fast. A Chunkwise encoder is adopted in our system to handle real-time speech inputs. The proposed online Transformer ASR system has been evaluated on Wall Street Journal (WSJ) and AIShell-1 datasets, yielding 5.5% word error rate (WER) and 7.1% character error rate (CER) respectively, with only a minor decay in performance when compared to the offline systems.

eess.AS↗

arXiv2019

Automated Attack and Defense Framework for 5G Security on Physical and Logical Layers

The 5th generation (5G) network adopts a great number of revolutionary technologies to fulfill continuously increasing requirements of a variety of applications, including ultra-high bandwidth, ultra-low latency, ultra-massive device access, ultra-reliability, and so on. Correspondingly, traditional security focuses on the core network, and the logical (non-physical) layer is no longer suitable for the 5G network. 5G security presents a tendency to extend from the network center to the network edge and from the logical layer to the physical layer. The physical layer security is also an essential part of 5G security. However, the security of each layer in 5G is mostly studied separately, which causes a lack of comprehensive analysis for security issues across layers. Meanwhile, potential security threats are lack of automated solutions. This article explores the 5G security by combining the physical layer and the logical layer from the perspective of automated attack and defense, and dedicate to provide automated solution framework for 5G security.

cs.NI↗

arXiv2018

End-to-end Speech Recognition with Adaptive Computation Steps

In this paper, we present Adaptive Computation Steps (ACS) algo-rithm, which enables end-to-end speech recognition models to dy-namically decide how many frames should be processed to predict a linguistic output. The model that applies ACS algorithm follows the encoder-decoder framework, while unlike the attention-based mod-els, it produces alignments independently at the encoder side using the correlation between adjacent frames. Thus, predictions can be made as soon as sufficient acoustic information is received, which makes the model applicable in online cases. Besides, a small change is made to the decoding stage of the encoder-decoder framework, which allows the prediction to exploit bidirectional contexts. We verify the ACS algorithm on a Mandarin speech corpus AIShell-1, and it achieves a 31.2% CER in the online occasion, compared to the 32.4% CER of the attention-based model. To fully demonstrate the advantage of ACS algorithm, offline experiments are conducted, in which our ACS model achieves an 18.7% CER, outperforming the attention-based counterpart with the CER of 22.0%.

eess.AS↗

arXiv2017

Assay of low-background stainless steel by smelting for the neutrino experiment at Jinping

To ensure compliance with the experimental requirement for ultra-low background, in this study the radioactivity of stainless steels manufactured by smelting is thoroughly investigated. Raw materials, stage samples, and commercial samples are investigated by glow discharge mass spectrometry (GDMS) and/or with high-purity germanium detectors (HPGe) at both the ground level and/or the China Jinping Underground Laboratory. Custom-made stainless steel samples are found to have radioactivity levels comparable to those in other low-background experiments. The comprehensive results regarding the radioactivity level in materials to be used in the proposed Jinping Neutrino Experiment are reported.

physics.ins-det↗

arXiv2016

Letter of Intent: Jinping Neutrino Experiment

Jinping Neutrino Experiment (Jinping) is proposed to significantly improve measurements on solar neutrinos and geoneutrinos in China Jinping Laboratory - a lab with a number of unparalleled features, thickest overburden, lowest reactor neutrino background, etc., which identify it as the world-best low-energy neutrino laboratory. The proposed experiment will have target mass of 4 kilotons of liquid scintillator or water-based liquid scintillator, with a fiducial mass of 2 kilotons for neutrino-electron scattering events and 3 kilotons for inverse-beta interaction events. A number of initial sensitivities studies have been carried out, including on the transition phase for the solar neutrinos oscillation from the vacuum to the matter effect, the discovery of solar neutrinos from the carbon-nitrogen-oxygen (CNO) cycle, the resolution of the high and low metallicity hypotheses, and the unambiguous separation on U and Th cascade decays from the dominant crustal anti-electron neutrinos in China.

physics.ins-det↗

arXiv2015

Separation of Scintillation and Cherenkov Lights in Linear Alkyl Benzene

To separate scintillation and Cherenkov lights in water-based liquid scintillator detectors is a desired feature for future neutrino and proton decay researches. Linear alkyl benzene (LAB) is one important ingredient of a water-based liquid scintillator being developed. In this paper we observed a good separation of scintillation and Cherenkov lights in an LAB sample. The rising and decay times of the scintillation light of the LAB were measured to be $(7.7\pm3.0)\ \rm{ns}$ and $(36.6\pm2.4)\ \rm{ns}$, respectively, while the full width [-3$σ$, 3$σ$] of the Cherenkov light was 12 ns dominated by the time resolution of our photomultiplier tubes. The light yield of the scintillation was measured to be $(1.01\pm0.12)\times10^3\ \rm{photons}/\rm{MeV}$.

physics.ins-det↗