Searcharxiv⌕ Search

arXiv subjects

Kai Li

Publications and source records attributed to Kai Li.

At least 235 records · Page 13Linked to original sources
arXiv2024

Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving

Machine learning (ML) inference platforms are tasked with balancing two competing goals: ensuring high throughput given many requests, and delivering low-latency responses to support interactive applications. Unfortunately, existing platform knobs (e.g., batch sizes) fail to ease this fundamental tension, and instead only enable users to harshly trade off one property for the other. This paper explores an alternate strategy to taming throughput-latency tradeoffs by changing the granularity at which inference is performed. We present Apparate, a system that automatically applies and manages early exits (EEs) in ML models, whereby certain inputs can exit with results at intermediate layers. To cope with the time-varying overhead and accuracy challenges that EEs bring, Apparate repurposes exits to provide continual feedback that powers several novel runtime monitoring and adaptation strategies. Apparate lowers median response latencies by 40.5--91.5% and 10.0--24.2% for diverse CV and NLP classification workloads, and median time-per-token latencies by 22.6--77.9% for generative scenarios, without affecting throughputs or violating tight accuracy constraints.

cs.DC↗
arXiv2024

Learning to Localize Actions in Instructional Videos with LLM-Based Multi-Pathway Text-Video Alignment

Learning to localize temporal boundaries of procedure steps in instructional videos is challenging due to the limited availability of annotated large-scale training videos. Recent works focus on learning the cross-modal alignment between video segments and ASR-transcripted narration texts through contrastive learning. However, these methods fail to account for the alignment noise, i.e., irrelevant narrations to the instructional task in videos and unreliable timestamps in narrations. To address these challenges, this work proposes a novel training framework. Motivated by the strong capabilities of Large Language Models (LLMs) in procedure understanding and text summarization, we first apply an LLM to filter out task-irrelevant information and summarize task-related procedure steps (LLM-steps) from narrations. To further generate reliable pseudo-matching between the LLM-steps and the video for training, we propose the Multi-Pathway Text-Video Alignment (MPTVA) strategy. The key idea is to measure alignment between LLM-steps and videos via multiple pathways, including: (1) step-narration-video alignment using narration timestamps, (2) direct step-to-video alignment based on their long-term semantic similarity, and (3) direct step-to-video alignment focusing on short-term fine-grained semantic similarity learned from general video domains. The results from different pathways are fused to generate reliable pseudo step-video matching. We conducted extensive experiments across various tasks and problem settings to evaluate our proposed method. Our approach surpasses state-of-the-art methods in three downstream tasks: procedure step grounding, step localization, and narration grounding by 5.9\%, 3.1\%, and 2.8\%.

cs.CV↗
arXiv2024

Photometric and Spectroscopic analysis of eight totally eclipsing contact binaries with small mass ratios

This paper selected eight totally eclipsing contact binaries for photometric and spectroscopic studies, spectral data were analyzed by ULySS, and photometric data were analyzed using PHOEBE through MCMC sampling. We used two methods to calculate the initial values for running MCMC: one method is a new approach proposed by ourselves to model light curves without spots, while the other method is the genetic algorithm (GA) which can determine physical parameters with spot. Due to the results, these eight targets are all small mass ratio contact binary stars with a mass ratio below 0.25. There are four systems exhibiting O'Connell effect. By adding a dark spot on the primary component, the ideal fitting can be obtained. Meanwhile, it was found that two systems are shallow contact binaries, while the remaining six are moderate contact binaries. An O-C analysis of the eight eclipsing binary stars revealed that seven of them exhibit long-term changes. Four of them display a long-term decreasing trend, while the other three show a long-term increasing trend, and two targets exhibit periodic variations. The decrease in period may be caused by the transfer of matter from the more massive component to the less massive component, while the increase in period may be caused by the transfer of matter from the less massive component to the more massive component. The absolute physical parameters, orbital angular momentum, initial masses, and ages of these eight systems were calculated. Additionally, their mass-luminosity and mass-radius distributions were analyzed.

astro-ph.SR↗
arXiv2024

SafeEar: Content Privacy-Preserving Audio Deepfake Detection

Text-to-Speech (TTS) and Voice Conversion (VC) models have exhibited remarkable performance in generating realistic and natural audio. However, their dark side, audio deepfake poses a significant threat to both society and individuals. Existing countermeasures largely focus on determining the genuineness of speech based on complete original audio recordings, which however often contain private content. This oversight may refrain deepfake detection from many applications, particularly in scenarios involving sensitive information like business secrets. In this paper, we propose SafeEar, a novel framework that aims to detect deepfake audios without relying on accessing the speech content within. Our key idea is to devise a neural audio codec into a novel decoupling model that well separates the semantic and acoustic information from audio samples, and only use the acoustic information (e.g., prosody and timbre) for deepfake detection. In this way, no semantic content will be exposed to the detector. To overcome the challenge of identifying diverse deepfake audio without semantic clues, we enhance our deepfake detector with real-world codec augmentation. Extensive experiments conducted on four benchmark datasets demonstrate SafeEar's effectiveness in detecting various deepfake techniques with an equal error rate (EER) down to 2.02%. Simultaneously, it shields five-language speech content from being deciphered by both machine and human auditory analysis, demonstrated by word error rates (WERs) all above 93.93% and our user study. Furthermore, our benchmark constructed for anti-deepfake and anti-content recovery evaluation helps provide a basis for future research in the realms of audio privacy preservation and deepfake detection.

cs.CR↗
arXiv2024

SPMamba: State-space model is all you need in speech separation

Existing CNN-based speech separation models face local receptive field limitations and cannot effectively capture long time dependencies. Although LSTM and Transformer-based speech separation models can avoid this problem, their high complexity makes them face the challenge of computational resources and inference efficiency when dealing with long audio. To address this challenge, we introduce an innovative speech separation method called SPMamba. This model builds upon the robust TF-GridNet architecture, replacing its traditional BLSTM modules with bidirectional Mamba modules. These modules effectively model the spatiotemporal relationships between the time and frequency dimensions, allowing SPMamba to capture long-range dependencies with linear computational complexity. Specifically, the bidirectional processing within the Mamba modules enables the model to utilize both past and future contextual information, thereby enhancing separation performance. Extensive experiments conducted on public datasets, including WSJ0-2Mix, WHAM!, and Libri2Mix, as well as the newly constructed Echo2Mix dataset, demonstrated that SPMamba significantly outperformed existing state-of-the-art models, achieving superior results while also reducing computational complexity. These findings highlighted the effectiveness of SPMamba in tackling the intricate challenges of speech separation in complex environments.

cs.SD↗
arXiv2024

Machine Anomalous Sound Detection Using Spectral-temporal Modulation Representations Derived from Machine-specific Filterbanks

Early detection of factory machinery malfunctions is crucial in industrial applications. In machine anomalous sound detection (ASD), different machines exhibit unique vibration-frequency ranges based on their physical properties. Meanwhile, the human auditory system is adept at tracking both temporal and spectral dynamics of machine sounds. Consequently, integrating the computational auditory models of the human auditory system with machine-specific properties can be an effective approach to machine ASD. We first quantified the frequency importances of four types of machines using the Fisher ratio (F-ratio). The quantified frequency importances were then used to design machine-specific non-uniform filterbanks (NUFBs), which extract the log non-uniform spectrum (LNS) feature. The designed NUFBs have a narrower bandwidth and higher filter distribution density in frequency regions with relatively high F-ratios. Finally, spectral and temporal modulation representations derived from the LNS feature were proposed. These proposed LNS feature and modulation representations are input into an autoencoder neural-network-based detector for ASD. The quantification results from the training set of the Malfunctioning Industrial Machine Investigation and Inspection dataset with a signal-to-noise (SNR) of 6 dB reveal that the distinguishing information between normal and anomalous sounds of different machines is encoded non-uniformly in the frequency domain. By highlighting these important frequency regions using NUFBs, the LNS feature can significantly enhance performance using the metric of AUC (area under the receiver operating characteristic curve) under various SNR conditions. Furthermore, modulation representations can further improve performance. Specifically, temporal modulation is effective for fans, pumps, and sliders, while spectral modulation is particularly effective for valves.

cs.SD↗
arXiv2024

Capillary-driven migration of droplets on conical fibers

A droplet placed on a hydrophilic conical fiber tends to move toward the end of larger radii due to capillary action. Experimental investigations are performed to explore the dynamics of droplets with varying viscosities and volumes on different fibers at the microscale. Droplets are found to accelerate initially and subsequently decelerate during migration. A dynamic model is developed to capture dynamics of the droplet migration, addressing the limitations of previous equilibrium-based scaling laws. Both experimental results and theoretical predictions indicate that droplets on more divergent fibers experience a longer acceleration phase. Additionally, gravitational effects are pronounced on fibers with small cone angles, exerting a substantial influence on droplet migration even below the capillary scale. Moreover, droplets move more slowly on dry fibers compared to those prewetted with the same liquid, primarily attributed to the increased friction. The experiments reveal the formation of a residual liquid film after droplet migration on dry fibers, leading to considerable volume loss in the droplets. To encompass the intricacies of migration on dry fibers, the model is refined to incorporate a higher friction coefficient and variable droplet volumes, providing a more comprehensive depiction of the underlying physics.

physics.flu-dyn↗
arXiv2024

Enhancing Multi-Stream Beamforming Through CQIs For 5G NR FDD Massive MIMO Communications: A Tuning-Free Scheme

In the fifth-generation new radio (5G NR) frequency division duplex (FDD) massive multiple-input and multiple-output (MIMO) systems, downlink beamforming relies on the acquisition of downlink channel state information (CSI). Codebook based limited feedback schemes have been proposed and widely used in practice to recover the downlink CSI with low communication overhead. In such schemes, the performance of downlink beamforming is determined by the codebook design and the codebook indicator feedback. However, limited by the quantization quality of the codebook, directly utilizing the codeword indicated by the feedback as the beamforming vector cannot achieve high performance. Therefore, other feedback values, such as channel qualification indicator (CQI), should be considered to enhance beamforming. In this paper, we present the relation between CQI and the optimal beamforming vectors, based on which an empirical Bayes based intelligent tuning-free algorithm is devised to learn the optimal beamforming vector and the associated regularization parameter. The proposed algorithm can handle different communication scenarios of MIMO systems, including single stream and multiple streams data transmission scenarios. Numerical results have shown the excellent performance of the proposed algorithm in terms of both beamforming vector acquisition and regularization parameter learning.

eess.SP↗
arXiv2024

Dynamical Transition due to Feedback-induced Skin Effect

The traditional dynamical phase transition refers to the appearance of singularities in an observable with respect to a control parameter for a late-time state or singularities in the rate function of the Loschmidt echo with respect to time. Here, we study the many-body dynamics in a continuously monitored free fermion system with conditional feedback under open boundary conditions. We surprisingly find a novel dynamical transition from a logarithmic scaling of the entanglement entropy to an area-law scaling as time evolves. The transition, which is noticeably different from the conventional dynamical phase transition, arises from the competition between the bulk dynamics and boundary skin effects. In addition, we find that while quasidisorder or disorder cannot drive a transition for the steady state, a transition occurs for the maximum entanglement entropy during the time evolution, which agrees well with the entanglement transition for the steady state of the dynamics under periodic boundary conditions.

quant-ph↗
arXiv2024

High Performance MoS2 Phototransistors Photogated by PN Junction

Photodetectors based on two-dimensional (2D) atomically thin semiconductors suffer from low light absorption, limiting their potential for practical applications. In this work, we demonstrate a high-performance MoS2 phototransistors by integrating few-layer MoS2 on a PN junction formed in a silicon (Si) substrate. The photovoltage created in the PN junction under light illumination electrically gates the MoS2 channel, creating a strong photoresponse in MoS2. We present an analytical model for the photoresponse of our device and show that it is in good agreement with measured experimental photocurrent in MoS2 and photovoltage in the Si PN junction. This device structure separates light absorption and electrical response functions, which provides us an opportunity to design new types of photodetectors. For example, incorporating ferroelectric materials into the gate structure can produce a negative capacitance that boosts gate voltage, enabling low power, high sensitivity phototransistor; this, combined with separating light absorption and electrical functions, enables advanced high-performance photodetectors.

cond-mat.mes-hall↗
arXiv2024

Improved Electrical Conductivity of Copper and Nitrogen Functionalized Carbon Nanotubes

In this work, we investigate the electrical conductivity of carbon nanotubes (CNTs), with a particular focus on the effects of doping. Using a first-principles approach, we study the electronic structure, phonon dispersion, and electron-phonon scattering to understand the finite-temperature electrical transport properties in CNTs. Our study covers both prototypical metallic and semiconducting CNTs, with special emphasis on the influence of typical defects such as vacancies and the incorporation of copper or nitrogen, such as pyridinic N, pyrrolic N, graphitic N, and oxidized N. Our theoretical study shows significant improvements in the electrical conduction properties of copper-CNT composites, especially when semiconducting CNTs are functionalized with nitrogen. Doping is found to cause significant changes in the electronic density of states near the Fermi level, which affects the electrical conductivity. Calculations show that certain types of functional groups, such as N-pyrrolic, result in a ~30-fold increase in the conductivity of semiconducting CNTs compared to Cu-incorporated CNTs alone. For metallic CNTs, the conductivity is in agreement with existing experimental data, and our prediction of significant increases in conductivity with N-pyrrolic functional group is consistent with recent experimental results, demonstrating the effectiveness of doping in modifying conductivity. Our study provides valuable insight into the electronic properties of doped CNTs and contributes to the development of ultra-high conductivity CNT composites.

cond-mat.mtrl-sci↗
arXiv2024

Enhanced Equilibria-Solving via Private Information Pre-Branch Structure in Adversarial Team Games

In ex ante coordinated adversarial team games (ATGs), a team competes against an adversary, and the team members are only allowed to coordinate their strategies before the game starts. The team-maxmin equilibrium with correlation (TMECor) is a suitable solution concept for ATGs. One class of TMECor-solving methods transforms the problem into solving NE in two-player zero-sum games, leveraging well-established tools for the latter. However, existing methods are fundamentally action-based, resulting in poor generalizability and low solving efficiency due to the exponential growth in the size of the transformed game. To address the above issues, we propose an efficient game transformation method based on private information, where all team members are represented by a single coordinator. We designed a structure called private information pre-branch, which makes decisions considering all possible private information from teammates. We prove that the size of the game transformed by our method is exponentially reduced compared to the current state-of-the-art. Moreover, we demonstrate equilibria equivalence. Experimentally, our method achieves a significant speedup of 182.89$\times$ to 694.44$\times$ in scenarios where the current state-of-the-art method can work, such as small-scale Kuhn poker and Leduc poker. Furthermore, our method is applicable to larger games and those with dynamically changing private information, such as Goofspiel.

cs.GT↗
arXiv2024

Performance testing of a novel short axis photomultiplier tube for the HUNT project

Photomultiplier tubes (PMTs) with large-area cathodes are increasingly being used in cosmic-ray experiments to enhance detection efficiency. The optical modules (OMs) of the High-Energy Underwater Neutrino Telescope (HUNT) have employed a brand new N6205 20-inch microchannel plate photomultiplier tube (MCP-PMT) developed by the North Night Vision Science & Technology (Nanjing) Research Institute Co. Ltd. (NNVT). In order to make the 20-inch PMT fit into the 23-inch diameter pressure-resistant glass sphere, NNVT improved the internal structure of PMT and shortened the height of PMT by more than 10~cm. The first batch of these PMTs has been delivered for preliminary research work. This paper describes a specific PMT testing platform built for the first batch of 15 MCP-PMTs, and some performance parameters of PMT, such as peak-to-valley ratio, TTS and nonliniearity, are measured. The measurement results show that the new PMT still has good performance and can meet the requirements of HUNT project.

hep-ex↗
arXiv2024

A Course Shared Task on Evaluating LLM Output for Clinical Questions

This paper presents a shared task that we organized at the Foundations of Language Technology (FoLT) course in 2023/2024 at the Technical University of Darmstadt, which focuses on evaluating the output of Large Language Models (LLMs) in generating harmful answers to health-related clinical questions. We describe the task design considerations and report the feedback we received from the students. We expect the task and the findings reported in this paper to be relevant for instructors teaching natural language processing (NLP) and designing course assignments.

cs.CL↗
arXiv2024

Rethinking Video-Text Understanding: Retrieval from Counterfactually Augmented Data

Recent video-text foundation models have demonstrated strong performance on a wide variety of downstream video understanding tasks. Can these video-text models genuinely understand the contents of natural videos? Standard video-text evaluations could be misleading as many questions can be inferred merely from the objects and contexts in a single frame or biases inherent in the datasets. In this paper, we aim to better assess the capabilities of current video-text models and understand their limitations. We propose a novel evaluation task for video-text understanding, namely retrieval from counterfactually augmented data (RCAD), and a new Feint6K dataset. To succeed on our new evaluation task, models must derive a comprehensive understanding of the video from cross-frame reasoning. Analyses show that previous video-text foundation models can be easily fooled by counterfactually augmented data and are far behind human-level performance. In order to narrow the gap between video-text models and human performance on RCAD, we identify a key limitation of current contrastive approaches on video-text data and introduce LLM-teacher, a more effective approach to learn action semantics by leveraging knowledge obtained from a pretrained large language model. Experiments and analyses show that our approach successfully learn more discriminative action embeddings and improves results on Feint6K when applied to multiple video-text models. Our Feint6K dataset and project page is available at https://feint6k.github.io.

cs.CV↗
arXiv2024

Fusion Flow-enhanced Graph Pooling Residual Networks for Unmanned Aerial Vehicles Surveillance in Day and Night Dual Visions

Recognizing unauthorized Unmanned Aerial Vehicles (UAVs) within designated no-fly zones throughout the day and night is of paramount importance, where the unauthorized UAVs pose a substantial threat to both civil and military aviation safety. However, recognizing UAVs day and night with dual-vision cameras is nontrivial, since red-green-blue (RGB) images suffer from a low detection rate under an insufficient light condition, such as on cloudy or stormy days, while black-and-white infrared (IR) images struggle to capture UAVs that overlap with the background at night. In this paper, we propose a new optical flow-assisted graph-pooling residual network (OF-GPRN), which significantly enhances the UAV detection rate in day and night dual visions. The proposed OF-GPRN develops a new optical fusion to remove superfluous backgrounds, which improves RGB/IR imaging clarity. Furthermore, OF-GPRN extends optical fusion by incorporating a graph residual split attention network and a feature pyramid, which refines the perception of UAVs, leading to a higher success rate in UAV detection. A comprehensive performance evaluation is conducted using a benchmark UAV catch dataset. The results indicate that the proposed OF-GPRN elevates the UAV mean average precision (mAP) detection rate to 87.8%, marking a 17.9% advancement compared to the residual graph neural network (ResGCN)-based approach.

cs.CV↗
arXiv2024

DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models

Diffusion models have achieved great success in synthesizing high-quality images. However, generating high-resolution images with diffusion models is still challenging due to the enormous computational costs, resulting in a prohibitive latency for interactive applications. In this paper, we propose DistriFusion to tackle this problem by leveraging parallelism across multiple GPUs. Our method splits the model input into multiple patches and assigns each patch to a GPU. However, naively implementing such an algorithm breaks the interaction between patches and loses fidelity, while incorporating such an interaction will incur tremendous communication overhead. To overcome this dilemma, we observe the high similarity between the input from adjacent diffusion steps and propose displaced patch parallelism, which takes advantage of the sequential nature of the diffusion process by reusing the pre-computed feature maps from the previous timestep to provide context for the current step. Therefore, our method supports asynchronous communication, which can be pipelined by computation. Extensive experiments show that our method can be applied to recent Stable Diffusion XL with no quality degradation and achieve up to a 6.1$\times$ speedup on eight NVIDIA A100s compared to one. Our code is publicly available at https://github.com/mit-han-lab/distrifuser.

cs.CV↗