Searcharxiv⌕ Search

arXiv subjects

Tao Li

Publications and source records attributed to Tao Li.

At least 343 records · Page 19Linked to original sources
arXiv2023

Octopus: A Heterogeneous In-network Computing Accelerator Enabling Deep Learning for network

Deep learning (DL) for network models have achieved excellent performance in the field and are becoming a promising component in future intelligent network system. Programmable in-network computing device has great potential to deploy DL for network models, however, existing device cannot afford to run a DL model. The main challenges of data-plane supporting DL-based network models lie in computing power, task granularity, model generality and feature extracting. To address above problems, we propose Octopus: a heterogeneous in-network computing accelerator enabling DL for network models. A feature extractor is designed for fast and efficient feature extracting. Vector accelerator and systolic array work in a heterogeneous collaborative way, offering low-latency-highthroughput general computing ability for packet-and-flow-based tasks. Octopus also contains on-chip memory fabric for storage and connecting, and Risc-V core for global controlling. The proposed Octopus accelerator design is implemented on FPGA. Functionality and performance of Octopus are validated in several use-cases, achieving performance of 31Mpkt/s feature extracting, 207ns packet-based computing latency, and 90kflow/s flow-based computing throughput.

cs.AR↗
arXiv2023

Designing Policies for Truth: Combating Misinformation with Transparency and Information Design

Misinformation has become a growing issue on online social platforms (OSPs), especially during elections or pandemics. To combat this, OSPs have implemented various policies, such as tagging, to notify users about potentially misleading information. However, these policies are often transparent and therefore susceptible to being exploited by content creators, who may not be willing to invest effort into producing authentic content, causing the viral spread of misinformation. Instead of mitigating the reach of existing misinformation, this work focuses on a solution of prevention, aiming to stop the spread of misinformation before it has a chance to gain momentum. We propose a Bayesian persuaded branching process ($\operatorname{BP}^2$) to model the strategic interactions among the OSP, the content creator, and the user. The misinformation spread on OSP is modeled by a multi-type branching process, where users' positive and negative comments influence the misinformation spreading. Using a Lagrangian induced by Bayesian plausibility, we characterize the OSP's optimal policy under the perfect Bayesian equilibrium. The convexity of the Lagrangian implies that the OSP's optimal policy is simply the fully informative tagging policy: revealing the content's accuracy to the user. Such a tagging policy solicits the best effort from the content creator in reducing misinformation, even though the OSP exerts no direct control over the content creator. We corroborate our findings using numerical simulations.

cs.GT↗
arXiv2023

METTS: Multilingual Emotional Text-to-Speech by Cross-speaker and Cross-lingual Emotion Transfer

Previous multilingual text-to-speech (TTS) approaches have considered leveraging monolingual speaker data to enable cross-lingual speech synthesis. However, such data-efficient approaches have ignored synthesizing emotional aspects of speech due to the challenges of cross-speaker cross-lingual emotion transfer - the heavy entanglement of speaker timbre, emotion, and language factors in the speech signal will make a system produce cross-lingual synthetic speech with an undesired foreign accent and weak emotion expressiveness. This paper proposes the Multilingual Emotional TTS (METTS) model to mitigate these problems, realizing both cross-speaker and cross-lingual emotion transfer. Specifically, METTS takes DelightfulTTS as the backbone model and proposes the following designs. First, to alleviate the foreign accent problem, METTS introduces multi-scale emotion modeling to disentangle speech prosody into coarse-grained and fine-grained scales, producing language-agnostic and language-specific emotion representations, respectively. Second, as a pre-processing step, formant shift-based information perturbation is applied to the reference signal for better disentanglement of speaker timbre in the speech. Third, a vector quantization-based emotion matcher is designed for reference selection, leading to decent naturalness and emotion diversity in cross-lingual synthetic speech. Experiments demonstrate the good design of METTS.

eess.AS↗
arXiv2023

Calibration of a Micromegas-based Gaseous Time Projection Chamber Using Cosmic Ray Muons

We report the calibration of a gaseous Time Projection Chamber based on Micromegas charge readout modules with cosmic ray muons, utilizing their penetrating power and relatively uniform energy deposition per unit length. Muon events were selected through track reconstruction to characterize detector performances, such as the drift velocity, electron lifetime, detector gain, and electric field distortion. The evolution of detector performances with gas purities and electric drift fields over a 50-day data-taking cycle was measured by the muon calibration method. For instance, the drift velocity degraded with gas impurities from ${3.40\pm 0.07 ~ cm/μs}$ to ${3.06\pm 0.06 ~ cm/μs}$, and then recovered with gas purification. A ${^{137}Cs}$ calibration source was also placed inside the detector as a reference for muon calibrations.

physics.ins-det↗
arXiv2023

DVPT: Dynamic Visual Prompt Tuning of Large Pre-trained Models for Medical Image Analysis

Limited labeled data makes it hard to train models from scratch in medical domain, and an important paradigm is pre-training and then fine-tuning. Large pre-trained models contain rich representations, which can be adapted to downstream medical tasks. However, existing methods either tune all the parameters or the task-specific layers of the pre-trained models, ignoring the input variations of medical images, and thus they are not efficient or effective. In this work, we aim to study parameter-efficient fine-tuning (PEFT) for medical image analysis, and propose a dynamic visual prompt tuning method, named DVPT. It can extract knowledge beneficial to downstream tasks from large models with a few trainable parameters. Firstly, the frozen features are transformed by an lightweight bottleneck layer to learn the domain-specific distribution of downstream medical tasks, and then a few learnable visual prompts are used as dynamic queries and then conduct cross-attention with the transformed features, attempting to acquire sample-specific knowledge that are suitable for each sample. Finally, the features are projected to original feature dimension and aggregated with the frozen features. This DVPT module can be shared between different Transformer layers, further reducing the trainable parameters. To validate DVPT, we conduct extensive experiments with different pre-trained models on medical classification and segmentation tasks. We find such PEFT method can not only efficiently adapt the pre-trained models to the medical domain, but also brings data efficiency with partial labeled data. For example, with 0.5\% extra trainable parameters, our method not only outperforms state-of-the-art PEFT methods, even surpasses the full fine-tuning by more than 2.20\% Kappa score on medical classification task. It can saves up to 60\% labeled data and 99\% storage cost of ViT-B/16.

cs.CV↗
arXiv2023

A First Order Meta Stackelberg Method for Robust Federated Learning

Previous research has shown that federated learning (FL) systems are exposed to an array of security risks. Despite the proposal of several defensive strategies, they tend to be non-adaptive and specific to certain types of attacks, rendering them ineffective against unpredictable or adaptive threats. This work models adversarial federated learning as a Bayesian Stackelberg Markov game (BSMG) to capture the defender's incomplete information of various attack types. We propose meta-Stackelberg learning (meta-SL), a provably efficient meta-learning algorithm, to solve the equilibrium strategy in BSMG, leading to an adaptable FL defense. We demonstrate that meta-SL converges to the first-order $\varepsilon$-equilibrium point in $O(\varepsilon^{-2})$ gradient iterations, with $O(\varepsilon^{-4})$ samples needed per iteration, matching the state of the art. Empirical evidence indicates that our meta-Stackelberg framework performs exceptionally well against potent model poisoning and backdoor attacks of an uncertain nature.

cs.LG↗
arXiv2023

Anisotropic Klemens model for the thermal conductivity tensor

With the constraint from Onsager reciprocity relations, here we generalize the Klemens model for phonon-phonon Umklapp scattering from isotropic to anisotropic. Combined with the anisotropic Debye dispersion, this anisotropic Klemens model leads to analytical expressions for heat transfer along both ab-plane (k_ab) and c-axis (k_c), suitable for both layered and chainlike materials with any anisotropy ratio of the dispersion and the scattering. The model is justified by comparison with experimental k_ab and k_c of bulk graphite at high temperatures, as well as the thickness-dependent k_c of graphite thin films at room temperature.

physics.app-ph↗
arXiv2023

Interpretable Style Transfer for Text-to-Speech with ControlVAE and Diffusion Bridge

With the demand for autonomous control and personalized speech generation, the style control and transfer in Text-to-Speech (TTS) is becoming more and more important. In this paper, we propose a new TTS system that can perform style transfer with interpretability and high fidelity. Firstly, we design a TTS system that combines variational autoencoder (VAE) and diffusion refiner to get refined mel-spectrograms. Specifically, a two-stage and a one-stage system are designed respectively, to improve the audio quality and the performance of style transfer. Secondly, a diffusion bridge of quantized VAE is designed to efficiently learn complex discrete style representations and improve the performance of style transfer. To have a better ability of style transfer, we introduce ControlVAE to improve the reconstruction quality and have good interpretability simultaneously. Experiments on LibriTTS dataset demonstrate that our method is more effective than baseline models.

cs.SD↗
arXiv2023

Reconstruction of the event vertex in the PandaX-III experiment with convolution neural network

The tracks left by charged particles in a gaseous time projection chamber~(TPC) incorporate important information about the interaction process and drift of electrons in gas. The electron diffusion information carried by the tracks is an effective signature to reconstruct $z_0$, the vertex position in drift direction at which the event takes place. In this paper, we propose to reconstruct $z_0$ with convolution neural network~(CNN) in the PandaX-III experiment. A CNN model VGGZ0net is built and validated with Monte Carlo simulation data. It gives $z_0$ with a 11~cm precision for the events above 2~MeV uniformly distributed along a drift distance of 120~cm, and then the electron lifetime can be deduced. The energy resolution of detector is significantly improved after the electron lifetime correction, i.e., from 10.1\% to 4.0\% FWHM at the Q-value of double beta decay of $^{136}$Xe for the scenario with an electron lifetime of 6.5~ms. The CNN model is also successfully applied to the experimental data of the PandaX-III prototype detector for $z_0$ reconstruction.

hep-ex↗
arXiv2023

An Extensible Plug-and-Play Method for Multi-Aspect Controllable Text Generation

Recently, multi-aspect controllable text generation that controls the generated text in multiple aspects (e.g., sentiment, topic, and keywords) has attracted increasing attention. Although methods based on parameter efficient tuning like prefix-tuning could achieve multi-aspect controlling in a plug-and-play way, the mutual interference of multiple prefixes leads to significant degeneration of constraints and limits their extensibility to training-time unseen aspect combinations. In this work, we provide a theoretical lower bound for the interference and empirically found that the interference grows with the number of layers where prefixes are inserted. Based on these analyses, we propose using trainable gates to normalize the intervention of prefixes to restrain the growing interference. As a result, controlling training-time unseen combinations of aspects can be realized by simply concatenating corresponding plugins such that new constraints can be extended at a lower cost. In addition, we propose a unified way to process both categorical and free-form constraints. Experiments on text generation and machine translation demonstrate the superiority of our approach over baselines on constraint accuracy, text quality, and extensibility.

cs.CL↗
arXiv2023

Fast-speed and low-power-consumption optical phased array based on thin-film lithium niobate platform

Fast scanning-speed and low-power-consumption are becoming progressively more and more important in realizing high-performance chiplet optical phased arrays (OPAs). Here, we establish an integrated OPA based on thin-film lithium niobate-on-insulator (LNOI) platform to access these outstanding performances. Significantly, a lithium niobate (LN) OPA chip is implemented by 32/48 channels LN waveguides enabled by electro-optic modulations, which showcases the low power consumption (1.11nJ/π}) and fast operation speed (14.4 ns) promising the advantage of the LNOI platform for integrated OPAs. As results, we experimentally achieved a beam steering with a 62.2°*8.8° field of view (FOV) and a beam divergence of 2.4°*1.2°. Moreover, by employing sparse aperiodic arrays in waveguides design we obtained a significant reduction of lateral divergence to 0.33° for the radiation beam. This work demonstrate that remarkable advantage of LNOI platform for power-saving and scalable OPA chips for various applications.

physics.optics↗
arXiv2023

Accurate ignition detection of solid fuel particles using machine learning

In the present work, accurate determination of single-particle ignition is focused on using high-speed optical diagnostics combined with machine learning approaches. Ignition of individual particles in a laminar flow reactor are visualized by simultaneous 10 kHz OH-LIF and DBI measurements. Two coal particle sizes of 90-125μm and 160-200μm are investigated in conventional air and oxy-fuel conditions with increasing oxygen concentrations. Ignition delay times are first evaluated with threshold methods, revealing obvious deviations compared to the ground truth detected by the human eye. Then, residual networks (ResNet) and feature pyramidal networks (FPN) are trained on the ground truth and applied to predict the ignition time.~Both networks are capable of detecting ignition with significantly higher accuracy and precision. Besides, influences of input data and depth of networks on the prediction performance of a trained model are examined.~The current study shows that the hierarchical feature extraction of the convolutions networks clearly facilitates data evaluation for high-speed optical measurements and could be transferred to other solid fuel experiments with similar boundary conditions.

cs.LG↗
arXiv2023

Sky-GVINS: a Sky-segmentation Aided GNSS-Visual-Inertial System for Robust Navigation in Urban Canyons

Integrating Global Navigation Satellite Systems (GNSS) in Simultaneous Localization and Mapping (SLAM) systems draws increasing attention to a global and continuous localization solution. Nonetheless, in dense urban environments, GNSS-based SLAM systems will suffer from the Non-Line-Of-Sight (NLOS) measurements, which might lead to a sharp deterioration in localization results. In this paper, we propose to detect the sky area from the up-looking camera to improve GNSS measurement reliability for more accurate position estimation. We present Sky-GVINS: a sky-aware GNSS-Visual-Inertial system based on a recent work called GVINS. Specifically, we adopt a global threshold method to segment the sky regions and non-sky regions in the fish-eye sky-pointing image and then project satellites to the image using the geometric relationship between satellites and the camera. After that, we reject satellites in non-sky regions to eliminate NLOS signals. We investigated various segmentation algorithms for sky detection and found that the Otsu algorithm reported the highest classification rate and computational efficiency, despite the algorithm's simplicity and ease of implementation. To evaluate the effectiveness of Sky-GVINS, we built a ground robot and conducted extensive real-world experiments on campus. Experimental results show that our method improves localization accuracy in both open areas and dense urban environments compared to the baseline method. Finally, we also conduct a detailed analysis and point out possible further directions for future research. For detailed information, visit our project website at https://github.com/SJTU-ViSYS/Sky-GVINS.

cs.RO↗
arXiv2023

AutoQNN: An End-to-End Framework for Automatically Quantizing Neural Networks

Exploring the expected quantizing scheme with suitable mixed-precision policy is the key point to compress deep neural networks (DNNs) in high efficiency and accuracy. This exploration implies heavy workloads for domain experts, and an automatic compression method is needed. However, the huge search space of the automatic method introduces plenty of computing budgets that make the automatic process challenging to be applied in real scenarios. In this paper, we propose an end-to-end framework named AutoQNN, for automatically quantizing different layers utilizing different schemes and bitwidths without any human labor. AutoQNN can seek desirable quantizing schemes and mixed-precision policies for mainstream DNN models efficiently by involving three techniques: quantizing scheme search (QSS), quantizing precision learning (QPL), and quantized architecture generation (QAG). QSS introduces five quantizing schemes and defines three new schemes as a candidate set for scheme search, and then uses the differentiable neural architecture search (DNAS) algorithm to seek the layer- or model-desired scheme from the set. QPL is the first method to learn mixed-precision policies by reparameterizing the bitwidths of quantizing schemes, to the best of our knowledge. QPL optimizes both classification loss and precision loss of DNNs efficiently and obtains the relatively optimal mixed-precision model within limited model size and memory footprint. QAG is designed to convert arbitrary architectures into corresponding quantized ones without manual intervention, to facilitate end-to-end neural network quantization. We have implemented AutoQNN and integrated it into Keras. Extensive experiments demonstrate that AutoQNN can consistently outperform state-of-the-art quantization.

cs.LG↗
arXiv2023

Is Stochastic Mirror Descent Vulnerable to Adversarial Delay Attacks? A Traffic Assignment Resilience Study

\textit{Intelligent Navigation Systems} (INS) are exposed to an increasing number of informational attack vectors, which often intercept through the communication channels between the INS and the transportation network during the data collecting process. To measure the resilience of INS, we use the concept of a Wardrop Non-Equilibrium Solution (WANES), which is characterized by the probabilistic outcome of learning within a bounded number of interactions. By using concentration arguments, we have discovered that any bounded feedback delaying attack only degrades the systematic performance up to order $\tilde{\mathcal{O}}(\sqrt{{d^3}{T^{-1}}})$ along the traffic flow trajectory within the Delayed Mirror Descent (DMD) online-learning framework. This degradation in performance can occur with only mild assumptions imposed. Our result implies that learning-based INS infrastructures can achieve Wardrop Non-equilibrium even when experiencing a certain period of disruption in the information structure. These findings provide valuable insights for designing defense mechanisms against possible jamming attacks across different layers of the transportation ecosystem.

cs.LG↗
arXiv2023

Investigating Catastrophic Overfitting in Fast Adversarial Training: A Self-fitting Perspective

Although fast adversarial training provides an efficient approach for building robust networks, it may suffer from a serious problem known as catastrophic overfitting (CO), where multi-step robust accuracy suddenly collapses to zero. In this paper, we for the first time decouple single-step adversarial examples into data-information and self-information, which reveals an interesting phenomenon called "self-fitting". Self-fitting, i.e., the network learns the self-information embedded in single-step perturbations, naturally leads to the occurrence of CO. When self-fitting occurs, the network experiences an obvious "channel differentiation" phenomenon that some convolution channels accounting for recognizing self-information become dominant, while others for data-information are suppressed. In this way, the network can only recognize images with sufficient self-information and loses generalization ability to other types of data. Based on self-fitting, we provide new insights into the existing methods to mitigate CO and extend CO to multi-step adversarial training. Our findings reveal a self-learning mechanism in adversarial training and open up new perspectives for suppressing different kinds of information to mitigate CO.

cs.LG↗
arXiv2023

Multi-Speaker Expressive Speech Synthesis via Multiple Factors Decoupling

This paper aims to synthesize the target speaker's speech with desired speaking style and emotion by transferring the style and emotion from reference speech recorded by other speakers. We address this challenging problem with a two-stage framework composed of a text-to-style-and-emotion (Text2SE) module and a style-and-emotion-to-wave (SE2Wave) module, bridging by neural bottleneck (BN) features. To further solve the multi-factor (speaker timbre, speaking style and emotion) decoupling problem, we adopt the multi-label binary vector (MBV) and mutual information (MI) minimization to respectively discretize the extracted embeddings and disentangle these highly entangled factors in both Text2SE and SE2Wave modules. Moreover, we introduce a semi-supervised training strategy to leverage data from multiple speakers, including emotion-labeled data, style-labeled data, and unlabeled data. To better transfer the fine-grained expression from references to the target speaker in non-parallel transfer, we introduce a reference-candidate pool and propose an attention-based reference selection approach. Extensive experiments demonstrate the good design of our model.

eess.AS↗