arXiv subjects

Peng Li

Publications and source records attributed to Peng Li.

At least 217 records · Page 12Linked to original sources

arXiv2024

Spiking Transformer Hardware Accelerators in 3D Integration

Spiking neural networks (SNNs) are powerful models of spatiotemporal computation and are well suited for deployment on resource-constrained edge devices and neuromorphic hardware due to their low power consumption. Leveraging attention mechanisms similar to those found in their artificial neural network counterparts, recently emerged spiking transformers have showcased promising performance and efficiency by capitalizing on the binary nature of spiking operations. Recognizing the current lack of dedicated hardware support for spiking transformers, this paper presents the first work on 3D spiking transformer hardware architecture and design methodology. We present an architecture and physical design co-optimization approach tailored specifically for spiking transformers. Through memory-on-logic and logic-on-logic stacking enabled by 3D integration, we demonstrate significant energy and delay improvements compared to conventional 2D CMOS integration.

cs.NE↗

arXiv2024

PointCG: Self-supervised Point Cloud Learning via Joint Completion and Generation

The core of self-supervised point cloud learning lies in setting up appropriate pretext tasks, to construct a pre-training framework that enables the encoder to perceive 3D objects effectively. In this paper, we integrate two prevalent methods, masked point modeling (MPM) and 3D-to-2D generation, as pretext tasks within a pre-training framework. We leverage the spatial awareness and precise supervision offered by these two methods to address their respective limitations: ambiguous supervision signals and insensitivity to geometric information. Specifically, the proposed framework, abbreviated as PointCG, consists of a Hidden Point Completion (HPC) module and an Arbitrary-view Image Generation (AIG) module. We first capture visible points from arbitrary views as inputs by removing hidden points. Then, HPC extracts representations of the inputs with an encoder and completes the entire shape with a decoder, while AIG is used to generate rendered images based on the visible points' representations. Extensive experiments demonstrate the superiority of the proposed method over the baselines in various downstream tasks. Our code will be made available upon acceptance.

cs.CV↗

arXiv2024

StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understanding

The rapid development of Multimodal Large Language Models (MLLMs) has expanded their capabilities from image comprehension to video understanding. However, most of these MLLMs focus primarily on offline video comprehension, necessitating extensive processing of all video frames before any queries can be made. This presents a significant gap compared to the human ability to watch, listen, think, and respond to streaming inputs in real time, highlighting the limitations of current MLLMs. In this paper, we introduce StreamingBench, the first comprehensive benchmark designed to evaluate the streaming video understanding capabilities of MLLMs. StreamingBench assesses three core aspects of streaming video understanding: (1) real-time visual understanding, (2) omni-source understanding, and (3) contextual understanding. The benchmark consists of 18 tasks, featuring 900 videos and 4,500 human-curated QA pairs. Each video features five questions presented at different time points to simulate a continuous streaming scenario. We conduct experiments on StreamingBench with 13 open-source and proprietary MLLMs and find that even the most advanced proprietary MLLMs like Gemini 1.5 Pro and GPT-4o perform significantly below human-level streaming video understanding capabilities. We hope our work can facilitate further advancements for MLLMs, empowering them to approach human-level video comprehension and interaction in more realistic scenarios.

cs.CV↗

arXiv2024

$B\to K\bar K(πη)h$ decays in the presence of isovector scalar resonances $a_0(980,1450)$

Different from the previous treatment in a two-body framework, we introduce the dimeson distribution amplitudes (DAs) to describe the strong dynamics between the S-wave resonances $a_0(980, 1450)$ and the $K\bar K (πη)$ pair, where the Gegenbauer coefficient required is determined from the experimental data on the time-like form factors involved. The branching ratios and direct CP asymmetries of the decays $B \to a^{(\prime)}_0 h \to K\bar K(πη) h$, with $a_0=a_0(980)$, $a^{\prime}_0=a_0(1450)$ and $h$ referring to a pion or a kaon, are then calculated in the perturbative QCD (PQCD) approach. We find that the branching ratios of the corresponding quasi-two-body decays $B\to a^{(\prime)}_0 K$ obtained with the narrow width approximation are closer to those predicted in the QCD factorization (QCDF) approach compared to the previous PQCD calculations, no matter a three-body or a two-body framework is assumed. Furthermore, all our predictions for these $B\to a^{(\prime)}_0 K$ decays are below the current experimental upper limits except for those of decays $B^0\to a^{(\prime)-}_0K^+$, which are (slightly) larger than the upper limits. Under the narrow width approximation, the branching ratios of the decays $B^+\to a^{(\prime)+}_0π^0$, $B^0\to a^{(\prime)+}_0π^-$ and $B^0\to a^{(\prime)0}_0π^0$ are comparable to or agree well with the previous PQCD and the QCDF calculations. While for the decays $B^+\to a^{(\prime)0}_0π^+$ and $B^0\to a^{(\prime)-}_0π^+$, their branching ratios are predicted to be unexpectedly large, for example, the obtained branching ratio of decay $B^+\to a^0_0π^+$ is even higher than the current experimental upper limit.

hep-ph↗

arXiv2024

Trustworthy Federated Learning: Privacy, Security, and Beyond

While recent years have witnessed the advancement in big data and Artificial Intelligence (AI), it is of much importance to safeguard data privacy and security. As an innovative approach, Federated Learning (FL) addresses these concerns by facilitating collaborative model training across distributed data sources without transferring raw data. However, the challenges of robust security and privacy across decentralized networks catch significant attention in dealing with the distributed data in FL. In this paper, we conduct an extensive survey of the security and privacy issues prevalent in FL, underscoring the vulnerability of communication links and the potential for cyber threats. We delve into various defensive strategies to mitigate these risks, explore the applications of FL across different sectors, and propose research directions. We identify the intricate security challenges that arise within the FL frameworks, aiming to contribute to the development of secure and efficient FL systems.

cs.CR↗

arXiv2024

Toward a More Populous Online Platform: The Economic Impacts of Compensated Reviews

Many companies nowadays offer compensation to online reviews (called compensated reviews), expecting to increase the volume of their non-compensated reviews and overall rating. Does this strategy work? On what subjects or topics does this strategy work the best? These questions have still not been answered in the literature but draw substantial interest from the industry. In this paper, we study the effect of compensated reviews on non-compensated reviews by utilizing online reviews on 1,240 auto shipping companies over a ten-year period from a transportation website. Because some online reviews have missing information on their compensation status, we first develop a classification algorithm to differentiate compensated reviews from non-compensated reviews by leveraging a machine learning-based identification process, drawing upon the unique features of the compensated reviews. From the classification results, we empirically investigate the effects of compensated reviews on non-compensated. Our results indicate that the number of compensated reviews does indeed increase the number of non-compensated reviews. In addition, the ratings of compensated reviews positively affect the ratings of non-compensated reviews. Moreover, if the compensated reviews feature the topic or subject of a car shipping function, the positive effect of compensated reviews on non-compensated ones is the strongest. Besides methodological contributions in text classification and empirical modeling, our study provides empirical evidence on how to prove the effectiveness of compensated online reviews in terms of improving the platform's overall online reviews and ratings. Also, it suggests a guideline for utilizing compensated reviews to their full strength, that is, with regard to featuring certain topics or subjects in these reviews to achieve the best outcome.

econ.GN↗

arXiv2024

Efficient Deep Learning Board: Training Feedback Is Not All You Need

Current automatic deep learning (i.e., AutoDL) frameworks rely on training feedback from actual runs, which often hinder their ability to provide quick and clear performance predictions for selecting suitable DL systems. To address this issue, we propose EfficientDL, an innovative deep learning board designed for automatic performance prediction and component recommendation. EfficientDL can quickly and precisely recommend twenty-seven system components and predict the performance of DL models without requiring any training feedback. The magic of no training feedback comes from our proposed comprehensive, multi-dimensional, fine-grained system component dataset, which enables us to develop a static performance prediction model and comprehensive optimized component recommendation algorithm (i.e., α\b{eta}-BO search), removing the dependency on actually running parameterized models during the traditional optimization search process. The simplicity and power of EfficientDL stem from its compatibility with most DL models. For example, EfficientDL operates seamlessly with mainstream models such as ResNet50, MobileNetV3, EfficientNet-B0, MaxViT-T, Swin-B, and DaViT-T, bringing competitive performance improvements. Besides, experimental results on the CIFAR-10 dataset reveal that EfficientDL outperforms existing AutoML tools in both accuracy and efficiency (approximately 20 times faster along with 1.31% Top-1 accuracy improvement than the cutting-edge methods). Source code, pretrained models, and datasets are available at https://github.com/OpenSELab/EfficientDL.

cs.LG↗

arXiv2024

Semileptonic and nonleptonic decays of $B_{u,d,s,c}^{*}$ in the covariant light-front approach

The semileptonic and nonleptonic decays of the b-flavor vector mesons $B^{*}_{u,d,s}$ and $B_{c}^{*}$ are investigated within the covariant light-front quark model (CLFQM). By calculating the form factors of the transitions $B_{u, d, s, c}^{*}\to P$ under the CLFQM, with $P$ denoting a pseudoscalar meson, i.e., $π, K, η_c(1S,2S), D_{(s)}, B_{(s)}$, we predict and discuss several physical observables, including the branching ratios, polarization fractions $f_{L}, f_{\|}$, and forward-backward asymmetries $A_{FB}$. The total widths of the single-photon radiative decay channels for these b-flavor vector mesons are estimated using their partial widths. In these considered decays, one can find that the semileptonic decays $B_{s}^{*0}\to D_{s}^{-}\ell^{\prime+}ν_{\ell^\prime}$ and $B_{c}^{*+}\to B_{s}^{0}\ell^{\prime+}ν_{\ell^\prime}, η_{c}\ell^{\prime+}ν_{\ell^\prime}$, with $\ell^\prime$ being $e$ or $τ$, and the nonleptonic channels $B_{c}^{*+}\to B^0_{s} π^{+}, B^0_{s} ρ^{+}$ have the largest branching ratios, which can reach up to the $10^{-7}$ order, and are most likely to be accessible at the future high-luminosity LHCb and Belle-II experiments.

hep-ph↗

arXiv2024

Dual-AEB: Synergizing Rule-Based and Multimodal Large Language Models for Effective Emergency Braking

Automatic Emergency Braking (AEB) systems are a crucial component in ensuring the safety of passengers in autonomous vehicles. Conventional AEB systems primarily rely on closed-set perception modules to recognize traffic conditions and assess collision risks. To enhance the adaptability of AEB systems in open scenarios, we propose Dual-AEB, a system combines an advanced multimodal large language model (MLLM) for comprehensive scene understanding and a conventional rule-based rapid AEB to ensure quick response times. To the best of our knowledge, Dual-AEB is the first method to incorporate MLLMs within AEB systems. Through extensive experimentation, we have validated the effectiveness of our method. The source code will be available at https://github.com/ChipsICU/Dual-AEB.

cs.RO↗

arXiv2024

Optimized Magnetic Resonance Fingerprinting Using Ziv-Zakai Bound

Magnetic Resonance Fingerprinting (MRF) has emerged as a promising quantitative imaging technique within the field of Magnetic Resonance Imaging (MRI), offers comprehensive insights into tissue properties by simultaneously acquiring multiple tissue parameter maps in a single acquisition. Sequence optimization is crucial for improving the accuracy and efficiency of MRF. In this work, a novel framework for MRF sequence optimization is proposed based on the Ziv-Zakai bound (ZZB). Unlike the Cramér-Rao bound (CRB), which aims to enhance the quality of a single fingerprint signal with deterministic parameters, ZZB provides insights into evaluating the minimum mismatch probability for pairs of fingerprint signals within the specified parameter range in MRF. Specifically, the explicit ZZB is derived to establish a lower bound for the discrimination error in the fingerprint signal matching process within MRF. This bound illuminates the intrinsic limitations of MRF sequences, thereby fostering a deeper understanding of existing sequence performance. Subsequently, an optimal experiment design problem based on ZZB was formulated to ascertain the optimal scheme of acquisition parameters, maximizing discrimination power of MRF between different tissue types. Preliminary numerical experiments show that the optimized ZZB scheme outperforms both the conventional and CRB schemes in terms of the reconstruction accuracy of multiple parameter maps.

eess.IV↗

arXiv2024

Towards the Mitigation of Confirmation Bias in Semi-supervised Learning: a Debiased Training Perspective

Semi-supervised learning (SSL) commonly exhibits confirmation bias, where models disproportionately favor certain classes, leading to errors in predicted pseudo labels that accumulate under a self-training paradigm. Unlike supervised settings, which benefit from a rich, static data distribution, SSL inherently lacks mechanisms to correct this self-reinforced bias, necessitating debiased interventions at each training step. Although the generation of debiased pseudo labels has been extensively studied, their effective utilization remains underexplored. Our analysis indicates that data from biased classes should have a reduced influence on parameter updates, while more attention should be given to underrepresented classes. To address these challenges, we introduce TaMatch, a unified framework for debiased training in SSL. TaMatch employs a scaling ratio derived from both a prior target distribution and the model's learning status to estimate and correct bias at each training step. This ratio adjusts the raw predictions on unlabeled data to produce debiased pseudo labels. In the utilization phase, these labels are differently weighted according to their predicted class, enhancing training equity and minimizing class bias. Additionally, TaMatch dynamically adjust the target distribution in response to the model's learning progress, facilitating robust handling of practical scenarios where the prior distribution is unknown. Empirical evaluations show that TaMatch significantly outperforms existing state-of-the-art methods across a range of challenging image classification tasks, highlighting the critical importance of both the debiased generation and utilization of pseudo labels in SSL.

cs.LG↗

arXiv2024

DataGpt-SQL-7B: An Open-Source Language Model for Text-to-SQL

In addressing the pivotal role of translating natural language queries into SQL commands, we propose a suite of compact, fine-tuned models and self-refine mechanisms to democratize data access and analysis for non-expert users, mitigating risks associated with closed-source Large Language Models. Specifically, we constructed a dataset of over 20K sample for Text-to-SQL as well as the preference dateset, to improve the efficiency in the domain of SQL generation. To further ensure code validity, a code corrector was integrated into the model. Our system, DataGpt-sql, achieved 87.2\% accuracy on the spider-dev, respectively, showcasing the effectiveness of our solution in text-to-SQL conversion tasks. Our code, data, and models are available at \url{https://github.com/CainiaoTechAi/datagpt-sql-7b}

cs.AI↗

arXiv2024

Unveiling High Selectivity Origin of Pt-Bi Catalysts for Alkaline Methanol Electrooxidation via CO-free pathway

A long-standing puzzle for methanol electrooxidation is how to achieve a CO-free pathway and accurately understand the origin of electrocatalytic selectivity. Herein, we unequivocally demonstrate that the Bi-modified Pt/C follows a CO-free dominated pathway during alkaline methanol electrooxidation, and unveil the formaldehyde (HCHO) intermediate as a critical factor influencing pathway selectivity. These findings are substantiated by kinetic isotope effects, formate Faradaic efficiency, in situ spectroscopy, ab initio molecular dynamic simulations, and density functional theory calculations. Bi modification significantly increases the HCHO dehydrogenation barrier, which facilitates its desorption and subsequent conversion to the H2COOH- anion at the alkaline interface, intrinsically avoiding CO formation. More specifically, the formation of ensemble sites featuring V-shaped Bi-Pt-Bi configuration inhibits the cleavage of C-H bond, and the weak OH binding energy at Bi adatoms effectively prevents blockage of oxygenated species, allowing such ensemble sites to fulfill their functional role. Our study opens up a novel dimension for designing advanced CO-free catalysts.

physics.chem-ph↗

arXiv2024

DS2TA: Denoising Spiking Transformer with Attenuated Spatiotemporal Attention

Vision Transformers (ViT) are current high-performance models of choice for various vision applications. Recent developments have given rise to biologically inspired spiking transformers that thrive in ultra-low power operations on neuromorphic hardware, however, without fully unlocking the potential of spiking neural networks. We introduce DS2TA, a Denoising Spiking transformer with attenuated SpatioTemporal Attention, designed specifically for vision applications. DS2TA introduces a new spiking attenuated spatiotemporal attention mechanism that considers input firing correlations occurring in both time and space, thereby fully harnessing the computational power of spiking neurons at the core of the transformer architecture. Importantly, DS2TA facilitates parameter-efficient spatiotemporal attention computation without introducing extra weights. DS2TA employs efficient hashmap-based nonlinear spiking attention denoisers to enhance the robustness and expressive power of spiking attention maps. DS2TA demonstrates state-of-the-art performances on several widely adopted static image and dynamic neuromorphic datasets. Operated over 4 time steps, DS2TA achieves 94.92% top-1 accuracy on CIFAR10 and 77.47% top-1 accuracy on CIFAR100, as well as 79.1% and 94.44% on CIFAR10-DVS and DVS-Gesture using 10 time steps.

cs.NE↗

arXiv2024

FedLF: Adaptive Logit Adjustment and Feature Optimization in Federated Long-Tailed Learning

Federated learning offers a paradigm to the challenge of preserving privacy in distributed machine learning. However, datasets distributed across each client in the real world are inevitably heterogeneous, and if the datasets can be globally aggregated, they tend to be long-tailed distributed, which greatly affects the performance of the model. The traditional approach to federated learning primarily addresses the heterogeneity of data among clients, yet it fails to address the phenomenon of class-wise bias in global long-tailed data. This results in the trained model focusing on the head classes while neglecting the equally important tail classes. Consequently, it is essential to develop a methodology that considers classes holistically. To address the above problems, we propose a new method FedLF, which introduces three modifications in the local training phase: adaptive logit adjustment, continuous class centred optimization, and feature decorrelation. We compare seven state-of-the-art methods with varying degrees of data heterogeneity and long-tailed distribution. Extensive experiments on benchmark datasets CIFAR-10-LT and CIFAR-100-LT demonstrate that our approach effectively mitigates the problem of model performance degradation due to data heterogeneity and long-tailed distribution. our code is available at https://github.com/18sym/FedLF.

cs.LG↗

arXiv2024

HiPrompt: Tuning-free Higher-Resolution Generation with Hierarchical MLLM Prompts

The potential for higher-resolution image generation using pretrained diffusion models is immense, yet these models often struggle with issues of object repetition and structural artifacts especially when scaling to 4K resolution and higher. We figure out that the problem is caused by that, a single prompt for the generation of multiple scales provides insufficient efficacy. In response, we propose HiPrompt, a new tuning-free solution that tackles the above problems by introducing hierarchical prompts. The hierarchical prompts offer both global and local guidance. Specifically, the global guidance comes from the user input that describes the overall content, while the local guidance utilizes patch-wise descriptions from MLLMs to elaborately guide the regional structure and texture generation. Furthermore, during the inverse denoising process, the generated noise is decomposed into low- and high-frequency spatial components. These components are conditioned on multiple prompt levels, including detailed patch-wise descriptions and broader image-level prompts, facilitating prompt-guided denoising under hierarchical semantic guidance. It further allows the generation to focus more on local spatial regions and ensures the generated images maintain coherent local and global semantics, structures, and textures with high definition. Extensive experiments demonstrate that HiPrompt outperforms state-of-the-art works in higher-resolution image generation, significantly reducing object repetition and enhancing structural quality.

cs.CV↗

arXiv2024

Frequency ratio of the $^{229\mathrm{m}}$Th nuclear isomeric transition and the $^{87}$Sr atomic clock

Optical atomic clocks$^{1,2}$ use electronic energy levels to precisely keep track of time. A clock based on nuclear energy levels promises a next-generation platform for precision metrology and fundamental physics studies. Thorium-229 nuclei exhibit a uniquely low energy nuclear transition within reach of state-of-the-art vacuum ultraviolet (VUV) laser light sources and have therefore been proposed for construction of the first nuclear clock$^{3,4}$. However, quantum state-resolved spectroscopy of the $^{229m}$Th isomer to determine the underlying nuclear structure and establish a direct frequency connection with existing atomic clocks has yet to be performed. Here, we use a VUV frequency comb to directly excite the narrow $^{229}$Th nuclear clock transition in a solid-state CaF$_2$ host material and determine the absolute transition frequency. We stabilize the fundamental frequency comb to the JILA $^{87}$Sr clock$^2$ and coherently upconvert the fundamental to its 7th harmonic in the VUV range using a femtosecond enhancement cavity. This VUV comb establishes a frequency link between nuclear and electronic energy levels and allows us to directly measure the frequency ratio of the $^{229}$Th nuclear clock transition and the $^{87}$Sr atomic clock. We also precisely measure the nuclear quadrupole splittings and extract intrinsic properties of the isomer. These results mark the start of nuclear-based solid-state optical clock and demonstrate the first comparison of nuclear and atomic clocks for fundamental physics studies. This work represents a confluence of precision metrology, ultrafast strong field physics, nuclear physics, and fundamental physics.

physics.atom-ph↗

arXiv2024

Contrastive Learning with Consistent Representations

Contrastive learning demonstrates great promise for representation learning. Data augmentations play a critical role in contrastive learning by providing informative views of the data without necessitating explicit labels. Nonetheless, the efficacy of current methodologies heavily hinges on the quality of employed data augmentation (DA) functions, often chosen manually from a limited set of options. While exploiting diverse data augmentations is appealing, the complexities inherent in both DAs and representation learning can lead to performance deterioration. Addressing this challenge and facilitating the systematic incorporation of diverse data augmentations, this paper proposes Contrastive Learning with Consistent Representations CoCor. At the heart of CoCor is a novel consistency metric termed DA consistency. This metric governs the mapping of augmented input data to the representation space, ensuring that these instances are positioned optimally in a manner consistent with the applied intensity of the DA. Moreover, we propose to learn the optimal mapping locations as a function of DA, all while preserving a desired monotonic property relative to DA intensity. Experimental results demonstrate that CoCor notably enhances the generalizability and transferability of learned representations in comparison to baseline methods.

cs.CV↗