arXiv subjects

Pei Zhang

Publications and source records attributed to Pei Zhang.

At least 91 records · Page 5Linked to original sources

arXiv2024

Neuromorphic Shack-Hartmann wave normal sensing

The Shack-Hartmann wavefront sensor is widely employed in adaptive optics systems to measure optical aberrations. However, simultaneously achieving high sensitivity and large dynamic range is still challenging, limiting the performance of diagnosing fast-changing turbulence. To overcome this limitation, we propose neuromorphic Shack-Hartmann wave normal sensing (NeuroSH). NeuroSH is a unifying framework that harnesses the computational neuromorphic imaging paradigm to extract the high-dimensional wave normal from temporal diversity measurements. Both numerical analysis and experimental verification demonstrate the feasibility of NeuroSH. To the best of our knowledge, the proposed NeuroSH is the first scheme to surpass the optical dynamic range limitation under challenging dynamic scenarios, thereby advancing ultra-fast turbulence mitigation technology for cutting-edge imagers.

physics.optics↗

arXiv2024

Anchor-based Multi-view Subspace Clustering with Hierarchical Feature Descent

Multi-view clustering has attracted growing attention owing to its capabilities of aggregating information from various sources and its promising horizons in public affairs. Up till now, many advanced approaches have been proposed in recent literature. However, there are several ongoing difficulties to be tackled. One common dilemma occurs while attempting to align the features of different views. {Moreover, due to the fact that many existing multi-view clustering algorithms stem from spectral clustering, this results to cubic time complexity w.r.t. the number of dataset. However, we propose Anchor-based Multi-view Subspace Clustering with Hierarchical Feature Descent(MVSC-HFD) to tackle the discrepancy among views through hierarchical feature descent and project to a common subspace( STAGE 1), which reveals dependency of different views. We further reduce the computational complexity to linear time cost through a unified sampling strategy in the common subspace( STAGE 2), followed by anchor-based subspace clustering to learn the bipartite graph collectively( STAGE 3). }Extensive experimental results on public benchmark datasets demonstrate that our proposed model consistently outperforms the state-of-the-art techniques.

cs.CV↗

arXiv2024

Neuromorphic Imaging and Classification with Graph Learning

Bio-inspired neuromorphic cameras asynchronously record pixel brightness changes and generate sparse event streams. They can capture dynamic scenes with little motion blur and more details in extreme illumination conditions. Due to the multidimensional address-event structure, most existing vision algorithms cannot properly handle asynchronous event streams. While several event representations and processing methods have been developed to address such an issue, they are typically driven by a large number of events, leading to substantial overheads in runtime and memory. In this paper, we propose a new graph representation of the event data and couple it with a Graph Transformer to perform accurate neuromorphic classification. Extensive experiments show that our approach leads to better results and excels at the challenging realistic situations where only a small number of events and limited computational resources are available, paving the way for neuromorphic applications embedded into mobile facilities.

cs.CV↗

arXiv2024

Event Encryption: Rethinking Privacy Exposure for Neuromorphic Imaging

Bio-inspired neuromorphic cameras sense illumination changes on a per-pixel basis and generate spatiotemporal streaming events within microseconds in response, offering visual information with high temporal resolution over a high dynamic range. Such devices often serve in surveillance systems due to their applicability and robustness in environments with high dynamics and harsh lighting, where they can still supply clearer recordings than traditional imaging. In other words, when it comes to privacy-relevant cases, neuromorphic cameras also expose more sensitive data and pose serious security threats. Therefore, asynchronous event streams necessitate careful encryption before transmission and usage. This work discusses several potential attack scenarios and approaches event encryption from the perspective of neuromorphic noise removal, in which we inversely introduce well-crafted noise into raw events until they are obfuscated. Our evaluations show that the encrypted events can effectively protect information from attacks of low-level visual reconstruction and high-level neuromorphic reasoning, and thus feature dependable privacy-preserving competence. The proposed solution gives impetus to the security of event data and paves the way to a highly encrypted technique for privacy-protective neuromorphic imaging.

eess.SP↗

arXiv2024

Neuromorphic Imaging with Joint Image Deblurring and Event Denoising

Neuromorphic imaging reacts to per-pixel brightness changes of a dynamic scene with high temporal precision and responds with asynchronous streaming events as a result. It also often supports a simultaneous output of an intensity image. Nevertheless, the raw events typically involve a large amount of noise due to the high sensitivity of the sensor, while capturing fast-moving objects at low frame rates results in blurry images. These deficiencies significantly degrade human observation and machine processing. Fortunately, the two information sources are inherently complementary -- events with microsecond-level temporal resolution, which are triggered by the edges of objects recorded in a latent sharp image, can supply rich motion details missing from the blurry one. In this work, we bring the two types of data together and introduce a simple yet effective unifying algorithm to jointly reconstruct blur-free images and noise-robust events in an iterative coarse-to-fine fashion. Specifically, an event-regularized prior offers precise high-frequency structures and dynamic features for blind deblurring, while image gradients serve as a kind of faithful supervision in regulating neuromorphic noise removal. Comprehensively evaluated on real and synthetic samples, such a synergy delivers superior reconstruction quality for both images with severe motion blur and raw event streams with a storm of noise, and also exhibits greater robustness to challenging realistic scenarios such as varying levels of illumination, contrast and motion magnitude. Meanwhile, it can be driven by much fewer events and holds a competitive edge at computational time overhead, rendering itself preferable as available computing resources are limited. Our solution gives impetus to the improvement of both sensing data and paves the way for highly accurate neuromorphic reasoning and analysis.

eess.IV↗

arXiv2023

Transferring a molecular foundation model for polymer property predictions

Transformer-based large language models have remarkable potential to accelerate design optimization for applications such as drug development and materials discovery. Self-supervised pretraining of transformer models requires large-scale datasets, which are often sparsely populated in topical areas such as polymer science. State-of-the-art approaches for polymers conduct data augmentation to generate additional samples but unavoidably incurs extra computational costs. In contrast, large-scale open-source datasets are available for small molecules and provide a potential solution to data scarcity through transfer learning. In this work, we show that using transformers pretrained on small molecules and fine-tuned on polymer properties achieve comparable accuracy to those trained on augmented polymer datasets for a series of benchmark prediction tasks.

cs.LG↗

arXiv2023

DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies

In the upcoming decade, deep learning may revolutionize the natural sciences, enhancing our capacity to model and predict natural occurrences. This could herald a new era of scientific exploration, bringing significant advancements across sectors from drug development to renewable energy. To answer this call, we present DeepSpeed4Science initiative (deepspeed4science.ai) which aims to build unique capabilities through AI system technology innovations to help domain experts to unlock today's biggest science mysteries. By leveraging DeepSpeed's current technology pillars (training, inference and compression) as base technology enablers, DeepSpeed4Science will create a new set of AI system technologies tailored for accelerating scientific discoveries by addressing their unique complexity beyond the common technical approaches used for accelerating generic large language models (LLMs). In this paper, we showcase the early progress we made with DeepSpeed4Science in addressing two of the critical system challenges in structural biology research.

cs.AI↗

arXiv2023

Normality Learning-based Graph Anomaly Detection via Multi-Scale Contrastive Learning

Graph anomaly detection (GAD) has attracted increasing attention in machine learning and data mining. Recent works have mainly focused on how to capture richer information to improve the quality of node embeddings for GAD. Despite their significant advances in detection performance, there is still a relative dearth of research on the properties of the task. GAD aims to discern the anomalies that deviate from most nodes. However, the model is prone to learn the pattern of normal samples which make up the majority of samples. Meanwhile, anomalies can be easily detected when their behaviors differ from normality. Therefore, the performance can be further improved by enhancing the ability to learn the normal pattern. To this end, we propose a normality learning-based GAD framework via multi-scale contrastive learning networks (NLGAD for abbreviation). Specifically, we first initialize the model with the contrastive networks on different scales. To provide sufficient and reliable normal nodes for normality learning, we design an effective hybrid strategy for normality selection. Finally, the model is refined with the only input of reliable normal nodes and learns a more accurate estimate of normality so that anomalous nodes can be more easily distinguished. Eventually, extensive experiments on six benchmark graph datasets demonstrate the effectiveness of our normality learning-based scheme on GAD. Notably, the proposed algorithm improves the detection performance (up to 5.89% AUC gain) compared with the state-of-the-art methods. The source code is released at https://github.com/FelixDJC/NLGAD.

cs.LG↗

arXiv2023

Efficient Multi-View Graph Clustering with Local and Global Structure Preservation

Anchor-based multi-view graph clustering (AMVGC) has received abundant attention owing to its high efficiency and the capability to capture complementary structural information across multiple views. Intuitively, a high-quality anchor graph plays an essential role in the success of AMVGC. However, the existing AMVGC methods only consider single-structure information, i.e., local or global structure, which provides insufficient information for the learning task. To be specific, the over-scattered global structure leads to learned anchors failing to depict the cluster partition well. In contrast, the local structure with an improper similarity measure results in potentially inaccurate anchor assignment, ultimately leading to sub-optimal clustering performance. To tackle the issue, we propose a novel anchor-based multi-view graph clustering framework termed Efficient Multi-View Graph Clustering with Local and Global Structure Preservation (EMVGC-LG). Specifically, a unified framework with a theoretical guarantee is designed to capture local and global information. Besides, EMVGC-LG jointly optimizes anchor construction and graph learning to enhance the clustering quality. In addition, EMVGC-LG inherits the linear complexity of existing AMVGC methods respecting the sample number, which is time-economical and scales well with the data size. Extensive experiments demonstrate the effectiveness and efficiency of our proposed method.

cs.LG↗

arXiv2023

Random-Walk Metaball-Imaging Discrete Element Lattice Boltzmann Method for 3D Solute Transport in Fluid-Particle Systems with Complex Granular Morphologies

Solute transport in fluid-particle systems is a fundamental process in numerous scientific and engineering disciplines. The simulation of it necessitates the consideration of solid particles with intricate shapes and sizes. To address this challenge, this study proposes the Random-Walk Metaball-Imaging Discrete Element Lattice Boltzmann Method (RW-MI-DELBM). In this model, we reconstruct particle geometries with the Metaball-Imaging algorithm, capture the particle behavior using the Discrete Element Method (DEM), simulate fluid behavior by the Lattice Boltzmann Method (LBM), and represent solute behavior through the Random Walk Method (RWM). Through the integration of these techniques with specially designed boundary conditions, we achieve to simulate the solute transport in fluid-particle systems comprising complex particle morphologies. Thorough validations, including analytical soluutions and experiments, are performed to assess the robustness and accuracy of this framework. The results demonstrate that the proposed framework can accurately capture the complex dynamics of solute transport under strict mass conservation. In particular, an investigation is carried out to assess the influence of particle morphologies on solute transport in a 3D oscillator, with a focus on identifying correlations between shape features and dispersion coefficients. Notably, all selected shape features exhibited strong correlations with the dispersion coefficient, indicating the significant influence of particle shapes on transport phenomena. However, due to the complexity of the relationship and the limited number of simulations, no clear patterns could be observed. Further comprehensive analyses incorporating a broader range of shape features and varying conditions are necessary to fully comprehend their collective influence on the dispersion coefficient.

cs.CE↗

arXiv2023

PolyLM: An Open Source Polyglot Large Language Model

Large language models (LLMs) demonstrate remarkable ability to comprehend, reason, and generate following nature language instructions. However, the development of LLMs has been primarily focused on high-resource languages, such as English, thereby limiting their applicability and research in other languages. Consequently, we present PolyLM, a multilingual LLM trained on 640 billion (B) tokens, avaliable in two model sizes: 1.7B and 13B. To enhance its multilingual capabilities, we 1) integrate bilingual data into training data; and 2) adopt a curriculum learning strategy that increases the proportion of non-English data from 30% in the first stage to 60% in the final stage during pre-training. Further, we propose a multilingual self-instruct method which automatically generates 132.7K diverse multilingual instructions for model fine-tuning. To assess the model's performance, we collect several existing multilingual tasks, including multilingual understanding, question answering, generation, and translation. Extensive experiments show that PolyLM surpasses other open-source models such as LLaMA and BLOOM on multilingual tasks while maintaining comparable performance in English. Our models, alone with the instruction data and multilingual benchmark, are available at: \url{https://modelscope.cn/models/damo/nlp_polylm_13b_text_generation}.

cs.CL↗

arXiv2023

Anticipated BSDEs driven by fractional Brownian motion with time-delayed generator

This paper discusses a new type of anticipated backward stochastic differential equation with a time-delayed generator (DABSDEs, for short) driven by fractional Brownian motion, also known as fractional BSDEs, with Hurst parameter $H\in(1/2,1)$, which extends the results of the anticipated backward stochastic differential equation to the case of the drive is fractional Brownian motion instead of a standard Brownian motion and in which the generator considers not only the present and future times but also the past time. By using the fixed point theorem, we will demonstrate the existence and uniqueness of the solutions to these equations. Moreover, we shall establish a comparison theorem for the solutions.

math.PR↗

arXiv2023

A spectral method for a Fokker-Planck equation in neuroscience with applications in neural networks with learning rules

In this work, we consider the Fokker-Planck equation of the Nonlinear Noisy Leaky Integrate-and-Fire (NNLIF) model for neuron networks. Due to the firing events of neurons at the microscopic level, this Fokker-Planck equation contains dynamic boundary conditions involving specific internal points. To efficiently solve this problem and explore the properties of the unknown, we construct a flexible numerical scheme for the Fokker-Planck equation in the framework of spectral methods that can accurately handle the dynamic boundary condition. This numerical scheme is stable with suitable choices of test function spaces, and asymptotic preserving, and it is easily extendable to variant models with multiple time scales. We also present extensive numerical examples to verify the scheme properties, including order of convergence and time efficiency, and explore unique properties of the model, including blow-up phenomena for the NNLIF model and learning and discriminative properties for the NNLIF model with learning rules.

math.NA↗

arXiv2023

Characterization and Generation of 3D Realistic Geological Particles with Metaball Descriptor based on X-Ray Computed Tomography

The morphology of geological particles is crucial in determining its granular characteristics and assembly responses. In this paper, Metaball-function based solutions are proposed for morphological characterization and generation of three-dimensional realistic particles according to the X-ray Computed Tomography (XRCT) images. For characterization, we develop a geometric-based Metaball-Imaging algorithm. This algorithm can capture the main contour of parental particles with a series of non-overlapping spheres and refine surface-texture details through gradient search. Four types of particles, hundreds of samples, are applied for evaluations. The result shows good matches on key morphological indicators(i.e., volume, surface area, sphericity, circularity, corey-shape factor, nominal diameter and surface-equivalent-sphere diameter), confirming its characterization precision. For generation, we propose the Metaball Variational Autoencoder. Assisted by deep neural networks, this method can generate 3D particles in Metaball form, while retaining coessential morphological features with parental particles. Additionally, this method allows for control over the generated shapes through an arithmetic pattern, enabling the generation of particles with specific shapes. Two sets of XRCT images different in sample number and geometric features are chosen as parental data. On each training set, one thousand particles are generated for validations. The generation fidelity is demonstrated through comparisons of morphologies and shape-feature distributions between generated and parental particles. Examples are also provided to demonstrate controllability on the generated shapes. With Metaball-based simulations frameworks previously proposed by the authors, these methods have the potential to provide valuable insights into the properties and behavior of actual geological particles.

eess.IV↗

arXiv2022

PigV$^2$: Monitoring Pig Vital Signs through Ground Vibrations Induced by Heartbeat and Respiration

Pig vital sign monitoring (e.g., estimating the heart rate (HR) and respiratory rate (RR)) is essential to understand the stress level of the sow and detect the onset of parturition. It helps to maximize peri-natal survival and improve animal well-being in swine production. The existing approach mainly relies on manual measurement, which is labor-intensive and only provides a few points of information. Other sensing modalities such as wearables and cameras are developed to enable more continuous measurement, but are still limited due to animal discomfort, data transfer, and storage challenges. In this paper, we introduce PigV$^2$, the first system to monitor pig heart rate and respiratory rate through ground vibrations. Our approach leverages the insight that both heartbeat and respiration generate ground vibrations when the sow is lying on the floor. We infer vital information by sensing and analyzing these vibrations. The main challenge in developing PigV$^2$ is the overlap of vital- and non-vital-related information in the vibration signals, including pig movements, pig postures, pig-to-sensor distances, and so on. To address this issue, we first characterize their effects, extract their current status, and then reduce their impact by adaptively interpolating vital rates over multiple sensors. PigV$^2$ is evaluated through a real-world deployment with 30 pigs. It has 3.4% and 8.3% average errors in monitoring the HR and RR of the sows, respectively.

eess.SP↗

arXiv2022

Optimized architectures for universal quantum state transformations using photonic path and polarization

An arbitrary lossless transformation in high-dimensional quantum space can be decomposed into elementary operations which are easy to implement, and an effective decomposition algorithm is important for constructing high-dimensional systems. Here, we present two optimized architectures to effectively realize an arbitrary unitary transformation by using the photonic path and polarization based on the existing decomposition algorithm. In the first architecture, the number of required interferometers is reduced by half compared with previous works. In the second architecture, by using the high-dimensional X gate, all the elementary operations are transferred to the operations which act locally on the photonic polarization in the same path. Such an architecture could be of significance in polarization-based applications. Both architectures maintain the symmetric layout. Our work facilitates the optical implementation of high-dimensional transformations and could have potential applications in high-dimensional quantum computation and quantum communication.

quant-ph↗

arXiv2022

Graph Anomaly Detection via Multi-Scale Contrastive Learning Networks with Augmented View

Graph anomaly detection (GAD) is a vital task in graph-based machine learning and has been widely applied in many real-world applications. The primary goal of GAD is to capture anomalous nodes from graph datasets, which evidently deviate from the majority of nodes. Recent methods have paid attention to various scales of contrastive strategies for GAD, i.e., node-subgraph and node-node contrasts. However, they neglect the subgraph-subgraph comparison information which the normal and abnormal subgraph pairs behave differently in terms of embeddings and structures in GAD, resulting in sub-optimal task performance. In this paper, we fulfill the above idea in the proposed multi-view multi-scale contrastive learning framework with subgraph-subgraph contrast for the first practice. To be specific, we regard the original input graph as the first view and generate the second view by graph augmentation with edge modifications. With the guidance of maximizing the similarity of the subgraph pairs, the proposed subgraph-subgraph contrast contributes to more robust subgraph embeddings despite of the structure variation. Moreover, the introduced subgraph-subgraph contrast cooperates well with the widely-adopted node-subgraph and node-node contrastive counterparts for mutual GAD performance promotions. Besides, we also conduct sufficient experiments to investigate the impact of different graph augmentation approaches on detection performance. The comprehensive experimental results well demonstrate the superiority of our method compared with the state-of-the-art approaches and the effectiveness of the multi-view subgraph pair contrastive strategy for the GAD task.

cs.LG↗

arXiv2022

Competency-Aware Neural Machine Translation: Can Machine Translation Know its Own Translation Quality?

Neural machine translation (NMT) is often criticized for failures that happen without awareness. The lack of competency awareness makes NMT untrustworthy. This is in sharp contrast to human translators who give feedback or conduct further investigations whenever they are in doubt about predictions. To fill this gap, we propose a novel competency-aware NMT by extending conventional NMT with a self-estimator, offering abilities to translate a source sentence and estimate its competency. The self-estimator encodes the information of the decoding procedure and then examines whether it can reconstruct the original semantics of the source sentence. Experimental results on four translation tasks demonstrate that the proposed method not only carries out translation tasks intact but also delivers outstanding performance on quality estimation. Without depending on any reference or annotated data typically required by state-of-the-art metric and quality estimation methods, our model yields an even higher correlation with human quality judgments than a variety of aforementioned methods, such as BLEURT, COMET, and BERTScore. Quantitative and qualitative analyses show better robustness of competency awareness in our model.

cs.CL↗