arXiv subjects

Hongyu Sun

Publications and source records attributed to Hongyu Sun.

30 records · Page 2Linked to original sources

arXiv2024

Parameter-efficient Prompt Learning for 3D Point Cloud Understanding

This paper presents a parameter-efficient prompt tuning method, named PPT, to adapt a large multi-modal model for 3D point cloud understanding. Existing strategies are quite expensive in computation and storage, and depend on time-consuming prompt engineering. We address the problems from three aspects. Firstly, a PromptLearner module is devised to replace hand-crafted prompts with learnable contexts to automate the prompt tuning process. Then, we lock the pre-trained backbone instead of adopting the full fine-tuning paradigm to substantially improve the parameter efficiency. Finally, a lightweight PointAdapter module is arranged near target tasks to enhance prompt tuning for 3D point cloud understanding. Comprehensive experiments are conducted to demonstrate the superior parameter and data efficiency of the proposed method.Meanwhile, we obtain new records on 4 public datasets and multiple 3D tasks, i.e., point cloud recognition, few-shot learning, and part segmentation. The implementation is available at https://github.com/auniquesun/PPT.

cs.CV↗

arXiv2023

Phase Neural Operator for Multi-Station Picking of Seismic Arrivals

Seismic wave arrival time measurements form the basis for numerous downstream applications. State-of-the-art approaches for phase picking use deep neural networks to annotate seismograms at each station independently, yet human experts annotate seismic data by examining the whole network jointly. Here, we introduce a general-purpose network-wide phase picking algorithm based on a recently developed machine learning paradigm called Neural Operator. Our model, called PhaseNO, leverages the spatio-temporal contextual information to pick phases simultaneously for any seismic network geometry. This results in superior performance over leading baseline algorithms by detecting many more earthquakes, picking more phase arrivals, while also greatly improving measurement accuracy. Following similar trends being seen across the domains of artificial intelligence, our approach provides but a glimpse of the potential gains from fully-utilizing the massive seismic datasets being collected worldwide.

physics.geo-ph↗

arXiv2023

Study on the Impacts of Hazardous Behaviors on Autonomous Vehicle Collision Rates Based on Humanoid Scenario Generation in CARLA

Testing of function safety and Safety Of The Intended Functionality (SOTIF) is important for autonomous vehicles (AVs). It is hard to test the AV's hazard response in the real world because it would involve hazards to passengers and other road users. This paper studied on virtual testing of AV on the CARLA platform and proposed a Humanoid Scenario Generation (HSG) scheme to investigate the impacts of hazardous behaviors on AV collision rates. The HSG scheme breakthrough the current limitation on the rarity and reproducibility of real scenes. By accurately capturing five prominent human driver behaviors that directly contribute to vehicle collisions in the real world, the methodology significantly enhances the realism and diversity of the simulation, as evidenced by collision rate statistics across various traffic scenarios. Thus, the modular framework allows for customization, and its seamless integration within the CARLA platform ensures compatibility with existing tools. Ultimately, the comparison results demonstrate that all vehicles that exhibited hazardous behaviors followed the predefined random speed distribution and the effectiveness of the HSG was validated by the distinct characteristics displayed by these behaviors.

cs.RO↗

arXiv2023

A Survey on Automated Program Repair Techniques

With the rapid development and large-scale popularity of program software, modern society increasingly relies on software systems. However, the problems exposed by software have also come to the fore. Software defect has become an important factor troubling developers. In this context, Automated Program Repair (APR) techniques have emerged, aiming to automatically fix software defect problems and reduce manual debugging work. In particular, benefiting from the advances in deep learning, numerous learning-based APR techniques have emerged in recent years, which also bring new opportunities for APR research. To give researchers a quick overview of APR techniques' complete development and future opportunities, we revisit the evolution of APR techniques and discuss in depth the latest advances in APR research. In this paper, the development of APR techniques is introduced in terms of four different patch generation schemes: search-based, constraint-based, template-based, and learning-based. Moreover, we propose a uniform set of criteria to review and compare each APR tool, summarize the advantages and disadvantages of APR techniques, and discuss the current state of APR development. Furthermore, we introduce the research on the related technical areas of APR that have also provided a strong motivation to advance APR development. Finally, we analyze current challenges and future directions, especially highlighting the critical opportunities that large language models bring to APR research.

cs.SE↗

arXiv2023

ViewFormer: View Set Attention for Multi-view 3D Shape Understanding

This paper presents ViewFormer, a simple yet effective model for multi-view 3d shape recognition and retrieval. We systematically investigate the existing methods for aggregating multi-view information and propose a novel ``view set" perspective, which minimizes the relation assumption about the views and releases the representation flexibility. We devise an adaptive attention model to capture pairwise and higher-order correlations of the elements in the view set. The learned multi-view correlations are aggregated into an expressive view set descriptor for recognition and retrieval. Experiments show the proposed method unleashes surprising capabilities across different tasks and datasets. For instance, with only 2 attention blocks and 4.8M learnable parameters, ViewFormer reaches 98.8% recognition accuracy on ModelNet40 for the first time, exceeding previous best method by 1.1% . On the challenging RGBD dataset, our method achieves 98.4% recognition accuracy, which is a 4.1% absolute improvement over the strongest baseline. ViewFormer also sets new records in several evaluation dimensions of 3D shape retrieval defined on the SHREC'17 benchmark.

cs.CV↗

arXiv2023

AirBirds: A Large-scale Challenging Dataset for Bird Strike Prevention in Real-world Airports

One fundamental limitation to the research of bird strike prevention is the lack of a large-scale dataset taken directly from real-world airports. Existing relevant datasets are either small in size or not dedicated for this purpose. To advance the research and practical solutions for bird strike prevention, in this paper, we present a large-scale challenging dataset AirBirds that consists of 118,312 time-series images, where a total of 409,967 bounding boxes of flying birds are manually, carefully annotated. The average size of all annotated instances is smaller than 10 pixels in 1920x1080 images. Images in the dataset are captured over 4 seasons of a whole year by a network of cameras deployed at a real-world airport, covering diverse bird species, lighting conditions and 13 meteorological scenarios. To the best of our knowledge, it is the first large-scale image dataset that directly collects flying birds in real-world airports for bird strike prevention. This dataset is publicly available at https://airbirdsdata.github.io/.

cs.CV↗

arXiv2023

ViPFormer: Efficient Vision-and-Pointcloud Transformer for Unsupervised Pointcloud Understanding

Recently, a growing number of work design unsupervised paradigms for point cloud processing to alleviate the limitation of expensive manual annotation and poor transferability of supervised methods. Among them, CrossPoint follows the contrastive learning framework and exploits image and point cloud data for unsupervised point cloud understanding. Although the promising performance is presented, the unbalanced architecture makes it unnecessarily complex and inefficient. For example, the image branch in CrossPoint is $\sim$8.3x heavier than the point cloud branch leading to higher complexity and latency. To address this problem, in this paper, we propose a lightweight Vision-and-Pointcloud Transformer (ViPFormer) to unify image and point cloud processing in a single architecture. ViPFormer learns in an unsupervised manner by optimizing intra-modal and cross-modal contrastive objectives. Then the pretrained model is transferred to various downstream tasks, including 3D shape classification and semantic segmentation. Experiments on different datasets show ViPFormer surpasses previous state-of-the-art unsupervised methods with higher accuracy, lower model complexity and runtime latency. Finally, the effectiveness of each component in ViPFormer is validated by extensive ablation studies. The implementation of the proposed method is available at https://github.com/auniquesun/ViPFormer.

cs.CV↗

arXiv2022

Cargo Ecosystem Dependency-Vulnerability Knowledge Graph Construction and Vulnerability Propagation Study

Currently, little is known about the structure of the Cargo ecosystem and the potential for vulnerability propagation. Many empirical studies generalize third-party dependency governance strategies from a single software ecosystem to other ecosystems but ignore the differences in the technical structures of different software ecosystems, making it difficult to directly generalize security governance strategies from other ecosystems to the Cargo ecosystem. To fill the gap in this area, this paper constructs a knowledge graph of dependency vulnerabilities for the Cargo ecosystem using techniques related to knowledge graphs to address this challenge. This paper is the first large-scale empirical study in a related research area to address vulnerability propagation in the Cargo ecosystem. This paper proposes a dependency-vulnerability knowledge graph parsing algorithm to determine the vulnerability propagation path and propagation range and empirically studies the characteristics of vulnerabilities in the Cargo ecosystem, the propagation range, and the factors that cause vulnerability propagation. Our research has found that the Cargo ecosystem's security vulnerabilities are primarily memory-related. 18% of the libraries affected by the vulnerability is still affected by the vulnerability in the latest version of the library. The number of versions affected by the propagation of the vulnerabilities is 19.78% in the entire Cargo ecosystem. This paper looks at the characteristics and propagation factors triggering vulnerabilities in the Cargo ecosystem. It provides some practical resolution strategies for administrators of the Cargo community, developers who use Cargo to manage third-party libraries, and library owners. This paper provides new ideas for improving the overall security of the Cargo ecosystem.

cs.CR↗

arXiv2022

Accelerating Time-Reversal Imaging with Neural Operators for Real-time Earthquake Locations

Earthquake hypocenters form the basis for a wide array of seismological analyses. Pick-based earthquake location workflows rely on the accuracy of phase pickers and may be biased when dealing with complex earthquake sequences in heterogeneous media. Time-reversal imaging of passive seismic sources with the cross-correlation imaging condition has potential for earthquake location with high accuracy and high resolution, but carries a large computational cost. Here we present an alternative deep-learning approach for earthquake location by combining the benefits of neural operators for wave propagation and time reversal imaging with multi-station waveform recordings. A U-shaped neural operator is trained to propagate seismic waves with various source time functions and thus can predict a backpropagated wavefield for each station in negligible time. These wavefields can either be stacked or correlated to locate earthquakes from the resulting source images. Compared with other waveform-based deep-learning location methods, time reversal imaging accounts for physical laws of wave propagation and is expected to achieve accurate earthquake location. We demonstrate the method with the 2D acoustic wave equation on both synthetic and field data. The results show that our method can efficiently obtain high resolution and high accuracy correlation-based time reversal imaging of earthquake sources. Moreover, our approach is adaptable to the number and geometry of seismic stations, which opens new strategies for real-time earthquake location and monitoring with dense seismic networks.

physics.geo-ph↗

arXiv2020

Deep learning for low frequency extrapolation of multicomponent data in elastic full waveform inversion

Full waveform inversion (FWI) strongly depends on an accurate starting model to succeed. This is particularly true in the elastic regime: The cycle-skipping phenomenon is more severe in elastic FWI compared to acoustic FWI, due to the short S-wave wavelength. In this paper, we extend our work on extrapolated FWI (EFWI) by proposing to synthesize the low frequencies of multi-component elastic seismic records, and use those "artificial" low frequencies to seed the frequency sweep of elastic FWI. Our solution involves deep learning: we separately train the same convolutional neural network (CNN) on two training datasets, one with vertical components and one with horizontal components of particle velocities, to extrapolate the low frequencies of elastic data. The architecture of this CNN is designed with a large receptive field, by either large convolutional kernels or dilated convolution. Numerical examples on the Marmousi2 model show that the 2-4Hz low frequency data extrapolated from band-limited data above 4Hz provide good starting models for elastic FWI of P-wave and S-wave velocities. Additionally, we study the generalization ability of the proposed neural network over different physical models. For elastic test data, collecting the training dataset by elastic simulation shows better extrapolation accuracy than acoustic simulation, i.e., a smaller generalization gap.

physics.geo-ph↗

arXiv2019

Extrapolated full waveform inversion with deep learning

The lack of low frequency information and a good initial model can seriously affect the success of full waveform inversion (FWI), due to the inherent cycle skipping problem. Computational low frequency extrapolation is in principle the most direct way to address this issue. By considering bandwidth extension as a regression problem in machine learning, we propose an architecture of convolutional neural network (CNN) to automatically extrapolate the missing low frequencies without preprocessing and post-processing steps. The bandlimited recordings are the inputs of the CNN and, in our numerical experiments, a neural network trained from enough samples can predict a reasonable approximation to the seismograms in the unobserved low frequency band, both in phase and in amplitude. The numerical experiments considered are set up on simulated P-wave data. In extrapolated FWI (EFWI), the low-wavenumber components of the model are determined from the extrapolated low frequencies, before proceeding with a frequency sweep of the bandlimited data. The proposed deep-learning method of low-frequency extrapolation shows adequate generalizability for the initialization step of EFWI. Numerical examples show that the neural network trained on several submodels of the Marmousi model is able to predict the low frequencies for the BP 2004 benchmark model. Additionally, the neural network can robustly process seismic data with uncertainties due to the existence of noise, poorly-known source wavelet, and different finite-difference scheme in the forward modeling operator. Finally, this approach is not subject to the structural limitations of other methods for bandwidth extension, and seems to offer a tantalizing solution to the problem of properly initializing FWI.

physics.geo-ph↗

arXiv2019

Local Crystal Misorientation Influences Non-Radiative Recombination

We use ultrasensitive electron backscatter diffraction (EBSD) to map the local crystal orientations, grains, and grain boundaries in CH3NH3PbI3 (MAPI) perovskite thin films. Although the true grain structure is broadly consistent with the morphology visible in scanning electron microscopy (SEM), the inverse pole figure maps taken with EBSD reveal grain structure and internal misorientation that is otherwise hidden. Local crystal misorientation is consistent with the presence of local strain which varies from one grain to the next. We acquire co-aligned confocal optical photoluminescence (PL) microscopy images on the same MAPI samples used for EBSD. We correlate optical and EBSD data, showing that PL is anticorrelated with the local grain orientation spread, suggesting that grains with higher degrees of crystalline orientational heterogeneity (local strain) exhibit more non-radiative recombination. We find that larger grains tend to have larger grain orientation spread, consistent with higher degrees of strain and non-radiative recombination.

cond-mat.mtrl-sci↗