arXiv subjects

Fei Ma

Publications and source records attributed to Fei Ma.

At least 109 records · Page 6Linked to original sources

arXiv2024

Cloud Adversarial Example Generation for Remote Sensing Image Classification

Most existing adversarial attack methods for remote sensing images merely add adversarial perturbations or patches, resulting in unnatural modifications. Clouds are common atmospheric effects in remote sensing images. Generating clouds on these images can produce adversarial examples better aligning with human perception. In this paper, we propose a Perlin noise based cloud generation attack method. Common Perlin noise based cloud generation is a random, non-optimizable process, which cannot be directly used to attack the target models. We design a Perlin Gradient Generator Network (PGGN), which takes a gradient parameter vector as input and outputs the grids of Perlin noise gradient vectors at different scales. After a series of computations based on the gradient vectors, cloud masks at corresponding scales can be produced. These cloud masks are then weighted and summed depending on a mixing coefficient vector and a scaling factor to produce the final cloud masks. The gradient vector, coefficient vector and scaling factor are collectively represented as a cloud parameter vector, transforming the cloud generation into a black-box optimization problem. The Differential Evolution (DE) algorithm is employed to solve for the optimal solution of the cloud parameter vector, achieving a query-based black-box attack. Detailed experiments confirm that this method has strong attack capabilities and achieves high query efficiency. Additionally, we analyze the transferability of the generated adversarial examples and their robustness in adversarial defense scenarios.

cs.CV↗

arXiv2024

SegTalker: Segmentation-based Talking Face Generation with Mask-guided Local Editing

Audio-driven talking face generation aims to synthesize video with lip movements synchronized to input audio. However, current generative techniques face challenges in preserving intricate regional textures (skin, teeth). To address the aforementioned challenges, we propose a novel framework called SegTalker to decouple lip movements and image textures by introducing segmentation as intermediate representation. Specifically, given the mask of image employed by a parsing network, we first leverage the speech to drive the mask and generate talking segmentation. Then we disentangle semantic regions of image into style codes using a mask-guided encoder. Ultimately, we inject the previously generated talking segmentation and style codes into a mask-guided StyleGAN to synthesize video frame. In this way, most of textures are fully preserved. Moreover, our approach can inherently achieve background separation and facilitate mask-guided facial local editing. In particular, by editing the mask and swapping the region textures from a given reference image (e.g. hair, lip, eyebrows), our approach enables facial editing seamlessly when generating talking face video. Experiments demonstrate that our proposed approach can effectively preserve texture details and generate temporally consistent video while remaining competitive in lip synchronization. Quantitative and qualitative results on the HDTF and MEAD datasets illustrate the superior performance of our method over existing methods.

cs.CV↗

arXiv2024

GaussianPU: A Hybrid 2D-3D Upsampling Framework for Enhancing Color Point Clouds via 3D Gaussian Splatting

Dense colored point clouds enhance visual perception and are of significant value in various robotic applications. However, existing learning-based point cloud upsampling methods are constrained by computational resources and batch processing strategies, which often require subdividing point clouds into smaller patches, leading to distortions that degrade perceptual quality. To address this challenge, we propose a novel 2D-3D hybrid colored point cloud upsampling framework (GaussianPU) based on 3D Gaussian Splatting (3DGS) for robotic perception. This approach leverages 3DGS to bridge 3D point clouds with their 2D rendered images in robot vision systems. A dual scale rendered image restoration network transforms sparse point cloud renderings into dense representations, which are then input into 3DGS along with precise robot camera poses and interpolated sparse point clouds to reconstruct dense 3D point clouds. We have made a series of enhancements to the vanilla 3DGS, enabling precise control over the number of points and significantly boosting the quality of the upsampled point cloud for robotic scene understanding. Our framework supports processing entire point clouds on a single consumer-grade GPU, such as the NVIDIA GeForce RTX 3090, eliminating the need for segmentation and thus producing high-quality, dense colored point clouds with millions of points for robot navigation and manipulation tasks. Extensive experimental results on generating million-level point cloud data validate the effectiveness of our method, substantially improving the quality of colored point clouds and demonstrating significant potential for applications involving large-scale point clouds in autonomous robotics and human-robot interaction scenarios.

cs.RO↗

arXiv2024

Enhancing and Accelerating Large Language Models via Instruction-Aware Contextual Compression

Large Language Models (LLMs) have garnered widespread attention due to their remarkable performance across various tasks. However, to mitigate the issue of hallucinations, LLMs often incorporate retrieval-augmented pipeline to provide them with rich external knowledge and context. Nevertheless, challenges stem from inaccurate and coarse-grained context retrieved from the retriever. Supplying irrelevant context to the LLMs can result in poorer responses, increased inference latency, and higher costs. This paper introduces a method called Instruction-Aware Contextual Compression, which filters out less informative content, thereby accelerating and enhancing the use of LLMs. The experimental results demonstrate that Instruction-Aware Contextual Compression notably reduces memory consumption and minimizes generation latency while maintaining performance levels comparable to those achieved with the use of the full context. Specifically, we achieved a 50% reduction in context-related costs, resulting in a 5% reduction in inference memory usage and a 2.2-fold increase in inference speed, with only a minor drop of 0.047 in Rouge-1. These findings suggest that our method strikes an effective balance between efficiency and performance.

cs.CL↗

arXiv2024

Learn To Learn More Precisely

Meta-learning has been extensively applied in the domains of few-shot learning and fast adaptation, achieving remarkable performance. While Meta-learning methods like Model-Agnostic Meta-Learning (MAML) and its variants provide a good set of initial parameters for the model, the model still tends to learn shortcut features, which leads to poor generalization. In this paper, we propose the formal conception of "learn to learn more precisely", which aims to make the model learn precise target knowledge from data and reduce the effect of noisy knowledge, such as background and noise. To achieve this target, we proposed a simple and effective meta-learning framework named Meta Self-Distillation(MSD) to maximize the consistency of learned knowledge, enhancing the models' ability to learn precise target knowledge. In the inner loop, MSD uses different augmented views of the same support data to update the model respectively. Then in the outer loop, MSD utilizes the same query data to optimize the consistency of learned knowledge, enhancing the model's ability to learn more precisely. Our experiment demonstrates that MSD exhibits remarkable performance in few-shot classification tasks in both standard and augmented scenarios, effectively boosting the accuracy and consistency of knowledge learned by the model.

cs.LG↗

arXiv2024

Generative Technology for Human Emotion Recognition: A Scope Review

Affective computing stands at the forefront of artificial intelligence (AI), seeking to imbue machines with the ability to comprehend and respond to human emotions. Central to this field is emotion recognition, which endeavors to identify and interpret human emotional states from different modalities, such as speech, facial images, text, and physiological signals. In recent years, important progress has been made in generative models, including Autoencoder, Generative Adversarial Network, Diffusion Model, and Large Language Model. These models, with their powerful data generation capabilities, emerge as pivotal tools in advancing emotion recognition. However, up to now, there remains a paucity of systematic efforts that review generative technology for emotion recognition. This survey aims to bridge the gaps in the existing literature by conducting a comprehensive analysis of over 320 research papers until June 2024. Specifically, this survey will firstly introduce the mathematical principles of different generative models and the commonly used datasets. Subsequently, through a taxonomy, it will provide an in-depth analysis of how generative techniques address emotion recognition based on different modalities in several aspects, including data augmentation, feature extraction, semi-supervised learning, cross-domain, etc. Finally, the review will outline future research directions, emphasizing the potential of generative models to advance the field of emotion recognition and enhance the emotional intelligence of AI systems.

cs.LG↗

arXiv2024

Assortativity in networks

The degree-degree correlation is crucial in understanding the structural properties of and dynamics occurring upon network, and is often measured by the assortativity coefficient $r$. In this paper, we first study this measure in detail and conclude that $r$ belongs to an asymmetric range $[-1,1)$ rather than the widely-cited $[-1,1]$. Among which, we verify that star is the unique tree network that achieves the lower bound of index $r$. Next, we obtain that all the resultant networks based on several widely-used kinds of edge-based iterative operations are disassortative if seed model has negative $r$, and also generate a family of growing neutral networks. Then, we propose an edge-based iterative operation to construct growing assortative network when seed is assortative, and further extend it to work well in general setting. Lastly, we establish a sufficient condition for existence of neutral tree network, accordingly, not only find out a representative of any order neutral tree network for the first time, but also are the first to create growing neutral tree networks as well. Also, we obtain $8n/9$ neutral non-tree graphs of distinct order as $n\rightarrow\infty$.

physics.soc-ph↗

arXiv2024

Monaural speech enhancement on drone via Adapter based transfer learning

Monaural Speech enhancement on drones is challenging because the ego-noise from the rotating motors and propellers leads to extremely low signal-to-noise ratios at onboard microphones. Although recent masking-based deep neural network methods excel in monaural speech enhancement, they struggle in the challenging drone noise scenario. Furthermore, existing drone noise datasets are limited, causing models to overfit. Considering the harmonic nature of drone noise, this paper proposes a frequency domain bottleneck adapter to enable transfer learning. Specifically, the adapter's parameters are trained on drone noise while retaining the parameters of the pre-trained Frequency Recurrent Convolutional Recurrent Network (FRCRN) fixed. Evaluation results demonstrate the proposed method can effectively enhance speech quality. Moreover, it is a more efficient alternative to fine-tuning models for various drone types, which typically requires substantial computational resources.

eess.AS↗

arXiv2024

Graph Set-colorings And Hypergraphs In Topological Coding

In order to make more complex number-based strings from topological coding for defending against the intelligent attacks equipped with quantum computing and providing effective protection technology for the age of quantum computing, we will introduce set-colored graphs admitting set-colorings that has been considerable cryptanalytic significance, and especially related with hypergraphs. We use the set-coloring of graphs to reflect the intersection of elements, and add other constraint requirements to express more connections between sets (as hyperedges). Since we try to find some easy and effective techniques based on graph theory for practical application, we use intersected-graphs admitting set-colorings defined on hyperedge sets to observe topological structures of hypergraphs, string-type Topcode-matrix, set-type Topcode-matrix, graph-type Topcode-matrix, hypergraph-type Topcode-matrix, matrix-type Topcode-matrix \emph{etc}. We will show that each connected graph is the intersected-graph of some hypergraph and investigate hypergraph's connectivity, colorings of hypergraphs, hypergraph homomorphism, hypernetworks, scale-free network generator, compound hypergraphs having their intersected-graphs with vertices to be hypergraphs (for high-dimensional extension diagram). Naturally, we get various graphic lattices, such as edge-coincided intersected-graph lattice, vertex-coincided intersected-graph lattice, edge-hamiltonian graphic lattice, hypergraph lattice and intersected-network lattice. Many techniques in this article can be translated into polynomial algorithms, since we are aiming to apply hypergraphs and graph set-colorings to homomorphic encryption and asymmetric cryptograph.

cs.CR↗

arXiv2024

Advancing large-scale thin-film PPLN nonlinear photonics with segmented tunable micro-heaters

Thin-film periodically poled lithium niobate (TF-PPLN) devices have recently gained prominence for efficient wavelength conversion processes in both classical and quantum applications. However, the patterning and poling of TF-PPLN devices today are mostly performed at chip scales, presenting a significant bottleneck for future large-scale nonlinear photonic systems that require the integration of multiple nonlinear components with consistent performance and low cost. Here, we take a pivotal step towards this goal by developing a wafer-scale TF-PPLN nonlinear photonic platform, leveraging ultraviolet stepper lithography and an automated poling process. To address the inhomogeneous broadening of the quasi-phase matching (QPM) spectrum induced by film thickness variations across the wafer, we propose and demonstrate segmented thermal optic tuning modules that can precisely adjust and align the QPM peak wavelengths in each section. \hl{Using the segmented micro-heaters, we show the successful realignment of inhomogeneously broadened multi-peak QPM spectra with up to 57$\%$ enhancement of conversion efficiency. We achieve a high normalized conversion efficiency of 3802$\%$W$^{-1}$cm$^{-2}$ in a 6 mm long PPLN waveguide, recovering 84$\%$ of the theoretically predicted efficiency in this device.} The advanced fabrication techniques and segmented tuning architectures presented herein pave the way for wafer-scale integration of complex functional nonlinear photonic circuits with applications in quantum information processing, precision sensing and metrology, and low-noise-figure optical signal amplification.

physics.optics↗

arXiv2024

A circular microphone array with virtual microphones based on acoustics-informed neural networks

Acoustic beamforming aims to focus acoustic signals to a specific direction and suppress undesirable interferences from other directions. Despite its flexibility and steerability, beamforming with circular microphone arrays suffers from significant performance degradation at frequencies corresponding to zeros of the Bessel functions. To conquer this constraint, baffled or concentric circular microphone arrays have been studied; however, the former needs a bulky baffle that interferes with the original sound field whereas the latter requires more microphones that increase the complexity and cost, both of which are undesirable in practical applications. To tackle this challenge, this paper proposes a circular microphone array equipped with virtual microphones, which resolves the performance degradation commonly associated with circular microphone arrays without resorting to physical modifications. The sound pressures at the virtual microphones are predicted from those measured by the physical microphones based on an acoustics-informed neural network, and then the sound pressures measured by the physical microphones and those predicted at the virtual microphones are integrated to design the beamformer. Experimental results demonstrate that the proposed approach not only eliminates the performance degradation but also suppresses spatial aliasing at high frequencies, thereby underscoring its promising potential.

eess.AS↗

arXiv2024

Sound Field Reconstruction Using a Compact Acoustics-informed Neural Network

Sound field reconstruction (SFR) augments the information of a sound field captured by a microphone array. Conventional SFR methods using basis function decomposition are straightforward and computationally efficient, but may require more microphones than needed to measure the sound field. Recent studies show that pure data-driven and learning-based methods are promising in some SFR tasks, but they are usually computationally heavy and may fail to reconstruct a physically valid sound field. This paper proposes a compact acoustics-informed neural network (AINN) method for SFR, whereby the Helmholtz equation is exploited to regularize the neural network. As opposed to pure data-driven approaches that solely rely on measured sound pressures, the integration of the Helmholtz equation improves robustness of the neural network against variations during the measurement processes and prompts the generation of physically valid reconstructions. The AINN is designed to be compact, and is able to predict not only the sound pressures but also sound pressure gradients within a spatial region of interest based on measured sound pressures along the boundary. Numerical experiments with acoustic transfer functions measured in different environments demonstrate the superiority of the AINN method over the traditional cylinder harmonic decomposition and the singular value decomposition methods.

eess.AS↗

arXiv2024

Retrosynthesis Prediction via Search in (Hyper) Graph

Predicting reactants from a specified core product stands as a fundamental challenge within organic synthesis, termed retrosynthesis prediction. Recently, semi-template-based methods and graph-edits-based methods have achieved good performance in terms of both interpretability and accuracy. However, due to their mechanisms these methods cannot predict complex reactions, e.g., reactions with multiple reaction center or attaching the same leaving group to more than one atom. In this study we propose a semi-template-based method, the \textbf{Retro}synthesis via \textbf{S}earch \textbf{i}n (Hyper) \textbf{G}raph (RetroSiG) framework to alleviate these limitations. In the proposed method, we turn the reaction center identification and the leaving group completion tasks as tasks of searching in the product molecular graph and leaving group hypergraph respectively. As a semi-template-based method RetroSiG has several advantages. First, RetroSiG is able to handle the complex reactions mentioned above by its novel search mechanism. Second, RetroSiG naturally exploits the hypergraph to model the implicit dependencies between leaving groups. Third, RetroSiG makes full use of the prior, i.e., one-hop constraint. It reduces the search space and enhances overall performance. Comprehensive experiments demonstrated that RetroSiG achieved competitive results. Furthermore, we conducted experiments to show the capability of RetroSiG in predicting complex reactions. Ablation experiments verified the efficacy of specific elements, such as the one-hop constraint and the leaving group hypergraph.

q-bio.QM↗

arXiv2024

Observation of scale-free localized states induced by non-Hermitian defects

Wave localization is a fundamental phenomenon that appears universally in both natural materials and artificial structures and plays a crucial role in understanding the various physical properties of a system. Usually, a localized state has an exponential profile with a localization length independent of the system size. Here, we experimentally demonstrate a new class of localized states called scale-free localized states, which has an unfixed localization length scaling linearly with the system size. Using circuit lattices, we observe that a non-Hermitian defect added to a Hermitian lattice induces an extensive number of states with scale-free localization. Furthermore, we demonstrate that, in a lattice with a parity-time-symmetric non-Hermitian defect, the scale-free localization emerges because of spontaneous parity-time symmetry breaking. Our results uncover a new type of localized states and extend the study of defect physics to the non-Hermitian regime.

cond-mat.mes-hall↗

arXiv2024

Your blush gives you away: detecting hidden mental states with remote photoplethysmography and thermal imaging

Multimodal emotion recognition techniques are increasingly essential for assessing mental states. Image-based methods, however, tend to focus predominantly on overt visual cues and often overlook subtler mental state changes. Psychophysiological research has demonstrated that HR and skin temperature are effective in detecting ANS activities, thereby revealing these subtle changes. However, traditional HR tools are generally more costly and less portable, while skin temperature analysis usually necessitates extensive manual processing. Advances in remote-PPG and automatic thermal ROI detection algorithms have been developed to address these issues, yet their accuracy in practical applications remains limited. This study aims to bridge this gap by integrating r-PPG with thermal imaging to enhance prediction performance. Ninety participants completed a 20-minute questionnaire to induce cognitive stress, followed by watching a film aimed at eliciting moral elevation. The results demonstrate that the combination of r-PPG and thermal imaging effectively detects emotional shifts. Using r-PPG alone, the prediction accuracy was 77% for cognitive stress and 61% for moral elevation, as determined by SVM. Thermal imaging alone achieved 79% accuracy for cognitive stress and 78% for moral elevation, utilizing a RF algorithm. An early fusion strategy of these modalities significantly improved accuracies, achieving 87% for cognitive stress and 83% for moral elevation using RF. Further analysis, which utilized statistical metrics and explainable machine learning methods including SHAP, highlighted key features and clarified the relationship between cardiac responses and facial temperature variations. Notably, it was observed that cardiovascular features derived from r-PPG models had a more pronounced influence in data fusion, despite thermal imaging's higher predictive accuracy in unimodal analysis.

cs.CY↗

arXiv2023

Top-K Pooling with Patch Contrastive Learning for Weakly-Supervised Semantic Segmentation

Weakly Supervised Semantic Segmentation (WSSS) using only image-level labels has gained significant attention due to cost-effectiveness. Recently, Vision Transformer (ViT) based methods without class activation map (CAM) have shown greater capability in generating reliable pseudo labels than previous methods using CAM. However, the current ViT-based methods utilize max pooling to select the patch with the highest prediction score to map the patch-level classification to the image-level one, which may affect the quality of pseudo labels due to the inaccurate classification of the patches. In this paper, we introduce a novel ViT-based WSSS method named top-K pooling with patch contrastive learning (TKP-PCL), which employs a top-K pooling layer to alleviate the limitations of previous max pooling selection. A patch contrastive error (PCE) is also proposed to enhance the patch embeddings to further improve the final results. The experimental results show that our approach is very efficient and outperforms other state-of-the-art WSSS methods on the PASCAL VOC 2012 dataset.

cs.CV↗

arXiv2023

Type-II Apollonian Model

The family of planar graphs is a particularly important family and models many real-world networks. In this paper, we propose a principled framework based on the widely-known Apollonian packing process to generate new planar network, i.e., Type-II Apollonian network $\mathcal{A}_{t}$. The manipulation is different from that of the typical Apollonian network, and is proceeded in terms of the iterative addition of triangle instead of vertex. As a consequence, network $\mathcal{A}_{t}$ turns out to be hamiltonian and eulerian, however, the typical Apollonian network is not. Then, we in-depth study some fundamental structural properties on network $\mathcal{A}_{t}$, and verify that network $\mathcal{A}_{t}$ is sparse like most real-world networks, has scale-free feature and small-world property, and exhibits disassortative mixing structure. Next, we design an effective algorithm for solving the problem of how to enumerate spanning trees on network $\mathcal{A}_{t}$, and derive the asymptotic solution of the spanning tree entropy, which suggests that Type-II Apollonian network is more reliable to a random removal of edges than the typical Apollonian network. Additionally, we study trapping problem on network $\mathcal{A}_{t}$, and use average trapping time as metric to show that Type-II Apollonian network $\mathcal{A}_{t}$ has better structure for fast information diffusion than the typical Apollonian network.

physics.soc-ph↗

arXiv2023

Spatial Upsampling of Head-Related Transfer Functions Using a Physics-Informed Neural Network

Head-related transfer function (HRTF) capture the information that a person uses to localize sound sources in space, and thus is crucial for creating personalized virtual acoustic experiences. However, practical HRTF measurement systems may only measure a person's HRTFs sparsely, and this necessitates HRTF upsampling. This paper proposes a physics-informed neural network (PINN) method for HRTF upsampling. The PINN exploits the Helmholtz equation, the governing equation of acoustic wave propagation, for regularizing the upsampling process. This helps the generation of physically valid upsamplings which generalize beyond the measured HRTF. Furthermore, the size (width and depth) of the PINN is set according to the Helmholtz equation and its solutions, the spherical harmonics (SHs). This makes the PINN have an appropriate level of expressive power and thus does not suffer from the over-fitting problem. Since the PINN is designed independent of any specific HRTF dataset, it offers more generalizability compared to pure data-driven methods. Numerical experiments confirm the better performance of the PINN method for HRTF upsampling in both interpolation and extrapolation scenarios in comparison with the SH method and the HRTF field method.

eess.AS↗