arXiv subjects

Bo Peng

Publications and source records attributed to Bo Peng.

At least 217 records · Page 12Linked to original sources

arXiv2024

How to Evaluate Semantic Communications for Images with ViTScore Metric?

Semantic communications (SC) have been expected to be a new paradigm shifting to catalyze the next generation communication, whose main concerns shift from accurate bit transmission to effective semantic information exchange in communications. However, the previous and widely-used metrics for images are not applicable to evaluate the image semantic similarity in SC. Classical metrics to measure the similarity between two images usually rely on the pixel level or the structural level, such as the PSNR and the MS-SSIM. Straightforwardly using some tailored metrics based on deep-learning methods in CV community, such as the LPIPS, is infeasible for SC. To tackle this, inspired by BERTScore in NLP community, we propose a novel metric for evaluating image semantic similarity, named Vision Transformer Score (ViTScore). We prove theoretically that ViTScore has 3 important properties, including symmetry, boundedness, and normalization, which make ViTScore convenient and intuitive for image measurement. To evaluate the performance of ViTScore, we compare ViTScore with 3 typical metrics (PSNR, MS-SSIM, and LPIPS) through 4 classes of experiments: (i) correlation with BERTScore through evaluation of image caption downstream CV task, (ii) evaluation in classical image communications, (iii) evaluation in image semantic communication systems, and (iv) evaluation in image semantic communication systems with semantic attack. Experimental results demonstrate that ViTScore is robust and efficient in evaluating the semantic similarity of images. Particularly, ViTScore outperforms the other 3 typical metrics in evaluating the image semantic changes by semantic attack, such as image inverse with Generative Adversarial Networks (GANs). This indicates that ViTScore is an effective performance metric when deployed in SC scenarios.

cs.CV↗

arXiv2024

Counterfactual Explanations for Face Forgery Detection via Adversarial Removal of Artifacts

Highly realistic AI generated face forgeries known as deepfakes have raised serious social concerns. Although DNN-based face forgery detection models have achieved good performance, they are vulnerable to latest generative methods that have less forgery traces and adversarial attacks. This limitation of generalization and robustness hinders the credibility of detection results and requires more explanations. In this work, we provide counterfactual explanations for face forgery detection from an artifact removal perspective. Specifically, we first invert the forgery images into the StyleGAN latent space, and then adversarially optimize their latent representations with the discrimination supervision from the target detection model. We verify the effectiveness of the proposed explanations from two aspects: (1) Counterfactual Trace Visualization: the enhanced forgery images are useful to reveal artifacts by visually contrasting the original images and two different visualization methods; (2) Transferable Adversarial Attacks: the adversarial forgery images generated by attacking the detection model are able to mislead other detection models, implying the removed artifacts are general. Extensive experiments demonstrate that our method achieves over 90% attack success rate and superior attack transferability. Compared with naive adversarial noise methods, our method adopts both generative and discriminative model priors, and optimize the latent representations in a synthesis-by-analysis way, which forces the search of counterfactual explanations on the natural face manifold. Thus, more general counterfactual traces can be found and better adversarial attack transferability can be achieved.

cs.CV↗

arXiv2024

NeRF2Points: Large-Scale Point Cloud Generation From Street Views' Radiance Field Optimization

Neural Radiance Fields (NeRF) have emerged as a paradigm-shifting methodology for the photorealistic rendering of objects and environments, enabling the synthesis of novel viewpoints with remarkable fidelity. This is accomplished through the strategic utilization of object-centric camera poses characterized by significant inter-frame overlap. This paper explores a compelling, alternative utility of NeRF: the derivation of point clouds from aggregated urban landscape imagery. The transmutation of street-view data into point clouds is fraught with complexities, attributable to a nexus of interdependent variables. First, high-quality point cloud generation hinges on precise camera poses, yet many datasets suffer from inaccuracies in pose metadata. Also, the standard approach of NeRF is ill-suited for the distinct characteristics of street-view data from autonomous vehicles in vast, open settings. Autonomous vehicle cameras often record with limited overlap, leading to blurring, artifacts, and compromised pavement representation in NeRF-based point clouds. In this paper, we present NeRF2Points, a tailored NeRF variant for urban point cloud synthesis, notable for its high-quality output from RGB inputs alone. Our paper is supported by a bespoke, high-resolution 20-kilometer urban street dataset, designed for point cloud generation and evaluation. NeRF2Points adeptly navigates the inherent challenges of NeRF-based point cloud synthesis through the implementation of the following strategic innovations: (1) Integration of Weighted Iterative Geometric Optimization (WIGO) and Structure from Motion (SfM) for enhanced camera pose accuracy, elevating street-view data precision. (2) Layered Perception and Integrated Modeling (LPiM) is designed for distinct radiance field modeling in urban environments, resulting in coherent point cloud representations.

cs.CV↗

arXiv2024

Derivative-free tree optimization for complex systems

A tremendous range of design tasks in materials, physics, and biology can be formulated as finding the optimum of an objective function depending on many parameters without knowing its closed-form expression or the derivative. Traditional derivative-free optimization techniques often rely on strong assumptions about objective functions, thereby failing at optimizing non-convex systems beyond 100 dimensions. Here, we present a tree search method for derivative-free optimization that enables accelerated optimal design of high-dimensional complex systems. Specifically, we introduce stochastic tree expansion, dynamic upper confidence bound, and short-range backpropagation mechanism to evade local optimum, iteratively approximating the global optimum using machine learning models. This development effectively confronts the dimensionally challenging problems, achieving convergence to global optima across various benchmark functions up to 2,000 dimensions, surpassing the existing methods by 10- to 20-fold. Our method demonstrates wide applicability to a wide range of real-world complex systems spanning materials, physics, and biology, considerably outperforming state-of-the-art algorithms. This enables efficient autonomous knowledge discovery and facilitates self-driving virtual laboratories. Although we focus on problems within the realm of natural science, the advancements in optimization techniques achieved herein are applicable to a broader spectrum of challenges across all quantitative disciplines.

cs.LG↗

arXiv2024

Unveiling the origin of unconventional moire ferroelectricity

Interfacial ferroelectricity emerges in heterostructures consisting of nonpolar van der Waals (vdW) layers, greatly expanding the scope of two dimensional ferroelectrics. In particular, the unconventional moire ferroelectricity observed in bilayer graphene/boron nitride (BN) heterostructures, exhibits promising functionalities with topological current, superconductivity and synaptic responses. However, the debate about its mechanism - correlation driven charge transfer between two graphene layers - limits device reproducibility and hence large-scale production. Here by designing a single-layer graphene encapsulated by lattice-mismatched WSe2, we identify the ferroelectricity as stemming from - instead of graphene moire bands - the particular BN, where interfacial sliding ferroelectricity must play a role. With similar structures, multilayer twisted MoS2 is found to reproduce the ferroelectricity. The key is a conductive moire ferroelectric, where the screened gate and the pinned domain wall together result in unchanged electronic states, i.e. anomalous screening. The intimate connection to interfacial sliding ferroelectricity thus provides advantages of diverse choices of constituent materials and robust polarization switching while preserving the unique anomalous screening, paving the way to reproducible and reliable memory-based devices in artificial intelligence.

cond-mat.mtrl-sci↗

arXiv2024

Radiometric propulsion: Advancing with the order-of-magnitude enhancement through graphene aerogel-coated vanes

Radiometer is a light-induced aerodynamic propulsive device under the rarefied gas environment, which holds great potential for the next-gen near-space flight. However, its practical applications are hindered by the weak propulsion forces on the conventional radiometer vanes. Herein, this material-aerodynamics cross-disciplinary study develops novel radiometer vanes with graphene aerogel coatings, which for the first time realize an order of magnitude enhancement in radiometric propulsion. The improvement is manifested as up to 29.7 times faster rotation speed at a low pressure of 0.2 Pa, 13.8 times faster at the pressure (1.5 Pa) with maximum speeds, and 4 orders of magnitude broader operating pressure range (10E-4 - 10E2 Pa). Direct Simulation Monte Carlo calculations reveal that the outstanding performance is ascribed to the improved temperature gradient and gas-solid momentum transfer efficiency tailored by surface porous microstructures. Moreover, we demonstrate a stable and long-term levitation prototype under both 1 sun irradiation and a rarefied gas environment.

physics.app-ph↗

arXiv2024

Artifact Feature Purification for Cross-domain Detection of AI-generated Images

In the era of AIGC, the fast development of visual content generation technologies, such as diffusion models, bring potential security risks to our society. Existing generated image detection methods suffer from performance drop when faced with out-of-domain generators and image scenes. To relieve this problem, we propose Artifact Purification Network (APN) to facilitate the artifact extraction from generated images through the explicit and implicit purification processes. For the explicit one, a suspicious frequency-band proposal method and a spatial feature decomposition method are proposed to extract artifact-related features. For the implicit one, a training strategy based on mutual information estimation is proposed to further purify the artifact-related features. Experiments show that for cross-generator detection, the average accuracy of APN is 5.6% ~ 16.4% higher than the previous 10 methods on GenImage dataset and 1.7% ~ 50.1% on DiffusionForensics dataset. For cross-scene detection, APN maintains its high performance. Via visualization analysis, we find that the proposed method extracts flexible forgery patterns and condenses the forgery information diluted in irrelevant features. We also find that the artifact features APN focuses on across generators and scenes are global and diverse. The code will be available on GitHub.

cs.CV↗

arXiv2024

Photo-induced electronic and spin topological phase transitions in monolayer bismuth

Ultrathin bismuth exhibits rich physics including strong spin-orbit coupling, ferroelectricity, nontrivial topology, and light-induced structural dynamics. We use \textit{ab initio} calculations to show that light can induce structural transitions to four transient phases in bismuth monolayers. These light-induced phases exhibit nontrivial topological character, which we illustrate using the recently introduced concept of spin bands and spin-resolved Wilson loops. Specifically, we find that the topology changes via the closing of the electron and spin band gaps during photo-induced structural phase transitions, leading to distinct edge states. Our study provides strategies to tailor electronic and spin topology via ultrafast control of photo-excited carriers and associated structural dynamics.

cond-mat.mtrl-sci↗

arXiv2024

Observation of unconventional van der Waals multiferroics near room temperature

The search for two-dimensional (2D) van der Waals (vdW) multiferroics is an exciting yet challenging endeavor. Room-temperature 2D vdW few-layer multiferroic is a much bigger insurmountable obstacle. Here we report the discovery of an unconventional 2D vdW multiferroic with out-of-plane ferroelectric polarization and long-range magnetic orders in trilayer NiI2 device from 10 K to 295 K. The evolutions of magnetic domains with magnetic field, and the evolutions between ferroelectric and antiferroelectric phase have been unambiguously observed. More significantly, we realize a robust mutual control of magnetism and ferroelectricity at room temperature. The magnetic domains are manipulated by a small voltage ranging from 1 V to 6 V at 0 T and 295 K. This work opens opportunities for exploring multiferroic physics at the limit of few atomic layers.

cond-mat.mtrl-sci↗

arXiv2024

Picasso: Memory-Efficient Graph Coloring Using Palettes With Applications in Quantum Computing

A coloring of a graph is an assignment of colors to vertices such that no two neighboring vertices have the same color. The need for memory-efficient coloring algorithms is motivated by their application in computing clique partitions of graphs arising in quantum computations where the objective is to map a large set of Pauli strings into a compact set of unitaries. We present Picasso, a randomized memory-efficient iterative parallel graph coloring algorithm with theoretical sublinear space guarantees under practical assumptions. The parameters of our algorithm provide a trade-off between coloring quality and resource consumption. To assist the user, we also propose a machine learning model to predict the coloring algorithm's parameters considering these trade-offs. We provide a sequential and a parallel implementation of the proposed algorithm. We perform an experimental evaluation on a 64-core AMD CPU equipped with 512 GB of memory and an Nvidia A100 GPU with 40GB of memory. For a small dataset where existing coloring algorithms can be executed within the 512 GB memory budget, we show up to 68x memory savings. On massive datasets we demonstrate that GPU-accelerated Picasso can process inputs with 49.5x more Pauli strings (vertex set in our graph) and 2,478x more edges than state-of-the-art parallel approaches.

cs.DC↗

arXiv2024

Towards Assessing the Synthetic-to-Measured Adversarial Vulnerability of SAR ATR

Recently, there has been increasing concern about the vulnerability of deep neural network (DNN)-based synthetic aperture radar (SAR) automatic target recognition (ATR) to adversarial attacks, where a DNN could be easily deceived by clean input with imperceptible but aggressive perturbations. This paper studies the synthetic-to-measured (S2M) transfer setting, where an attacker generates adversarial perturbation based solely on synthetic data and transfers it against victim models trained with measured data. Compared with the current measured-to-measured (M2M) transfer setting, our approach does not need direct access to the victim model or the measured SAR data. We also propose the transferability estimation attack (TEA) to uncover the adversarial risks in this more challenging and practical scenario. The TEA makes full use of the limited similarity between the synthetic and measured data pairs for blind estimation and optimization of S2M transferability, leading to feasible surrogate model enhancement without mastering the victim model and data. Comprehensive evaluations based on the publicly available synthetic and measured paired labeled experiment (SAMPLE) dataset demonstrate that the TEA outperforms state-of-the-art methods and can significantly enhance various attack algorithms in computer vision and remote sensing applications. Codes and data are available at https://github.com/scenarri/S2M-TEA.

cs.CV↗

arXiv2024

A Uniformly Selected Sample of Low-mass Black Holes in Seyfert 1 Galaxies. III. Radio sources from the SKA pathfinders and beyond

Occupying the intermediate-mass regime of the accretion--jet parameter space, radio continuum emission from active galactic nuclei with black hole mass M_BH <~ 10^6 Msun (low-mass AGNs) is a valuable probe to the physics of relativistic jets. Yet the number of low-mass AGNs with radio detection is rather limited so far (~ 40 in total). In this work we make two efforts to search for radio counterparts for the largest sample of optically selected low-mass AGNs. First, we collect counterparts from the recent data releases of SKA pathfinders such as LOFAR Two-metre Sky Survey (LoTSS). Additionally, we deeply mine in Faint Images of the Radio Sky at Twenty-Centimeters (FIRST), fitting the FIRST images of the optical AGNs with an elaborate procedure optimized to detect faint radio sources. We have obtained 151 radio sources (mainly from the SKA pathfinders), including 102 new reliable sources (S/N >= 5) and 23 new candidates (3.5 <= S/N < 5). The majority of these new sources (119 of 125) have flux densities lower than the threshold of the official FIRST catalog. The new sources have rest-frame 20 cm power (P_20cm) from 1.98 x 10^20 to 1.29 x 10^23 W/Hz. For low-z Seyfert galaxies P_20cm correlates with M_BH intrinsically and positively, yet only marginally with Eddington ratio L/L_EDD. In terms of the logN--logS relation for the expanding Universe, the limiting flux density for the completeness of our LoTSS sources turns out to be 0.45 mJy at 1.4 GHz; i.e., complete to such a flux-density level that is four times deeper than the official FIRST catalog.

astro-ph.GA↗

arXiv2024

URHand: Universal Relightable Hands

Existing photorealistic relightable hand models require extensive identity-specific observations in different views, poses, and illuminations, and face challenges in generalizing to natural illuminations and novel identities. To bridge this gap, we present URHand, the first universal relightable hand model that generalizes across viewpoints, poses, illuminations, and identities. Our model allows few-shot personalization using images captured with a mobile phone, and is ready to be photorealistically rendered under novel illuminations. To simplify the personalization process while retaining photorealism, we build a powerful universal relightable prior based on neural relighting from multi-view images of hands captured in a light stage with hundreds of identities. The key challenge is scaling the cross-identity training while maintaining personalized fidelity and sharp details without compromising generalization under natural illuminations. To this end, we propose a spatially varying linear lighting model as the neural renderer that takes physics-inspired shading as input feature. By removing non-linear activations and bias, our specifically designed lighting model explicitly keeps the linearity of light transport. This enables single-stage training from light-stage data while generalizing to real-time rendering under arbitrary continuous illuminations across diverse identities. In addition, we introduce the joint learning of a physically based model and our neural relighting model, which further improves fidelity and generalization. Extensive experiments show that our approach achieves superior performance over existing methods in terms of both quality and generalizability. We also demonstrate quick personalization of URHand from a short phone scan of an unseen identity.

cs.CV↗

arXiv2023

Boosting photocatalytic water splitting of polymeric C$_{60}$ by reduced dimensionality from 2D monolayer to 1D chain

Recent synthesis of monolayer fullerene networks [$Nature$ 606, 507 (2022)] provides new opportunities for photovoltaics and photocatalysis because of their versatile crystal structures for further tailoring of electronic, optical and chemical function. To shed light on the structural aspects of photocatalytic water splitting performance of fullerene nanomaterials, we compare the photocatalytic properties of individual polymeric fullerene chains and monolayer fullerene networks from first principles calculations. It is found that the photocatalytic efficiency can be further optimised by reducing dimensionality from 2D to 1D. The conduction band edge of the polymeric C$_{60}$ chain provides a much higher external potential for the hydrogen reduction reaction than its monolayer counterparts over a wider range of pH values, and the surface active sites in the 1D chain are two times more than those in the 2D networks from a thermodynamic perspective. These observations render the 1D fullerene polymer a more promising candidate as a photocatalyst for the hydrogen evolution reaction than monolayer fullerene networks.

cond-mat.mtrl-sci↗

arXiv2023

Integrating Subsystem Embedding Subalgebras and Coupled Cluster Green's Function: A Theoretical Foundation for Quantum Embedding in Excitation Manifold

In this study, we introduce a novel approach to coupled-cluster Green's function (CCGF) embedding by seamlessly integrating conventional CCGF theory with the state-of-the-art sub-system embedding sub-algebras coupled cluster (SES-CC) formalism. This integration focuses primarily on delineating the characteristics of the sub-system and the corresponding segments of the Green's function, defined explicitly by active orbitals. Crucially, our work involves the adaptation of the SES-CC paradigm, addressing the left eigenvalue problem through a distinct form of Hamiltonian similarity transformation. This advancement not only facilitates a comprehensive representation of the interaction between the embedded sub-system and its surrounding environment but also paves the way for the quantum mechanical description of multiple embedded domains, particularly by employing the emergent quantum flow algorithms. Our theoretical underpinnings further set the stage for a generalization to multiple embedded sub-systems. This expansion holds significant promise for the exploration and application of non-equilibrium quantum systems, enhancing the understanding of system-environment interactions. In doing so, the research underscores the potential of SES-CC embedding within the realm of quantum computations and multi-scale simulations, promising a good balance between accuracy and computational efficiency.

quant-ph↗

arXiv2023

RS-Corrector: Correcting the Racial Stereotypes in Latent Diffusion Models

Recent text-conditioned image generation models have demonstrated an exceptional capacity to produce diverse and creative imagery with high visual quality. However, when pre-trained on billion-sized datasets randomly collected from the Internet, where potential biased human preferences exist, these models tend to produce images with common and recurring stereotypes, particularly for certain racial groups. In this paper, we conduct an initial analysis of the publicly available Stable Diffusion model and its derivatives, highlighting the presence of racial stereotypes. These models often generate distorted or biased images for certain racial groups, emphasizing stereotypical characteristics. To address these issues, we propose a framework called "RS-Corrector", designed to establish an anti-stereotypical preference in the latent space and update the latent code for refined generated results. The correction process occurs during the inference stage without requiring fine-tuning of the original model. Extensive empirical evaluations demonstrate that the introduced \themodel effectively corrects the racial stereotypes of the well-trained Stable Diffusion model while leaving the original model unchanged.

cs.CV↗

arXiv2023

Qutrit Circuits and Algebraic Relations: A Pathway to Efficient Spin-1 Hamiltonian Simulation

Quantum information processing has witnessed significant advancements through the application of qubit-based techniques within universal gate sets. Recently, exploration beyond the qubit paradigm to $d$-dimensional quantum units or qudits has opened new avenues for improving computational efficiency. This paper delves into the qudit-based approach, particularly addressing the challenges presented in the high-fidelity implementation of qudit-based circuits due to increased complexity. As an innovative approach towards enhancing qudit circuit fidelity, we explore algebraic relations, such as the Yang-Baxter-like turnover equation, that may enable circuit compression and optimization. The paper introduces the turnover relation for the three-qutrit time propagator and its potential use in reducing circuit depth. We further investigate whether this relation can be generalized for higher-dimensional quantum circuits, including a focused study on the one-dimensional spin-1 Heisenberg model. Our work outlines both rigorous and numerically efficient approaches to potentially achieve this generalization, providing a foundation for further explorations in the field of qudit-based quantum computing.

quant-ph↗

arXiv2023

Learning Dense Correspondence for NeRF-Based Face Reenactment

Face reenactment is challenging due to the need to establish dense correspondence between various face representations for motion transfer. Recent studies have utilized Neural Radiance Field (NeRF) as fundamental representation, which further enhanced the performance of multi-view face reenactment in photo-realism and 3D consistency. However, establishing dense correspondence between different face NeRFs is non-trivial, because implicit representations lack ground-truth correspondence annotations like mesh-based 3D parametric models (e.g., 3DMM) with index-aligned vertexes. Although aligning 3DMM space with NeRF-based face representations can realize motion control, it is sub-optimal for their limited face-only modeling and low identity fidelity. Therefore, we are inspired to ask: Can we learn the dense correspondence between different NeRF-based face representations without a 3D parametric model prior? To address this challenge, we propose a novel framework, which adopts tri-planes as fundamental NeRF representation and decomposes face tri-planes into three components: canonical tri-planes, identity deformations, and motion. In terms of motion control, our key contribution is proposing a Plane Dictionary (PlaneDict) module, which efficiently maps the motion conditions to a linear weighted addition of learnable orthogonal plane bases. To the best of our knowledge, our framework is the first method that achieves one-shot multi-view face reenactment without a 3D parametric model prior. Extensive experiments demonstrate that we produce better results in fine-grained motion control and identity preservation than previous methods.

cs.CV↗