arXiv subjects

Peng Wang

Publications and source records attributed to Peng Wang.

At least 541 records · Page 30Linked to original sources

arXiv2023

Effects of spatial dimensionality and band tilting on the longitudinal optical conductivities in Dirac bands

We report a unified theory based on linear response, for analyzing the longitudinal optical conductivity (LOC) of materials with tilted Dirac cones. Depending on the tilt parameter $t$, the Dirac electrons have four phases: untilted, type-I, type-II, and type-III; the Dirac dispersion can be isotropic or anisotropic; the spatial dimension of the material can be one-, two-, or three-dimensions (1D, 2D and 3D). The interband LOCs and intraband LOCs in $d$ dimension (with $d\ge2$) are found to scale as $σ_{0}ω^{d-2}$ and $σ_{0}μ^{d-1}δ(ω)$, respectively, where $ω$ is the frequency and $μ$ the chemical potential. The interband LOC vanishes in 1D due to lack of extra spatial dimension. In contrast, the interband LOCs in 2D and 3D are nonvanishing and share many similar properties. A universal and robust fixed point of interband LOCs appears at $ω=2μ$ no matter $d=2$ or $d=3$, which can be intuitively understood by the geometric structures of Fermi surface and energy resonance contour. The intraband LOCs and the carrier density for 2D and 3D tilted Dirac bands are both closely related to the geometric structure of Fermi surface and the cutoff of integration. The angular dependence of LOCs is found to characterize both spatial dimensionality and band tilting and the constant asymptotic background values of LOC reflect features of Dirac bands. The LOCs in the anisotropic tilted Dirac cone can be connected to its isotropic counterpart by a ratio that consists of Fermi velocities for both 2D and 3D. Most of the findings are universal for tilted Dirac materials and hence valid for a great many Dirac materials in the spatial dimensions of physical interest.

cond-mat.mes-hall↗

arXiv2023

PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Prediction

We propose a Pose-Free Large Reconstruction Model (PF-LRM) for reconstructing a 3D object from a few unposed images even with little visual overlap, while simultaneously estimating the relative camera poses in ~1.3 seconds on a single A100 GPU. PF-LRM is a highly scalable method utilizing the self-attention blocks to exchange information between 3D object tokens and 2D image tokens; we predict a coarse point cloud for each view, and then use a differentiable Perspective-n-Point (PnP) solver to obtain camera poses. When trained on a huge amount of multi-view posed data of ~1M objects, PF-LRM shows strong cross-dataset generalization ability, and outperforms baseline methods by a large margin in terms of pose prediction accuracy and 3D reconstruction quality on various unseen evaluation datasets. We also demonstrate our model's applicability in downstream text/image-to-3D task with fast feed-forward inference. Our project website is at: https://totoro97.github.io/pf-lrm .

cs.CV↗

arXiv2023

SCL-VI: Self-supervised Context Learning for Visual Inspection of Industrial Defects

The unsupervised visual inspection of defects in industrial products poses a significant challenge due to substantial variations in product surfaces. Current unsupervised models struggle to strike a balance between detecting texture and object defects, lacking the capacity to discern latent representations and intricate features. In this paper, we present a novel self-supervised learning algorithm designed to derive an optimal encoder by tackling the renowned jigsaw puzzle. Our approach involves dividing the target image into nine patches, tasking the encoder with predicting the relative position relationships between any two patches to extract rich semantics. Subsequently, we introduce an affinity-augmentation method to accentuate differences between normal and abnormal latent representations. Leveraging the classic support vector data description algorithm yields final detection results. Experimental outcomes demonstrate that our proposed method achieves outstanding detection and segmentation performance on the widely used MVTec AD dataset, with rates of 95.8% and 96.8%, respectively, establishing a state-of-the-art benchmark for both texture and object defects. Comprehensive experimentation underscores the effectiveness of our approach in diverse industrial applications.

cs.CV↗

arXiv2023

Attribute-Aware Deep Hashing with Self-Consistency for Large-Scale Fine-Grained Image Retrieval

Our work focuses on tackling large-scale fine-grained image retrieval as ranking the images depicting the concept of interests (i.e., the same sub-category labels) highest based on the fine-grained details in the query. It is desirable to alleviate the challenges of both fine-grained nature of small inter-class variations with large intra-class variations and explosive growth of fine-grained data for such a practical task. In this paper, we propose attribute-aware hashing networks with self-consistency for generating attribute-aware hash codes to not only make the retrieval process efficient, but also establish explicit correspondences between hash codes and visual attributes. Specifically, based on the captured visual representations by attention, we develop an encoder-decoder structure network of a reconstruction task to unsupervisedly distill high-level attribute-specific vectors from the appearance-specific visual representations without attribute annotations. Our models are also equipped with a feature decorrelation constraint upon these attribute vectors to strengthen their representative abilities. Then, driven by preserving original entities' similarity, the required hash codes can be generated from these attribute-specific vectors and thus become attribute-aware. Furthermore, to combat simplicity bias in deep hashing, we consider the model design from the perspective of the self-consistency principle and propose to further enhance models' self-consistency by equipping an additional image reconstruction path. Comprehensive quantitative experiments under diverse empirical settings on six fine-grained retrieval datasets and two generic retrieval datasets show the superiority of our models over competing methods.

cs.IR↗

arXiv2023

FollowEval: A Multi-Dimensional Benchmark for Assessing the Instruction-Following Capability of Large Language Models

The effective assessment of the instruction-following ability of large language models (LLMs) is of paramount importance. A model that cannot adhere to human instructions might be not able to provide reliable and helpful responses. In pursuit of this goal, various benchmarks have been constructed to evaluate the instruction-following capacity of these models. However, these benchmarks are limited to a single language and are constructed using automated approaches, which restricts their applicability and the quality of the test examples they contain. To bridge this gap, we introduce the FollowEval benchmark in this paper. This benchmark is composed of instances in both English and Chinese, and all test examples are crafted by human experts. Furthermore, the FollowEval benchmark is designed to assess LLMs across five critical dimensions of instruction following: string manipulation, commonsense reasoning, logical reasoning, spatial reasoning, and response constraints. To enhance the complexity and present a sufficient challenge, each test example is designed to evaluate more than one dimension. We have evaluated various LLMs using the FollowEval benchmark and found that their performance significantly lags behind that of humans. This highlights the considerable room for improvement in the instruction-following ability of these models.

cs.CL↗

arXiv2023

Infrared Imaging of Magnetic Octupole Domains in Non-collinear Antiferromagnets

Magnetic structure plays a pivotal role in the functionality of antiferromagnets (AFMs), which not only can be employed to encode digital data but also yields novel phenomena. Despite its growing significance, visualizing the antiferromagnetic domain structure remains a challenge, particularly for non-collinear AFMs. Currently, the observation of magnetic domains in non-collinear antiferromagnetic materials is feasible only in Mn$_{3}$Sn, underscoring the limitations of existing techniques that necessitate distinct methods for in-plane and out-of-plane magnetic domain imaging. In this study, we present a versatile method for imaging the antiferromagnetic domain structure in a series of non-collinear antiferromagnetic materials by utilizing the anomalous Ettingshausen effect (AEE), which resolves both the magnetic octupole moments parallel and perpendicular to the sample surface. Temperature modulation due to the AEE originating from different magnetic domains is measured by the lock-in thermography, revealing distinct behaviors of octupole domains in different antiferromagnets. This work delivers an efficient technique for the visualization of magnetic domains in non-collinear AFMs, which enables comprehensive study of the magnetization process at the microscopic level and paves the way for potential advancements in applications.

cond-mat.mtrl-sci↗

arXiv2023

DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Model

We propose \textbf{DMV3D}, a novel 3D generation approach that uses a transformer-based 3D large reconstruction model to denoise multi-view diffusion. Our reconstruction model incorporates a triplane NeRF representation and can denoise noisy multi-view images via NeRF reconstruction and rendering, achieving single-stage 3D generation in $\sim$30s on single A100 GPU. We train \textbf{DMV3D} on large-scale multi-view image datasets of highly diverse objects using only image reconstruction losses, without accessing 3D assets. We demonstrate state-of-the-art results for the single-image reconstruction problem where probabilistic modeling of unseen object parts is required for generating diverse reconstructions with sharp textures. We also show high-quality text-to-3D generation results outperforming previous 3D diffusion models. Our project website is at: https://justimyhxu.github.io/projects/dmv3d/ .

cs.CV↗

arXiv2023

Beamforming Performances of Holographic Surfaces

In this paper, we investigate the beamforming performances of holographic surfaces implemented as lossless antenna arrays with less than half-wavelength spacing. We first develop a method to quantify the mutual coupling effect among the antennas in an array. The developed coupling model is general and applicable to arrays with arbitrary distribution of any type of antennas with arbitrary structure, physical size and radiation power pattern. In particular, it reduces to a neat analytical expression for arbitrarily deployed isotropic antenna arrays. We then discuss the beamforming design for holographic surfaces, and in particular provide analytical beamforming characterizations for arrays with two arbitrarily spaced isotropic antennas. Numerical results indicate that, by accounting for the mutual coupling effect between antennas, the array densification by packing more antennas in a given surface aperture can significantly enhance both the beamforming gain and spatial resolution of the system. The beamforming gain enhancement and beamwidth reduction can be several dBs higher than, and more than half of, those achieved by the conventional half-wavelength spaced antenna arrays in the same surface aperture. The gains of densification become saturated when the antenna spacing is below a critical value, and the saturated gain reduces as the surface aperture increases.

cs.IT↗

arXiv2023

Rotating wave approximation and renormalized perturbation theory

The rotating wave approximation (RWA) plays a central role in the quantum dynamics of two-level systems. We derive corrections to the RWA using the renormalization group approach to asymptotic analysis. We study both the Rabi and Jaynes-Cummings models and compare our analytical results with numerical calculations.

quant-ph↗

arXiv2023

A Directional Monitoring Approach of Sequential Incomplete Wind Power Curves with Copula-based Variational Inference

Wind turbines often work under complex conditions which result in performance degradation. Accurate performance degradation monitoring is essential to ensure the reliable operation of wind turbines and reduce the maintenance costs. Wind turbine power curve monitoring is an effective way to detect performance degradation. However, due to the intermittency and fluctuation of wind speed, the wind speed range varies at different time periods, making power curves difficult to compare. Motivated by this, we proposed copula-based variational inference framework and used it to establish a sequential incomplete wind power curve estimation algorithm. First, a monotone power curve is constructed based on copula-based variational inference and integrated spline regression model. Besides, the prior distribution of model parameters are sequentially updated. Then, a directional control chart based on a new statistic named KLdivergence factor is constructed to monitor wind turbine performance degradation. The real data of a wind farm in the east of the United Kingdom shows that the proposed method can both improve the accuracy of wind turbine power curve modeling and monitor wind turbine performance degradation more precisely and comprehensively than the existing approaches.

stat.AP↗

arXiv2023

PERF: Panoramic Neural Radiance Field from a Single Panorama

Neural Radiance Field (NeRF) has achieved substantial progress in novel view synthesis given multi-view images. Recently, some works have attempted to train a NeRF from a single image with 3D priors. They mainly focus on a limited field of view with a few occlusions, which greatly limits their scalability to real-world 360-degree panoramic scenarios with large-size occlusions. In this paper, we present PERF, a 360-degree novel view synthesis framework that trains a panoramic neural radiance field from a single panorama. Notably, PERF allows 3D roaming in a complex scene without expensive and tedious image collection. To achieve this goal, we propose a novel collaborative RGBD inpainting method and a progressive inpainting-and-erasing method to lift up a 360-degree 2D scene to a 3D scene. Specifically, we first predict a panoramic depth map as initialization given a single panorama and reconstruct visible 3D regions with volume rendering. Then we introduce a collaborative RGBD inpainting approach into a NeRF for completing RGB images and depth maps from random views, which is derived from an RGB Stable Diffusion model and a monocular depth estimator. Finally, we introduce an inpainting-and-erasing strategy to avoid inconsistent geometry between a newly-sampled view and reference views. The two components are integrated into the learning of NeRFs in a unified optimization framework and achieve promising results. Extensive experiments on Replica and a new dataset PERF-in-the-wild demonstrate the superiority of our PERF over state-of-the-art methods. Our PERF can be widely used for real-world applications, such as panorama-to-3D, text-to-3D, and 3D scene stylization applications. Project page and code are available at https://perf-project.github.io/ and https://github.com/perf-project/PeRF.

cs.CV↗

arXiv2023

Generalized Neural Collapse for a Large Number of Classes

Neural collapse provides an elegant mathematical characterization of learned last layer representations (a.k.a. features) and classifier weights in deep classification models. Such results not only provide insights but also motivate new techniques for improving practical deep models. However, most of the existing empirical and theoretical studies in neural collapse focus on the case that the number of classes is small relative to the dimension of the feature space. This paper extends neural collapse to cases where the number of classes are much larger than the dimension of feature space, which broadly occur for language models, retrieval systems, and face recognition applications. We show that the features and classifier exhibit a generalized neural collapse phenomenon, where the minimum one-vs-rest margins is maximized.We provide empirical study to verify the occurrence of generalized neural collapse in practical deep neural networks. Moreover, we provide theoretical study to show that the generalized neural collapse provably occurs under unconstrained feature model with spherical constraint, under certain technical conditions on feature dimension and number of classes.

cs.LG↗

arXiv2023

Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond

In this work, we introduce the Qwen-VL series, a set of large-scale vision-language models (LVLMs) designed to perceive and understand both texts and images. Starting from the Qwen-LM as a foundation, we endow it with visual capacity by the meticulously designed (i) visual receptor, (ii) input-output interface, (iii) 3-stage training pipeline, and (iv) multilingual multimodal cleaned corpus. Beyond the conventional image description and question-answering, we implement the grounding and text-reading ability of Qwen-VLs by aligning image-caption-box tuples. The resulting models, including Qwen-VL and Qwen-VL-Chat, set new records for generalist models under similar model scales on a broad range of visual-centric benchmarks (e.g., image captioning, question answering, visual grounding) and different settings (e.g., zero-shot, few-shot). Moreover, on real-world dialog benchmarks, our instruction-tuned Qwen-VL-Chat also demonstrates superiority compared to existing vision-language chatbots. Code, demo and models are available at https://github.com/QwenLM/Qwen-VL.

cs.CV↗

arXiv2023

Knowledge Rumination for Pre-trained Language Models

Previous studies have revealed that vanilla pre-trained language models (PLMs) lack the capacity to handle knowledge-intensive NLP tasks alone; thus, several works have attempted to integrate external knowledge into PLMs. However, despite the promising outcome, we empirically observe that PLMs may have already encoded rich knowledge in their pre-trained parameters but fail to fully utilize them when applying them to knowledge-intensive tasks. In this paper, we propose a new paradigm dubbed Knowledge Rumination to help the pre-trained language model utilize that related latent knowledge without retrieving it from the external corpus. By simply adding a prompt like "As far as I know" to the PLMs, we try to review related latent knowledge and inject them back into the model for knowledge consolidation. We apply the proposed knowledge rumination to various language models, including RoBERTa, DeBERTa, and GPT-3. Experimental results on six commonsense reasoning tasks and GLUE benchmarks demonstrate the effectiveness of our proposed approach, which proves that the knowledge stored in PLMs can be better exploited to enhance performance. Code is available in https://github.com/zjunlp/knowledge-rumination.

cs.CL↗

arXiv2023

Observations of Orbiting Hot Spots around Naked Singularities

Recently, it has been reported that photons can traverse naked singularities in the Janis-Newman-Winicour and Born-Infeld spacetimes when these singularities are appropriately regularized. In this paper, we investigate observational signatures of hot spots orbiting these naked singularities, with a focus on discerning them from black holes. In contrast to Schwarzschild black holes, we unveil the presence of multiple additional image tracks within critical curves in time integrated images capturing a complete orbit of hot spots. Moreover, these new images manifest as a more pronounced second-highest peak in temporal magnitudes when observed at low inclinations.

gr-qc↗

arXiv2023

A Trustworthy and Consistent Blockchain Oracle Scheme for Industrial Internet of Things

Blockchain provides decentralization and trustlessness features for the Industrial Internet of Things (IIoT), which expands the application scenarios of IIoT. To address the problem that the blockchain cannot actively obtain off-chain data, the blockchain oracle is proposed as a bridge between the blockchain and external data. However, the existing oracle schemes are difficult to solve the problem of low quality of service caused by frequent data changes and heterogeneous devices in IIoT, and the current oracle node selection schemes are difficult to balance security and quality of service. To tackle these problems, this paper proposes a secure and reliable oracle scheme that can obtain high-quality off-chain data. Specifically, we first design an oracle node selection algorithm based on Verifiable Random Function (VRF) and reputation mechanism to securely select high-quality nodes. Second, we propose a data filtering algorithm based on a sliding window to further improve the consistency of the collected data. We verify the security of the proposed scheme through security analysis. The experimental results show that the proposed scheme can effectively improve the service quality of the oracle.

cs.CR↗

arXiv2023

Focus on the Sound around You: Monaural Target Speaker Extraction via Distance and Speaker Information

Previously, Target Speaker Extraction (TSE) has yielded outstanding performance in certain application scenarios for speech enhancement and source separation. However, obtaining auxiliary speaker-related information is still challenging in noisy environments with significant reverberation. inspired by the recently proposed distance-based sound separation, we propose the near sound (NS) extractor, which leverages distance information for TSE to reliably extract speaker information without requiring previous speaker enrolment, called speaker embedding self-enrollment (SESE). Full- & sub-band modeling is introduced to enhance our NS-Extractor's adaptability towards environments with significant reverberation. Experimental results on several cross-datasets demonstrate the effectiveness of our improvements and the excellent performance of our proposed NS-Extractor in different application scenarios.

cs.SD↗

arXiv2023

Human-centric Behavior Description in Videos: New Benchmark and Model

In the domain of video surveillance, describing the behavior of each individual within the video is becoming increasingly essential, especially in complex scenarios with multiple individuals present. This is because describing each individual's behavior provides more detailed situational analysis, enabling accurate assessment and response to potential risks, ensuring the safety and harmony of public places. Currently, video-level captioning datasets cannot provide fine-grained descriptions for each individual's specific behavior. However, mere descriptions at the video-level fail to provide an in-depth interpretation of individual behaviors, making it challenging to accurately determine the specific identity of each individual. To address this challenge, we construct a human-centric video surveillance captioning dataset, which provides detailed descriptions of the dynamic behaviors of 7,820 individuals. Specifically, we have labeled several aspects of each person, such as location, clothing, and interactions with other elements in the scene, and these people are distributed across 1,012 videos. Based on this dataset, we can link individuals to their respective behaviors, allowing for further analysis of each person's behavior in surveillance videos. Besides the dataset, we propose a novel video captioning approach that can describe individual behavior in detail on a person-level basis, achieving state-of-the-art results. To facilitate further research in this field, we intend to release our dataset and code.

cs.CV↗