Searcharxiv⌕ Search

arXiv subjects

Peng Wang

Publications and source records attributed to Peng Wang.

At least 415 records · Page 23Linked to original sources

Meta-Exploiting Frequency Prior for Cross-Domain Few-Shot Learning

Meta-learning offers a promising avenue for few-shot learning (FSL), enabling models to glean a generalizable feature embedding through episodic training on synthetic FSL tasks in a source domain. Yet, in practical scenarios where the target task diverges from that in the source domain, meta-learning based method is susceptible to over-fitting. To overcome this, we introduce a novel framework, Meta-Exploiting Frequency Prior for Cross-Domain Few-Shot Learning, which is crafted to comprehensively exploit the cross-domain transferable image prior that each image can be decomposed into complementary low-frequency content details and high-frequency robust structural characteristics. Motivated by this insight, we propose to decompose each query image into its high-frequency and low-frequency components, and parallel incorporate them into the feature embedding network to enhance the final category prediction. More importantly, we introduce a feature reconstruction prior and a prediction consistency prior to separately encourage the consistency of the intermediate feature as well as the final category prediction between the original query image and its decomposed frequency components. This allows for collectively guiding the network's meta-learning process with the aim of learning generalizable image feature embeddings, while not introducing any extra computational cost in the inference phase. Our framework establishes new state-of-the-art results on multiple cross-domain few-shot learning benchmarks.

cs.CV↗

Efficient Adaptation of Pre-trained Vision Transformer via Householder Transformation

A common strategy for Parameter-Efficient Fine-Tuning (PEFT) of pre-trained Vision Transformers (ViTs) involves adapting the model to downstream tasks by learning a low-rank adaptation matrix. This matrix is decomposed into a product of down-projection and up-projection matrices, with the bottleneck dimensionality being crucial for reducing the number of learnable parameters, as exemplified by prevalent methods like LoRA and Adapter. However, these low-rank strategies typically employ a fixed bottleneck dimensionality, which limits their flexibility in handling layer-wise variations. To address this limitation, we propose a novel PEFT approach inspired by Singular Value Decomposition (SVD) for representing the adaptation matrix. SVD decomposes a matrix into the product of a left unitary matrix, a diagonal matrix of scaling values, and a right unitary matrix. We utilize Householder transformations to construct orthogonal matrices that efficiently mimic the unitary matrices, requiring only a vector. The diagonal values are learned in a layer-wise manner, allowing them to flexibly capture the unique properties of each layer. This approach enables the generation of adaptation matrices with varying ranks across different layers, providing greater flexibility in adapting pre-trained models. Experiments on standard downstream vision tasks demonstrate that our method achieves promising fine-tuning performance.

cs.CV↗

A Full-duplex Speech Dialogue Scheme Based On Large Language Models

We present a generative dialogue system capable of operating in a full-duplex manner, allowing for seamless interaction. It is based on a large language model (LLM) carefully aligned to be aware of a perception module, a motor function module, and the concept of a simple finite state machine (called neural FSM) with two states. The perception and motor function modules operate in tandem, allowing the system to speak and listen to the user simultaneously. The LLM generates textual tokens for inquiry responses and makes autonomous decisions to start responding to, wait for, or interrupt the user by emitting control tokens to the neural FSM. All these tasks of the LLM are carried out as next token prediction on a serialized view of the dialogue in real-time. In automatic quality evaluations simulating real-life interaction, the proposed system reduces the average conversation response latency by more than threefold compared with LLM-based half-duplex dialogue systems while responding within less than 500 milliseconds in more than 50% of evaluated interactions. Running an LLM with only 8 billion parameters, our system exhibits an 8% higher interruption precision rate than the best available commercial LLM for voice-based dialogue.

cs.CL↗

Spin-induced Scalar Clouds around Kerr-Newman Black Holes

Recent studies have demonstrated that a scalar field non-minimally coupled to the electromagnetic field can experience a spin-induced tachyonic instability near Kerr-Newman black holes, potentially driving the formation of scalar clouds. In this paper, we construct such scalar clouds for both fundamental and excited modes, detailing their existence domains and wave functions. Our results indicate that a sufficiently strong coupling between the scalar and electromagnetic fields is essential for sustaining scalar clouds. Within the strong coupling regime, black holes that rotate either too slowly or too rapidly are unable to support scalar clouds. Furthermore, we observe that scalar cloud wave functions are concentrated near the black hole's poles. These findings provide a foundation for future investigations of spin-induced scalarized Kerr-Newman black holes.

gr-qc↗

Visual Prompt Tuning in Null Space for Continual Learning

Existing prompt-tuning methods have demonstrated impressive performances in continual learning (CL), by selecting and updating relevant prompts in the vision-transformer models. On the contrary, this paper aims to learn each task by tuning the prompts in the direction orthogonal to the subspace spanned by previous tasks' features, so as to ensure no interference on tasks that have been learned to overcome catastrophic forgetting in CL. However, different from the orthogonal projection in the traditional CNN architecture, the prompt gradient orthogonal projection in the ViT architecture shows completely different and greater challenges, i.e., 1) the high-order and non-linear self-attention operation; 2) the drift of prompt distribution brought by the LayerNorm in the transformer block. Theoretically, we have finally deduced two consistency conditions to achieve the prompt gradient orthogonal projection, which provide a theoretical guarantee of eliminating interference on previously learned knowledge via the self-attention mechanism in visual prompt tuning. In practice, an effective null-space-based approximation solution has been proposed to implement the prompt gradient orthogonal projection. Extensive experimental results demonstrate the effectiveness of anti-forgetting on four class-incremental benchmarks with diverse pre-trained baseline models, and our approach achieves superior performances to state-of-the-art methods. Our code is available at https://github.com/zugexiaodui/VPTinNSforCL.

cs.CV↗

Towards Robust Blockchain Price Oracle: A Study on Human-Centric Node Selection Strategy and Incentive Mechanism

As a trusted middleware connecting the blockchain and the real world, the blockchain oracle can obtain trusted real-time price information for financial applications such as payment and settlement, and asset valuation on the blockchain. However, the current oracle schemes face the dilemma of security and service quality in the process of node selection, and the implicit interest relationship in financial applications leads to a significant conflict of interest between the task publisher and the executor, which reduces the participation enthusiasm of both parties and system security. Therefore, this paper proposes an anonymous node selection scheme that anonymously selects nodes with high reputations to participate in tasks to ensure the security and service quality of nodes. Then, this paper also details the interest requirements and behavioral motives of all parties in the payment settlement and asset valuation scenarios. Under the hypothesis of rational man, an incentive mechanism based on the Stackelberg game is proposed. It can achieve equilibrium under the pursuit of the revenue of task publishers and executors, thereby ensuring the revenue of all types of users and improving the enthusiasm for participation. Finally, we verify the security of the proposed scheme through security analysis. The experimental results show that the proposed scheme can reduce the variance of obtaining price data by about 55\% while ensuring security, and meeting the revenue of all parties.

cs.CR↗

SEMSO: A Secure and Efficient Multi-Data Source Blockchain Oracle

In recent years, blockchain oracle, as the key link between blockchain and real-world data interaction, has greatly expanded the application scope of blockchain. In particular, the emergence of the Multi-Data Source (MDS) oracle has greatly improved the reliability of the oracle in the case of untrustworthy data sources. However, the current MDS oracle scheme requires nodes to obtain data redundantly from multiple data sources to guarantee data reliability, which greatly increases the resource overhead and response time of the system. Therefore, in this paper, we propose a Secure and Efficient Multi-data Source Oracle framework (SEMSO), which nodes only need to access one data source to ensure the reliability of final data. First, we design a new off-chain data aggregation protocol TBLS, to guarantee data source diversity and reliability at low cost. Second, according to the rational man assumption, the data source selection task of nodes is modeled and solved based on the Bayesian game under incomplete information to maximize the node's revenue while improving the success rate of TBLS aggregation and system response speed. Security analysis verifies the reliability of the proposed scheme, and experiments show that under the same environmental assumptions, SEMSO takes into account data diversity while reducing the response time by 23.5\%.

cs.CR↗

CAMEL: Physically Inspired Crosstalk-Aware Mapping and gatE scheduLing for Frequency-Tunable Quantum Chips

Crosstalk represents a formidable obstacle in quantum computing. When quantum gates are executed parallelly, the resonance of qubit frequencies can lead to residual coupling, compromising the fidelity. Existing crosstalk solutions encounter difficulties in mitigating crosstalk and decoherence when dealing with parallel two-qubit gates in frequency-tunable quantum chips. Inspired by the physical properties of frequency-tunable quantum chips, we introduce a Crosstalk-Aware Mapping and gatE Scheduling (CAMEL) approach to address these challenges. CAMEL aims to mitigate crosstalk of parallel two-qubit gates and suppress decoherence. Utilizing the features of the tunable coupler, the CAMEL approach integrates a pulse compensation method for crosstalk mitigation. Furthermore, we present a compilation framework, including two steps. Firstly, we devise a qubit mapping approach that accounts for both crosstalk and decoherence. Secondly, we introduce a gate timing scheduling approach capable of prioritizing the execution of the largest set of crosstalk-free parallel gates to shorten quantum circuit execution times. Evaluation results demonstrate the effectiveness of CAMEL in mitigating crosstalk compared to crosstalk-agnostic methods. Furthermore, in contrast to approaches serializing crosstalk gates, CAMEL successfully suppresses decoherence. Finally, CAMEL exhibits better performance over dynamic-frequency awareness in low-complexity hardware.

quant-ph↗

Two Birds With One Stone: Enhancing Communication and Sensing via Multi-Functional RIS

In this article, we propose new network architectures that integrate multi-functional reconfigurable intelligent surfaces (MF-RISs) into 6G networks to enhance both communication and sensing capabilities. Firstly, we elaborate how to leverage MF-RISs for improving communication performance in different communication modes including unicast, mulitcast, and broadcast and for different multi-access schemes. Next, we emphasize synergistic benefits of integrating MF-RISs with wireless sensing, enabling more accurate and efficient target detection in 6G networks. Furthermore, we present two schemes that utilize MF-RISs to enhance the performance of integrated sensing and communication (ISAC). We also study multi-objective optimization to achieve the optimal trade-off between communication and sensing performance. Finally, we present numerical results to show the performance improvements offered by MF-RISs compared to conventional RISs in ISAC. We also outline key research directions for MF-RIS under the ambition of 6G.

eess.SP↗

Wrong-of-Thought: An Integrated Reasoning Framework with Multi-Perspective Verification and Wrong Information

Chain-of-Thought (CoT) has become a vital technique for enhancing the performance of Large Language Models (LLMs), attracting increasing attention from researchers. One stream of approaches focuses on the iterative enhancement of LLMs by continuously verifying and refining their reasoning outputs for desired quality. Despite its impressive results, this paradigm faces two critical issues: (1) Simple verification methods: The current paradigm relies solely on a single verification method. (2) Wrong Information Ignorance: Traditional paradigms directly ignore wrong information during reasoning and refine the logic paths from scratch each time. To address these challenges, we propose Wrong-of-Thought (WoT), which includes two core modules: (1) Multi-Perspective Verification: A multi-perspective verification method for accurately refining the reasoning process and result, and (2) Wrong Information Utilization: Utilizing wrong information to alert LLMs and reduce the probability of LLMs making same mistakes. Experiments on 8 popular datasets and 5 LLMs demonstrate that WoT surpasses all previous baselines. In addition, WoT exhibits powerful capabilities in difficult computation tasks.

cs.CL↗

Distilling Instruction-following Abilities of Large Language Models with Task-aware Curriculum Planning

Instruction tuning aims to align large language models (LLMs) with open-domain instructions and human-preferred responses. While several studies have explored autonomous approaches to distilling and annotating instructions from powerful proprietary LLMs, such as ChatGPT, they often neglect the impact of the distributions and characteristics of tasks, together with the varying difficulty of instructions in training sets. This oversight can lead to imbalanced knowledge capabilities and poor generalization powers of student LLMs. To address these challenges, we introduce Task-Aware Curriculum Planning for Instruction Refinement (TAPIR), a multi-round distillation framework that utilizes an oracle LLM to select instructions that are difficult for a student LLM to follow. To balance the student's capabilities, task distributions in training sets are adjusted with responses automatically refined according to their corresponding tasks. In addition, by incorporating curriculum planning, our approach systematically escalates the difficulty levels of tasks, progressively enhancing the student LLM's capabilities. We rigorously evaluate TAPIR using several widely recognized benchmarks (such as AlpacaEval 2.0, MT-Bench, etc.) and multiple student LLMs. Empirical results demonstrate that student LLMs, trained with our method and less training data, outperform larger instruction-tuned models and strong distillation baselines.

cs.CL↗

Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution

We present the Qwen2-VL Series, an advanced upgrade of the previous Qwen-VL models that redefines the conventional predetermined-resolution approach in visual processing. Qwen2-VL introduces the Naive Dynamic Resolution mechanism, which enables the model to dynamically process images of varying resolutions into different numbers of visual tokens. This approach allows the model to generate more efficient and accurate visual representations, closely aligning with human perceptual processes. The model also integrates Multimodal Rotary Position Embedding (M-RoPE), facilitating the effective fusion of positional information across text, images, and videos. We employ a unified paradigm for processing both images and videos, enhancing the model's visual perception capabilities. To explore the potential of large multimodal models, Qwen2-VL investigates the scaling laws for large vision-language models (LVLMs). By scaling both the model size-with versions at 2B, 8B, and 72B parameters-and the amount of training data, the Qwen2-VL Series achieves highly competitive performance. Notably, the Qwen2-VL-72B model achieves results comparable to leading models such as GPT-4o and Claude3.5-Sonnet across various multimodal benchmarks, outperforming other generalist models. Code is available at https://github.com/QwenLM/Qwen2-VL .

cs.CV↗

A performance portable, fully implicit Landau collision operator with batched linear solvers

Modern accelerators use hierarchical parallel programming models that enable massive multithreading within a processing element (PE), with multiple PEs per device driven by traditional processes. Batching is a technique for exposing PE-level parallelism in algorithms that have traditionally run on MPI processes or multiple threads within a single process. Opportunities for batching arise in, for example, kinetic discretizations of magnetized plasmas where collisions are advanced in velocity space at each spatial point independently. This paper builds on previous work on a high-performance, fully nonlinear, Landau collision operator by batching the linear solver, as well as batching the spatial point problems and adding new support for multiple grids for multiscale, multi-species problems. An anisotropic relaxation verification test that agrees well with previous published results and analytical models is presented. The performance results from NVIDIA A100 and AMD MI250X nodes are presented with hardware utilization analysis for each architecture. The entire implicit Landau operator time advance is implemented in Kokkos for performance portability, running entirely on the device and is available in the PETSc numerical library.

physics.plasm-ph↗

The Beyond-Halo Mass Effects of the Cosmic Web Environment on Galaxies

Galaxy properties primarily depend on their host halo mass. Halo mass, in turn, depends on the cosmic web environment. We explore if the effect of the cosmic web on galaxy properties is entirely transitive via host halo mass, or if the cosmic web has an effect independent of mass. The secondary galaxy bias, sometimes referred to as ``galaxy assembly bias'', is the beyond-mass component of the galaxy-halo connection. We investigate the link between the cosmic web environment and the secondary galaxy bias in simulations. We measure the secondary galaxy bias through the following summary statistics: projected two-point correlation function, $\wprp$, and counts-in-cylinders statistics, $\Pncic$. First, we examine the extent to which the secondary galaxy bias can be accounted for with a measure of the environment as a secondary halo property. We find that the total secondary galaxy bias preferentially places galaxies in more strongly clustered haloes. In particular, haloes at fixed mass tend to host more galaxies when they are more strongly associated with nodes or filaments. This tendency accounts for a significant portion, but not the entirety, of the total secondary galaxy bias effect. Second, we quantify how the secondary galaxy bias behaves differently depending on the host halo proximity to nodes and filaments. We find that the total secondary galaxy bias is relatively stronger in haloes more associated with nodes or filaments. We emphasise the importance of removing halo mass effects when considering the cosmic web environment as a factor in the galaxy-halo connection.

astro-ph.GA↗

ALAAMEE: Open-source software for fitting autologistic actor attribute models

The autologistic actor attribute model (ALAAM) is a model for social influence, derived from the more widely known exponential-family random graph model (ERGM). ALAAMs can be used to estimate parameters corresponding to multiple forms of social contagion associated with network structure and actor covariates. This work introduces ALAAMEE, open-source Python software for estimation, simulation, and goodness-of-fit testing for ALAAM models. ALAAMEE implements both the stochastic approximation and equilibrium expectation (EE) algorithms for ALAAM parameter estimation, including estimation from snowball sampled network data. It implements data structures and statistics for undirected, directed, and bipartite networks. We use a simulation study to assess the accuracy of the EE algorithm for ALAAM parameter estimation and statistical inference, and demonstrate the use of ALAAMEE with empirical examples using both small (fewer than 100 nodes) and large (more than 10 000 nodes) networks.

stat.CO↗

VL-Reader: Vision and Language Reconstructor is an Effective Scene Text Recognizer

Text recognition is an inherent integration of vision and language, encompassing the visual texture in stroke patterns and the semantic context among the character sequences. Towards advanced text recognition, there are three key challenges: (1) an encoder capable of representing the visual and semantic distributions; (2) a decoder that ensures the alignment between vision and semantics; and (3) consistency in the framework during pre-training, if it exists, and fine-tuning. Inspired by masked autoencoding, a successful pre-training strategy in both vision and language, we propose an innovative scene text recognition approach, named VL-Reader. The novelty of the VL-Reader lies in the pervasive interplay between vision and language throughout the entire process. Concretely, we first introduce a Masked Visual-Linguistic Reconstruction (MVLR) objective, which aims at simultaneously modeling visual and linguistic information. Then, we design a Masked Visual-Linguistic Decoder (MVLD) to further leverage masked vision-language context and achieve bi-modal feature interaction. The architecture of VL-Reader maintains consistency from pre-training to fine-tuning. In the pre-training stage, VL-Reader reconstructs both masked visual and text tokens, while in the fine-tuning stage, the network degrades to reconstruct all characters from an image without any masked regions. VL-reader achieves an average accuracy of 97.1% on six typical datasets, surpassing the SOTA by 1.1%. The improvement was even more significant on challenging datasets. The results demonstrate that vision and language reconstructor can serve as an effective scene text recognizer.

cs.CV↗

Dispersive wave propagation in disordered flexible fibers enhances stress attenuation

We experimentally and computationally analyze impact-shock-induced stress wave propagation in packings of disordered flexible fibers. We find that dispersive wave propagation, associated with large stress attenuation, occurs much more prevalently in systems with larger fiber aspect ratios and moderate fiber flexibility. We trace these features to the microstructural properties of fiber contact chains and the energy-trapping abilities of deformable fibers. These findings provide new insights into physics of the shock-impacted flexible fiber packings and open the way towards an improved granular-material-based damping technology.

cond-mat.soft↗

CorrespondentDream: Enhancing 3D Fidelity of Text-to-3D using Cross-View Correspondences

Leveraging multi-view diffusion models as priors for 3D optimization have alleviated the problem of 3D consistency, e.g., the Janus face problem or the content drift problem, in zero-shot text-to-3D models. However, the 3D geometric fidelity of the output remains an unresolved issue; albeit the rendered 2D views are realistic, the underlying geometry may contain errors such as unreasonable concavities. In this work, we propose CorrespondentDream, an effective method to leverage annotation-free, cross-view correspondences yielded from the diffusion U-Net to provide additional 3D prior to the NeRF optimization process. We find that these correspondences are strongly consistent with human perception, and by adopting it in our loss design, we are able to produce NeRF models with geometries that are more coherent with common sense, e.g., more smoothed object surface, yielding higher 3D fidelity. We demonstrate the efficacy of our approach through various comparative qualitative results and a solid user study.

cs.CV↗