arXiv subjects

Ping Wang

Publications and source records attributed to Ping Wang.

At least 145 records · Page 8Linked to original sources

arXiv2024

Hypergraph Multi-modal Large Language Model: Exploiting EEG and Eye-tracking Modalities to Evaluate Heterogeneous Responses for Video Understanding

Understanding of video creativity and content often varies among individuals, with differences in focal points and cognitive levels across different ages, experiences, and genders. There is currently a lack of research in this area, and most existing benchmarks suffer from several drawbacks: 1) a limited number of modalities and answers with restrictive length; 2) the content and scenarios within the videos are excessively monotonous, transmitting allegories and emotions that are overly simplistic. To bridge the gap to real-world applications, we introduce a large-scale Subjective Response Indicators for Advertisement Videos dataset, namely SRI-ADV. Specifically, we collected real changes in Electroencephalographic (EEG) and eye-tracking regions from different demographics while they viewed identical video content. Utilizing this multi-modal dataset, we developed tasks and protocols to analyze and evaluate the extent of cognitive understanding of video content among different users. Along with the dataset, we designed a Hypergraph Multi-modal Large Language Model (HMLLM) to explore the associations among different demographics, video elements, EEG, and eye-tracking indicators. HMLLM could bridge semantic gaps across rich modalities and integrate information beyond different modalities to perform logical reasoning. Extensive experimental evaluations on SRI-ADV and other additional video-based generative performance benchmarks demonstrate the effectiveness of our method. The codes and dataset will be released at https://github.com/mininglamp-MLLM/HMLLM.

cs.CV↗

arXiv2024

Nanoscale Engineering of Wurtzite Ferroelectrics: Unveiling Phase Transition and Ferroelectric Switching in ScAlN Nanowires

The pursuit of extreme device miniaturization and the exploration of novel physical phenomena have spurred significant interest in crystallographic phase control and ferroelectric switching in reduced dimensions. Recently, wurtzite ferroelectrics have emerged as a new class of functional materials, offering intriguing piezoelectric and ferroelectric properties, CMOS compatibility, and seamless integration with mainstream semiconductor technology. However, the exploration of crystallographic phase and ferroelectric switching in reduced dimensions, especially in nanostructures, has remained a largely uncharted territory. In this study, we present the first comprehensive investigation into the crystallographic phase transition of ScAlN nanowires across the full Sc compositional range. While a gradual transition from wurtzite to cubic phase was observed with increasing Sc composition, we further demonstrated that a highly ordered wurtzite phase ScAlN could be confined at the ScAlN/GaN interface for Sc contents surpassing what is possible in conventional films, holding great potential to addressing the fundamental high coercive field of wurtzite ferroelectrics. In addition, we provide the first evidence of ferroelectric switching in ScAlN nanowires, a result that holds significant implications for future device miniaturization. Our demonstration of tunable ferroelectric ScAlN nanowires opens new possibilities for nanoscale, domain, alloy, strain, and quantum engineering of wurtzite ferroelectrics, representing a significant stride towards the development of next-generation, miniaturized devices based on wurtzite ferroelectrics.

cond-mat.mtrl-sci↗

arXiv2024

HMDN: Hierarchical Multi-Distribution Network for Click-Through Rate Prediction

As the recommendation service needs to address increasingly diverse distributions, such as multi-population, multi-scenario, multitarget, and multi-interest, more and more recent works have focused on multi-distribution modeling and achieved great progress. However, most of them only consider modeling in a single multi-distribution manner, ignoring that mixed multi-distributions often coexist and form hierarchical relationships. To address these challenges, we propose a flexible modeling paradigm, named Hierarchical Multi-Distribution Network (HMDN), which efficiently models these hierarchical relationships and can seamlessly integrate with existing multi-distribution methods, such as Mixture of-Experts (MoE) and Dynamic-Weight (DW) models. Specifically, we first design a hierarchical multi-distribution representation refinement module, employing a multi-level residual quantization to obtain fine-grained hierarchical representation. Then, the refined hierarchical representation is integrated into the existing single multi-distribution models, seamlessly expanding them into mixed multi-distribution models. Experimental results on both public and industrial datasets validate the effectiveness and flexibility of HMDN.

cs.LG↗

arXiv2024

Multiobjective Vehicle Routing Optimization with Time Windows: A Hybrid Approach Using Deep Reinforcement Learning and NSGA-II

This paper proposes a weight-aware deep reinforcement learning (WADRL) approach designed to address the multiobjective vehicle routing problem with time windows (MOVRPTW), aiming to use a single deep reinforcement learning (DRL) model to solve the entire multiobjective optimization problem. The Non-dominated sorting genetic algorithm-II (NSGA-II) method is then employed to optimize the outcomes produced by the WADRL, thereby mitigating the limitations of both approaches. Firstly, we design an MOVRPTW model to balance the minimization of travel cost and the maximization of customer satisfaction. Subsequently, we present a novel DRL framework that incorporates a transformer-based policy network. This network is composed of an encoder module, a weight embedding module where the weights of the objective functions are incorporated, and a decoder module. NSGA-II is then utilized to optimize the solutions generated by WADRL. Finally, extensive experimental results demonstrate that our method outperforms the existing and traditional methods. Due to the numerous constraints in VRPTW, generating initial solutions of the NSGA-II algorithm can be time-consuming. However, using solutions generated by the WADRL as initial solutions for NSGA-II significantly reduces the time required for generating initial solutions. Meanwhile, the NSGA-II algorithm can enhance the quality of solutions generated by WADRL, resulting in solutions with better scalability. Notably, the weight-aware strategy significantly reduces the training time of DRL while achieving better results, enabling a single DRL model to solve the entire multiobjective optimization problem.

cs.AI↗

arXiv2024

Hierarchical Separable Video Transformer for Snapshot Compressive Imaging

Transformers have achieved the state-of-the-art performance on solving the inverse problem of Snapshot Compressive Imaging (SCI) for video, whose ill-posedness is rooted in the mixed degradation of spatial masking and temporal aliasing. However, previous Transformers lack an insight into the degradation and thus have limited performance and efficiency. In this work, we tailor an efficient reconstruction architecture without temporal aggregation in early layers and Hierarchical Separable Video Transformer (HiSViT) as building block. HiSViT is built by multiple groups of Cross-Scale Separable Multi-head Self-Attention (CSS-MSA) and Gated Self-Modulated Feed-Forward Network (GSM-FFN) with dense connections, each of which is conducted within a separate channel portions at a different scale, for multi-scale interactions and long-range modeling. By separating spatial operations from temporal ones, CSS-MSA introduces an inductive bias of paying more attention within frames instead of between frames while saving computational overheads. GSM-FFN further enhances the locality via gated mechanism and factorized spatial-temporal convolutions. Extensive experiments demonstrate that our method outperforms previous methods by $\!>\!0.5$ dB with comparable or fewer parameters and complexity. The source codes and pretrained models are released at https://github.com/pwangcs/HiSViT.

cs.CV↗

arXiv2024

Hypergraph based Understanding for Document Semantic Entity Recognition

Semantic entity recognition is an important task in the field of visually-rich document understanding. It distinguishes the semantic types of text by analyzing the position relationship between text nodes and the relation between text content. The existing document understanding models mainly focus on entity categories while ignoring the extraction of entity boundaries. We build a novel hypergraph attention document semantic entity recognition framework, HGA, which uses hypergraph attention to focus on entity boundaries and entity categories at the same time. It can conduct a more detailed analysis of the document text representation analyzed by the upstream model and achieves a better performance of semantic information. We apply this method on the basis of GraphLayoutLM to construct a new semantic entity recognition model HGALayoutLM. Our experiment results on FUNSD, CORD, XFUND and SROIE show that our method can effectively improve the performance of semantic entity recognition tasks based on the original model. The results of HGALayoutLM on FUNSD and XFUND reach the new state-of-the-art results.

cs.AI↗

arXiv2024

Semantics-Preserved Distortion for Personal Privacy Protection in Information Management

In recent years, machine learning - particularly deep learning - has significantly impacted the field of information management. While several strategies have been proposed to restrict models from learning and memorizing sensitive information from raw texts, this paper suggests a more linguistically-grounded approach to distort texts while maintaining semantic integrity. To this end, we leverage Neighboring Distribution Divergence, a novel metric to assess the preservation of semantic meaning during distortion. Building on this metric, we present two distinct frameworks for semantic-preserving distortion: a generative approach and a substitutive approach. Our evaluations across various tasks, including named entity recognition, constituency parsing, and machine reading comprehension, affirm the plausibility and efficacy of our distortion technique in personal privacy protection. We also test our method against attribute attacks in three privacy-focused assignments within the NLP domain, and the findings underscore the simplicity and efficacy of our data-based improvement approach over structural improvement approaches. Moreover, we explore privacy protection in a specific medical information management scenario, showing our method effectively limits sensitive data memorization, underscoring its practicality.

cs.CL↗

arXiv2024

The Music Maestro or The Musically Challenged, A Massive Music Evaluation Benchmark for Large Language Models

Benchmark plays a pivotal role in assessing the advancements of large language models (LLMs). While numerous benchmarks have been proposed to evaluate LLMs' capabilities, there is a notable absence of a dedicated benchmark for assessing their musical abilities. To address this gap, we present ZIQI-Eval, a comprehensive and large-scale music benchmark specifically designed to evaluate the music-related capabilities of LLMs. ZIQI-Eval encompasses a wide range of questions, covering 10 major categories and 56 subcategories, resulting in over 14,000 meticulously curated data entries. By leveraging ZIQI-Eval, we conduct a comprehensive evaluation over 16 LLMs to evaluate and analyze LLMs' performance in the domain of music. Results indicate that all LLMs perform poorly on the ZIQI-Eval benchmark, suggesting significant room for improvement in their musical capabilities. With ZIQI-Eval, we aim to provide a standardized and robust evaluation framework that facilitates a comprehensive assessment of LLMs' music-related abilities. The dataset is available at GitHub\footnote{https://github.com/zcli-charlie/ZIQI-Eval} and HuggingFace\footnote{https://huggingface.co/datasets/MYTH-Lab/ZIQI-Eval}.

cs.SD↗

arXiv2024

Zone Evaluation: Revealing Spatial Bias in Object Detection

A fundamental limitation of object detectors is that they suffer from "spatial bias", and in particular perform less satisfactorily when detecting objects near image borders. For a long time, there has been a lack of effective ways to measure and identify spatial bias, and little is known about where it comes from and what degree it is. To this end, we present a new zone evaluation protocol, extending from the traditional evaluation to a more generalized one, which measures the detection performance over zones, yielding a series of Zone Precisions (ZPs). For the first time, we provide numerical results, showing that the object detectors perform quite unevenly across the zones. Surprisingly, the detector's performance in the 96% border zone of the image does not reach the AP value (Average Precision, commonly regarded as the average detection performance in the entire image zone). To better understand spatial bias, a series of heuristic experiments are conducted. Our investigation excludes two intuitive conjectures about spatial bias that the object scale and the absolute positions of objects barely influence the spatial bias. We find that the key lies in the human-imperceptible divergence in data patterns between objects in different zones, thus eventually forming a visible performance gap between the zones. With these findings, we finally discuss a future direction for object detection, namely, spatial disequilibrium problem, aiming at pursuing a balanced detection ability over the entire image zone. By broadly evaluating 10 popular object detectors and 5 detection datasets, we shed light on the spatial bias of object detectors. We hope this work could raise a focus on detection robustness. The source codes, evaluation protocols, and tutorials are publicly available at https://github.com/Zzh-tju/ZoneEval.

cs.CV↗

arXiv2024

DVS-RG: Differential Variable Speed Limits Control using Deep Reinforcement Learning with Graph State Representation

Variable speed limit (VSL) control is an established yet challenging problem to improve freeway traffic mobility and alleviate bottlenecks by customizing speed limits at proper locations based on traffic conditions. Recent advances in deep reinforcement learning (DRL) have shown promising results in solving VSL control problems by interacting with sophisticated environments. However, the modeling of these methods ignores the inherent graph structure of the traffic state which can be a key factor for more efficient VSL control. Graph structure can not only capture the static spatial feature but also the dynamic temporal features of traffic. Therefore, we propose the DVS-RG: DRL-based differential variable speed limit controller with graph state representation. DVS-RG provides distinct speed limits per lane in different locations dynamically. The road network topology and traffic information(e.g., occupancy, speed) are integrated as the state space of DVS-RG so that the spatial features can be learned. The normalization reward which combines efficiency and safety is used to train the VSL controller to avoid excessive inefficiencies or low safety. The results obtained from the simulation study on SUMO show that DRL-RG achieves higher traffic efficiency (the average waiting time reduced to 68.44\%) and improves the safety measures (the number of potential collision reduced by 15.93\% ) compared to state-of-the-art DRL methods.

eess.SY↗

arXiv2024

Blood Works for Graphene Production

Blood, a ubiquitous and fundamental carbohydrate material composed of plasma, red blood cells, white blood cells, and platelets, has been playing an important role in biology, life science, history, and religious study, while graphene has garnered significant attention due to its exceptional properties and extensive range of potential applications. Achieving environmentally friendly, cost-effective growth using hybrid precursors and obtaining high-quality graphene through a straightforward CVD process has been traditionally considered mutually exclusive. This study demonstrates that we can produce high-quality graphene domains with controlled thickness through a one-step growth process at atmospheric pressure using blood as a precursor. Raman spectroscopy confirms the uniformity of the blood-grown graphene films, and observing the half-integer quantum Hall effect in the measured devices highlights its outstanding electronic properties. This unprecedented approach opens possibilities for blood application, facilitating an unconventional route in graphene growth applications.

cond-mat.mes-hall↗

arXiv2024

Gate control of 2D magnetism in tri- and four-layers $\rm CrI_3$/graphene heterostructures

We conduct experimental studies on the electrical transport properties of monolayer graphene directly covered by a few layers of $\rm CrI_3$. We do not observe the expected magnetic exchange coupling in the graphene but instead discover proximity effects featuring gate and magnetic field tunability. The tunability of gate voltage is manifested in the alignment of the lowest conduction band of $\rm CrI_3$ and the Fermi level of graphene, which can be controlled by the gate voltage. The coexistence of the normal and atypical quantum Hall effects in our device also corresponds to gate-control modulation doping. The lowest conduction band depends on the magnetic states of the $\rm CrI_3$ and can be altered by the magnetic field, which corresponds to the resistance loops during back-and-forth sweeps of the magnetic field. Our results serve as a reference for exploiting the magnetic proximity effects in graphene.

cond-mat.mes-hall↗

arXiv2024

Scaling and Universality in the Temporal Occurrence of Repeating FRBs

Fast Radio Bursts (FRBs) are energetic phenomena that have significant implications for understanding fundamental physics and the universe. Recent observations of FRB 121102, FRB 20220912A, and FRB 20201124A by the Five-hundred-meter Aperture Spherical Telescope (FAST) showed high burst rates and distinctive energy distribution and temporal properties. In this study, we examine these observations to investigate the scale invariance of the waiting times between bursts for intervals longer than approximately 1 second. Our analysis revealed a unified scaling law for these longer intervals, which is similar to the behavior of solar flares. This discovery inspires us to suggest a dual analogy of the FRB scenario across the entire time intervals: with earthquake dynamics at subsecond scales and with solar flare dynamics beyond the one-second threshold. This threshold potentially aligns with the dynamic time scale of neutron star crusts, offering insight of the occurrence of FRBs into the internal processes of neutron stars.

astro-ph.HE↗

arXiv2024

Deep Optics for Video Snapshot Compressive Imaging

Video snapshot compressive imaging (SCI) aims to capture a sequence of video frames with only a single shot of a 2D detector, whose backbones rest in optical modulation patterns (also known as masks) and a computational reconstruction algorithm. Advanced deep learning algorithms and mature hardware are putting video SCI into practical applications. Yet, there are two clouds in the sunshine of SCI: i) low dynamic range as a victim of high temporal multiplexing, and ii) existing deep learning algorithms' degradation on real system. To address these challenges, this paper presents a deep optics framework to jointly optimize masks and a reconstruction network. Specifically, we first propose a new type of structural mask to realize motion-aware and full-dynamic-range measurement. Considering the motion awareness property in measurement domain, we develop an efficient network for video SCI reconstruction using Transformer to capture long-term temporal dependencies, dubbed Res2former. Moreover, sensor response is introduced into the forward model of video SCI to guarantee end-to-end model training close to real system. Finally, we implement the learned structural masks on a digital micro-mirror device. Experimental results on synthetic and real data validate the effectiveness of the proposed framework. We believe this is a milestone for real-world video SCI. The source code and data are available at https://github.com/pwangcs/DeepOpticsSCI.

cs.CV↗

arXiv2024

Dual-Scale Transformer for Large-Scale Single-Pixel Imaging

Single-pixel imaging (SPI) is a potential computational imaging technique which produces image by solving an illposed reconstruction problem from few measurements captured by a single-pixel detector. Deep learning has achieved impressive success on SPI reconstruction. However, previous poor reconstruction performance and impractical imaging model limit its real-world applications. In this paper, we propose a deep unfolding network with hybrid-attention Transformer on Kronecker SPI model, dubbed HATNet, to improve the imaging quality of real SPI cameras. Specifically, we unfold the computation graph of the iterative shrinkagethresholding algorithm (ISTA) into two alternative modules: efficient tensor gradient descent and hybrid-attention multiscale denoising. By virtue of Kronecker SPI, the gradient descent module can avoid high computational overheads rooted in previous gradient descent modules based on vectorized SPI. The denoising module is an encoder-decoder architecture powered by dual-scale spatial attention for high- and low-frequency aggregation and channel attention for global information recalibration. Moreover, we build a SPI prototype to verify the effectiveness of the proposed method. Extensive experiments on synthetic and real data demonstrate that our method achieves the state-of-the-art performance. The source code and pre-trained models are available at https://github.com/Gang-Qu/HATNet-SPI.

cs.CV↗

arXiv2024

High quality Fe1+yTe synthesized by chemical vapor deposition with conspicuous vortex flow

Two-dimensional (2D) materials provide an ideal platform to explore novel superconducting behavior including Ising superconductivity, topological superconductivity and Majorana bound states in different 2D stoichiometric Ta-, Nb-, and Fe-based crystals. However, tuning the element content in 2D compounds for regulating their superconductivity has not been realized. In this work, we report the synthesis of high quality Fe1+yTe with tunable Fe content by chemical vapor deposition (CVD). The quality and composition of Fe1+yTe are characterized by Raman spectroscopy, X-ray photoelectron spectroscopy (XPS) and scanning transmission electron microscopy (STEM). The superconducting behavior of Fe1+yTe crystals with varying Fe contents is observed. The superconducting transition of selected Fe1.13Te sample is sharp (ΔTc = 1 K), while Fe1.43Te with a high-Fe content shows a relative broad superconducting transition (ΔTc = 2.6 K) at zero magnetic field. Significantly, the conspicuous vortex flow and a transition from a 3D vortex liquid state to a 2D vortex liquid state is observed in Fe1.43Te sample. Our work highlights the tunability of the superconducting properties of Fe1+yTe and sheds light on the vortex dynamics in Fe-based superconductors, which facilitates us to understand the intrinsic mechanisms of high-temperature superconductivity.

physics.app-ph↗

arXiv2024

SCINeRF: Neural Radiance Fields from a Snapshot Compressive Image

In this paper, we explore the potential of Snapshot Compressive Imaging (SCI) technique for recovering the underlying 3D scene representation from a single temporal compressed image. SCI is a cost-effective method that enables the recording of high-dimensional data, such as hyperspectral or temporal information, into a single image using low-cost 2D imaging sensors. To achieve this, a series of specially designed 2D masks are usually employed, which not only reduces storage requirements but also offers potential privacy protection. Inspired by this, to take one step further, our approach builds upon the powerful 3D scene representation capabilities of neural radiance fields (NeRF). Specifically, we formulate the physical imaging process of SCI as part of the training of NeRF, allowing us to exploit its impressive performance in capturing complex scene structures. To assess the effectiveness of our method, we conduct extensive evaluations using both synthetic data and real data captured by our SCI system. Extensive experimental results demonstrate that our proposed approach surpasses the state-of-the-art methods in terms of image reconstruction and novel view image synthesis. Moreover, our method also exhibits the ability to restore high frame-rate multi-view consistent images by leveraging SCI and the rendering capabilities of NeRF. The code is available at https://github.com/WU-CVGL/SCINeRF.

eess.IV↗

arXiv2024

RoboKeyGen: Robot Pose and Joint Angles Estimation via Diffusion-based 3D Keypoint Generation

Estimating robot pose and joint angles is significant in advanced robotics, enabling applications like robot collaboration and online hand-eye calibration.However, the introduction of unknown joint angles makes prediction more complex than simple robot pose estimation, due to its higher dimensionality.Previous methods either regress 3D keypoints directly or utilise a render&compare strategy. These approaches often falter in terms of performance or efficiency and grapple with the cross-camera gap problem.This paper presents a novel framework that bifurcates the high-dimensional prediction task into two manageable subtasks: 2D keypoints detection and lifting 2D keypoints to 3D. This separation promises enhanced performance without sacrificing the efficiency innate to keypoint-based techniques.A vital component of our method is the lifting of 2D keypoints to 3D keypoints. Common deterministic regression methods may falter when faced with uncertainties from 2D detection errors or self-occlusions.Leveraging the robust modeling potential of diffusion models, we reframe this issue as a conditional 3D keypoints generation task. To bolster cross-camera adaptability, we introduce theNormalised Camera Coordinate Space (NCCS), ensuring alignment of estimated 2D keypoints across varying camera intrinsics.Experimental results demonstrate that the proposed method outperforms the state-of-the-art render\&compare method and achieves higher inference speed.Furthermore, the tests accentuate our method's robust cross-camera generalisation capabilities.We intend to release both the dataset and code in https://nimolty.github.io/Robokeygen/

cs.RO↗