arXiv subjects

Dong Wang

Publications and source records attributed to Dong Wang.

At least 307 records · Page 17Linked to original sources

arXiv2024

AlignVSR: Audio-Visual Cross-Modal Alignment for Visual Speech Recognition

Visual Speech Recognition (VSR) aims to recognize corresponding text by analyzing visual information from lip movements. Due to the high variability and weak information of lip movements, VSR tasks require effectively utilizing any information from any source and at any level. In this paper, we propose a VSR method based on audio-visual cross-modal alignment, named AlignVSR. The method leverages the audio modality as an auxiliary information source and utilizes the global and local correspondence between the audio and visual modalities to improve visual-to-text inference. Specifically, the method first captures global alignment between video and audio through a cross-modal attention mechanism from video frames to a bank of audio units. Then, based on the temporal correspondence between audio and video, a frame-level local alignment loss is introduced to refine the global alignment, improving the utility of the audio information. Experimental results on the LRS2 and CNVSRC.Single datasets consistently show that AlignVSR outperforms several mainstream VSR methods, demonstrating its superior and robust performance.

cs.SD↗

arXiv2024

NeuralMAG: Fast and Generalizable Micromagnetic Simulation with Deep Neural Nets

Micromagnetics has made significant strides, particularly due to its wide-ranging applications in magnetic storage design. Numerical simulation is a cornerstone of micromagnetics research, relying on first-principle rules to compute the dynamic evolution of micromagnetic systems based on the renowned LLG equation, named after Landau, Lifshitz, and Gilbert. However, simulations are often hindered by their slow speed. Although Fast-Fourier transformation (FFT) calculations reduce the computational complexity to O(NlogN), it remains impractical for large-scale simulations. In this paper, we introduce NeuralMAG, a deep learning approach to micromagnetic simulation. Our approach follows the LLG iterative framework but accelerates demagnetizing field computation through the employment of a U-shaped neural network (Unet). The Unet architecture comprises an encoder that extracts aggregated spins at various scales and learns the local interaction at each scale, followed by a decoder that accumulates the local interactions at different scales to approximate the global convolution. This divide-and-accumulate scheme achieves a time complexity of O(N), significantly enhancing the speed and feasibility of large-scale simulations. Unlike existing neural methods, NeuralMAG concentrates on the core computation rather than an end-to-end approximation for a specific task, making it inherently generalizable. To validate the new approach, we trained a single model and evaluated it on two micromagnetics tasks with various sample sizes, shapes, and material settings.

cs.LG↗

arXiv2024

Developer Reactions to Protestware in Open Source Software: The cases of color.js and es5.ext

There is growing concern about maintainers self-sabotaging their work in order to take political or economic stances, a practice referred to as "protestware". Our objective is to understand the discourse around discussions on such an attack, how it is received by the community, and whether developers respond to the attack in a timely manner. We study two notable protestware cases i.e., colors.js and es5-ext. Results indicate that protestware discussions are spread more quickly on the GitHub platform, while security vulnerabilities are faster on social media. By establishing a taxonomy of protestware discussions, we identify posts that express stances and provide technical mitigation instructions. We applied a thematic analysis to 684 protestware related posts to identify five major themes during the discussions: i. disseminate and response, ii. stance, iii. reputation, iv. communicative styles, v. rights and ethics. This work sheds light on the nuanced landscape of protestware discussions, offering insights for both researchers and developers into maintaining a healthy balance between the political or social actions of developers and the collective well-being of the open-source community.

cs.SE↗

arXiv2024

KOI: Accelerating Online Imitation Learning via Hybrid Key-state Guidance

Online Imitation Learning struggles with the gap between extensive online exploration space and limited expert trajectories, hindering efficient exploration due to inaccurate reward estimation. Inspired by the findings from cognitive neuroscience, we hypothesize that an agent could estimate precise task-aware reward for efficient online exploration, through decomposing the target task into the objectives of "what to do" and the mechanisms of "how to do". In this work, we introduce the hybrid Key-state guided Online Imitation (KOI) learning method, which leverages the integration of semantic and motion key states as guidance for reward estimation. Initially, we utilize visual-language models to extract semantic key states from expert trajectory, indicating the objectives of "what to do". Within the intervals between semantic key states, optical flow is employed to capture motion key states to understand the mechanisms of "how to do". By integrating a thorough grasp of hybrid key states, we refine the trajectory-matching reward computation, accelerating online imitation learning with task-aware exploration. We evaluate not only the success rate of the tasks in the Meta-World and LIBERO environments, but also the trend of variance during online imitation learning, proving that our method is more sample efficient. We also conduct real-world robotic manipulation experiments to validate the efficacy of our method, demonstrating the practical applicability of our KOI method. Videos and code are available at https://gewu-lab.github.io/Keystate_Online_Imitation/.

cs.RO↗

arXiv2024

Train Once, Deploy Anywhere: Matryoshka Representation Learning for Multimodal Recommendation

Despite recent advancements in language and vision modeling, integrating rich multimodal knowledge into recommender systems continues to pose significant challenges. This is primarily due to the need for efficient recommendation, which requires adaptive and interactive responses. In this study, we focus on sequential recommendation and introduce a lightweight framework called full-scale Matryoshka representation learning for multimodal recommendation (fMRLRec). Our fMRLRec captures item features at different granularities, learning informative representations for efficient recommendation across multiple dimensions. To integrate item features from diverse modalities, fMRLRec employs a simple mapping to project multimodal item features into an aligned feature space. Additionally, we design an efficient linear transformation that embeds smaller features into larger ones, substantially reducing memory requirements for large-scale training on recommendation data. Combined with improved state space modeling techniques, fMRLRec scales to different dimensions and only requires one-time training to produce multiple models tailored to various granularities. We demonstrate the effectiveness and efficiency of fMRLRec on multiple benchmark datasets, which consistently achieves superior performance over state-of-the-art baseline methods. We make our code and data publicly available at https://github.com/yueqirex/fMRLRec.

cs.IR↗

arXiv2024

Proof-of-principle experimental demonstration of quantum secure imaging based on quantum key distribution

We present a quantum secure imaging (QSI) scheme based on the phase encoding and weak + vacuum decoy-state BB84 protocol of quantum key distribution (QKD). It allows us to implement a computational ghost imaging (CGI) system with more simplified equipment and reconstructed algorithms by using a digital micro-mirror device (DMD) to preset the specific spatial distribution of the light intensity. What's more, the quantum bit error rate (QBER) and the secure key rate analytical functions of QKD are used to see through the intercept-resend jamming attacks and ensure the authenticity of the imaging information. In the experiment, we obtained the image of the object quickly and efficiently by measuring the signal photon counts with single-photon detector (SPD), and achieved a secure key rate of 571.0 bps and a secure QBER of 3.99%, which is well below the lower bound of QBER of 14.51$\%$. Besides, our imaging system uses a laser with invisible wavelength of 1550 nm, whose intensity is low as single-photon, that can realize weak-light imaging and is immune to the stray light or air turbulence, thus it will become a better choice for quantum security radar against intercept-resend jamming attacks.

quant-ph↗

arXiv2024

Phase-coding quantum-key-distribution system based on Sagnac-Mach-Zehnder interferometers

Stability and robustness are important criteria to evaluate the performance of a quantum-key-distribution (QKD) system in real-life applications. However, the inherent birefringence effect of the fiber channel and disturbance caused by the variation of the deployment environment of the QKD system tremendously decreases its performance. To eliminate this adverse impact, we propose a polarization-insensitive phase-coding QKD system based on Sagnac-Mach-Zehnder interferometers. Verified by theoretical analysis and experimental tests, this QKD system is robust against channel polarization disturbance. The robustness and long-term stability of the QKD system is confirmed with a 10-day continuous operation over a 12.6-dB channel, which consists of a 50-km fiber spool and a polarization scrambler (2 rad/s). As a result, an average quantum bit error rate of 0.958% and a sustained secure key rate of 3.68 kbps are obtained. Moreover, the secure key rate of the QKD system for a typical channel loss of 10 dB reaches 6.89 kbps, and the achievable maximum transmission distance exceeds 125 km.

quant-ph↗

arXiv2024

Quantitative Analysis of Audio-Visual Tasks: An Information-Theoretic Perspective

In the field of spoken language processing, audio-visual speech processing is receiving increasing research attention. Key components of this research include tasks such as lip reading, audio-visual speech recognition, and visual-to-speech synthesis. Although significant success has been achieved, theoretical analysis is still insufficient for audio-visual tasks. This paper presents a quantitative analysis based on information theory, focusing on information intersection between different modalities. Our results show that this analysis is valuable for understanding the difficulties of audio-visual processing tasks as well as the benefits that could be obtained by modality integration.

cs.SD↗

arXiv2024

Quantifying genuine tripartite entanglement by reshaping the state

Although genuine multipartite entanglement (GME), as one quantum resource, is indispensable in quantum information processing, most of the existing measures cannot detect GME faithfully. In this paper, we present a novel GME measure, namely the minimum pairwise concurrence (MPC), by introducing pairwise entanglement, which characters the entanglement between two single-qubit subsystems of a multipartite system without tracing out the remaining qubit. The pairwise entanglement can be obtained by combining the entanglement of reduced subsystem and three-tangle. Compared with the existing measures, the MPC measure outperforms the previous ones in many aspects. Due to its fine properties, it thus is believed that the MPC could be one of good candidates in achieving potential quantum tasks and also facilitate the understanding for GME.

quant-ph↗

arXiv2024

Narrowing band gap chemically and physically: Conductive dense hydrocarbon

Band gap energy of an organic molecule can be reduced by intermolecular interaction enhancement, and thus, certain polycyclic aromatic hydrocarbons (PAHs), which are insulators with wide band gaps, are expected to undergo insulator-metal transitions by simple compression. Such a pressure-induced electronic transition can be exploited to transform non-metallic organic materials into states featuring intriguing electronic characteristics such as high-temperature superconductivity. Numerous attempts have been made to metalize various small PAHs, but so far only pressure-induced amorphization well below the megabar region was observed. The wide band gap energy of the small PAHs and low chemical stability under simple compression are the bottlenecks. We have investigated the band gap energy evolution and the crystal structural compression of the large PAH molecules, where the band gap energy is significantly reduced by increasing the number of π-electrons and improved chemical stability with fully benzenoid molecular structure. Herein, we present a pressure-induced transition in dicoronylene, C48H20, an insulator at ambient conditions that transforms into a semi-metallic state above 23.0 GPa with a three-order-of-magnitude reduction in resistivity. In-situ UV-visible absorption, transport property measurement, Raman spectroscopy, X-ray diffraction and density functional theory calculations were performed to provide tentative explanations to the alterations in its electronic structure at high pressure. The discovery of an electronic transition at pressures well below the megabar is a promising step towards realization of a single component purely hydrocarbon molecular metal in the near future.

cond-mat.mtrl-sci↗

arXiv2024

Quantum Characteristics Near Event Horizons

We investigate quantum characteristics around Schwarzschild black hole, exploring various quantum resources and their interplay in curved space-time. Our analysis reveals intriguing behaviors of quantum coherence, global and genuine multipartite entanglement, first-order coherence, and mutual information in different scenarios. Initially, we consider three particles shared among Alice, Bob, and Charlie in a Minkowski space far from the event horizon, where these particles are correlated via GHZ-type correlation. While Alice's particle remains in Minkowski space, Bob and Charlie accelerate towards the event horizon, experiencing black hole evaporation and generating antiparticles correlated via the Hawking effect. We employ the Kruskal basis formulation to derive a penta-partite pure state shared among particles inside and outside the event horizon. Investigating different scenarios among particles both inside and outside the event horizon, we observe how quantum resources evolve and distribute among consideration of different particles with Hawking temperature and mode frequency. The trade-off relationship between first-order coherence and concurrence fill persists, indicating the intricate interplay between coherence and entanglement. Notably, the mutual information between external observers and particles inside the black hole becomes non-zero, deepening our understanding of quantum effects in curved space-time and shedding light on the quantum nature of the black hole. We believe that these findings will pave the way for future investigations into the fundamental quantum mechanical aspects of gravity under extreme environments.

quant-ph↗

arXiv2024

Computing optimal partition problems via Lagrange multiplier approach

In this paper, we consider numerical approximations for the optimal partition problem using Lagrange multipliers. By rewriting it into constrained gradient flows, three and four steps numerical schemes based on the Lagrange multiplier approach \cite{ChSh22,ChSh_II22} are proposed to solve the constrained gradient system. Numerical schemes proposed for the constrained gradient flows satisfy the nice properties of orthogonality-preserving, norm-preserving, positivity-preserving and energy dissipating. The proposed schemes are very efficient in which only linear Poisson equations are solved at each time step. Extensive numerical results in 2D and 3D for optimal partition problem are presented to validate the effectiveness and accuracy of the proposed numerical schemes.

math.OC↗

arXiv2024

Intertwined Biases Across Social Media Spheres: Unpacking Correlations in Media Bias Dimensions

Media bias significantly shapes public perception by reinforcing stereotypes and exacerbating societal divisions. Prior research has often focused on isolated media bias dimensions such as \textit{political bias} or \textit{racial bias}, neglecting the complex interrelationships among various bias dimensions across different topic domains. Moreover, we observe that models trained on existing media bias benchmarks fail to generalize effectively on recent social media posts, particularly in certain bias identification tasks. This shortfall primarily arises because these benchmarks do not adequately reflect the rapidly evolving nature of social media content, which is characterized by shifting user behaviors and emerging trends. In response to these limitations, our research introduces a novel dataset collected from YouTube and Reddit over the past five years. Our dataset includes automated annotations for YouTube content across a broad spectrum of bias dimensions, such as gender, racial, and political biases, as well as hate speech, among others. It spans diverse domains including politics, sports, healthcare, education, and entertainment, reflecting the complex interplay of biases across different societal sectors. Through comprehensive statistical analysis, we identify significant differences in bias expression patterns and intra-domain bias correlations across these domains. By utilizing our understanding of the correlations among various bias dimensions, we lay the groundwork for creating advanced systems capable of detecting multiple biases simultaneously. Overall, our dataset advances the field of media bias identification, contributing to the development of tools that promote fairer media consumption. The comprehensive awareness of existing media bias fosters more ethical journalism, promotes cultural sensitivity, and supports a more informed and equitable public discourse.

cs.SI↗

arXiv2024

Flexible Beam Coverage Optimization for Movable-Antenna Array

Fluid antennas (FAs) and movable antennas (MAs) have attracted increasing attention in wireless communications recently. As compared to the conventional fixed-position antennas (FPAs), their geometry can be dynamically reconfigured, such that more flexible beamforming can be achieved for signal coverage and/or interference nulling. In this paper, we investigate the use of MAs to achieve uniform coverage for multiple regions with arbitrary number and width in the spatial domain. In particular, we aim to jointly optimize the MAs weights and positions within a linear array to maximize the minimum beam gain over the desired spatial regions. However, the resulting problem is non-convex and difficult to be optimally solved. To tackle this difficulty, we propose an alternating optimization (AO) algorithm to obtain a high-quality suboptimal solution, where the MAs weights and positions are alternately optimized by applying successive convex approximation (SCA) technique. Numerical results show that our proposed MAbased beam coverage scheme can achieve much better performance than conventional FPAs.

eess.SP↗

arXiv2024

Joint Beamforming and Antenna Position Optimization for Movable Antenna-Assisted Spectrum Sharing

Fluid antennas (FAs) and movable antennas (MAs) have drawn increasing attention in wireless communications recently due to their ability to create favorable channel conditions via local antenna movement within a confined region. In this letter, we advance their application for cognitive radio to facilitate efficient spectrum sharing between primary and secondary communication systems. In particular, we aim to jointly optimize the transmit beamforming and MA positions at a secondary transmitter (ST) to maximize the received signal power at a secondary receiver (SR) subject to the constraints on its imposed co-channel interference power with multiple primary receivers (PRs). However, such an optimization problem is difficult to be optimally solved due to the highly nonlinear functions of the received signal/interference power at the SR/all PRs in terms of the MA positions. To drive useful insights, we first perform theoretical analyses to unveil MAs' capability to achieve maximum-ratio transmission with the SR and effective interference mitigation for all PRs at the same time. To solve the MA position optimization problem, we propose an alternating optimization (AO) algorithm to obtain a high-quality suboptimal solution. Numerical results demonstrate that our proposed algorithms can significantly outperform the conventional fixed-position antennas (FPAs) and other baseline schemes.

eess.SP↗

arXiv2024

HiBid: A Cross-Channel Constrained Bidding System with Budget Allocation by Hierarchical Offline Deep Reinforcement Learning

Online display advertising platforms service numerous advertisers by providing real-time bidding (RTB) for the scale of billions of ad requests every day. The bidding strategy handles ad requests cross multiple channels to maximize the number of clicks under the set financial constraints, i.e., total budget and cost-per-click (CPC), etc. Different from existing works mainly focusing on single channel bidding, we explicitly consider cross-channel constrained bidding with budget allocation. Specifically, we propose a hierarchical offline deep reinforcement learning (DRL) framework called ``HiBid'', consisted of a high-level planner equipped with auxiliary loss for non-competitive budget allocation, and a data augmentation enhanced low-level executor for adaptive bidding strategy in response to allocated budgets. Additionally, a CPC-guided action selection mechanism is introduced to satisfy the cross-channel CPC constraint. Through extensive experiments on both the large-scale log data and online A/B testing, we confirm that HiBid outperforms six baselines in terms of the number of clicks, CPC satisfactory ratio, and return-on-investment (ROI). We also deploy HiBid on Meituan advertising platform to already service tens of thousands of advertisers every day.

cs.LG↗

arXiv2024

Selecting Initial Seeds for Better JVM Fuzzing

Literature in traditional program fuzzing has confirmed that effectiveness is largely impacted by redundancy among initial seeds, thereby proposing a series of seed selection methods. JVM fuzzing, compared to traditional ones, presents unique characteristics, including large-scale and intricate code, and programs with both syntactic and semantic features. However, it remains unclear whether the existing seed selection methods are suitable for JVM fuzzing and whether utilizing program features can enhance effectiveness. To address this, we devise a total of 10 initial seed selection methods, comprising coverage-based, prefuzz-based, and program-feature-based methods. We then conduct an empirical study on three JVM implementations to extensively evaluate the performance of the seed selection methods within two SOTA fuzzing techniques (JavaTailor and VECT). Specifically, we examine performance from three aspects: (i) effectiveness and efficiency using widely studied initial seeds, (ii) effectiveness using the programs in the wild, and (iii) the ability to detect new bugs. Evaluation results first show that the program-feature-based method that utilizes the control flow graph not only has a significantly lower time overhead (i.e., 30s), but also outperforms other methods, achieving 142% to 269% improvement compared to the full set of initial seeds. Second, results reveal that the initial seed selection greatly improves the quality of wild programs and exhibits complementary effectiveness by detecting new behaviors. Third, results demonstrate that given the same testing period, initial seed selection improves the JVM fuzzing techniques by detecting more unknown bugs. Particularly, 21 out of the 25 detected bugs have been confirmed or fixed by developers. This work takes the first look at initial seed selection in JVM fuzzing, confirming its importance in fuzzing effectiveness and efficiency.

cs.SE↗

arXiv2024

MambaVT: Spatio-Temporal Contextual Modeling for robust RGB-T Tracking

Existing RGB-T tracking algorithms have made remarkable progress by leveraging the global interaction capability and extensive pre-trained models of the Transformer architecture. Nonetheless, these methods mainly adopt imagepair appearance matching and face challenges of the intrinsic high quadratic complexity of the attention mechanism, resulting in constrained exploitation of temporal information. Inspired by the recently emerged State Space Model Mamba, renowned for its impressive long sequence modeling capabilities and linear computational complexity, this work innovatively proposes a pure Mamba-based framework (MambaVT) to fully exploit spatio-temporal contextual modeling for robust visible-thermal tracking. Specifically, we devise the long-range cross-frame integration component to globally adapt to target appearance variations, and introduce short-term historical trajectory prompts to predict the subsequent target states based on local temporal location clues. Extensive experiments show the significant potential of vision Mamba for RGB-T tracking, with MambaVT achieving state-of-the-art performance on four mainstream benchmarks while requiring lower computational costs. We aim for this work to serve as a simple yet strong baseline, stimulating future research in this field. The code and pre-trained models will be made available.

cs.CV↗