arXiv subjects

Dong Wang

Publications and source records attributed to Dong Wang.

At least 361 records · Page 20Linked to original sources

arXiv2024

Nonreciprocal Photon-Phonon Entanglement in Kerr-Modified Spinning Cavity Magnomechanics

Cavity magnomechanics has shown great potential in studying macroscopic quantum effects, especially for quantum entanglement, which is a key resource for quantum information science. Here we propose to realize magnon mediated nonreciprocal photon-phonon entanglement, which exhibits asymmetry when opposite magnetic or driving fields are respectively applied to the magnons with the Kerr effect or the photons with the Sagnac effect. We find that the mean magnon number can selectively exhibit nonreciprocal linear or nonlinear (bistable) behavior with the strength of the strong driving field on the cavity. Assisted by this driving field, the magnon-phonon coupling is greatly enhanced, leading to the nonreciprocal photon-phonon entanglement via the swapping interaction between the magnons and photons. This nonreciprocal entanglement can be significantly enhanced with the magnon Kerr and Sagnac effects. Given the available parameters, the nonreciprocal photon-phonon entanglement can be preserved at $\sim3$ K, showing remarkable resilience against the bath temperature. The result reveals that our paper holds promise in developing various nonreciprocal devices with both the magnon Kerr and Sagnac effects in cavity magnomechanics.

quant-ph↗

arXiv2024

GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting

In this paper, we introduce \textbf{GS-SLAM} that first utilizes 3D Gaussian representation in the Simultaneous Localization and Mapping (SLAM) system. It facilitates a better balance between efficiency and accuracy. Compared to recent SLAM methods employing neural implicit representations, our method utilizes a real-time differentiable splatting rendering pipeline that offers significant speedup to map optimization and RGB-D rendering. Specifically, we propose an adaptive expansion strategy that adds new or deletes noisy 3D Gaussians in order to efficiently reconstruct new observed scene geometry and improve the mapping of previously observed areas. This strategy is essential to extend 3D Gaussian representation to reconstruct the whole scene rather than synthesize a static object in existing methods. Moreover, in the pose tracking process, an effective coarse-to-fine technique is designed to select reliable 3D Gaussian representations to optimize camera pose, resulting in runtime reduction and robust estimation. Our method achieves competitive performance compared with existing state-of-the-art real-time methods on the Replica, TUM-RGBD datasets. Project page: https://gs-slam.github.io/.

cs.CV↗

arXiv2024

Deployable Reinforcement Learning with Variable Control Rate

Deploying controllers trained with Reinforcement Learning (RL) on real robots can be challenging: RL relies on agents' policies being modeled as Markov Decision Processes (MDPs), which assume an inherently discrete passage of time. The use of MDPs results in that nearly all RL-based control systems employ a fixed-rate control strategy with a period (or time step) typically chosen based on the developer's experience or specific characteristics of the application environment. Unfortunately, the system should be controlled at the highest, worst-case frequency to ensure stability, which can demand significant computational and energy resources and hinder the deployability of the controller on onboard hardware. Adhering to the principles of reactive programming, we surmise that applying control actions only when necessary enables the use of simpler hardware and helps reduce energy consumption. We challenge the fixed frequency assumption by proposing a variant of RL with variable control rate. In this approach, the policy decides the action the agent should take as well as the duration of the time step associated with that action. In our new setting, we expand Soft Actor-Critic (SAC) to compute the optimal policy with a variable control rate, introducing the Soft Elastic Actor-Critic (SEAC) algorithm. We show the efficacy of SEAC through a proof-of-concept simulation driving an agent with Newtonian kinematics. Our experiments show higher average returns, shorter task completion times, and reduced computational resources when compared to fixed rate policies.

cs.RO↗

arXiv2024

Open-Vocabulary Federated Learning with Multimodal Prototyping

Existing federated learning (FL) studies usually assume the training label space and test label space are identical. However, in real-world applications, this assumption is too ideal to be true. A new user could come up with queries that involve data from unseen classes, and such open-vocabulary queries would directly defect such FL systems. Therefore, in this work, we explicitly focus on the under-explored open-vocabulary challenge in FL. That is, for a new user, the global server shall understand her/his query that involves arbitrary unknown classes. To address this problem, we leverage the pre-trained vision-language models (VLMs). In particular, we present a novel adaptation framework tailored for VLMs in the context of FL, named as Federated Multimodal Prototyping (Fed-MP). Fed-MP adaptively aggregates the local model weights based on light-weight client residuals, and makes predictions based on a novel multimodal prototyping mechanism. Fed-MP exploits the knowledge learned from the seen classes, and robustifies the adapted VLM to unseen categories. Our empirical evaluation on various datasets validates the effectiveness of Fed-MP.

cs.CL↗

arXiv2024

Unified Sequence-to-Sequence Learning for Single- and Multi-Modal Visual Object Tracking

In this paper, we introduce a new sequence-to-sequence learning framework for RGB-based and multi-modal object tracking. First, we present SeqTrack for RGB-based tracking. It casts visual tracking as a sequence generation task, forecasting object bounding boxes in an autoregressive manner. This differs from previous trackers, which depend on the design of intricate head networks, such as classification and regression heads. SeqTrack employs a basic encoder-decoder transformer architecture. The encoder utilizes a bidirectional transformer for feature extraction, while the decoder generates bounding box sequences autoregressively using a causal transformer. The loss function is a plain cross-entropy. Second, we introduce SeqTrackv2, a unified sequence-to-sequence framework for multi-modal tracking tasks. Expanding upon SeqTrack, SeqTrackv2 integrates a unified interface for auxiliary modalities and a set of task-prompt tokens to specify the task. This enables it to manage multi-modal tracking tasks using a unified model and parameter set. This sequence learning paradigm not only simplifies the tracking framework, but also showcases superior performance across 14 challenging benchmarks spanning five single- and multi-modal tracking tasks. The code and models are available at https://github.com/chenxin-dlut/SeqTrackv2.

cs.CV↗

arXiv2024

HPL-ESS: Hybrid Pseudo-Labeling for Unsupervised Event-based Semantic Segmentation

Event-based semantic segmentation has gained popularity due to its capability to deal with scenarios under high-speed motion and extreme lighting conditions, which cannot be addressed by conventional RGB cameras. Since it is hard to annotate event data, previous approaches rely on event-to-image reconstruction to obtain pseudo labels for training. However, this will inevitably introduce noise, and learning from noisy pseudo labels, especially when generated from a single source, may reinforce the errors. This drawback is also called confirmation bias in pseudo-labeling. In this paper, we propose a novel hybrid pseudo-labeling framework for unsupervised event-based semantic segmentation, HPL-ESS, to alleviate the influence of noisy pseudo labels. In particular, we first employ a plain unsupervised domain adaptation framework as our baseline, which can generate a set of pseudo labels through self-training. Then, we incorporate offline event-to-image reconstruction into the framework, and obtain another set of pseudo labels by predicting segmentation maps on the reconstructed images. A noisy label learning strategy is designed to mix the two sets of pseudo labels and enhance the quality. Moreover, we propose a soft prototypical alignment module to further improve the consistency of target domain features. Extensive experiments show that our proposed method outperforms existing state-of-the-art methods by a large margin on the DSEC-Semantic dataset (+5.88% accuracy, +10.32% mIoU), which even surpasses several supervised methods.

cs.CV↗

arXiv2024

Evidence-Driven Retrieval Augmented Response Generation for Online Misinformation

The proliferation of online misinformation has posed significant threats to public interest. While numerous online users actively participate in the combat against misinformation, many of such responses can be characterized by the lack of politeness and supporting facts. As a solution, text generation approaches are proposed to automatically produce counter-misinformation responses. Nevertheless, existing methods are often trained end-to-end without leveraging external knowledge, resulting in subpar text quality and excessively repetitive responses. In this paper, we propose retrieval augmented response generation for online misinformation (RARG), which collects supporting evidence from scientific sources and generates counter-misinformation responses based on the evidences. In particular, our RARG consists of two stages: (1) evidence collection, where we design a retrieval pipeline to retrieve and rerank evidence documents using a database comprising over 1M academic articles; (2) response generation, in which we align large language models (LLMs) to generate evidence-based responses via reinforcement learning from human feedback (RLHF). We propose a reward function to maximize the utilization of the retrieved evidence while maintaining the quality of the generated text, which yields polite and factual responses that clearly refutes misinformation. To demonstrate the effectiveness of our method, we study the case of COVID-19 and perform extensive experiments with both in- and cross-domain datasets, where RARG consistently outperforms baselines by generating high-quality counter-misinformation responses.

cs.CL↗

arXiv2024

Computational modeling of the physical features that influence breast cancer invasion into adipose tissue

Breast cancer invasion into adipose tissue strongly influences disease progression and metastasis. The degree of cancer cell invasion into adipose tissue depends on numerous biochemical and physical properties of cancer cells, adipocytes, and other key components of adipose tissue. We model breast cancer invasion into adipose tissue as a physical process by carrying out simulations of active, cohesive spherical particles (cancer cells) invading into confluent packings of deformable polyhedra (adipocytes). We quantify the degree of invasion by calculating the interfacial area $A_t$ between cancer cells and adipocytes. We determine the long-time value of $A_t$ versus the activity and strength of the cohesion between cancer cells, as well as mechanical properties of the adipocytes and extracellular matrix (ECM) in which the adipocytes are embedded. We show that the degree of invasion collapses onto a master curve by plotting it versus a dimensionless energy scale $E_c$, which grows linearly with mean-square fluctuations and persistence time of the cancer cell velocities, is inversely proportional to the pressure of the system, and has an offset that increases with the cancer cell cohesive energy. The condition, $E_c \gg 1$, indicates that cancer cells will invade the adipose tissue, whereas for $E_c \ll 1$, the cancer cells and adipocytes remain demixed. We also show that constraints on adipocyte positions by the ECM decrease $A_t$ relative to that obtained for unconstrained adipocytes. Finally, spatial heterogeneity in structural and mechanical properties of the adipocytes in the presence of ECM impedes invasion relative to adipose tissue with uniform properties.

physics.bio-ph↗

arXiv2024

Implicit Event-RGBD Neural SLAM

Implicit neural SLAM has achieved remarkable progress recently. Nevertheless, existing methods face significant challenges in non-ideal scenarios, such as motion blur or lighting variation, which often leads to issues like convergence failures, localization drifts, and distorted mapping. To address these challenges, we propose EN-SLAM, the first event-RGBD implicit neural SLAM framework, which effectively leverages the high rate and high dynamic range advantages of event data for tracking and mapping. Specifically, EN-SLAM proposes a differentiable CRF (Camera Response Function) rendering technique to generate distinct RGB and event camera data via a shared radiance field, which is optimized by learning a unified implicit representation with the captured event and RGBD supervision. Moreover, based on the temporal difference property of events, we propose a temporal aggregating optimization strategy for the event joint tracking and global bundle adjustment, capitalizing on the consecutive difference constraints of events, significantly enhancing tracking accuracy and robustness. Finally, we construct the simulated dataset DEV-Indoors and real captured dataset DEV-Reals containing 6 scenes, 17 sequences with practical motion blur and lighting changes for evaluations. Experimental results show that our method outperforms the SOTA methods in both tracking ATE and mapping ACC with a real-time 17 FPS in various challenging environments. Project page: https://delinqu.github.io/EN-SLAM.

cs.CV↗

arXiv2024

A modularized algorithmic framework for interface related optimization problems using characteristic functions

In this paper, we consider the algorithms and convergence for a general optimization problem, which has a wide range of applications in image segmentation, topology optimization, flow network formulation, and surface reconstruction. In particular, the problem focuses on interface related optimization problems where the interface is implicitly described by characteristic functions of the corresponding domains. Under such representation and discretization, the problem is then formulated into a discretized optimization problem where the objective function is concave with respect to characteristic functions and convex with respect to state variables. We show that under such structure, the iterative scheme based on alternative minimization can converge to a local minimizer. Extensive numerical examples are performed to support the theory.

math.OC↗

arXiv2024

DemiNet: Dependency-Aware Multi-Interest Network with Self-Supervised Graph Learning for Click-Through Rate Prediction

In this paper, we propose a novel model named DemiNet (short for DEpendency-Aware Multi-Interest Network) to address the above two issues. To be specific, we first consider various dependency types between item nodes and perform dependency-aware heterogeneous attention for denoising and obtaining accurate sequence item representations. Secondly, for multiple interests extraction, multi-head attention is conducted on top of the graph embedding. To filter out noisy inter-item correlations and enhance the robustness of extracted interests, self-supervised interest learning is introduced to the above two steps. Thirdly, to aggregate the multiple interests, interest experts corresponding to different interest routes give rating scores respectively, while a specialized network assigns the confidence of each score. Experimental results on three real-world datasets demonstrate that the proposed DemiNet significantly improves the overall recommendation performance over several state-of-the-art baselines. Further studies verify the efficacy and interpretability benefits brought by the fine-grained user interest modeling.

cs.IR↗

arXiv2024

Federated Recommendation via Hybrid Retrieval Augmented Generation

Federated Recommendation (FR) emerges as a novel paradigm that enables privacy-preserving recommendations. However, traditional FR systems usually represent users/items with discrete identities (IDs), suffering from performance degradation due to the data sparsity and heterogeneity in FR. On the other hand, Large Language Models (LLMs) as recommenders have proven effective across various recommendation scenarios. Yet, LLM-based recommenders encounter challenges such as low inference efficiency and potential hallucination, compromising their performance in real-world scenarios. To this end, we propose GPT-FedRec, a federated recommendation framework leveraging ChatGPT and a novel hybrid Retrieval Augmented Generation (RAG) mechanism. GPT-FedRec is a two-stage solution. The first stage is a hybrid retrieval process, mining ID-based user patterns and text-based item features. Next, the retrieved results are converted into text prompts and fed into GPT for re-ranking. Our proposed hybrid retrieval mechanism and LLM-based re-rank aims to extract generalized features from data and exploit pretrained knowledge within LLM, overcoming data sparsity and heterogeneity in FR. In addition, the RAG approach also prevents LLM hallucination, improving the recommendation performance for real-world users. Experimental results on diverse benchmark datasets demonstrate the superior performance of GPT-FedRec against state-of-the-art baseline methods.

cs.IR↗

arXiv2024

Unstoppable Attack: Label-Only Model Inversion via Conditional Diffusion Model

Model inversion attacks (MIAs) aim to recover private data from inaccessible training sets of deep learning models, posing a privacy threat. MIAs primarily focus on the white-box scenario where attackers have full access to the model's structure and parameters. However, practical applications are usually in black-box scenarios or label-only scenarios, i.e., the attackers can only obtain the output confidence vectors or labels by accessing the model. Therefore, the attack models in existing MIAs are difficult to effectively train with the knowledge of the target model, resulting in sub-optimal attacks. To the best of our knowledge, we pioneer the research of a powerful and practical attack model in the label-only scenario. In this paper, we develop a novel MIA method, leveraging a conditional diffusion model (CDM) to recover representative samples under the target label from the training set. Two techniques are introduced: selecting an auxiliary dataset relevant to the target model task and using predicted labels as conditions to guide training CDM; and inputting target label, pre-defined guidance strength, and random noise into the trained attack model to generate and correct multiple results for final selection. This method is evaluated using Learned Perceptual Image Patch Similarity as a new metric and as a judgment basis for deciding the values of hyper-parameters. Experimental results show that this method can generate similar and accurate samples to the target label, outperforming generators of previous approaches.

cs.AI↗

arXiv2024

Safe Offline Reinforcement Learning with Real-Time Budget Constraints

Aiming at promoting the safe real-world deployment of Reinforcement Learning (RL), research on safe RL has made significant progress in recent years. However, most existing works in the literature still focus on the online setting where risky violations of the safety budget are likely to be incurred during training. Besides, in many real-world applications, the learned policy is required to respond to dynamically determined safety budgets (i.e., constraint threshold) in real time. In this paper, we target at the above real-time budget constraint problem under the offline setting, and propose Trajectory-based REal-time Budget Inference (TREBI) as a novel solution that models this problem from the perspective of trajectory distribution and solves it through diffusion model planning. Theoretically, we prove an error bound of the estimation on the episodic reward and cost under the offline setting and thus provide a performance guarantee for TREBI. Empirical results on a wide range of simulation tasks and a real-world large-scale advertising application demonstrate the capability of TREBI in solving real-time budget constraint problems under offline settings.

cs.LG↗

arXiv2024

Quantum coherence and entanglement under the influence of decoherence

In this work, we delve into the dynamic traits of the relative entropy of quantum coherence (REQC) as the quantum system interacts with the different noisy channels, drawing comparisons with entanglement (concurrence). The research results demonstrate the broader prevalence and stronger robustness of the REQC as opposed to concurrence. It's worth noting that the bit flip channel cannot uphold a constant nonzero frozen the REQC, besides, the concurrence follows a pattern of temporary reduction to zero, followed by recovery after a certain time span. More importantly, the REQC maintains its presence consistently until reaching a critical threshold, whereas concurrence experiences completely attenuation to zero under the influence of phase damping and amplitude damping channels.

quant-ph↗

arXiv2024

Kinematic-aware Prompting for Generalizable Articulated Object Manipulation with LLMs

Generalizable articulated object manipulation is essential for home-assistant robots. Recent efforts focus on imitation learning from demonstrations or reinforcement learning in simulation, however, due to the prohibitive costs of real-world data collection and precise object simulation, it still remains challenging for these works to achieve broad adaptability across diverse articulated objects. Recently, many works have tried to utilize the strong in-context learning ability of Large Language Models (LLMs) to achieve generalizable robotic manipulation, but most of these researches focus on high-level task planning, sidelining low-level robotic control. In this work, building on the idea that the kinematic structure of the object determines how we can manipulate it, we propose a kinematic-aware prompting framework that prompts LLMs with kinematic knowledge of objects to generate low-level motion trajectory waypoints, supporting various object manipulation. To effectively prompt LLMs with the kinematic structure of different objects, we design a unified kinematic knowledge parser, which represents various articulated objects as a unified textual description containing kinematic joints and contact location. Building upon this unified description, a kinematic-aware planner model is proposed to generate precise 3D manipulation waypoints via a designed kinematic-aware chain-of-thoughts prompting method. Our evaluation spanned 48 instances across 16 distinct categories, revealing that our framework not only outperforms traditional methods on 8 seen categories but also shows a powerful zero-shot capability for 8 unseen articulated object categories. Moreover, the real-world experiments on 7 different object categories prove our framework's adaptability in practical scenarios. Code is released at https://github.com/GeWu-Lab/LLM_articulated_object_manipulation/tree/main.

cs.RO↗

arXiv2024

A Disruptive Research Playbook for Studying Disruptive Innovations

As researchers, we are now witnessing a fundamental change in our technologically-enabled world due to the advent and diffusion of highly disruptive technologies such as generative AI, Augmented Reality (AR) and Virtual Reality (VR). In particular, software engineering has been profoundly affected by the transformative power of disruptive innovations for decades, with a significant impact of technical advancements on social dynamics due to its the socio-technical nature. In this paper, we reflect on the importance of formulating and addressing research in software engineering through a socio-technical lens, thus ensuring a holistic understanding of the complex phenomena in this field. We propose a research playbook with the goal of providing a guide to formulate compelling and socially relevant research questions and to identify the appropriate research strategies for empirical investigations, with an eye on the long-term implications of technologies or their use. We showcase how to apply the research playbook. Firstly, we show how it can be used retrospectively to reflect on a prior disruptive technology, Stack Overflow, and its impact on software development. Secondly, we show it can be used to question the impact of two current disruptive technologies: AI and AR/VR. Finally, we introduce a specialized GPT model to support the researcher in framing future investigations. We conclude by discussing the broader implications of adopting the playbook for both researchers and practitioners in software engineering and beyond.

cs.SE↗

arXiv2024

Quantum Light Generation based on GaN Microring towards Fully On-chip Source

Integrated quantum light source is increasingly desirable in large-scale quantum information processing.~Despite recent remarkable advances, new material platform is constantly being explored for the fully on-chip integration of quantum light generation, active and passive manipulation, and detection. Here, for the first time, we demonstrate a gallium nitride (GaN) microring based quantum light generation in the telecom C-band, which has potential towards the monolithic integration of quantum light source.~In our demonstration, the GaN microring has a free spectral range of 330 GHz and a near-zero anomalous dispersion region of over 100 nm. The generation of energy-time entangled photon pair is demonstrated with a typical raw two-photon interference visibility of 95.5$\pm$6.5%, which is further configured to generate heralded single photon with a typical heralded second-order auto-correlation $g^{(2)}_{H}(0)$ of 0.045$\pm$0.001. Our results pave the way for developing chip-scale quantum photonic circuit.

quant-ph↗