Searcharxiv⌕ Search

arXiv subjects

Yue Hu

Publications and source records attributed to Yue Hu.

At least 181 records · Page 10Linked to original sources
arXiv2024

Pragmatic Communication in Multi-Agent Collaborative Perception

Collaborative perception allows each agent to enhance its perceptual abilities by exchanging messages with others. It inherently results in a trade-off between perception ability and communication costs. Previous works transmit complete full-frame high-dimensional feature maps among agents, resulting in substantial communication costs. To promote communication efficiency, we propose only transmitting the information needed for the collaborator's downstream task. This pragmatic communication strategy focuses on three key aspects: i) pragmatic message selection, which selects task-critical parts from the complete data, resulting in spatially and temporally sparse feature vectors; ii) pragmatic message representation, which achieves pragmatic approximation of high-dimensional feature vectors with a task-adaptive dictionary, enabling communicating with integer indices; iii) pragmatic collaborator selection, which identifies beneficial collaborators, pruning unnecessary communication links. Following this strategy, we first formulate a mathematical optimization framework for the perception-communication trade-off and then propose PragComm, a multi-agent collaborative perception system with two key components: i) single-agent detection and tracking and ii) pragmatic collaboration. The proposed PragComm promotes pragmatic communication and adapts to a wide range of communication conditions. We evaluate PragComm for both collaborative 3D object detection and tracking tasks in both real-world, V2V4Real, and simulation datasets, OPV2V and V2X-SIM2.0. PragComm consistently outperforms previous methods with more than 32.7K times lower communication volume on OPV2V. Code is available at github.com/PhyllisH/PragComm.

cs.CV↗
arXiv2024

Velocity Gradient and Stellar Polarization: Magnetic Field Tomography towards the L1688 Cloud

Magnetic fields are a defining yet enigmatic aspect of the interstellar medium (ISM), with their three-dimensional mapping posing a substantial challenge. In this study, we harness the innovative Velocity Gradient Technique (VGT), underpinned by magnetohydrodynamic (MHD) turbulence theories, to elucidate the magnetic field structure by applying it to the atomic neutral hydrogen (HI) emission line and the molecular tracer $^{12}$CO. We construct the tomography of the magnetic field in the low-mass star-forming region L1688, utilizing two approaches: (1) VGT-HI combined with the Galactic rotational curve, and (2) stellar polarization paired with precise star parallax measurements. Our analysis reveals that the magnetic field orientations deduced from stellar polarization undergo a distinct directional change in the vicinity of L1688, providing evidence that the misalignment between VGT-HI and stellar polarization stems from the influence of the molecular cloud's magnetic field on the polarization of starlight. When comparing VGT-$^{12}$CO to stellar polarization and Planck polarization data, we observe that VGT-$^{12}$CO effectively reconciles the misalignment noted with VGT-HI, showing statistical alignment with Planck polarization measurements. This indicates that VGT-$^{12}$CO could be integrated with VGT-HI, offering vital insights into the magnetic fields of molecular clouds, thereby enhancing the accuracy of our 3D magnetic field reconstructions.

astro-ph.GA↗
arXiv2023

DAP: Domain-aware Prompt Learning for Vision-and-Language Navigation

Following language instructions to navigate in unseen environments is a challenging task for autonomous embodied agents. With strong representation capabilities, pretrained vision-and-language models are widely used in VLN. However, most of them are trained on web-crawled general-purpose datasets, which incurs a considerable domain gap when used for VLN tasks. To address the problem, we propose a novel and model-agnostic domain-aware prompt learning (DAP) framework. For equipping the pretrained models with specific object-level and scene-level cross-modal alignment in VLN tasks, DAP applies a low-cost prompt tuning paradigm to learn soft visual prompts for extracting in-domain image semantics. Specifically, we first generate a set of in-domain image-text pairs with the help of the CLIP model. Then we introduce soft visual prompts in the input space of the visual encoder in a pretrained model. DAP injects in-domain visual knowledge into the visual encoder of the pretrained model in an efficient way. Experimental results on both R2R and REVERIE show the superiority of DAP compared to existing state-of-the-art methods.

cs.CV↗
arXiv2023

Context-I2W: Mapping Images to Context-dependent Words for Accurate Zero-Shot Composed Image Retrieval

Different from Composed Image Retrieval task that requires expensive labels for training task-specific models, Zero-Shot Composed Image Retrieval (ZS-CIR) involves diverse tasks with a broad range of visual content manipulation intent that could be related to domain, scene, object, and attribute. The key challenge for ZS-CIR tasks is to learn a more accurate image representation that has adaptive attention to the reference image for various manipulation descriptions. In this paper, we propose a novel context-dependent mapping network, named Context-I2W, for adaptively converting description-relevant Image information into a pseudo-word token composed of the description for accurate ZS-CIR. Specifically, an Intent View Selector first dynamically learns a rotation rule to map the identical image to a task-specific manipulation view. Then a Visual Target Extractor further captures local information covering the main targets in ZS-CIR tasks under the guidance of multiple learnable queries. The two complementary modules work together to map an image to a context-dependent pseudo-word token without extra supervision. Our model shows strong generalization ability on four ZS-CIR tasks, including domain conversion, object composition, object manipulation, and attribute manipulation. It obtains consistent and significant performance boosts ranging from 1.88% to 3.60% over the best methods and achieves new state-of-the-art results on ZS-CIR. Our code is available at https://github.com/Pter61/context-i2w.

cs.CV↗
arXiv2023

Prompt-based Context- and Domain-aware Pretraining for Vision and Language Navigation

Pretrained visual-language models have extensive world knowledge and are widely used in visual and language navigation (VLN). However, they are not sensitive to indoor scenarios for VLN tasks. Another challenge for VLN is how the agent understands the contextual relations between actions on a path and performs cross-modal alignment sequentially. In this paper, we propose a novel Prompt-bAsed coNtext- and inDoor-Aware (PANDA) pretraining framework to address these problems. It performs prompting in two stages. In the indoor-aware stage, we apply an efficient tuning paradigm to learn deep visual prompts from an indoor dataset, in order to augment pretrained models with inductive biases towards indoor environments. This can enable more sample-efficient adaptation for VLN agents. Furthermore, in the context-aware stage, we design a set of hard context prompts to capture the sequence-level semantics in the instruction. They enable further tuning of the pretrained models via contrastive learning. Experimental results on both R2R and REVERIE show the superiority of PANDA compared to existing state-of-the-art methods.

cs.CV↗
arXiv2023

Helping Language Models Learn More: Multi-dimensional Task Prompt for Few-shot Tuning

Large language models (LLMs) can be used as accessible and intelligent chatbots by constructing natural language queries and directly inputting the prompt into the large language model. However, different prompt' constructions often lead to uncertainty in the answers and thus make it hard to utilize the specific knowledge of LLMs (like ChatGPT). To alleviate this, we use an interpretable structure to explain the prompt learning principle in LLMs, which certificates that the effectiveness of language models is determined by position changes of the task's related tokens. Therefore, we propose MTPrompt, a multi-dimensional task prompt learning method consisting based on task-related object, summary, and task description information. By automatically building and searching for appropriate prompts, our proposed MTPrompt achieves the best results on few-shot samples setting and five different datasets. In addition, we demonstrate the effectiveness and stability of our method in different experimental settings and ablation experiments. In interaction with large language models, embedding more task-related information into prompts will make it easier to stimulate knowledge embedded in large language models.

cs.CL↗
arXiv2023

RoboSync: Efficient Real-Time Operating System for Social Robots with Customizable Behaviour

Traditional robotic systems require complex implementations that are not always accessible or easy to use for Human-Robot Interaction (HRI) application developers. With the aim of simplifying the implementation of HRI applications, this paper introduces a novel real-time operating system (RTOS) designed for customizable HRI - RoboSync. By creating multi-level abstraction layers, the system enables users to define complex emotional and behavioral models without needing deep technical expertise. The system's modular architecture comprises a behavior modeling layer, a machine learning plugin configuration layer, a sensor checks customization layer, a scheduler that fits the need of HRI, and a communication and synchronization layer. This approach not only promotes ease of use without highly specialized skills but also ensures real-time responsiveness and adaptability. The primary functionality of the RTOS has been implemented for proof of concept and was tested on a CortexM4 microcontroller, demonstrating its potential for a wide range of lightweight simple-to-implement social robotics applications.

cs.RO↗
arXiv2023

Probing Three-Dimensional Magnetic Fields: II -- An Interpretable Convolutional Neural Network

Observing 3D magnetic fields, including orientation and strength, within the interstellar medium is vital but notoriously difficult. However, recent advances in our understanding of anisotropic magnetohydrodynamic (MHD) turbulence demonstrate that MHD turbulence and 3D magnetic fields leave their imprints on the intensity features of spectroscopic observations. Leveraging these theoretical frameworks, we propose a novel Convolutional Neural Network (CNN) model to extract this embedded information, enabling the probe of 3D magnetic fields. This model examines not only the plane-of-the-sky magnetic field orientation ($ϕ$), but also the magnetic field's inclination angle ($γ$) relative to the line-of-sight, and the total magnetization level (M$_A^{-1}$) of the cloud. We train the model using synthetic emission lines of $^{13}$CO (J = 1 - 0) and C$^{18}$O (J = 1 - 0), generated from 3D MHD simulations that span conditions from sub-Alfvénic to super-Alfvénic molecular clouds. Our tests confirm that the CNN model effectively reconstructs the 3D magnetic field topology and magnetization. The median uncertainties are under $5^\circ$ for both $ϕ$ and $γ$, and less than 0.2 for M$_A$ in sub-Alfvénic conditions (M$_A\approx0.5$). In super-Alfvénic scenarios (M$_A\approx2.0$), they are under $15^\circ$ for $ϕ$ and $γ$, and 1.5 for M$_A$. We applied this trained CNN model to the L1478 molecular cloud. Results show a strong agreement between the CNN-predicted magnetic field orientation and that derived from Planck 353 GHz polarization data. The CNN approach enabled us to construct the 3D magnetic field map for L1478, revealing a global inclination angle of $\approx76^\circ$ and a global M$_A$ of $\approx1.07$.

astro-ph.GA↗
arXiv2023

The Impact of Robots' Facial Emotional Expressions on Light Physical Exercises

To address the global challenge of population aging, our goal is to enhance successful aging through the introduction of robots capable of assisting in daily physical activities and promoting light exercises, which would enhance the cognitive and physical well-being of older adults. Previous studies have shown that facial expressions can increase engagement when interacting with robots. This study aims to investigate how older adults perceive and interact with a robot capable of displaying facial emotions while performing a physical exercise task together. We employed a collaborative robotic arm with a flat panel screen to encourage physical exercise across three different facial emotion conditions. We ran the experiment with older adults aged between 66 and 88. Our findings suggest that individuals perceive robots exhibiting facial expressions as less competent than those without such expressions. Additionally, the presence of facial expressions does not appear to significantly impact participants' levels of engagement, unlike other state-of-the-art studies. This observation is likely linked to our study's emphasis on collaborative physical human-robot interaction (pHRI) applications, as opposed to socially oriented pHRI applications. Additionally, we foresee a requirement for more suitable non-verbal social behavior to effectively enhance participants' engagement levels.

cs.RO↗
arXiv2023

Augmented Kinesthetic Teaching: Enhancing Task Execution Efficiency through Intuitive Human Instructions

In this paper, we present a complete and efficient implementation of a knowledge-sharing augmented kinesthetic teaching approach for efficient task execution in robotics. Our augmented kinesthetic teaching method integrates intuitive human feedback, including verbal, gesture, gaze, and physical guidance, to facilitate the extraction of multiple layers of task information including control type, attention direction, input and output type, action state change trigger, etc., enhancing the adaptability and autonomy of robots during task execution. We propose an efficient Programming by Demonstration (PbD) framework for users with limited technical experience to teach the robot in an intuitive manner. The proposed framework provides an interface for such users to teach customized tasks using high-level commands, with the goal of achieving a smoother teaching experience and task execution. This is demonstrated with the sample task of pouring water.

cs.RO↗
arXiv2023

Charged vacancy in graphene: interplay between Landau levels and atomic collapse resonances

The interplay between a magnetic field and the Coulomb potential from a charged vacancy on the electron states in graphene is investigated within the tight-binding model. The Coulomb potential removes locally Landau level degeneracy, while the vacancy introduces a satellite level next to the normal Landau level. These satellite levels are found throughout the positive energy region, but in the negative energy region they turn into atomic collapse resonances. Crossings between Landau levels with different angular quantum number $m$ are found. Unlike the point impurity system in which an anticrossing occurs between Landau levels of the same $m$, in this work anticrossing is found between the normal Landau level and the vacancy induced level. The atomic collapse resonance hybridize with the Landau levels. The charge at which the lowest Landau level $m = -1, N = 1$ crosses increases $E = 0$ with enhancing magnetic field. Landau level scaling anomaly occurs when the charge is larger than the critical charge $β\approx0.6$ and this critical charge is independent of the magnetic field.

cond-mat.mes-hall↗
arXiv2023

Damping of MHD Turbulence in A Partially Ionized Medium

The coupling state between ions and neutrals in the interstellar medium plays a key role in the dynamics of magnetohydrodynamic (MHD) turbulence, but is challenging to study numerically. In this work, we investigate the damping of MHD turbulence in a partially ionized medium using 3D two-fluid (ions+neutrals) simulations generated with the AthenaK code. Specifically, we examine the velocity, density, and magnetic field statistics of the two-fluid MHD turbulence in different regimes of neutral-ion coupling. Our results demonstrate that when ions and neutrals are strongly coupled, the velocity statistics resemble those of single-fluid MHD turbulence. Both the velocity structures and kinetic energy spectra of ions and neutrals are similar, while their density structures can be significantly different. With an excess of small-scale sharp density fluctuations in ions, the density spectrum in ions is shallower than that of neutrals. When ions and neutrals are weakly coupled, the turbulence in ions is more severely damped due to the ion-neutral collisional friction than that in neutrals, resulting in a steep kinetic energy spectrum and density spectrum in ions compared to the Kolmogorov spectrum. We also find that the magnetic energy spectrum basically follows the shape of the kinetic energy spectrum of ions, irrespective of the coupling regime. In addition, we find large density fluctuations in ions and neutrals and thus spatially inhomogeneous ionization fractions. As a result, the neutral-ion decoupling and damping of MHD turbulence take place over a range of length scales.

astro-ph.GA↗
arXiv2023

Watermarking Vision-Language Pre-trained Models for Multi-modal Embedding as a Service

Recent advances in vision-language pre-trained models (VLPs) have significantly increased visual understanding and cross-modal analysis capabilities. Companies have emerged to provide multi-modal Embedding as a Service (EaaS) based on VLPs (e.g., CLIP-based VLPs), which cost a large amount of training data and resources for high-performance service. However, existing studies indicate that EaaS is vulnerable to model extraction attacks that induce great loss for the owners of VLPs. Protecting the intellectual property and commercial ownership of VLPs is increasingly crucial yet challenging. A major solution of watermarking model for EaaS implants a backdoor in the model by inserting verifiable trigger embeddings into texts, but it is only applicable for large language models and is unrealistic due to data and model privacy. In this paper, we propose a safe and robust backdoor-based embedding watermarking method for VLPs called VLPMarker. VLPMarker utilizes embedding orthogonal transformation to effectively inject triggers into the VLPs without interfering with the model parameters, which achieves high-quality copyright verification and minimal impact on model performance. To enhance the watermark robustness, we further propose a collaborative copyright verification strategy based on both backdoor trigger and embedding distribution, enhancing resilience against various attacks. We increase the watermark practicality via an out-of-distribution trigger selection approach, removing access to the model training data and thus making it possible for many real-world scenarios. Our extensive experiments on various datasets indicate that the proposed watermarking approach is effective and safe for verifying the copyright of VLPs for multi-modal EaaS and robust against model extraction attacks. Our code is available at https://github.com/Pter61/vlpmarker.

cs.CR↗
arXiv2023

S2F-NER: Exploring Sequence-to-Forest Generation for Complex Entity Recognition

Named Entity Recognition (NER) remains challenging due to the complex entities, like nested, overlapping, and discontinuous entities. Existing approaches, such as sequence-to-sequence (Seq2Seq) generation and span-based classification, have shown impressive performance on various NER subtasks, but they are difficult to scale to datasets with longer input text because of either exposure bias issue or inefficient computation. In this paper, we propose a novel Sequence-to-Forest generation paradigm, S2F-NER, which can directly extract entities in sentence via a Forest decoder that decode multiple entities in parallel rather than sequentially. Specifically, our model generate each path of each tree in forest autoregressively, where the maximum depth of each tree is three (which is the shortest feasible length for complex NER and is far smaller than the decoding length of Seq2Seq). Based on this novel paradigm, our model can elegantly mitigates the exposure bias problem and keep the simplicity of Seq2Seq. Experimental results show that our model significantly outperforms the baselines on three discontinuous NER datasets and on two nested NER datasets, especially for discontinuous entity recognition.

cs.CL↗
arXiv2023

EMMA-X: An EM-like Multilingual Pre-training Algorithm for Cross-lingual Representation Learning

Expressing universal semantics common to all languages is helpful in understanding the meanings of complex and culture-specific sentences. The research theme underlying this scenario focuses on learning universal representations across languages with the usage of massive parallel corpora. However, due to the sparsity and scarcity of parallel data, there is still a big challenge in learning authentic ``universals'' for any two languages. In this paper, we propose EMMA-X: an EM-like Multilingual pre-training Algorithm, to learn (X)Cross-lingual universals with the aid of excessive multilingual non-parallel data. EMMA-X unifies the cross-lingual representation learning task and an extra semantic relation prediction task within an EM framework. Both the extra semantic classifier and the cross-lingual sentence encoder approximate the semantic relation of two sentences, and supervise each other until convergence. To evaluate EMMA-X, we conduct experiments on XRETE, a newly introduced benchmark containing 12 widely studied cross-lingual tasks that fully depend on sentence-level representations. Results reveal that EMMA-X achieves state-of-the-art performance. Further geometric analysis of the built representation space with three requirements demonstrates the superiority of EMMA-X over advanced models.

cs.CL↗
arXiv2023

Exploiting Manifold Structured Data Priors for Improved MR Fingerprinting Reconstruction

Estimating tissue parameter maps with high accuracy and precision from highly undersampled measurements presents one of the major challenges in MR fingerprinting (MRF). Many existing works project the recovered voxel fingerprints onto the Bloch manifold to improve reconstruction performance. However, little research focuses on exploiting the latent manifold structure priors among fingerprints. To fill this gap, we propose a novel MRF reconstruction framework based on manifold structured data priors. Since it is difficult to directly estimate the fingerprint manifold structure, we model the tissue parameters as points on a low-dimensional parameter manifold. We reveal that the fingerprint manifold shares the same intrinsic topology as the parameter manifold, although being embedded in different Euclidean spaces. To exploit the non-linear and non-local redundancies in MRF data, we divide the MRF data into spatial patches, and the similarity measurement among data patches can be accurately obtained using the Euclidean distance between the corresponding patches in the parameter manifold. The measured similarity is then used to construct the graph Laplacian operator, which represents the fingerprint manifold structure. Thus, the fingerprint manifold structure is introduced in the reconstruction framework by using the low-dimensional parameter manifold. Additionally, we incorporate the locally low-rank prior in the reconstruction framework to further utilize the local correlations within each patch for improved reconstruction performance. We also adopt a GPU-accelerated NUFFT library to accelerate reconstruction in non-Cartesian sampling scenarios. Experimental results demonstrate that our method can achieve significantly improved reconstruction performance with reduced computational time over the state-of-the-art methods.

eess.IV↗
arXiv2023

Asynchrony-Robust Collaborative Perception via Bird's Eye View Flow

Collaborative perception can substantially boost each agent's perception ability by facilitating communication among multiple agents. However, temporal asynchrony among agents is inevitable in the real world due to communication delays, interruptions, and clock misalignments. This issue causes information mismatch during multi-agent fusion, seriously shaking the foundation of collaboration. To address this issue, we propose CoBEVFlow, an asynchrony-robust collaborative perception system based on bird's eye view (BEV) flow. The key intuition of CoBEVFlow is to compensate motions to align asynchronous collaboration messages sent by multiple agents. To model the motion in a scene, we propose BEV flow, which is a collection of the motion vector corresponding to each spatial location. Based on BEV flow, asynchronous perceptual features can be reassigned to appropriate positions, mitigating the impact of asynchrony. CoBEVFlow has two advantages: (i) CoBEVFlow can handle asynchronous collaboration messages sent at irregular, continuous time stamps without discretization; and (ii) with BEV flow, CoBEVFlow only transports the original perceptual features, instead of generating new perceptual features, avoiding additional noises. To validate CoBEVFlow's efficacy, we create IRregular V2V(IRV2V), the first synthetic collaborative perception dataset with various temporal asynchronies that simulate different real-world scenarios. Extensive experiments conducted on both IRV2V and the real-world dataset DAIR-V2X show that CoBEVFlow consistently outperforms other baselines and is robust in extremely asynchronous settings. The code is available at https://github.com/MediaBrain-SJTU/CoBEVFlow.

cs.CV↗
arXiv2023

Variation of Tensor Force due to Nuclear Medium Effect

The enhancement of $J^π(T)$=3$^{+}$(0) state with isospin $T=0$ excited by the tensor force in the free $^{6}$Li nucleus has been observed, for the first time, relative to a shrinkable excitation in the $^{6}$Li cluster component inside its host nucleus. Comparatively, the excitation of $J^π(T)$=0$^{+}$(1) state with isospin $T=1$ for these two $^{6}$Li formations take on an approximately equal excitation strength. The mechanism of such tensor force effect was proposed due to the intensive nuclear medium role on isospin $T$=0 state.

nucl-ex↗