Searcharxiv⌕ Search

arXiv subjects

Xinyu Li

Publications and source records attributed to Xinyu Li.

At least 163 records · Page 9Linked to original sources
arXiv2023

Passive Human Sensing Enhanced by Reconfigurable Intelligent Surface: Opportunities and Challenges

Reconfigurable intelligent surfaces (RISs) have flexible and exceptional performance in manipulating electromagnetic waves and customizing wireless channels. These capabilities enable them to provide a plethora of valuable activity-related information for promoting wireless human sensing. In this article, we present a comprehensive review of passive human sensing using radio frequency signals with the assistance of RISs. Specifically, we first introduce fundamental principles and physical platform of RISs. Subsequently, based on the specific applications, we categorize the state-of-the-art human sensing techniques into three types, including human imaging,localization, and activity recognition. Meanwhile, we would also investigate the benefits that RISs bring to these applications. Furthermore, we explore the application of RISs in human micro-motion sensing, and propose a vital signs monitoring system enhanced by RISs. Experimental results are presented to demonstrate the promising potential of RISs in sensing vital signs for manipulating individuals. Finally, we discuss the technical challenges and opportunities in this field.

eess.SP↗
arXiv2023

Efficiently Visualizing Large Graphs

Most existing graph visualization methods based on dimension reduction are limited to relatively small graphs due to performance issues. In this work, we propose a novel dimension reduction method for graph visualization, called t-Distributed Stochastic Graph Neighbor Embedding (t-SGNE). t-SGNE is specifically designed to visualize cluster structures in the graph. As a variant of the standard t-SNE method, t-SGNE avoids the time-consuming computations of pairwise similarity. Instead, it uses the neighbor structures of the graph to reduce the time complexity from quadratic to linear, thus supporting larger graphs. In addition, to suit t-SGNE, we combined Laplacian Eigenmaps with the shortest path algorithm in graphs to form the graph embedding algorithm ShortestPath Laplacian Eigenmaps Embedding (SPLEE). Performing SPLEE to obtain a high-dimensional embedding of the large-scale graph and then using t-SGNE to reduce its dimension for visualization, we are able to visualize graphs with up to 300K nodes and 1M edges within 5 minutes and achieve approximately 10% improvement in visualization quality. Codes and data are available at https://github.com/Charlie-XIAO/embedding-visualization-test.

cs.LG↗
arXiv2023

Motion-Guided Masking for Spatiotemporal Representation Learning

Several recent works have directly extended the image masked autoencoder (MAE) with random masking into video domain, achieving promising results. However, unlike images, both spatial and temporal information are important for video understanding. This suggests that the random masking strategy that is inherited from the image MAE is less effective for video MAE. This motivates the design of a novel masking algorithm that can more efficiently make use of video saliency. Specifically, we propose a motion-guided masking algorithm (MGM) which leverages motion vectors to guide the position of each mask over time. Crucially, these motion-based correspondences can be directly obtained from information stored in the compressed format of the video, which makes our method efficient and scalable. On two challenging large-scale video benchmarks (Kinetics-400 and Something-Something V2), we equip video MAE with our MGM and achieve up to +$1.3\%$ improvement compared to previous state-of-the-art methods. Additionally, our MGM achieves equivalent performance to previous video MAE using up to $66\%$ fewer training epochs. Lastly, we show that MGM generalizes better to downstream transfer learning and domain adaptation tasks on the UCF101, HMDB51, and Diving48 datasets, achieving up to +$4.9\%$ improvement compared to baseline methods.

cs.CV↗
arXiv2023

MEGA: Multimodal Alignment Aggregation and Distillation For Cinematic Video Segmentation

Previous research has studied the task of segmenting cinematic videos into scenes and into narrative acts. However, these studies have overlooked the essential task of multimodal alignment and fusion for effectively and efficiently processing long-form videos (>60min). In this paper, we introduce Multimodal alignmEnt aGgregation and distillAtion (MEGA) for cinematic long-video segmentation. MEGA tackles the challenge by leveraging multiple media modalities. The method coarsely aligns inputs of variable lengths and different modalities with alignment positional encoding. To maintain temporal synchronization while reducing computation, we further introduce an enhanced bottleneck fusion layer which uses temporal alignment. Additionally, MEGA employs a novel contrastive loss to synchronize and transfer labels across modalities, enabling act segmentation from labeled synopsis sentences on video shots. Our experimental results show that MEGA outperforms state-of-the-art methods on MovieNet dataset for scene segmentation (with an Average Precision improvement of +1.19%) and on TRIPOD dataset for act segmentation (with a Total Agreement improvement of +5.51%)

cs.CV↗
arXiv2023

Practical and Ethical Challenges of Large Language Models in Education: A Systematic Scoping Review

Educational technology innovations leveraging large language models (LLMs) have shown the potential to automate the laborious process of generating and analysing textual content. While various innovations have been developed to automate a range of educational tasks (e.g., question generation, feedback provision, and essay grading), there are concerns regarding the practicality and ethicality of these innovations. Such concerns may hinder future research and the adoption of LLMs-based innovations in authentic educational contexts. To address this, we conducted a systematic scoping review of 118 peer-reviewed papers published since 2017 to pinpoint the current state of research on using LLMs to automate and support educational tasks. The findings revealed 53 use cases for LLMs in automating education tasks, categorised into nine main categories: profiling/labelling, detection, grading, teaching support, prediction, knowledge representation, feedback, content generation, and recommendation. Additionally, we also identified several practical and ethical challenges, including low technological readiness, lack of replicability and transparency, and insufficient privacy and beneficence considerations. The findings were summarised into three recommendations for future studies, including updating existing innovations with state-of-the-art models (e.g., GPT-3/4), embracing the initiative of open-sourcing models/systems, and adopting a human-centred approach throughout the developmental process. As the intersection of AI and education is continuously evolving, the findings of this study can serve as an essential reference point for researchers, allowing them to leverage the strengths, learn from the limitations, and uncover potential research opportunities enabled by ChatGPT and other generative AI models.

cs.CL↗
arXiv2023

Controlling photons by phonons via giant atom in a waveguide QED setup

We investigate the single photon scattering in a phonon-photon hybrid system in the waveguide QED scheme. In our consideration, an artificial giant atom, which is dressed by the phonons in a surface acoustic wave resonator, interacts with a coupled resonator waveguide (CRW) nonlocally via two connecting sites. Together with the interference effect by the nonlocal coupling, the phonon serves as a controller to the transport of the photon in the waveguide. On the one hand, the coupling strength between the giant atom and the surface acoustic wave resonator modulates the width of the transmission valley or window in the near resonant regime. On the other hand, the two reflective peaks induced by the Rabi splitting degrade into a single one when the giant atom is large detuned from the surface acoustic resonator, which implies an effective dispersive coupling. Our study paves the way for the potential application of giant atoms in the hybrid system.

quant-ph↗
arXiv2023

DiffWA: Diffusion Models for Watermark Attack

With the rapid development of deep neural networks(DNNs), many robust blind watermarking algorithms and frameworks have been proposed and achieved good results. At present, the watermark attack algorithm can not compete with the watermark addition algorithm. And many watermark attack algorithms only care about interfering with the normal extraction of the watermark, and the watermark attack will cause great visual loss to the image. To this end, we propose DiffWA, a conditional diffusion model with distance guidance for watermark attack, which can restore the image while removing the embedded watermark. The core of our method is training an image-to-image conditional diffusion model on unwatermarked images and guiding the conditional model using a distance guidance when sampling so that the model will generate unwatermarked images which is similar to original images. We conducted experiments on CIFAR-10 using our proposed models. The results shows that the model can remove the watermark with good effect and make the bit error rate of watermark extraction higher than 0.4. At the same time, the attacked image will maintain good visual effect with PSNR more than 31 and SSIM more than 0.97 compared with the original image.

cs.MM↗
arXiv2023

Functional data clustering via information maximization

A new method for clustering functional data is proposed via information maximization. The proposed method learns a probabilistic classifier in an unsupervised manner so that mutual information (or squared loss mutual information) between data points and cluster assignments is maximized. A notable advantage of this proposed method is that it only involves continuous optimization of model parameters, which is simpler than discrete optimization of cluster assignments and avoids the disadvantages of generative models. Unlike some existing methods, the proposed method does not require estimating the probability densities of Karhunen-Lo`eve expansion scores under different clusters and also does not require the common eigenfunction assumption. The empirical performance and the applications of the proposed methods are demonstrated by simulation studies and real data analyses. In addition, the proposed method allows for out-of-sample clustering, and its effect is comparable with that of some supervised classifiers.

stat.AP↗
arXiv2023

Adversarial Training for Gradient Descent: Analysis Through its Continuous-time Approximation

Adversarial training has gained great popularity as one of the most effective defenses for deep neural network and more generally for gradient-based machine learning models against adversarial perturbations on data points. This paper establishes a continuous-time approximation for the mini-max game of adversarial training. This approximation approach allows for precise and analytical comparisons between stochastic gradient descent and its adversarial training counterpart; and confirms theoretically the robustness of adversarial training from a new gradient-flow viewpoint. The analysis is then corroborated through various analytical and numerical examples.

cs.LG↗
arXiv2023

A Computational Approach for Mapping Electrochemical Activity of Multi-Principal Element Alloys

Multi principal element alloys (MPEAs) comprise a unique class of metal alloys. MPEAs have been demonstrated to possess several exceptional properties, including, as most relevant to the present study, a high corrosion resistance. In the context of MPEA design, the vast number of potential alloying elements and the staggering number of elemental combinations favours a computational alloy design approach. In order to computationally assess the prospective corrosion performance of MPEA, an approach was developed in this study. A density functional theory (DFT) based Monte Carlo method was used for the development of MPEA structure, with the AlCrTiV alloy used as a model. High-throughput DFT calculations were performed to create training datasets for surface activity towards different adsorbate species: O2-, Cl- and H+. Machine learning (ML) with combined representation was then utilised to predict the adsorption and vacancy energies as descriptors for surface activity. The capability of the combined computational methods of MC, DFT and ML, as a virtual electrochemical performance simulator for MPEAs was established and may be useful in exploring other MPEAs.

cond-mat.mtrl-sci↗
arXiv2023

LooPy: A Research-Friendly Mix Framework for Music Information Retrieval on Electronic Dance Music

Music information retrieval (MIR) has gone through an explosive development with the advancement of deep learning in recent years. However, music genres like electronic dance music (EDM) has always been relatively less investigated compared to others. Considering its wide range of applications, we present a Python package for automated EDM audio generation as an infrastructure for MIR for EDM songs, to mitigate the difficulty of acquiring labelled data. It is a convenient tool that could be easily concatenated to the end of many symbolic music generation pipelines. Inside this package, we provide a framework to build professional-level templates that could render a well-produced track from specified melody and chords, or produce massive tracks given only a specific key by our probabilistic symbolic melody generator. Experiments show that our mixes could achieve the same quality of the original reference songs produced by world-famous artists, with respect to both subjective and objective criteria. Our code is accessible in this repository: https://github.com/Gariscat/loopy and the official site of the project is also online https://loopy4edm.com .

cs.SD↗
arXiv2023

Revisiting Multimodal Representation in Contrastive Learning: From Patch and Token Embeddings to Finite Discrete Tokens

Contrastive learning-based vision-language pre-training approaches, such as CLIP, have demonstrated great success in many vision-language tasks. These methods achieve cross-modal alignment by encoding a matched image-text pair with similar feature embeddings, which are generated by aggregating information from visual patches and language tokens. However, direct aligning cross-modal information using such representations is challenging, as visual patches and text tokens differ in semantic levels and granularities. To alleviate this issue, we propose a Finite Discrete Tokens (FDT) based multimodal representation. FDT is a set of learnable tokens representing certain visual-semantic concepts. Both images and texts are embedded using shared FDT by first grounding multimodal inputs to FDT space and then aggregating the activated FDT representations. The matched visual and semantic concepts are enforced to be represented by the same set of discrete tokens by a sparse activation constraint. As a result, the granularity gap between the two modalities is reduced. Through both quantitative and qualitative analyses, we demonstrate that using FDT representations in CLIP-style models improves cross-modal alignment and performance in visual recognition and vision-language downstream tasks. Furthermore, we show that our method can learn more comprehensive representations, and the learned FDT capture meaningful cross-modal correspondence, ranging from objects to actions and attributes.

cs.CV↗
arXiv2023

Bifurcation analysis of the Keynesian cross model

This study rigorously investigates the Keynesian cross model of a national economy with a focus on the dynamic relationship between government spending and economic equilibrium. The model consists of two ordinary differential equations regarding the rate of change of national income and the rate of consumer spending. Three dynamic relationships between national income and government spending are studied. This study aims to classify the stabilities of equilibrium states for the economy by discussing different cases of government spending. Furthermore, the implication of government spending on the national economy is investigated based on phase portraits and bifurcation analysis of the dynamical system in each scenario.

econ.GN↗
arXiv2023

Lessons Learnt from a Multimodal Learning Analytics Deployment In-the-wild

Multimodal Learning Analytics (MMLA) innovations make use of rapidly evolving sensing and artificial intelligence algorithms to collect rich data about learning activities that unfold in physical learning spaces. The analysis of these data is opening exciting new avenues for both studying and supporting learning. Yet, practical and logistical challenges commonly appear while deploying MMLA innovations "in-the-wild". These can span from technical issues related to enhancing the learning space with sensing capabilities, to the increased complexity of teachers' tasks and informed consent. These practicalities have been rarely discussed. This paper addresses this gap by presenting a set of lessons learnt from a 2-year human-centred MMLA in-the-wild study conducted with 399 students and 17 educators. The lessons learnt were synthesised into topics related to i) technological/physical aspects of the deployment; ii) multimodal data and interfaces; iii) the design process; iv) participation, ethics and privacy; and v) the sustainability of the deployment.

cs.HC↗
arXiv2023

CAT: Causal Audio Transformer for Audio Classification

The attention-based Transformers have been increasingly applied to audio classification because of their global receptive field and ability to handle long-term dependency. However, the existing frameworks which are mainly extended from the Vision Transformers are not perfectly compatible with audio signals. In this paper, we introduce a Causal Audio Transformer (CAT) consisting of a Multi-Resolution Multi-Feature (MRMF) feature extraction with an acoustic attention block for more optimized audio modeling. In addition, we propose a causal module that alleviates over-fitting, helps with knowledge transfer, and improves interpretability. CAT obtains higher or comparable state-of-the-art classification performance on ESC50, AudioSet and UrbanSound8K datasets, and can be easily generalized to other Transformer-based models.

cs.SD↗
arXiv2023

Nearest-Neighbor Inter-Intra Contrastive Learning from Unlabeled Videos

Contrastive learning has recently narrowed the gap between self-supervised and supervised methods in image and video domain. State-of-the-art video contrastive learning methods such as CVRL and $ρ$-MoCo spatiotemporally augment two clips from the same video as positives. By only sampling positive clips locally from a single video, these methods neglect other semantically related videos that can also be useful. To address this limitation, we leverage nearest-neighbor videos from the global space as additional positive pairs, thus improving positive key diversity and introducing a more relaxed notion of similarity that extends beyond video and even class boundaries. Our method, Inter-Intra Video Contrastive Learning (IIVCL), improves performance on a range of video tasks.

cs.CV↗
arXiv2023

An Intermediate-field Fast Radio Burst Model and the Quasi-periodic Oscillation

Quasi-periodic oscillation (QPO) signals are discovered in some fast radio bursts (FRBs) such as FRB 20191221A, as well as in the X-ray burst associated with the galactic FRB from SGR 1935+2154. We revisit the intermediate-field FRB model where the radio waves are generated as fast-magnetosonic waves through magnetic reconnection near the light cylinder. The current sheet in the magnetar wind is compressed by a low frequency pulse emitted from the inner magnetosphere to trigger magnetic reconnection. By incorporating the wave dynamics of the magnetosphere, we demonstrate how the FRB frequency, the single pulse width, and luminosity are determined by the period, magnetic field, QPO frequency and quake energetics of the magnetar. We find that this model can naturally and self-consistently interpret the X-ray/radio event from SGR 1935+2154 and the QPO in FRB 20191221A. It can also explain the observed wide energy range of repeating FRBs in a narrow bandwidth.

astro-ph.HE↗
arXiv2023

Construction of Wave Dark Matter Halos: Numerical Algorithm and Analytical Constraints

We present a wave generalization of the classic Schwarzschild method for constructing self-consistent halos -- such a halo consists of a suitable superposition of waves instead of particle orbits, chosen to yield a desired mean density profile. As an illustration, the method is applied to spherically symmetric halos. We derive an analytic relation between the particle distribution function and the wave superposition amplitudes, and show how it simplifies in the high energy (WKB) limit. We verify the stability of such constructed halos by numerically evolving the Schrödinger-Poisson system. The algorithm provides an efficient and accurate way to simulate the time-dependent halo substructures from wave interference. We use this method to construct halos with a variety of density profiles, all of which have a core from the ground-state wave function, though the core-halo relation need not be the standard one.

astro-ph.CO↗