arXiv subjects

Qing Yang

Publications and source records attributed to Qing Yang.

At least 109 records · Page 6Linked to original sources

arXiv2023

UAV-assisted Semantic Communication with Hybrid Action Reinforcement Learning

In this paper, we aim to explore the use of uplink semantic communications with the assistance of UAV in order to improve data collection effiicency for metaverse users in remote areas. To reduce the time for uplink data collection while balancing the trade-off between reconstruction quality and computational energy cost, we propose a hybrid action reinforcement learning (RL) framework to make decisions on semantic model scale, channel allocation, transmission power, and UAV trajectory. The variables are classified into discrete type and continuous type, which are optimized by two different RL agents to generate the combined action. Simulation results indicate that the proposed hybrid action reinforcement learning framework can effectively improve the efficiency of uplink semantic data collection under different parameter settings and outperforms the benchmark scenarios.

cs.NI↗

arXiv2023

PUNR: Pre-training with User Behavior Modeling for News Recommendation

News recommendation aims to predict click behaviors based on user behaviors. How to effectively model the user representations is the key to recommending preferred news. Existing works are mostly focused on improvements in the supervised fine-tuning stage. However, there is still a lack of PLM-based unsupervised pre-training methods optimized for user representations. In this work, we propose an unsupervised pre-training paradigm with two tasks, i.e. user behavior masking and user behavior generation, both towards effective user behavior modeling. Firstly, we introduce the user behavior masking pre-training task to recover the masked user behaviors based on their contextual behaviors. In this way, the model could capture a much stronger and more comprehensive user news reading pattern. Besides, we incorporate a novel auxiliary user behavior generation pre-training task to enhance the user representation vector derived from the user encoder. We use the above pre-trained user modeling encoder to obtain news and user representations in downstream fine-tuning. Evaluations on the real-world news benchmark show significant performance improvements over existing baselines.

cs.IR↗

arXiv2023

Key Frame Mechanism For Efficient Conformer Based End-to-end Speech Recognition

Recently, Conformer as a backbone network for end-to-end automatic speech recognition achieved state-of-the-art performance. The Conformer block leverages a self-attention mechanism to capture global information, along with a convolutional neural network to capture local information, resulting in improved performance. However, the Conformer-based model encounters an issue with the self-attention mechanism, as computational complexity grows quadratically with the length of the input sequence. Inspired by previous Connectionist Temporal Classification (CTC) guided blank skipping during decoding, we introduce intermediate CTC outputs as guidance into the downsampling procedure of the Conformer encoder. We define the frame with non-blank output as key frame. Specifically, we introduce the key frame-based self-attention (KFSA) mechanism, a novel method to reduce the computation of the self-attention mechanism using key frames. The structure of our proposed approach comprises two encoders. Following the initial encoder, we introduce an intermediate CTC loss function to compute the label frame, enabling us to extract the key frames and blank frames for KFSA. Furthermore, we introduce the key frame-based downsampling (KFDS) mechanism to operate on high-dimensional acoustic features directly and drop the frames corresponding to blank labels, which results in new acoustic feature sequences as input to the second encoder. By using the proposed method, which achieves comparable or higher performance than vanilla Conformer and other similar work such as Efficient Conformer. Meantime, our proposed method can discard more than 60\% useless frames during model training and inference, which will accelerate the inference speed significantly. This work code is available in {https://github.com/scufan1990/Key-Frame-Mechanism-For-Efficient-Conformer}

cs.SD↗

arXiv2023

Femtosecond electron diffraction reveals local disorder and local anharmonicity in thermoelectric SnSe

The microscopic arrangement of atoms and molecules is the determining factor in how materials behave and perform. Beyond the long-range periodicity, the local disorder with local structures deviating from the average lattice structure plays a vital role in determining the physical properties of the phonon, electron and spin subsystems in crystalline functional materials. Experimentally characterizing the 3D atomic configuration of such local disorder and correlating it with the advanced functions remain a big challenge. Time-domain evolution of the local disorder, either static or dynamical, is lost due to the characterization at equilibrium state with conventional probing techniques. With the combination of femtosecond electron diffraction, structure factor calculation and TDDFT-MD simulation, we exclusively identify the static local disorder and the local anharmonicity of it in thermoelectric SnSe. The ultrafast structural dynamics in time domain reveal a dominant static off-symmetry displacement of Sn (~0.4 angstrom) and the anharmonicity of this local disorder induces an ultrafast atomic displacement within 100 fs after photoexcitation. The microscopic picture of the local anharmonicity indicates a direct and first signature of the THz Einstein oscillators in real space. Therefore, a glass-like thermal transport channel with the local disorder, the Einstein oscillators and the local anharmonicity, updates the fundamental insight into the long-debated ultralow thermal conductivity in SnSe. The local disorder over one to a few unit cells is pervasive and indispensable in thermoelectric materials, multiferroic materials and correlated electronic materials. Our method of revealing the 3D local disorder and the local correlated interactions by ultrafast structural dynamics will inspire broad interest in construction of the structure-property relationship in material science.

cond-mat.mtrl-sci↗

arXiv2023

Towards Causal Representation Learning and Deconfounding from Indefinite Data

Owing to the cross-pollination between causal discovery and deep learning, non-statistical data (e.g., images, text, etc.) encounters significant conflicts in terms of properties and methods with traditional causal data. To unify these data types of varying forms, we redefine causal data from two novel perspectives and then propose three data paradigms. Among them, the indefinite data (like dialogues or video sources) induce low sample utilization and incapability of the distribution assumption, both leading to the fact that learning causal representation from indefinite data is, as of yet, largely unexplored. We design the causal strength variational model to settle down these two problems. Specifically, we leverage the causal strength instead of independent noise as the latent variable to construct evidence lower bound. By this design ethos, The causal strengths of different structures are regarded as a distribution and can be expressed as a 2D matrix. Moreover, considering the latent confounders, we disentangle the causal graph G into two relation subgraphs O and C. O contains pure relations between observed variables, while C represents the relations from latent variables to observed variables. We implement the above designs as a dynamic variational inference model, tailored to learn causal representation from indefinite data under latent confounding. Finally, we conduct comprehensive experiments on synthetic and real-world data to demonstrate the effectiveness of our method.

cs.LG↗

arXiv2023

Hierarchical Semantic Perceptual Listener Head Video Generation: A High-performance Pipeline

In dyadic speaker-listener interactions, the listener's head reactions along with the speaker's head movements, constitute an important non-verbal semantic expression together. The listener Head generation task aims to synthesize responsive listener's head videos based on audios of the speaker and reference images of the listener. Compared to the Talking-head generation, it is more challenging to capture the correlation clues from the speaker's audio and visual information. Following the ViCo baseline scheme, we propose a high-performance solution by enhancing the hierarchical semantic extraction capability of the audio encoder module and improving the decoder part, renderer and post-processing modules. Our solution gets the first place on the official leaderboard for the track of listening head generation. This paper is a technical report of ViCo@2023 Conversational Head Generation Challenge in ACM Multimedia 2023 conference.

cs.CV↗

arXiv2023

CGCE: A Chinese Generative Chat Evaluation Benchmark for General and Financial Domains

Generative chat models, such as ChatGPT and GPT-4, have revolutionized natural language generation (NLG) by incorporating instructions and human feedback to achieve significant performance improvements. However, the lack of standardized evaluation benchmarks for chat models, particularly for Chinese and domain-specific models, hinders their assessment and progress. To address this gap, we introduce the Chinese Generative Chat Evaluation (CGCE) benchmark, focusing on general and financial domains. The CGCE benchmark encompasses diverse tasks, including 200 questions in the general domain and 150 specific professional questions in the financial domain. Manual scoring evaluates factors such as accuracy, coherence, expression clarity, and completeness. The CGCE benchmark provides researchers with a standardized framework to assess and compare Chinese generative chat models, fostering advancements in NLG research.

cs.CL↗

arXiv2023

Self-QA: Unsupervised Knowledge Guided Language Model Alignment

Large-scale language models like ChatGPT and GPT-4 have gained attention for their impressive conversational and generative capabilities. However, the creation of supervised paired question-answering data for instruction tuning presents formidable challenges. This endeavor necessitates substantial human effort for data annotation and wrestles with issues concerning data quality, diversity, accuracy, and other related factors. To overcome these obstacles, we introduce an innovative framework named Self-QA, which replaces the traditional practice of human-written instruction seeds with a vast amount of unsupervised knowledge, enabling the model to generate a larger quantity of correct and domain-specific instruction data. The effectiveness of our proposed method is demonstrated through experiments conducted on unsupervised corpora from various domains.

cs.CL↗

arXiv2023

XuanYuan 2.0: A Large Chinese Financial Chat Model with Hundreds of Billions Parameters

In recent years, pre-trained language models have undergone rapid development with the emergence of large-scale models. However, there is a lack of open-sourced chat models specifically designed for the Chinese language, especially in the field of Chinese finance, at the scale of hundreds of billions. To address this gap, we introduce XuanYuan 2.0, the largest Chinese chat model to date, built upon the BLOOM-176B architecture. Additionally, we propose a novel training method called hybrid-tuning to mitigate catastrophic forgetting. By combining general-domain with domain-specific knowledge and integrating the stages of pre-training and fine-tuning, XuanYuan 2.0 is capable of providing accurate and contextually appropriate responses in the Chinese financial domain.

cs.CL↗

arXiv2023

Far-field Super-resolution Chemical Microscopy

Far-field chemical microscopy providing molecular electronic or vibrational fingerprint information opens a new window for the study of three-dimensional biological, material, and chemical systems. Chemical microscopy provides a nondestructive way of chemical identification without exterior labels. However, the diffraction limit of optics hindered it from discovering more details under the resolution limit. Recent development of super-resolution techniques gives enlightenment to open this door behind far-field chemical microscopy. Here, we review recent advances that have pushed the boundary of far-field chemical microscopy in terms of spatial resolution. We further highlight applications in biomedical research, material characterization, environmental study, cultural heritage conservation, and integrated chip inspection.

physics.optics↗

arXiv2023

PlanarTrack: A Large-scale Challenging Benchmark for Planar Object Tracking

Planar object tracking is a critical computer vision problem and has drawn increasing interest owing to its key roles in robotics, augmented reality, etc. Despite rapid progress, its further development, especially in the deep learning era, is largely hindered due to the lack of large-scale challenging benchmarks. Addressing this, we introduce PlanarTrack, a large-scale challenging planar tracking benchmark. Specifically, PlanarTrack consists of 1,000 videos with more than 490K images. All these videos are collected in complex unconstrained scenarios from the wild, which makes PlanarTrack, compared with existing benchmarks, more challenging but realistic for real-world applications. To ensure the high-quality annotation, each frame in PlanarTrack is manually labeled using four corners with multiple-round careful inspection and refinement. To our best knowledge, PlanarTrack, to date, is the largest and most challenging dataset dedicated to planar object tracking. In order to analyze the proposed PlanarTrack, we evaluate 10 planar trackers and conduct comprehensive comparisons and in-depth analysis. Our results, not surprisingly, demonstrate that current top-performing planar trackers degenerate significantly on the challenging PlanarTrack and more efforts are needed to improve planar tracking in the future. In addition, we further derive a variant named PlanarTrack$_{\mathbf{BB}}$ for generic object tracking from PlanarTrack. Our evaluation of 10 excellent generic trackers on PlanarTrack$_{\mathrm{BB}}$ manifests that, surprisingly, PlanarTrack$_{\mathrm{BB}}$ is even more challenging than several popular generic tracking benchmarks and more attention should be paid to handle such planar objects, though they are rigid. All benchmarks and evaluations will be released at the project webpage.

cs.CV↗

arXiv2023

Connectivity keeping edges of trees in 3-connected or 3-edge-connected graphs

Hasunuma [J. Graph Theory 102 (2023) 423-435] conjectured that for any tree $T$ of order $m$, every $k$-connected (or $k$-edge-connected) graph $G$ with minimum degree at least $k+m-1$ contains a tree $T'\cong T$ such that $G-E(T')$ is still $k$-connected (or $k$-edge connected). Hasunuma verified this conjecture for $k\leq 2$. In this paper, we confirm this conjecture for $k=3$.

math.CO↗

arXiv2023

Efficient reference-less transmission matrix retrieval for a multimode fiber using fast Fourier transform

Transmission matrix (TM) linearly maps the incident and transmitted complex fields, and has been used widely due to its ability to characterize scattering media. It is computationally demanding to reconstruct the TM from intensity images measured by a reference-less experimental setup. Removing reference beam for interference gains the advantage of simple experimental setup. However, the long computational time still limits its practical application. We propose an efficient reference-less TM retrieval method for multimode fiber (MMF). Our method adopts a data acquisition scheme which employs Fourier transform matrix in the design of the incident fields. We develop a nonlinear optimization algorithm to solve the TM retrieval problem in a parallel manner. The data acquisition scheme allows the algorithm to be implemented with fast Fourier transform (FFT), and hence achieves great efficiency improvement. Further, our method acquires intensity images at a defocus plane and correct the error of relative phase offset of TM recovered from the intensity images measured at one fixed plane. We validate the proposed TM retrieval method with both simulations and experiments. By using FFT, our TM retrieval algorithm achieves 1200x speed-up in computational time, and recovers $2286 \times 8192$ TM of a 0.22 NA and $50 \ μm$ diameter MMF with 124.9 seconds by a computer of 32 CPU cores. With the advantages of efficiency and the correction of phase offset, our method paves the way for the application of reference-less TM retrieval in real practice.

physics.optics↗

arXiv2023

RL-GA: A Reinforcement Learning-Based Genetic Algorithm for Electromagnetic Detection Satellite Scheduling Problem

The study of electromagnetic detection satellite scheduling problem (EDSSP) has attracted attention due to the detection requirements for a large number of targets. This paper proposes a mixed-integer programming model for the EDSSP problem and a genetic algorithm based on reinforcement learning (RL-GA). Numerous factors that affect electromagnetic detection are considered in the model, such as detection mode, bandwidth, and other factors. The RL-GA embeds a Q-learning method into an improved genetic algorithm, and the evolution of each individual depends on the decision of the agent. Q-learning is used to guide the population search process by choosing evolution operators. In this way, the search information can be effectively used by the reinforcement learning method. In the algorithm, we design a reward function to update the Q value. According to the problem characteristics, a new combination of is proposed. The RL-GA also uses an elite individual retention strategy to improve search performance. After that, a task time window selection algorithm (TTWSA) is proposed to evaluate the performance of population evolution. Several experiments are used to examine the scheduling effect of the proposed algorithm. Through the experimental verification of multiple instances, it can be seen that the RL-GA can solve the EDSSP problem effectively. Compared with the state-of-the-art algorithms, the RL-GA performs better in several aspects.

cs.NE↗

arXiv2023

UAV aided Metaverse over Wireless Communications: A Reinforcement Learning Approach

Metaverse is expected to create a virtual world closely connected with reality to provide users with immersive experience with the support of 5G high data rate communication technique. A huge amount of data in physical world needs to be synchronized to the virtual world to provide immersive experience for users, and there will be higher requirements on coverage to include more users into Metaverse. However, 5G signal suffers severe attenuation, which makes it more expensive to maintain the same coverage. Unmanned aerial vehicle (UAV) is a promising candidate technique for future implementation of Metaverse as a low-cost and high-mobility platform for communication devices. In this paper, we propose a proximal policy optimization (PPO) based double-agent cooperative reinforcement learning method for channel allocation and trajectory control of UAV to collect and synchronize data from the physical world to the virtual world, and expand the coverage of Metaverse services economically. Simulation results show that our proposed method is able to achieve better performance compared to the benchmark approaches.

eess.SY↗

arXiv2022

An Efficient Split Fine-tuning Framework for Edge and Cloud Collaborative Learning

To enable the pre-trained models to be fine-tuned with local data on edge devices without sharing data with the cloud, we design an efficient split fine-tuning (SFT) framework for edge and cloud collaborative learning. We propose three novel techniques in this framework. First, we propose a matrix decomposition-based method to compress the intermediate output of a neural network to reduce the communication volume between the edge device and the cloud server. Second, we eliminate particular links in the model without affecting the convergence performance in fine-tuning. Third, we implement our system atop PyTorch to allow users to easily extend their existing training scripts to enjoy the efficient edge and cloud collaborative learning. Experiments results on 9 NLP datasets show that our framework can reduce the communication traffic by 96 times with little impact on the model accuracy.

cs.DC↗

arXiv2022

An Empirical Study on Disentanglement of Negative-free Contrastive Learning

Negative-free contrastive learning methods have attracted a lot of attention with simplicity and impressive performances for large-scale pretraining. However, its disentanglement property remains unexplored. In this paper, we examine negative-free contrastive learning methods to study the disentanglement property empirically. We find that existing disentanglement metrics fail to make meaningful measurements for high-dimensional representation models, so we propose a new disentanglement metric based on Mutual Information between latent representations and data factors. With this proposed metric, we benchmark the disentanglement property of negative-free contrastive learning on both popular synthetic datasets and a real-world dataset CelebA. Our study shows that the investigated methods can learn a well-disentangled subset of representation. As far as we know, we are the first to extend the study of disentangled representation learning to high-dimensional representation space and introduce negative-free contrastive learning methods into this area. The source code of this paper is available at \url{https://github.com/noahcao/disentanglement_lib_med}.

cs.LG↗

arXiv2022

Gravitational wave source clustering in the luminosity distance space with the presence of peculiar velocity and lensing errors

GW number count can be used as a novel tracer of the large scale structure (LSS) in the luminosity distance space (LDS), just like galaxies in the redshift space. It is possible to obtain the $D_L-D_A$ duality relation with clustering effect. However, several LSS induced errors will contaminate the GW luminosity distance measurement, such as the peculiar velocity dispersion error of the host galaxy as well as the foreground lensing magnification. The distance uncertainties induced from these effects will degrade the GW clustering from a spectroscopic-like data down to a photometric-like data. In this paper, we investigate how these LSS induced distance errors modify our cosmological parameter precision inferred from the LDS clustering. We consider two of the next generation GW observatories, namely the Big Bang Observatory (BBO) and the Einstein Telescope (ET). We forecast the parameter estimation errors on the angular diameter distance $D_A$, luminosity distance space Hubble parameter $H_L$ and structure growth rate $f_Lσ_8$ with a Fisher matrix method. Generally speaking, the GW source clustering data can be used for cosmological studies below $D_L<5$ Gpc, while above this scale the lensing errors will increase significantly. We find that for BBO, it is possible to constrain the cosmological parameters with a relative error of $10^{-3}$ to $10^{-2}$ below $D_L<5$ Gpc. The velocity dispersion error is dominant in the low luminosity distance range, while the lensing magnification error is the bottleneck in the large luminosity distance range. To reduce the lensing error, we assumed a $50\%$ delensing efficiency. Even with this optimal assumption, the fractional error increased to $O(1)$ at luminosity distance $D_L=25$ Gpc. The results for ET are similar as those from BBO. Due to the GW source number in ET is less than that from BBO, the corresponding results also get a bit worse.

astro-ph.CO↗