arXiv subjects

Lei Sun

Publications and source records attributed to Lei Sun.

At least 127 records · Page 7Linked to original sources

arXiv2024

ODA: Observation-Driven Agent for integrating LLMs and Knowledge Graphs

The integration of Large Language Models (LLMs) and knowledge graphs (KGs) has achieved remarkable success in various natural language processing tasks. However, existing methodologies that integrate LLMs and KGs often navigate the task-solving process solely based on the LLM's analysis of the question, overlooking the rich cognitive potential inherent in the vast knowledge encapsulated in KGs. To address this, we introduce Observation-Driven Agent (ODA), a novel AI agent framework tailored for tasks involving KGs. ODA incorporates KG reasoning abilities via global observation, which enhances reasoning capabilities through a cyclical paradigm of observation, action, and reflection. Confronting the exponential explosion of knowledge during observation, we innovatively design a recursive observation mechanism. Subsequently, we integrate the observed knowledge into the action and reflection modules. Through extensive experiments, ODA demonstrates state-of-the-art performance on several datasets, notably achieving accuracy improvements of 12.87% and 8.9%.

cs.CL↗

arXiv2024

Preferred-Action-Optimized Diffusion Policies for Offline Reinforcement Learning

Offline reinforcement learning (RL) aims to learn optimal policies from previously collected datasets. Recently, due to their powerful representational capabilities, diffusion models have shown significant potential as policy models for offline RL issues. However, previous offline RL algorithms based on diffusion policies generally adopt weighted regression to improve the policy. This approach optimizes the policy only using the collected actions and is sensitive to Q-values, which limits the potential for further performance enhancement. To this end, we propose a novel preferred-action-optimized diffusion policy for offline RL. In particular, an expressive conditional diffusion model is utilized to represent the diverse distribution of a behavior policy. Meanwhile, based on the diffusion model, preferred actions within the same behavior distribution are automatically generated through the critic function. Moreover, an anti-noise preference optimization is designed to achieve policy improvement by using the preferred actions, which can adapt to noise-preferred actions for stable training. Extensive experiments demonstrate that the proposed method provides competitive or superior performance compared to previous state-of-the-art offline RL methods, particularly in sparse reward tasks such as Kitchen and AntMaze. Additionally, we empirically prove the effectiveness of anti-noise preference optimization.

cs.LG↗

arXiv2024

The UV luminosity function at 0.6 < z < 1 from UVCANDELS

UVCANDELS is a HST Cycle-26 Treasury Program awarded 164 orbits of primary ultraviolet (UV) F275W imaging and coordinated parallel optical F435W imaging in four CANDELS fields: GOODS-N, GOODS-S, EGS, and COSMOS, covering a total area of $\sim426$ arcmin$^2$. This is $\sim2.7$ times larger than the area covered by previous deep-field space UV data combined, reaching a depth of about 27 and 28 ABmag ($5σ$ in $0.2"$ apertures) for F275W and F435W, respectively. Along with the new photometric catalogs, we present an analysis of the rest-frame UV luminosity function (LF), relying on our UV-optimized aperture photometry method yielding a factor of $1.5\times$ increase than the H-isophot aperture photometry in the signal-to-noise ratios of galaxies in our F275W imaging. Using well tested photometric redshift measurements we identify 5810 galaxies at redshifts $0.6<z<1$, down to an absolute magnitude of $M_\text{UV} = -14.2$. In order to minimize the effect of uncertainties in estimating the completeness function, especially at the faint-end, we restrict our analysis to sources above $30\%$ completeness, which provides a final sample of 4726 galaxies at $-21.5<M_\text{UV}<-15.5$. We performed a maximum likelihood estimate to derive the best-fit parameters of the UV LF. We report a best-fit faint-end slope of $α= -1.359^{+0.041}_{-0.041}$ at $z \sim 0.8$. Creating sub-samples at $z\sim0.7$ and $z\sim0.9$, we observe a possible evolution of $α$ with redshift. The unobscured UV luminosity density at $M_\text{UV}<-10$ is derived as $ρ_\text{UV}=1.339^{+0.027}_{-0.030}\ (\times10^{26} \text{ergs/s/Hz/Mpc}^3)$ using our best-fit LF parameters. The new F275W and F435 photometric catalogs from UVCANDELS have been made publicly available on the Barbara A. Mikulski Archive for Space Telescopes (MAST).

astro-ph.GA↗

arXiv2024

AIS 2024 Challenge on Video Quality Assessment of User-Generated Content: Methods and Results

This paper reviews the AIS 2024 Video Quality Assessment (VQA) Challenge, focused on User-Generated Content (UGC). The aim of this challenge is to gather deep learning-based methods capable of estimating the perceptual quality of UGC videos. The user-generated videos from the YouTube UGC Dataset include diverse content (sports, games, lyrics, anime, etc.), quality and resolutions. The proposed methods must process 30 FHD frames under 1 second. In the challenge, a total of 102 participants registered, and 15 submitted code and models. The performance of the top-5 submissions is reviewed and provided here as a survey of diverse deep models for efficient video quality assessment of user-generated content.

cs.CV↗

arXiv2024

NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Results

This paper reviews the NTIRE 2024 Challenge on Shortform UGC Video Quality Assessment (S-UGC VQA), where various excellent solutions are submitted and evaluated on the collected dataset KVQ from popular short-form video platform, i.e., Kuaishou/Kwai Platform. The KVQ database is divided into three parts, including 2926 videos for training, 420 videos for validation, and 854 videos for testing. The purpose is to build new benchmarks and advance the development of S-UGC VQA. The competition had 200 participants and 13 teams submitted valid solutions for the final testing phase. The proposed solutions achieved state-of-the-art performances for S-UGC VQA. The project can be found at https://github.com/lixinustc/KVQChallenge-CVPR-NTIRE2024.

eess.IV↗

arXiv2024

Ultraviolet Positioning via TDOA: Error Analysis and System Prototype

This work performs the design, real-time hardware realization, and experimental evaluation of a positioning system by ultra-violet (UV) communication under photon-level signal detection. The positioning is based on time-difference of arrival (TDOA) principle. Time division-based transmission of synchronization sequence from three transmitters with known positions is applied. We investigate the positioning error via decomposing it into two parts, the transmitter-side timing error and the receiver-side synchronization error. The theoretical average error matches well with the simulation results, which indicates that theoretical fitting can provide reliable guidance and prediction for hardware experiments. We also conduct real-time hardware realization of the TDOA-based positioning system using Field Programmable Gate Array (FPGA), which is experimentally evaluated via outdoor experiments. Experimental results match well with the theoretical and simulation results.

eess.SY↗

arXiv2024

A Monte-Carlo Simulation on Resonant Scattering of X-ray Line Emission in Supernova Remnants

Resonant scattering (RS) of X-ray line emission in supernova remnants (SNRs) may modify the observed line profiles and fluxes and has potential impact on estimating the physical properties of the hot gas and hence on understanding the SNR physics, but has not been theoretically modeled ever. Here we present our Monte-Carlo simulation of RS effect on X-ray resonant-line emission, typified by O VII He$α$ r line, from SNRs. We employ the physical conditions characterized by the Sedov-Taylor solution and some basic parameters similar to those in Cygnus Loop. We show that the impact of RS effect is most significant near the edge of the remnant. The line profiles are predicted to be asymmetric because of different temperatures and photon production efficiencies of the expanding gas at different radii. We also predict the surface brightness of the line emission would decrease in the outer projected region but is slightly enhanced in the inner. The G-ratio of the OVII He$α$ triplet can be effectively elevated by RS in the outer region. We show that RS effect of the O VII He$α$ r line in the southwestern boundary region of Cygnus Loop is non-negligible. The observed OVII G-ratio $\sim$1.8 of the region could be achieved with RS taken into account for properly elevated O abundance from the previous estimates. Additional simulation performed for the SNRs in ejecta-dominated phase like Cas A shows that RS in the shocked ejecta may have some apparently effects on the observational properties of oxygen resonant lines.

astro-ph.HE↗

arXiv2024

Detect-Order-Construct: A Tree Construction based Approach for Hierarchical Document Structure Analysis

Document structure analysis (aka document layout analysis) is crucial for understanding the physical layout and logical structure of documents, with applications in information retrieval, document summarization, knowledge extraction, etc. In this paper, we concentrate on Hierarchical Document Structure Analysis (HDSA) to explore hierarchical relationships within structured documents created using authoring software employing hierarchical schemas, such as LaTeX, Microsoft Word, and HTML. To comprehensively analyze hierarchical document structures, we propose a tree construction based approach that addresses multiple subtasks concurrently, including page object detection (Detect), reading order prediction of identified objects (Order), and the construction of intended hierarchical structure (Construct). We present an effective end-to-end solution based on this framework to demonstrate its performance. To assess our approach, we develop a comprehensive benchmark called Comp-HRDoc, which evaluates the above subtasks simultaneously. Our end-to-end system achieves state-of-the-art performance on two large-scale document layout analysis datasets (PubLayNet and DocLayNet), a high-quality hierarchical document structure reconstruction dataset (HRDoc), and our Comp-HRDoc benchmark. The Comp-HRDoc benchmark will be released to facilitate further research in this field.

cs.CV↗

arXiv2024

Typing supernova remnant G352.7$-$0.1 using XMM-Newton X-ray observations

G352.7$-$0.1 is a mixed-morphology (MM) supernova remnant (SNR) with multiple radio arcs and has a disputed supernova origin. We conducted a spatially resolved spectroscopic study of the remnant with XMM-Newton X-ray data to investigate its explosion mechanism and explain its morphology. The global X-ray spectra of the SNR can be adequately reproduced using a metal-rich thermal plasma model with a temperature of $\sim 2$ keV and ionization timescale of $\sim 3\times 10^{10}~{\rm cm^{-3}~s}$. Through a comparison with various supernova nucleosynthesis models, we found that observed metal properties from Mg to Fe can be better described using core-collapse supernova models, while thermonuclear models fail to explain the observed high Mg/Si ratio. The best-fit supernova model suggests a $\sim 13$ $M_\odot$ progenitor star, consistent with previous estimates using the wind bubble size. We also discussed the possible mechanisms that may lead to SNR G352.7$-$0.1 being an MMSNR. By dividing the SNR into several regions, we found that the temperature and abundance do not significantly vary with regions, except for a decreased temperature and abundance in a region interacting with molecular clouds. The brightest X-ray emission of the SNR spatially matches with the inner radio structure, suggesting that the centrally filled X-ray morphology results from a projection effect.

astro-ph.HE↗

arXiv2024

Computational Imaging for Machine Perception: Transferring Semantic Segmentation beyond Aberrations

Semantic scene understanding with Minimalist Optical Systems (MOS) in mobile and wearable applications remains a challenge due to the corrupted imaging quality induced by optical aberrations. However, previous works only focus on improving the subjective imaging quality through the Computational Imaging (CI) technique, ignoring the feasibility of advancing semantic segmentation. In this paper, we pioneer the investigation of Semantic Segmentation under Optical Aberrations (SSOA) with MOS. To benchmark SSOA, we construct Virtual Prototype Lens (VPL) groups through optical simulation, generating Cityscapes-ab and KITTI-360-ab datasets under different behaviors and levels of aberrations. We look into SSOA via an unsupervised domain adaptation perspective to address the scarcity of labeled aberration data in real-world scenarios. Further, we propose Computational Imaging Assisted Domain Adaptation (CIADA) to leverage prior knowledge of CI for robust performance in SSOA. Based on our benchmark, we conduct experiments on the robustness of classical segmenters against aberrations. In addition, extensive evaluations of possible solutions to SSOA reveal that CIADA achieves superior performance under all aberration distributions, bridging the gap between computational imaging and downstream applications for MOS. The project page is at https://github.com/zju-jiangqi/CIADA.

cs.CV↗

arXiv2024

UniVIE: A Unified Label Space Approach to Visual Information Extraction from Form-like Documents

Existing methods for Visual Information Extraction (VIE) from form-like documents typically fragment the process into separate subtasks, such as key information extraction, key-value pair extraction, and choice group extraction. However, these approaches often overlook the hierarchical structure of form documents, including hierarchical key-value pairs and hierarchical choice groups. To address these limitations, we present a new perspective, reframing VIE as a relation prediction problem and unifying labels of different tasks into a single label space. This unified approach allows for the definition of various relation types and effectively tackles hierarchical relationships in form-like documents. In line with this perspective, we present UniVIE, a unified model that addresses the VIE problem comprehensively. UniVIE functions using a coarse-to-fine strategy. It initially generates tree proposals through a tree proposal network, which are subsequently refined into hierarchical trees by a relation decoder module. To enhance the relation prediction capabilities of UniVIE, we incorporate two novel tree constraints into the relation decoder: a tree attention mask and a tree level embedding. Extensive experimental evaluations on both our in-house dataset HierForms and a publicly available dataset SIBR, substantiate that our method achieves state-of-the-art results, underscoring the effectiveness and potential of our unified approach in advancing the field of VIE.

cs.CL↗

arXiv2024

Dynamic Relation Transformer for Contextual Text Block Detection

Contextual Text Block Detection (CTBD) is the task of identifying coherent text blocks within the complexity of natural scenes. Previous methodologies have treated CTBD as either a visual relation extraction challenge within computer vision or as a sequence modeling problem from the perspective of natural language processing. We introduce a new framework that frames CTBD as a graph generation problem. This methodology consists of two essential procedures: identifying individual text units as graph nodes and discerning the sequential reading order relationships among these units as graph edges. Leveraging the cutting-edge capabilities of DQ-DETR for node detection, our framework innovates further by integrating a novel mechanism, a Dynamic Relation Transformer (DRFormer), dedicated to edge generation. DRFormer incorporates a dual interactive transformer decoder that deftly manages a dynamic graph structure refinement process. Through this iterative process, the model systematically enhances the graph's fidelity, ultimately resulting in improved precision in detecting contextual text blocks. Comprehensive experimental evaluations conducted on both SCUT-CTW-Context and ReCTS-Context datasets substantiate that our method achieves state-of-the-art results, underscoring the effectiveness and potential of our graph generation framework in advancing the field of CTBD.

cs.CV↗

arXiv2023

Perceptual Quality Assessment for Video Frame Interpolation

The quality of frames is significant for both research and application of video frame interpolation (VFI). In recent VFI studies, the methods of full-reference image quality assessment have generally been used to evaluate the quality of VFI frames. However, high frame rate reference videos, necessities for the full-reference methods, are difficult to obtain in most applications of VFI. To evaluate the quality of VFI frames without reference videos, a no-reference perceptual quality assessment method is proposed in this paper. This method is more compatible with VFI application and the evaluation scores from it are consistent with human subjective opinions. A new quality assessment dataset for VFI was constructed through subjective experiments firstly, to assess the opinion scores of interpolated frames. The dataset was created from triplets of frames extracted from high-quality videos using 9 state-of-the-art VFI algorithms. The proposed method evaluates the perceptual coherence of frames incorporating the original pair of VFI inputs. Specifically, the method applies a triplet network architecture, including three parallel feature pipelines, to extract the deep perceptual features of the interpolated frame as well as the original pair of frames. Coherence similarities of the two-way parallel features are jointly calculated and optimized as a perceptual metric. In the experiments, both full-reference and no-reference quality assessment methods were tested on the new quality dataset. The results show that the proposed method achieves the best performance among all compared quality assessment methods on the dataset.

eess.IV↗

arXiv2023

YAYI 2: Multilingual Open-Source Large Language Models

As the latest advancements in natural language processing, large language models (LLMs) have achieved human-level language understanding and generation abilities in many real-world tasks, and even have been regarded as a potential path to the artificial general intelligence. To better facilitate research on LLMs, many open-source LLMs, such as Llama 2 and Falcon, have recently been proposed and gained comparable performances to proprietary models. However, these models are primarily designed for English scenarios and exhibit poor performances in Chinese contexts. In this technical report, we propose YAYI 2, including both base and chat models, with 30 billion parameters. YAYI 2 is pre-trained from scratch on a multilingual corpus which contains 2.65 trillion tokens filtered by our pre-training data processing pipeline. The base model is aligned with human values through supervised fine-tuning with millions of instructions and reinforcement learning from human feedback. Extensive experiments on multiple benchmarks, such as MMLU and CMMLU, consistently demonstrate that the proposed YAYI 2 outperforms other similar sized open-source models.

cs.CL↗

arXiv2023

The USTC-NERCSLIP Systems for the CHiME-7 DASR Challenge

This technical report details our submission system to the CHiME-7 DASR Challenge, which focuses on speaker diarization and speech recognition under complex multi-speaker scenarios. Additionally, it also evaluates the efficiency of systems in handling diverse array devices. To address these issues, we implemented an end-to-end speaker diarization system and introduced a rectification strategy based on multi-channel spatial information. This approach significantly diminished the word error rates (WER). In terms of recognition, we utilized publicly available pre-trained models as the foundational models to train our end-to-end speech recognition models. Our system attained a Macro-averaged diarization-attributed WER (DA-WER) of 21.01% on the CHiME-7 evaluation set, which signifies a relative improvement of 62.04% over the official baseline system.

eess.AS↗

arXiv2023

Conditional DETR for Fast Training Convergence

The recently-developed DETR approach applies the transformer encoder and decoder architecture to object detection and achieves promising performance. In this paper, we handle the critical issue, slow training convergence, and present a conditional cross-attention mechanism for fast DETR training. Our approach is motivated by that the cross-attention in DETR relies highly on the content embeddings for localizing the four extremities and predicting the box, which increases the need for high-quality content embeddings and thus the training difficulty. Our approach, named conditional DETR, learns a conditional spatial query from the decoder embedding for decoder multi-head cross-attention. The benefit is that through the conditional spatial query, each cross-attention head is able to attend to a band containing a distinct region, e.g., one object extremity or a region inside the object box. This narrows down the spatial range for localizing the distinct regions for object classification and box regression, thus relaxing the dependence on the content embeddings and easing the training. Empirical results show that conditional DETR converges 6.7x faster for the backbones R50 and R101 and 10x faster for stronger backbones DC5-R50 and DC5-R101. Code is available at https://github.com/Atten4Vis/ConditionalDETR.

cs.CV↗

arXiv2023

Key Gene Mining in Transcriptional Regulation for Specific Biological Processes with Small Sample Sizes Using Multi-network pipeline Transformer

Gene mining is an important topic in the field of life sciences, but traditional machine learning methods cannot consider the regulatory relationships between genes. Deep learning methods perform poorly in small sample sizes. This study proposed a deep learning method, called TransGeneSelector, that can mine critical regulatory genes involved in certain life processes using a small-sample transcriptome dataset. The method combines a WGAN-GP data augmentation network, a sample filtering network, and a Transformer classifier network, which successfully classified the state (germinating or dry seeds) of Arabidopsis thaliana seed in a dataset of 79 samples, showing performance comparable to that of Random Forests. Further, through the use of SHapley Additive exPlanations method, TransGeneSelector successfully mined genes involved in seed germination. Through the construction of gene regulatory networks and the enrichment analysis of KEGG, as well as RT-qPCR quantitative analysis, it was confirmed that these genes are at a more upstream regulatory level than those Random Forests mined, and the top 11 genes that were uniquely mined by TransGeneSelector were found to be related to the KAI2 signaling pathway, which is of great regulatory importance for germination-related genes. This study provides a practical tool for life science researchers to mine key genes from transcriptome data.

q-bio.QM↗

arXiv2023

Robust Table Structure Recognition with Dynamic Queries Enhanced Detection Transformer

We present a new table structure recognition (TSR) approach, called TSRFormer, to robustly recognizing the structures of complex tables with geometrical distortions from various table images. Unlike previous methods, we formulate table separation line prediction as a line regression problem instead of an image segmentation problem and propose a new two-stage dynamic queries enhanced DETR based separation line regression approach, named DQ-DETR, to predict separation lines from table images directly. Compared to Vallina DETR, we propose three improvements in DQ-DETR to make the two-stage DETR framework work efficiently and effectively for the separation line prediction task: 1) A new query design, named Dynamic Query, to decouple single line query into separable point queries which could intuitively improve the localization accuracy for regression tasks; 2) A dynamic queries based progressive line regression approach to progressively regressing points on the line which further enhances localization accuracy for distorted tables; 3) A prior-enhanced matching strategy to solve the slow convergence issue of DETR. After separation line prediction, a simple relation network based cell merging module is used to recover spanning cells. With these new techniques, our TSRFormer achieves state-of-the-art performance on several benchmark datasets, including SciTSR, PubTabNet, WTW and FinTabNet. Furthermore, we have validated the robustness and high localization accuracy of our approach to tables with complex structures, borderless cells, large blank spaces, empty or spanning cells as well as distorted or even curved shapes on a more challenging real-world in-house dataset.

cs.CV↗