arXiv subjects

Qi Li

Publications and source records attributed to Qi Li.

At least 307 records · Page 17Linked to original sources

arXiv2024

Deep Linear Array Pushbroom Image Restoration: A Degradation Pipeline and Jitter-Aware Restoration Network

Linear Array Pushbroom (LAP) imaging technology is widely used in the realm of remote sensing. However, images acquired through LAP always suffer from distortion and blur because of camera jitter. Traditional methods for restoring LAP images, such as algorithms estimating the point spread function (PSF), exhibit limited performance. To tackle this issue, we propose a Jitter-Aware Restoration Network (JARNet), to remove the distortion and blur in two stages. In the first stage, we formulate an Optical Flow Correction (OFC) block to refine the optical flow of the degraded LAP images, resulting in pre-corrected images where most of the distortions are alleviated. In the second stage, for further enhancement of the pre-corrected images, we integrate two jitter-aware techniques within the Spatial and Frequency Residual (SFRes) block: 1) introducing Coordinate Attention (CoA) to the SFRes block in order to capture the jitter state in orthogonal direction; 2) manipulating image features in both spatial and frequency domains to leverage local and global priors. Additionally, we develop a data synthesis pipeline, which applies Continue Dynamic Shooting Model (CDSM) to simulate realistic degradation in LAP images. Both the proposed JARNet and LAP image synthesis pipeline establish a foundation for addressing this intricate challenge. Extensive experiments demonstrate that the proposed two-stage method outperforms state-of-the-art image restoration models. Code is available at https://github.com/JHW2000/JARNet.

cs.CV↗

arXiv2024

Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems

Large language models (LLMs) have strong capabilities in solving diverse natural language processing tasks. However, the safety and security issues of LLM systems have become the major obstacle to their widespread application. Many studies have extensively investigated risks in LLM systems and developed the corresponding mitigation strategies. Leading-edge enterprises such as OpenAI, Google, Meta, and Anthropic have also made lots of efforts on responsible LLMs. Therefore, there is a growing need to organize the existing studies and establish comprehensive taxonomies for the community. In this paper, we delve into four essential modules of an LLM system, including an input module for receiving prompts, a language model trained on extensive corpora, a toolchain module for development and deployment, and an output module for exporting LLM-generated content. Based on this, we propose a comprehensive taxonomy, which systematically analyzes potential risks associated with each module of an LLM system and discusses the corresponding mitigation strategies. Furthermore, we review prevalent benchmarks, aiming to facilitate the risk assessment of LLM systems. We hope that this paper can help LLM participants embrace a systematic perspective to build their responsible LLM systems.

cs.CL↗

arXiv2023

MIVC: Multiple Instance Visual Component for Visual-Language Models

Vision-language models have been widely explored across a wide range of tasks and achieve satisfactory performance. However, it's under-explored how to consolidate entity understanding through a varying number of images and to align it with the pre-trained language models for generative tasks. In this paper, we propose MIVC, a general multiple instance visual component to bridge the gap between various image inputs with off-the-shelf vision-language models by aggregating visual representations in a permutation-invariant fashion through a neural network. We show that MIVC could be plugged into the visual-language models to improve the model performance consistently on visual question answering, classification and captioning tasks on a public available e-commerce dataset with multiple images per product. Furthermore, we show that the component provides insight into the contribution of each image to the downstream tasks.

cs.CV↗

arXiv2023

A Comprehensive Evaluation of Parameter-Efficient Fine-Tuning on Software Engineering Tasks

Pre-trained models (PTMs) have achieved great success in various Software Engineering (SE) downstream tasks following the ``pre-train then fine-tune'' paradigm. As fully fine-tuning all parameters of PTMs can be computationally expensive, a widely used solution is parameter-efficient fine-tuning (PEFT), which freezes PTMs while introducing extra parameters. Though work has been done to test PEFT methods in the SE field, a comprehensive evaluation is still lacking. This paper aims to fill in this gap by evaluating the effectiveness of five PEFT methods on eight PTMs and four SE downstream tasks. For different tasks and PEFT methods, we seek answers to the following research questions: 1) Is it more effective to use PTMs trained specifically on source code, or is it sufficient to use PTMs trained on natural language text? 2) What is the impact of varying model sizes? 3) How does the model architecture affect the performance? Besides effectiveness, we also discuss the efficiency of PEFT methods, concerning the costs of required training time and GPU resource consumption. We hope that our findings can provide a deeper understanding of PEFT methods on various PTMs and SE downstream tasks. All the codes and data are available at \url{https://github.com/zwtnju/PEFT.git}.

cs.SE↗

arXiv2023

Produce Once, Utilize Twice for Anomaly Detection

Visual anomaly detection aims at classifying and locating the regions that deviate from the normal appearance. Embedding-based methods and reconstruction-based methods are two main approaches for this task. However, they are either not efficient or not precise enough for the industrial detection. To deal with this problem, we derive POUTA (Produce Once Utilize Twice for Anomaly detection), which improves both the accuracy and efficiency by reusing the discriminant information potential in the reconstructive network. We observe that the encoder and decoder representations of the reconstructive network are able to stand for the features of the original and reconstructed image respectively. And the discrepancies between the symmetric reconstructive representations provides roughly accurate anomaly information. To refine this information, a coarse-to-fine process is proposed in POUTA, which calibrates the semantics of each discriminative layer by the high-level representations and supervision loss. Equipped with the above modules, POUTA is endowed with the ability to provide a more precise anomaly location than the prior arts. Besides, the representation reusage also enables to exclude the feature extraction process in the discriminative network, which reduces the parameters and improves the efficiency. Extensive experiments show that, POUTA is superior or comparable to the prior methods with even less cost. Furthermore, POUTA also achieves better performance than the state-of-the-art few-shot anomaly detection methods without any special design, showing that POUTA has strong ability to learn representations inherent in the training data.

cs.CV↗

arXiv2023

Charmonia in an unquenched quark model

In this work, we study the charmonium spectrum within an unquenched quark model including coupled-channel effects. In couple-channel calculations, we include all of the opened charmed meson channels with the once-subtracted method, meanwhile adopt a suppressed factor to soften the hard vertices given by the $^3P_0$ model in the high momentum region. We obtain a good description of both the masses and widths for the well-established states in the charmonium spectrum. Furthermore, we give predictions for the higher $S$-, $P$- and $D$-wave charmonium states up to mass region of $\sim 5.0$ GeV. The magnitude of mass shifts due to the coupled-channel effects is estimated to be about $10s$ MeV. Although many decay channels are opened for the higher charmonium states, they are relatively narrow states. Their widths scatter in the range of $\sim 10s-100$ MeV. Many charmonium-like states, such as $χ_{c1}(3872)$, $χ_{c1}(4274)$, $χ_{c0}(3915)$, $χ_{c0}(4500)$, $χ_{c0}(4700)$, $X(4160)$, $X(4350)$, $Y(4500)$, and $ψ(4660)$/$Y(4710)$, can be accommodated by the charmonium spectrum when the unquenched coupled-channel effects are carefully considered.

hep-ph↗

arXiv2023

Securing Graph Neural Networks in MLaaS: A Comprehensive Realization of Query-based Integrity Verification

The deployment of Graph Neural Networks (GNNs) within Machine Learning as a Service (MLaaS) has opened up new attack surfaces and an escalation in security concerns regarding model-centric attacks. These attacks can directly manipulate the GNN model parameters during serving, causing incorrect predictions and posing substantial threats to essential GNN applications. Traditional integrity verification methods falter in this context due to the limitations imposed by MLaaS and the distinct characteristics of GNN models. In this research, we introduce a groundbreaking approach to protect GNN models in MLaaS from model-centric attacks. Our approach includes a comprehensive verification schema for GNN's integrity, taking into account both transductive and inductive GNNs, and accommodating varying pre-deployment knowledge of the models. We propose a query-based verification technique, fortified with innovative node fingerprint generation algorithms. To deal with advanced attackers who know our mechanisms in advance, we introduce randomized fingerprint nodes within our design. The experimental evaluation demonstrates that our method can detect five representative adversarial model-centric attacks, displaying 2 to 4 times greater efficiency compared to baselines.

cs.CR↗

arXiv2023

BinGo: Identifying Security Patches in Binary Code with Graph Representation Learning

A timely software update is vital to combat the increasing security vulnerabilities. However, some software vendors may secretly patch their vulnerabilities without creating CVE entries or even describing the security issue in their change log. Thus, it is critical to identify these hidden security patches and defeat potential N-day attacks. Researchers have employed various machine learning techniques to identify security patches in open-source software, leveraging the syntax and semantic features of the software changes and commit messages. However, all these solutions cannot be directly applied to the binary code, whose instructions and program flow may dramatically vary due to different compilation configurations. In this paper, we propose BinGo, a new security patch detection system for binary code. The main idea is to present the binary code as code property graphs to enable a comprehensive understanding of program flow and perform a language model over each basic block of binary code to catch the instruction semantics. BinGo consists of four phases, namely, patch data pre-processing, graph extraction, embedding generation, and graph representation learning. Due to the lack of an existing binary security patch dataset, we construct such a dataset by compiling the pre-patch and post-patch source code of the Linux kernel. Our experimental results show BinGo can achieve up to 80.77% accuracy in identifying security patches between two neighboring versions of binary code. Moreover, BinGo can effectively reduce the false positives and false negatives caused by the different compilers and optimization levels.

cs.CR↗

arXiv2023

ActiveClean: Generating Line-Level Vulnerability Data via Active Learning

Deep learning vulnerability detection tools are increasing in popularity and have been shown to be effective. These tools rely on large volume of high quality training data, which are very hard to get. Most of the currently available datasets provide function-level labels, reporting whether a function is vulnerable or not vulnerable. However, for a vulnerability detection to be useful, we need to also know the lines that are relevant to the vulnerability. This paper makes efforts towards developing systematic tools and proposes. ActiveClean to generate the large volume of line-level vulnerability data from commits. That is, in addition to function-level labels, it also reports which lines in the function are likely responsible for vulnerability detection. In the past, static analysis has been applied to clean commits to generate line-level data. Our approach based on active learning, which is easy to use and scalable, provide a complementary approach to static analysis. We designed semantic and syntactic properties from commit lines and use them to train the model. We evaluated our approach on both Java and C datasets processing more than 4.3K commits and 119K commit lines. AcitveClean achieved an F1 score between 70-74. Further, we also show that active learning is effective by using just 400 training data to reach F1 score of 70.23. Using ActiveClean, we generate the line-level labels for the entire FFMpeg project in the Devign dataset, including 5K functions, and also detected incorrect function-level labels. We demonstrated that using our cleaned data, LineVul, a SOTA line-level vulnerability detection tool, detected 70 more vulnerable lines and 18 more vulnerable functions, and improved Top 10 accuracy from 66% to 73%.

cs.SE↗

arXiv2023

Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network with Graph Representation Learning

Biphasic face photo-sketch synthesis has significant practical value in wide-ranging fields such as digital entertainment and law enforcement. Previous approaches directly generate the photo-sketch in a global view, they always suffer from the low quality of sketches and complex photo variations, leading to unnatural and low-fidelity results. In this paper, we propose a novel Semantic-Driven Generative Adversarial Network to address the above issues, cooperating with Graph Representation Learning. Considering that human faces have distinct spatial structures, we first inject class-wise semantic layouts into the generator to provide style-based spatial information for synthesized face photos and sketches. Additionally, to enhance the authenticity of details in generated faces, we construct two types of representational graphs via semantic parsing maps upon input faces, dubbed the IntrA-class Semantic Graph (IASG) and the InteR-class Structure Graph (IRSG). Specifically, the IASG effectively models the intra-class semantic correlations of each facial semantic component, thus producing realistic facial details. To preserve the generated faces being more structure-coordinated, the IRSG models inter-class structural relations among every facial component by graph representation learning. To further enhance the perceptual quality of synthesized images, we present a biphasic interactive cycle training strategy by fully taking advantage of the multi-level feature consistency between the photo and sketch. Extensive experiments demonstrate that our method outperforms the state-of-the-art competitors on the CUFS and CUFSF datasets.

cs.CV↗

arXiv2023

Improving Unsupervised Relation Extraction by Augmenting Diverse Sentence Pairs

Unsupervised relation extraction (URE) aims to extract relations between named entities from raw text without requiring manual annotations or pre-existing knowledge bases. In recent studies of URE, researchers put a notable emphasis on contrastive learning strategies for acquiring relation representations. However, these studies often overlook two important aspects: the inclusion of diverse positive pairs for contrastive learning and the exploration of appropriate loss functions. In this paper, we propose AugURE with both within-sentence pairs augmentation and augmentation through cross-sentence pairs extraction to increase the diversity of positive pairs and strengthen the discriminative power of contrastive learning. We also identify the limitation of noise-contrastive estimation (NCE) loss for relation representation learning and propose to apply margin loss for sentence pairs. Experiments on NYT-FB and TACRED datasets demonstrate that the proposed relation representation learning and a simple K-Means clustering achieves state-of-the-art performance.

cs.CL↗

arXiv2023

CoRec: An Easy Approach for Coordination Recognition

In this paper, we observe and address the challenges of the coordination recognition task. Most existing methods rely on syntactic parsers to identify the coordinators in a sentence and detect the coordination boundaries. However, state-of-the-art syntactic parsers are slow and suffer from errors, especially for long and complicated sentences. To better solve the problems, we propose a pipeline model COordination RECognizer (CoRec). It consists of two components: coordinator identifier and conjunct boundary detector. The experimental results on datasets from various domains demonstrate the effectiveness and efficiency of the proposed method. Further experiments show that CoRec positively impacts downstream tasks, improving the yield of state-of-the-art Open IE models.

cs.CL↗

arXiv2023

Fractional quantum Hall interface induced by geometric singularity

The geometric response of quantum Hall liquids is an important aspect to understand their topological characteristics in addition to the electromagnetic response. According to the Wen-Zee theory, the topological spin is coupled to the curvature of the space in which the electrons reside. The presence of conical geometry provides a local isolated geometric singularity, making it suitable for exploring the geometric response. In the context of two-dimensional electrons in a perpendicular magnetic field, each Landau orbit occupies the same area. The cone geometry naturally provides a structure in which the distances between two adjacent orbits gradually change and can be easily adjusted by altering the tip angle. The presence of a cone tip introduces a geometric singularity that affects the electron density and interacts with the motion of electrons, which has been extensively studied. Furthermore, this type of geometry can automatically create a smooth interface or crossover between the crystalline charge-density-wave state and the liquid-like fractional quantum Hall state. In this work, the properties of this interface are studied from multiple perspectives, shedding light on the behavior of quantum Hall liquids in such geometric configurations.

cond-mat.mes-hall↗

arXiv2023

Revealing the preference for correcting separated aberrations in joint optic-image design

The joint design of the optical system and the downstream algorithm is a challenging and promising task. Due to the demand for balancing the global optimal of imaging systems and the computational cost of physical simulation, existing methods cannot achieve efficient joint design of complex systems such as smartphones and drones. In this work, starting from the perspective of the optical design, we characterize the optics with separated aberrations. Additionally, to bridge the hardware and software without gradients, an image simulation system is presented to reproduce the genuine imaging procedure of lenses with large field-of-views. As for aberration correction, we propose a network to perceive and correct the spatially varying aberrations and validate its superiority over state-of-the-art methods. Comprehensive experiments reveal that the preference for correcting separated aberrations in joint design is as follows: longitudinal chromatic aberration, lateral chromatic aberration, spherical aberration, field curvature, and coma, with astigmatism coming last. Drawing from the preference, a 10% reduction in the total track length of the consumer-level mobile phone lens module is accomplished. Moreover, this procedure spares more space for manufacturing deviations, realizing extreme-quality enhancement of computational photography. The optimization paradigm provides innovative insight into the practical joint design of sophisticated optical systems and post-processing algorithms.

physics.optics↗

arXiv2023

Towards Robust and Accurate Visual Prompting

Visual prompting, an efficient method for transfer learning, has shown its potential in vision tasks. However, previous works focus exclusively on VP from standard source models, it is still unknown how it performs under the scenario of a robust source model: Whether a visual prompt derived from a robust model can inherit the robustness while suffering from the generalization performance decline, albeit for a downstream dataset that is different from the source dataset? In this work, we get an affirmative answer of the above question and give an explanation on the visual representation level. Moreover, we introduce a novel technique named Prompt Boundary Loose (PBL) to effectively mitigates the suboptimal results of visual prompt on standard accuracy without losing (or even significantly improving) its adversarial robustness when using a robust model as source model. Extensive experiments across various datasets show that our findings are universal and demonstrate the significant benefits of our proposed method.

cs.CV↗

arXiv2023

An Empirical Study of Using ChatGPT for Fact Verification Task

ChatGPT has recently emerged as a powerful tool for performing diverse NLP tasks. However, ChatGPT has been criticized for generating nonfactual responses, raising concerns about its usability for sensitive tasks like fact verification. This study investigates three key research questions: (1) Can ChatGPT be used for fact verification tasks? (2) What are different prompts performance using ChatGPT for fact verification tasks? (3) For the best-performing prompt, what common mistakes does ChatGPT make? Specifically, this study focuses on conducting a comprehensive and systematic analysis by designing and comparing the performance of three different prompts for fact verification tasks on the benchmark FEVER dataset using ChatGPT.

cs.IR↗

arXiv2023

Secure Inter-domain Routing and Forwarding via Verifiable Forwarding Commitments

The Internet inter-domain routing system is vulnerable. On the control plane, the de facto Border Gateway Protocol (BGP) does not have built-in mechanisms to authenticate routing announcements, so an adversary can announce virtually arbitrary paths to hijack network traffic; on the data plane, it is difficult to ensure that actual forwarding path complies with the control plane decisions. The community has proposed significant research to secure the routing system. Yet, existing secure BGP protocols (e.g., BGPsec) are not incrementally deployable, and existing path authorization protocols are not compatible with the current Internet routing infrastructure. In this paper, we propose FC-BGP, the first secure Internet inter-domain routing system that can simultaneously authenticate BGP announcements and validate data plane forwarding in an efficient and incrementally-deployable manner. FC-BGP is built upon a novel primitive, name Forwarding Commitment, to certify an AS's routing intent on its directly connected hops. We analyze the security benefits of FC-BGP in the Internet at different deployment rates. Further, we implement a prototype of FC-BGP and extensively evaluate it over a large-scale overlay network with 100 virtual machines deployed globally. The results demonstrate that FC-BGP saves roughly 55% of the overhead required to validate BGP announcements compared with BGPsec, and meanwhile FC-BGP introduces a small overhead for building a globally-consistent view on the desirable forwarding paths.

cs.NI↗

arXiv2023

The Ground State Lattice Distortion of CsV$_{3}$Sb$_{5}$ Revealed by de Haas-van Alphen Oscillations

The recently discovered AV$_{3}$Sb$_{5}$ (A = K, Rb, Cs) compounds have garnered intense attention in the scientific community due to their unique characteristics as kagome superconductors coexisting with a charge density wave (CDW) order. To comprehend the electronic properties of this system, it is essential to understand the lattice distortions associated with the CDW order and the ground state electronic behavior. Here, we comprehensively examine the Fermi surface by a combination of angle-dependent torque magnetometry and density functional theory calculations. We observe magnetic breakdown in the de Haas-van Alphen oscillations under high magnetic fields. Additionally, by examining the angular and temperature variations in quantum oscillation frequencies, we gain insight into the evolution of the three-dimensional like Fermi surfaces and the cyclotron masses of the orbits, which are consistent with weak electron-phonon coupling. Notably, further comparisons indicate that the 2$\times$2$\times$2 Star-of-David (SoD) distortion is more compatible with both high frequency data above 1000\,T and low frequency data below 500\,T, while the 2$\times$2$\times$2 Tri-Hexagonal (TrH) distortion aligns well with experimental data at mid frequencies. This finding implies the inherent coexistence of both TrH and SoD 2$\times$2$\times$2 patterns within the CDW order. These observations provide key insights into the interplay among effective electronic dimensionality, CDW state, and superconductivity.

cond-mat.supr-con↗