arXiv subjects

Wei Lu

Publications and source records attributed to Wei Lu.

At least 217 records · Page 12Linked to original sources

arXiv2023

Know Where to Go: Make LLM a Relevant, Responsible, and Trustworthy Searcher

The advent of Large Language Models (LLMs) has shown the potential to improve relevance and provide direct answers in web searches. However, challenges arise in validating the reliability of generated results and the credibility of contributing sources, due to the limitations of traditional information retrieval algorithms and the LLM hallucination problem. Aiming to create a "PageRank" for the LLM era, we strive to transform LLM into a relevant, responsible, and trustworthy searcher. We propose a novel generative retrieval framework leveraging the knowledge of LLMs to foster a direct link between queries and online sources. This framework consists of three core modules: Generator, Validator, and Optimizer, each focusing on generating trustworthy online sources, verifying source reliability, and refining unreliable sources, respectively. Extensive experiments and evaluations highlight our method's superior relevance, responsibility, and trustfulness against various SOTA methods.

cs.IR↗

arXiv2023

The Weight Hierarchies of Linear Codes from Simplicial Complexes

The study of the generalized Hamming weight of linear codes is a significant research topic in coding theory as it conveys the structural information of the codes and determines their performance in various applications. However, determining the generalized Hamming weights of linear codes, especially the weight hierarchy, is generally challenging. In this paper, we investigate the generalized Hamming weights of a class of linear code $\C$ over $\bF_q$, which is constructed from defining sets. These defining sets are either special simplicial complexes or their complements in $\bF_q^m$. We determine the complete weight hierarchies of these codes by analyzing the maximum or minimum intersection of certain simplicial complexes and all $r$-dimensional subspaces of $\bF_q^m$, where $1\leq r\leq {\rm dim}_{\bF_q}(\C)$.

cs.IT↗

arXiv2023

Decomposed Prompt Tuning via Low-Rank Reparameterization

While prompt tuning approaches have achieved competitive performance with high efficiency, we observe that they invariably employ the same initialization process, wherein the soft prompt is either randomly initialized or derived from an existing embedding vocabulary. In contrast to these conventional methods, this study aims to investigate an alternative way to derive soft prompt. Our empirical studies show that the soft prompt typically exhibits a low intrinsic rank characteristic. With such observations, we propose decomposed prompt tuning, a novel approach that utilizes low-rank matrices to initialize the soft prompt. Through the low-rank reparameterization, our method significantly reduces the number of trainable parameters while maintaining effectiveness. Experimental results on the SuperGLUE benchmark in both high-resource and low-resource scenarios demonstrate the effectiveness of the proposed method.

cs.CL↗

arXiv2023

A modular table-top setup for ultrafast X-ray diffraction

We present a table-top setup for femtosecond time-resolved X-ray diffraction based on a Cu Kα (8.05 keV) laser driven plasma X-ray source. Due to its modular design it provides high accessibility to its individual components (e.g. X-ray optics and sample environment). The Kα-yield of the source is optimized using a pre-pulse scheme. A magnifying multilayer X-ray mirror with Montel-Helios geometry is used to collect the emitted radiation, resulting in a quasi-collimated flux of more than 105 Cu Kα photons/pulse impinging on the sample under investigation at a repetition rate of 10 Hz. A gas ionization chamber detector is placed right after the X-ray mirror and used for normalization of the diffraction signals enabling the measurement of relative signal changes of less than 1% even at the given low repetition rate. Time-resolved diffraction experiments on laser-excited epitaxial Bi films serve as an example to demonstrate the capabilities of the set-up. The set-up can also be used for Debye-Scherrer type measurements on poly-crystalline samples.

cond-mat.mtrl-sci↗

arXiv2023

H2CGL: Modeling Dynamics of Citation Network for Impact Prediction

The potential impact of a paper is often quantified by how many citations it will receive. However, most commonly used models may underestimate the influence of newly published papers over time, and fail to encapsulate this dynamics of citation network into the graph. In this study, we construct hierarchical and heterogeneous graphs for target papers with an annual perspective. The constructed graphs can record the annual dynamics of target papers' scientific context information. Then, a novel graph neural network, Hierarchical and Heterogeneous Contrastive Graph Learning Model (H2CGL), is proposed to incorporate heterogeneity and dynamics of the citation network. H2CGL separately aggregates the heterogeneous information for each year and prioritizes the highly-cited papers and relationships among references, citations, and the target paper. It then employs a weighted GIN to capture dynamics between heterogeneous subgraphs over years. Moreover, it leverages contrastive learning to make the graph representations more sensitive to potential citations. Particularly, co-cited or co-citing papers of the target paper with large citation gap are taken as hard negative samples, while randomly dropping low-cited papers could generate positive samples. Extensive experimental results on two scholarly datasets demonstrate that the proposed H2CGL significantly outperforms a series of baseline approaches for both previously and freshly published papers. Additional analyses highlight the significance of the proposed modules. Our codes and settings have been released on Github (https://github.com/ECNU-Text-Computing/H2CGL)

cs.DL↗

arXiv2023

Create and Find Flatness: Building Flat Training Spaces in Advance for Continual Learning

Catastrophic forgetting remains a critical challenge in the field of continual learning, where neural networks struggle to retain prior knowledge while assimilating new information. Most existing studies emphasize mitigating this issue only when encountering new tasks, overlooking the significance of the pre-task phase. Therefore, we shift the attention to the current task learning stage, presenting a novel framework, C&F (Create and Find Flatness), which builds a flat training space for each task in advance. Specifically, during the learning of the current task, our framework adaptively creates a flat region around the minimum in the loss landscape. Subsequently, it finds the parameters' importance to the current task based on their flatness degrees. When adapting the model to a new task, constraints are applied according to the flatness and a flat space is simultaneously prepared for the impending task. We theoretically demonstrate the consistency between the created and found flatness. In this manner, our framework not only accommodates ample parameter space for learning new tasks but also preserves the preceding knowledge of earlier tasks. Experimental results exhibit C&F's state-of-the-art performance as a standalone continual learning approach and its efficacy as a framework incorporating other methods. Our work is available at https://github.com/Eric8932/Create-and-Find-Flatness.

cs.LG↗

arXiv2023

Generative modeling, design and analysis of spider silk protein sequences for enhanced mechanical properties

Spider silks are remarkable materials characterized by superb mechanical properties such as strength, extensibility and lightweightedness. Yet, to date, limited models are available to fully explore sequence-property relationships for analysis and design. Here we propose a custom generative large-language model to enable design of novel spider silk protein sequences to meet complex combinations of target mechanical properties. The model, pretrained on a large set of protein sequences, is fine-tuned on ~1,000 major ampullate spidroin (MaSp) sequences for which associated fiber-level mechanical properties exist, to yield an end-to-end forward and inverse generative strategy. Performance is assessed through: (1), a novelty analysis and protein type classification for generated spidroin sequences through BLAST searches, (2) property evaluation and comparison with similar sequences, (3) comparison of molecular structures, as well as, and (4) a detailed sequence motif analyses. We generate silk sequences with property combinations that do not exist in nature, and develop a deep understanding the mechanistic roles of sequence patterns in achieving overarching key mechanical properties (elastic modulus, strength, toughness, failure strain). The model provides an efficient approach to expand the silkome dataset, facilitating further sequence-structure analyses of silks, and establishes a foundation for synthetic silk design and optimization.

cond-mat.mtrl-sci↗

arXiv2023

Two classes of reducible cyclic codes with large minimum symbol-pair distances

The high-density data storage technology aims to design high-capacity storage at a relatively low cost. In order to achieve this goal, symbol-pair codes were proposed by Cassuto and Blaum \cite{CB10,CB11} to handle channels that output pairs of overlapping symbols. Such a channel is called symbol-pair read channel, which introduce new concept called symbol-pair weight and minimum symbol-pair distance. In this paper, we consider the parameters of two classes of reducible cyclic codes under the symbol-pair metric. Based on the theory of cyclotomic numbers and Gaussian period over finite fields, we show the possible symbol-pair weights of these codes. Their minimum symbol-pair distances are twice the minimum Hamming distances under some conditions. Moreover, we obtain some three symbol-pair weight codes and determine their symbol-pair weight distribution. A class of MDS symbol-pair codes is also established. Among other results, we determine the values of some generalized cyclotomic numbers.

cs.IT↗

arXiv2023

DDH-QA: A Dynamic Digital Humans Quality Assessment Database

In recent years, large amounts of effort have been put into pushing forward the real-world application of dynamic digital human (DDH). However, most current quality assessment research focuses on evaluating static 3D models and usually ignores motion distortions. Therefore, in this paper, we construct a large-scale dynamic digital human quality assessment (DDH-QA) database with diverse motion content as well as multiple distortions to comprehensively study the perceptual quality of DDHs. Both model-based distortion (noise, compression) and motion-based distortion (binding error, motion unnaturalness) are taken into consideration. Ten types of common motion are employed to drive the DDHs and a total of 800 DDHs are generated in the end. Afterward, we render the video sequences of the distorted DDHs as the evaluation media and carry out a well-controlled subjective experiment. Then a benchmark experiment is conducted with the state-of-the-art video quality assessment (VQA) methods and the experimental results show that existing VQA methods are limited in assessing the perceptual loss of DDHs.

cs.CV↗

arXiv2023

EEP-3DQA: Efficient and Effective Projection-based 3D Model Quality Assessment

Currently, great numbers of efforts have been put into improving the effectiveness of 3D model quality assessment (3DQA) methods. However, little attention has been paid to the computational costs and inference time, which is also important for practical applications. Unlike 2D media, 3D models are represented by more complicated and irregular digital formats, such as point cloud and mesh. Thus it is normally difficult to perform an efficient module to extract quality-aware features of 3D models. In this paper, we address this problem from the aspect of projection-based 3DQA and develop a no-reference (NR) \underline{E}fficient and \underline{E}ffective \underline{P}rojection-based \underline{3D} Model \underline{Q}uality \underline{A}ssessment (\textbf{EEP-3DQA}) method. The input projection images of EEP-3DQA are randomly sampled from the six perpendicular viewpoints of the 3D model and are further spatially downsampled by the grid-mini patch sampling strategy. Further, the lightweight Swin-Transformer tiny is utilized as the backbone to extract the quality-aware features. Finally, the proposed EEP-3DQA and EEP-3DQA-t (tiny version) achieve the best performance than the existing state-of-the-art NR-3DQA methods and even outperforms most full-reference (FR) 3DQA methods on the point cloud and mesh quality assessment databases while consuming less inference time than the compared 3DQA methods.

cs.CV↗

arXiv2023

Meter-scale strong coupling between magnons and photons

We experimentally realize a meter-scale strong coupling effect between magnons and photons at room temperature, with a coherent coupling of 20 m and a dissipative coupling of 7.6 m. To this end, we integrate a saturable gain into a microwave cavity and then couple this active cavity to a magnon mode via a long coaxial cable. The gain compensates for the cavity dissipation, but preserves the cavity radiation that mediates the indirect photon-magnon coupling. It thus enables the long-range strong photon-magnon coupling. With full access to traveling waves, we demonstrate a remote control of photon-magnon coupling by modulating the phase and amplitude of traveling waves, rather than reconfiguring subsystems themselves. Our method for realizing long-range strong coupling in cavity magnonics provides a general idea for other physical systems. Our experimental achievements may promote the construction of information networks based on cavity magnonics.

quant-ph↗

arXiv2023

Control of the magnon-polariton hybridization with a microwave pump

Pump-induced magnon modes (PIMs) are recently discovered elementary excitations in ferrimagnets that offer significant tunability to spin dynamics. Here, we investigate the coupling between a PIM and cavity magnon polaritons (CMPs) by driving a cavity magnonic system away from equilibrium with a microwave pump. In our experiment, the Walker mode simultaneously couples with the PIM and cavity photons and thus combines two strongly coherent coupling processes in a single cavity structure. Such a PIM-CMP hybridization system acquires complementary properties from both the PIM and CMPs, allowing it to be freely manipulated by the magnetic field, the pump power and the pump frequency. These coherent manipulations exhibit unique behaviors beyond the intrinsic properties limited by the material nature and electromagnetic boundary conditions, thereby creating opportunities for extending the control of hybrid devices.

cond-mat.mes-hall↗

arXiv2023

Unveiling a Pump-Induced Magnon Mode via its Strong Interaction with Walker Modes

We observe a power-dependent anticrossing of Walker spin-wave modes under microwave pumping when a ferrimagnet is placed in a microwave waveguide that does not support any discrete photon mode. We interpret this unexpected anticrossing as the generation of a pump-induced magnon mode that couples strongly to the Walker modes of the ferrimagnet. This anticrossing inherits an excellent tunability from the pump, which allows us to control the anticrossing via the pump power, frequency, and waveform. Further, we realize a remarkable functionality of this anticrossing, namely, a microwave frequency comb, in terms of the nonlinear interaction that mixes the pump and probe frequencies. Such a frequency comb originates from the magnetic dynamics and thereby does not suffer from the charge noise. The unveiled hybrid magnonics driven away from its equilibrium enriches the utilization of anticrossing for coherent information processing.

cond-mat.mes-hall↗

arXiv2023

Research Explosion: More Effort to Climb onto Shoulders of the Giant

Fast-growing scientific publications present challenges to the scientific community. In this paper, we describe their implications to researchers. As references form explicit foundations for researchers to conduct a study, we investigate the evolution in reference patterns based on 60.8 million papers published from 1960 to 2015. The results demonstrate that recent papers contain more references than older ones, especially the well-cited papers compared with other papers. Well-cited papers receive 10 or more citations within 5 years of publication. Their references cover a longer period from classic research to very recent studies. Authors of well-cited papers are also farsighted to discover the reference papers with good potential to receive high citation numbers in near future. We also discover that the number of accumulative publications has a negative impact on next-5-year citation count for most fields except Chemistry, Materials science, Environmental science, Biology, and Engineering. Our findings suggest that researchers are expected to devote more effort to producing impactful research. Based on all these findings, we strongly advise against judging researchers simply based on the number of their publications. On the other hand, authors and reviewers should ensure that published papers contain adequate contributions. The code for our analysis is on GitHub: https://github.com/ECNU-Text-Computing/Research-Explosion.

cs.DL↗

arXiv2023

Enhancing Quality of Pose-varied Face Restoration with Local Weak Feature Sensing and GAN Prior

Facial semantic guidance (including facial landmarks, facial heatmaps, and facial parsing maps) and facial generative adversarial networks (GAN) prior have been widely used in blind face restoration (BFR) in recent years. Although existing BFR methods have achieved good performance in ordinary cases, these solutions have limited resilience when applied to face images with serious degradation and pose-varied (e.g., looking right, looking left, laughing, etc.) in real-world scenarios. In this work, we propose a well-designed blind face restoration network with generative facial prior. The proposed network is mainly comprised of an asymmetric codec and a StyleGAN2 prior network. In the asymmetric codec, we adopt a mixed multi-path residual block (MMRB) to gradually extract weak texture features of input images, which can better preserve the original facial features and avoid excessive fantasy. The MMRB can also be plug-and-play in other networks. Furthermore, thanks to the affluent and diverse facial priors of the StyleGAN2 model, we adopt it as the primary generator network in our proposed method and specially design a novel self-supervised training strategy to fit the distribution closer to the target and flexibly restore natural and realistic facial details. Extensive experiments on synthetic and real-world datasets demonstrate that our model performs superior to the prior art for face restoration and face super-resolution tasks.

cs.CV↗

arXiv2023

One Network, Many Masks: Towards More Parameter-Efficient Transfer Learning

Fine-tuning pre-trained language models for multiple tasks tends to be expensive in terms of storage. To mitigate this, parameter-efficient transfer learning (PETL) methods have been proposed to address this issue, but they still require a significant number of parameters and storage when being applied to broader ranges of tasks. To achieve even greater storage reduction, we propose PROPETL, a novel method that enables efficient sharing of a single PETL module which we call prototype network (e.g., adapter, LoRA, and prefix-tuning) across layers and tasks. We then learn binary masks to select different sub-networks from the shared prototype network and apply them as PETL modules into different layers. We find that the binary masks can determine crucial information from the network, which is often ignored in previous studies. Our work can also be seen as a type of pruning method, where we find that overparameterization also exists in the seemingly small PETL modules. We evaluate PROPETL on various downstream tasks and show that it can outperform other PETL methods with approximately 10% of the parameter storage required by the latter.

cs.CL↗

arXiv2023

Leveraging Training Data in Few-Shot Prompting for Numerical Reasoning

Chain-of-thought (CoT) prompting with large language models has proven effective in numerous natural language processing tasks, but designing prompts that generalize well to diverse problem types can be challenging, especially in the context of math word problem (MWP) solving. Additionally, it is common to have a large amount of training data that have a better diversity coverage but CoT annotations are not available, which limits the use of supervised learning techniques. To address these issues, we investigate two approaches to leverage the training data in a few-shot prompting scenario: dynamic program prompting and program distillation. Our approach is largely inspired by Gao et al., (2022), where they proposed to replace the CoT with the programs as the intermediate reasoning step. Such a prompting strategy allows us to accurately verify the answer correctness through program execution in MWP solving. Our dynamic program prompting involves annotating the training data by sampling correct programs from a large language model, while program distillation involves adapting a smaller model to the program-annotated training data. Our experiments on three standard MWP datasets demonstrate the effectiveness of these approaches, yielding significant improvements over previous baselines for prompting and fine-tuning. Our results suggest that leveraging a large amount of training data can improve the generalization ability of prompts and boost the performance of fine-tuned small models in MWP solving.

cs.CL↗

arXiv2023

Learning Multi-Step Reasoning by Solving Arithmetic Tasks

Mathematical reasoning is regarded as a necessary ability for Language Models (LMs). Recent works demonstrate large LMs' impressive performance in solving math problems. The success is attributed to their Chain-of-Thought (CoT) reasoning abilities, i.e., the ability to decompose complex questions into step-by-step reasoning chains, but such ability seems only to emerge from models with abundant parameters. This work investigates how to incorporate relatively small LMs with the capabilities of multi-step reasoning. We propose to inject such abilities by continually pre-training LMs on a synthetic dataset MsAT which is composed of Multi-step Arithmetic Tasks. Our experiments on four math word problem datasets show the effectiveness of the proposed method in enhancing LMs' math reasoning abilities.

cs.CL↗