Searcharxiv⌕ Search

arXiv subjects

Bowen Wang

Publications and source records attributed to Bowen Wang.

At least 91 records · Page 5Linked to original sources
arXiv2023

Causal Intervention for Measuring Confidence in Drug-Target Interaction Prediction

Identifying and discovering drug-target interactions(DTIs) are vital steps in drug discovery and development. They play a crucial role in assisting scientists in finding new drugs and accelerating the drug development process. Recently, knowledge graph and knowledge graph embedding (KGE) models have made rapid advancements and demonstrated impressive performance in drug discovery. However, such models lack authenticity and accuracy in drug target identification, leading to an increased misjudgment rate and reduced drug development efficiency. To address these issues, we focus on the problem of drug-target interactions, with knowledge mapping as the core technology. Specifically, a causal intervention-based confidence measure is employed to assess the triplet score to improve the accuracy of the drug-target interaction prediction model. Experimental results demonstrate that the developed confidence measurement method based on causal intervention can significantly enhance the accuracy of DTI link prediction, particularly for high-precision models. The predicted results are more valuable in guiding the design and development of subsequent drug development experiments, thereby significantly improving the efficiency of drug development.

q-bio.QM↗
arXiv2023

Instruct Me More! Random Prompting for Visual In-Context Learning

Large-scale models trained on extensive datasets, have emerged as the preferred approach due to their high generalizability across various tasks. In-context learning (ICL), a popular strategy in natural language processing, uses such models for different tasks by providing instructive prompts but without updating model parameters. This idea is now being explored in computer vision, where an input-output image pair (called an in-context pair) is supplied to the model with a query image as a prompt to exemplify the desired output. The efficacy of visual ICL often depends on the quality of the prompts. We thus introduce a method coined Instruct Me More (InMeMo), which augments in-context pairs with a learnable perturbation (prompt), to explore its potential. Our experiments on mainstream tasks reveal that InMeMo surpasses the current state-of-the-art performance. Specifically, compared to the baseline without learnable prompt, InMeMo boosts mIoU scores by 7.35 and 15.13 for foreground segmentation and single object detection tasks, respectively. Our findings suggest that InMeMo offers a versatile and efficient way to enhance the performance of visual ICL with lightweight training. Code is available at https://github.com/Jackieam/InMeMo.

cs.CV↗
arXiv2023

MPrompt: Exploring Multi-level Prompt Tuning for Machine Reading Comprehension

The large language models have achieved superior performance on various natural language tasks. One major drawback of such approaches is they are resource-intensive in fine-tuning new datasets. Soft-prompt tuning presents a resource-efficient solution to fine-tune the pre-trained language models (PLMs) while keeping their weight frozen. Existing soft prompt methods mainly focus on designing the input-independent prompts that steer the model to fit the domain of the new dataset. Those methods often ignore the fine-grained information about the task and context of the text. In this paper, we propose a multi-level prompt tuning (MPrompt) method for machine reading comprehension. It utilizes prompts at task-specific, domain-specific, and context-specific levels to enhance the comprehension of input semantics at different granularities. We also propose an independence constraint to steer each domain-specific prompt to focus on information within its domain to avoid redundancy. Moreover, we present a prompt generator that incorporates context-related knowledge in the prompt generation to enhance contextual relevancy. We conducted extensive experiments on 12 benchmarks of various QA formats and achieved an average improvement of 1.94\% over the state-of-the-art methods.

cs.CL↗
arXiv2023

CARE-MI: Chinese Benchmark for Misinformation Evaluation in Maternity and Infant Care

The recent advances in natural language processing (NLP), have led to a new trend of applying large language models (LLMs) to real-world scenarios. While the latest LLMs are astonishingly fluent when interacting with humans, they suffer from the misinformation problem by unintentionally generating factually false statements. This can lead to harmful consequences, especially when produced within sensitive contexts, such as healthcare. Yet few previous works have focused on evaluating misinformation in the long-form (LF) generation of LLMs, especially for knowledge-intensive topics. Moreover, although LLMs have been shown to perform well in different languages, misinformation evaluation has been mostly conducted in English. To this end, we present a benchmark, CARE-MI, for evaluating LLM misinformation in: 1) a sensitive topic, specifically the maternity and infant care domain; and 2) a language other than English, namely Chinese. Most importantly, we provide an innovative paradigm for building LF generation evaluation benchmarks that can be transferred to other knowledge-intensive domains and low-resourced languages. Our proposed benchmark fills the gap between the extensive usage of LLMs and the lack of datasets for assessing the misinformation generated by these models. It contains 1,612 expert-checked questions, accompanied with human-selected references. Using our benchmark, we conduct extensive experiments and found that current Chinese LLMs are far from perfect in the topic of maternity and infant care. In an effort to minimize the reliance on human resources for performance evaluation, we offer off-the-shelf judgment models for automatically assessing the LF output of LLMs given benchmark questions. Moreover, we compare potential solutions for LF generation evaluation and provide insights for building better automated metrics.

cs.CL↗
arXiv2023

TCRA-LLM: Token Compression Retrieval Augmented Large Language Model for Inference Cost Reduction

Since ChatGPT released its API for public use, the number of applications built on top of commercial large language models (LLMs) increase exponentially. One popular usage of such models is leveraging its in-context learning ability and generating responses given user queries leveraging knowledge obtained by retrieval augmentation. One problem of deploying commercial retrieval-augmented LLMs is the cost due to the additionally retrieved context that largely increases the input token size of the LLMs. To mitigate this, we propose a token compression scheme that includes two methods: summarization compression and semantic compression. The first method applies a T5-based model that is fine-tuned by datasets generated using self-instruct containing samples with varying lengths and reduce token size by doing summarization. The second method further compresses the token size by removing words with lower impact on the semantic. In order to adequately evaluate the effectiveness of the proposed methods, we propose and utilize a dataset called Food-Recommendation DB (FRDB) focusing on food recommendation for women around pregnancy period or infants. Our summarization compression can reduce 65% of the retrieval token size with further 0.3% improvement on the accuracy; semantic compression provides a more flexible way to trade-off the token size with performance, for which we can reduce the token size by 20% with only 1.6% of accuracy drop.

cs.CL↗
arXiv2023

AgentTuning: Enabling Generalized Agent Abilities for LLMs

Open large language models (LLMs) with great performance in various tasks have significantly advanced the development of LLMs. However, they are far inferior to commercial models such as ChatGPT and GPT-4 when acting as agents to tackle complex tasks in the real world. These agent tasks employ LLMs as the central controller responsible for planning, memorization, and tool utilization, necessitating both fine-grained prompting methods and robust LLMs to achieve satisfactory performance. Though many prompting methods have been proposed to complete particular agent tasks, there is lack of research focusing on improving the agent capabilities of LLMs themselves without compromising their general abilities. In this work, we present AgentTuning, a simple and general method to enhance the agent abilities of LLMs while maintaining their general LLM capabilities. We construct AgentInstruct, a lightweight instruction-tuning dataset containing high-quality interaction trajectories. We employ a hybrid instruction-tuning strategy by combining AgentInstruct with open-source instructions from general domains. AgentTuning is used to instruction-tune the Llama 2 series, resulting in AgentLM. Our evaluations show that AgentTuning enables LLMs' agent capabilities without compromising general abilities. The AgentLM-70B is comparable to GPT-3.5-turbo on unseen agent tasks, demonstrating generalized agent capabilities. We open source the AgentInstruct and AgentLM-7B, 13B, and 70B models at https://github.com/THUDM/AgentTuning, serving open and powerful alternatives to commercial LLMs for agent tasks.

cs.CL↗
arXiv2023

Improving Facade Parsing with Vision Transformers and Line Integration

Facade parsing stands as a pivotal computer vision task with far-reaching applications in areas like architecture, urban planning, and energy efficiency. Despite the recent success of deep learning-based methods in yielding impressive results on certain open-source datasets, their viability for real-world applications remains uncertain. Real-world scenarios are considerably more intricate, demanding greater computational efficiency. Existing datasets often fall short in representing these settings, and previous methods frequently rely on extra models to enhance accuracy, which requires much computation cost. In this paper, we introduce Comprehensive Facade Parsing (CFP), a dataset meticulously designed to encompass the intricacies of real-world facade parsing tasks. Comprising a total of 602 high-resolution street-view images, this dataset captures a diverse array of challenging scenarios, including sloping angles and densely clustered buildings, with painstakingly curated annotations for each image. We introduce a new pipeline known as Revision-based Transformer Facade Parsing (RTFP). This marks the pioneering utilization of Vision Transformers (ViT) in facade parsing, and our experimental results definitively substantiate its merit. We also design Line Acquisition, Filtering, and Revision (LAFR), an efficient yet accurate revision algorithm that can improve the segment result solely from simple line detection using prior knowledge of the facade. In ECP 2011, RueMonge 2014, and our CFP, we evaluate the superiority of our method.

cs.CV↗
arXiv2023

Fast and reconfigurable sort-in-memory system enabled by memristors

Sorting is fundamental and ubiquitous in modern computing systems. Hardware sorting systems are built based on comparison operations with Von Neumann architecture, but their performance are limited by the bandwidth between memory and comparison units and the performance of complementary metal-oxide-semiconductor (CMOS) based circuitry. Sort-in-memory (SIM) based on emerging memristors is desired but not yet available due to comparison operations that are challenging to be implemented within memristive memory. Here we report fast and reconfigurable SIM system enabled by digit read (DR) on 1-transistor-1-resistor (1T1R) memristor arrays. We develop DR tree node skipping (TNS) that support variable data quantity and data types, and extend TNS with multi-bank, bit-slice and multi-level strategies to enable cross-array TNS (CA-TNS) for practical adoptions. Experimented on benchmark sorting datasets, our memristor-enabled SIM system presents up to 3.32x~7.70x speedup, 6.23x~183.5x energy efficiency improvement and 2.23x~7.43x area reduction compared with state-of-the-art sorting systems. We apply such SIM system for shortest path search with Dijkstra's algorithm and neural network inference with in-situ pruning, demonstrating the capability in solving practical sorting tasks and the compatibility in integrating with other compute-in-memory (CIM) schemes. The comparison-free TNS/CA-TNS SIM enabled by memristors pushes sorting into a new paradigm of sort-in-memory for next-generation sorting systems.

cs.AR↗
arXiv2023

Dynamic Hybrid Beamforming Design for Dual-Function Radar-Communication Systems

This paper investigates dynamic hybrid beamforming (HBF) for a dual-function radar-communication (DFRC) system, where the DFRC base station (BS) simultaneously serves multiple single-antenna users and senses a target in the presence of multiple clutters. Particularly, we apply a HBF architecture with dynamic subarrays and double phase shifters in the DFRC BS. Aiming at maximizing the radar mutual information, we consider jointly designing the dynamic HBF of the DFRC system, subject to the constraints of communication quality of service (QoS), transmit power, and analog beamformer. To solve the complicated non-convex optimization, an efficient alternating optimization algorithm based on the majorization-minimization methods is developed. Simulation results verify the advancement of the considered HBF architecture and the effectiveness of the proposed design method.

eess.SP↗
arXiv2023

IncreLoRA: Incremental Parameter Allocation Method for Parameter-Efficient Fine-tuning

With the increasing size of pre-trained language models (PLMs), fine-tuning all the parameters in the model is not efficient, especially when there are a large number of downstream tasks, which incur significant training and storage costs. Many parameter-efficient fine-tuning (PEFT) approaches have been proposed, among which, Low-Rank Adaptation (LoRA) is a representative approach that injects trainable rank decomposition matrices into every target module. Yet LoRA ignores the importance of parameters in different modules. To address this problem, many works have been proposed to prune the parameters of LoRA. However, under limited training conditions, the upper bound of the rank of the pruned parameter matrix is still affected by the preset values. We, therefore, propose IncreLoRA, an incremental parameter allocation method that adaptively adds trainable parameters during training based on the importance scores of each module. This approach is different from the pruning method as it is not limited by the initial number of training parameters, and each parameter matrix has a higher rank upper bound for the same training overhead. We conduct extensive experiments on GLUE to demonstrate the effectiveness of IncreLoRA. The results show that our method owns higher parameter efficiency, especially when under the low-resource settings where our method significantly outperforms the baselines. Our code is publicly available.

cs.CL↗
arXiv2023

High-performance Thermoelectric Monolayer γ-GeSe and its Group-IV Monochalcogenide Isostructural Family

Recently synthesized novel phase of germanium selenide (γ-GeSe) adopts a hexagonal lattice and a surprisingly high conductivity than graphite. This triggers great interests in exploring its potential for thermoelectric applications. Herein, we explored the thermoelectric performance of monolayer γ-GeSe and other isostructural γ-phase of group-IV monochalcogenides γ-GeX (X = S, Se and Te) using the density functional theory and the Boltzmann transport theory. A superb thermoelectric performance of monolayer γ-GeSe is revealed with figure of merit ZT value up to 1.13-2.76 for n-type doping at a moderate carrier concentration of 4.73-2.58x10^12 cm-2 between 300 and 600 K. This superb performance is rooted in its rich pocket states and flat plateau levels around the electronic band edges, leading to promoted concentrations and electronic conductivity, and limited thermal conductivity. Our work suggests that monolayer γ-GeSe is a promising candidate for high performance medium-temperature thermoelectric applications.

cond-mat.mtrl-sci↗
arXiv2023

Phononic transport in 1T prime-MoTe2: anisotropic structure with an isotropic lattice thermal conductivity

Molybdenum ditelluride (MoTe2) is an unique transition metal dichalcogenide owing to its energetically comparable 1H and 1T prime phases. This implies a high chance of coexistence of 1H-1T prime heterostructures which poses great complexity in the measurement of the intrinsic lattice thermal conductivities (kappa). In this work, via first-principles calculations, we examine the lattice-wave propagation and thermal conduction in this highly structurally anisotropic 1T prime MoTe2. Our calculation shows that the 1T prime phase has a sound velocity of 2.13 km/s (longitudinal acoustic wave), much lower than that of the 1H phase (4.05 km /s), indicating a staggered transmission of lattice waves across the boundary from 1H to 1T prime phase. Interestingly, the highly anisotropic 1T prime MoTe2 shows nearly isotropic and limited kappa_L of 13.02 W/mK, owing to a large Gruneisen parameter of acoustic flexural mode, heavy masses of Mo and Te elements and a low phonon group velocity. Accumulative kappa_L as a function of mean free path (MFP) indicates phonons with MFP less than ~300 nm contribute 80% of kappa_L and an inflection point at ~600 nm. Our results will be critical for understanding of the size dependent kappa_L of nanostructured 1T prime MoTe2.

cond-mat.mtrl-sci↗
arXiv2023

Cramer-Rao Lower Bound Analysis for OTFS and OFDM Modulation Systems

The orthogonal time frequency space (OTFS) modulation as a promising signal representation attracts growingcinterest for integrated sensing and communication (ISAC), yet its merits over orthogonal frequency division multiplexing (OFDM) remain controversial. This paper devotes to a comprehensive comparison of OTFS and OFDM for sensing from the perspective of Cramer-Rao lower bounds (CRLB) analysis. To this end, we develop the cyclic prefix (CP)-Free and CP-added model for OFDM, while for OTFS, we consider the Zak transform based and the Two-Step conversion based models, respectively. Then we rephrase these four models into a unified matrix format to derive the CRLB of the delays and doppler shifts for multipath scenario. Numerical results demonstrate the superiority of OTFS modulation for sensing, and the effect of physical parameters for performance achievement.

eess.SP↗
arXiv2023

Learning Bottleneck Concepts in Image Classification

Interpreting and explaining the behavior of deep neural networks is critical for many tasks. Explainable AI provides a way to address this challenge, mostly by providing per-pixel relevance to the decision. Yet, interpreting such explanations may require expert knowledge. Some recent attempts toward interpretability adopt a concept-based framework, giving a higher-level relationship between some concepts and model decisions. This paper proposes Bottleneck Concept Learner (BotCL), which represents an image solely by the presence/absence of concepts learned through training over the target task without explicit supervision over the concepts. It uses self-supervision and tailored regularizers so that learned concepts can be human-understandable. Using some image classification tasks as our testbed, we demonstrate BotCL's potential to rebuild neural networks for better interpretability. Code is available at https://github.com/wbw520/BotCL and a simple demo is available at https://botcl.liangzhili.com/.

cs.CV↗
arXiv2023

Relative Entropy-Based Constant-Envelope Beamforming for Target Detection in Large-Scale MIMO Radar With Low-Resoultion ADCs

Hybrid digital/analog architecture and low-resolution analog-to-digital/digital-to-analog converters (ADCs /DACs) are two low-cost implementations for large-scale millimeter wave (mmWave) systems. In this paper, we investigate the problem of constant-envelope transmit beamforming for large-scale multiple-input multiple-output (MIMO) radar system, where the transmit array adopts a hybrid digital/analog architecture with a small number of RF chains and the receive array adopts a fully digital architecture with low-resolution ADCs. We derive the relative entropy between the probability density functions associated with the two test hypotheses under low-resolution ADCs. We formulate our optimization problem by maximizing the relative entropy, subject to the constant envelope and orthogonality constraints. To suboptimally solve the resultant problem, a two-stage framework is developed. In the first stage, we optimize the transmit power at the directions of the target and clutter. In the second stage, an efficient iterative algorithm based on majorization-minimization is presented to obtain the constant-envelope beamformer according to the attained transmit power. Specifically, we apply a quadratic function as the minorizer, leading to a low-complexity solution at each iteration. In addition, to further facilitate low-cost implementation of the constant-envelope beamformer, we consider the problem of one-bit beamforming design and propose an efficient iterative method based on the Nesterov-like gradient method to solve it. Numerical simulations are provided to demonstrate the effectiveness of the proposed schemes.

eess.SP↗
arXiv2023

DR-Label: Improving GNN Models for Catalysis Systems by Label Deconstruction and Reconstruction

Attaining the equilibrium state of a catalyst-adsorbate system is key to fundamentally assessing its effective properties, such as adsorption energy. Machine learning methods with finer supervision strategies have been applied to boost and guide the relaxation process of an atomic system and better predict its properties at the equilibrium state. In this paper, we present a novel graph neural network (GNN) supervision and prediction strategy DR-Label. The method enhances the supervision signal, reduces the multiplicity of solutions in edge representation, and encourages the model to provide node predictions that are graph structural variation robust. DR-Label first Deconstructs finer-grained equilibrium state information to the model by projecting the node-level supervision signal to each edge. Reversely, the model Reconstructs a more robust equilibrium state prediction by transforming edge-level predictions to node-level with a sphere-fitting algorithm. The DR-Label strategy was applied to three radically distinct models, each of which displayed consistent performance enhancements. Based on the DR-Label strategy, we further proposed DRFormer, which achieved a new state-of-the-art performance on the Open Catalyst 2020 (OC20) dataset and the Cu-based single-atom-alloyed CO adsorption (SAA) dataset. We expect that our work will highlight crucial steps for the development of a more accurate model in equilibrium state property prediction of a catalysis system.

cs.LG↗
arXiv2023

Electroweak corrections to Higgs boson production via W W fusion at the future LHeC

Precision measurement of quark Yukawa couplings is a crucial aspect of Higgs property study. Proposed as a future upgrade of the Large Hadron Collider (LHC), the Large Hadron electron Collider (LHeC) provides opportunities to probe quark Yukawa couplings with a high precision because of relatively low rate from QCD background as compared with that of the Higgs processes at the LHC. For this purpose, it is important to have a precision prediction of the Higgs production rate at the LHeC. The leading production channel of the Higgs boson at the LHeC is via weak boson fusion (WBF) which has the unique kinematic feature of forward tagging jets. Since there is no color exchange between the forward-backward objects in WBF processes, electroweak radiative corrections become particularly important. In this paper, we focus on the electroweak radiative corrections at next-to-leading order for Higgs boson production in the WBF channel at the LHeC. In the two renormalization schemes we use, the loop corrections are respectively at the level of 9% and 18% relative to the leading order result, and the next-to-leading calculation in both schemes agree up to a truncation error expected from the perturbative method. The size of the EW correction exceeds that from QCD radiations and is shown to be important for the study of Higgs phenomenology.

hep-ph↗
arXiv2023

SaFormer: A Conditional Sequence Modeling Approach to Offline Safe Reinforcement Learning

Offline safe RL is of great practical relevance for deploying agents in real-world applications. However, acquiring constraint-satisfying policies from the fixed dataset is non-trivial for conventional approaches. Even worse, the learned constraints are stationary and may become invalid when the online safety requirement changes. In this paper, we present a novel offline safe RL approach referred to as SaFormer, which tackles the above issues via conditional sequence modeling. In contrast to existing sequence models, we propose cost-related tokens to restrict the action space and a posterior safety verification to enforce the constraint explicitly. Specifically, SaFormer performs a two-stage auto-regression conditioned by the maximum remaining cost to generate feasible candidates. It then filters out unsafe attempts and executes the optimal action with the highest expected return. Extensive experiments demonstrate the efficacy of SaFormer featuring (1) competitive returns with tightened constraint satisfaction; (2) adaptability to the in-range cost values of the offline data without retraining; (3) generalizability for constraints beyond the current dataset.

cs.LG↗