Searcharxiv⌕ Search

arXiv subjects

Bo Wang

Publications and source records attributed to Bo Wang.

At least 469 records · Page 26Linked to original sources
arXiv2023

To Transformers and Beyond: Large Language Models for the Genome

In the rapidly evolving landscape of genomics, deep learning has emerged as a useful tool for tackling complex computational challenges. This review focuses on the transformative role of Large Language Models (LLMs), which are mostly based on the transformer architecture, in genomics. Building on the foundation of traditional convolutional neural networks and recurrent neural networks, we explore both the strengths and limitations of transformers and other LLMs for genomics. Additionally, we contemplate the future of genomic modeling beyond the transformer architecture based on current trends in research. The paper aims to serve as a guide for computational biologists and computer scientists interested in LLMs for genomic data. We hope the paper can also serve as an educational introduction and discussion for biologists to a fundamental shift in how we will be analyzing genomic data in the future.

q-bio.GN↗
arXiv2023

Disordered hyperuniformity signals functioning and resilience of self-organized vegetation patterns

In harsh environments, organisms may self-organize into spatially patterned systems in various ways. So far, studies of ecosystem spatial self-organization have primarily focused on apparent orders reflected by regular patterns. However, self-organized ecosystems may also have cryptic orders that can be unveiled only through certain quantitative analyses. Here we show that disordered hyperuniformity as a striking class of hidden orders can exist in spatially self-organized vegetation landscapes. By analyzing the high-resolution remotely sensed images across the American drylands, we demonstrate that it is not uncommon to find disordered hyperuniform vegetation states characterized by suppressed density fluctuations at long range. Such long-range hyperuniformity has been documented in a wide range of microscopic systems. Our finding contributes to expanding this domain to accommodate natural landscape ecological systems. We use theoretical modeling to propose that disordered hyperuniform vegetation patterning can arise from three generalized mechanisms prevalent in dryland ecosystems, including (1) critical absorbing states driven by an ecological legacy effect, (2) scale-dependent feedbacks driven by plant-plant facilitation and competition, and (3) density-dependent aggregation driven by plant-sediment feedbacks. Our modeling results also show that disordered hyperuniform patterns can help ecosystems cope with arid conditions with enhanced functioning of soil moisture acquisition. However, this advantage may come at the cost of slower recovery of ecosystem structure upon perturbations. Our work highlights that disordered hyperuniformity as a distinguishable but underexplored ecosystem self-organization state merits systematic studies to better understand its underlying mechanisms, functioning, and resilience.

q-bio.PE↗
arXiv2023

Variational operator learning: A unified paradigm marrying training neural operators and solving partial differential equations

Neural operators as novel neural architectures for fast approximating solution operators of partial differential equations (PDEs), have shown considerable promise for future scientific computing. However, the mainstream of training neural operators is still data-driven, which needs an expensive ground-truth dataset from various sources (e.g., solving PDEs' samples with the conventional solvers, real-world experiments) in addition to training stage costs. From a computational perspective, marrying operator learning and specific domain knowledge to solve PDEs is an essential step in reducing dataset costs and label-free learning. We propose a novel paradigm that provides a unified framework of training neural operators and solving PDEs with the variational form, which we refer to as the variational operator learning (VOL). Ritz and Galerkin approach with finite element discretization are developed for VOL to achieve matrix-free approximation of system functional and residual, then direct minimization and iterative update are proposed as two optimization strategies for VOL. Various types of experiments based on reasonable benchmarks about variable heat source, Darcy flow, and variable stiffness elasticity are conducted to demonstrate the effectiveness of VOL. With a label-free training set and a 5-label-only shift set, VOL learns solution operators with its test errors decreasing in a power law with respect to the amount of unlabeled data. To the best of the authors' knowledge, this is the first study that integrates the perspectives of the weak form and efficient iterative methods for solving sparse linear systems into the end-to-end operator learning task.

cs.LG↗
arXiv2023

CongFu: Conditional Graph Fusion for Drug Synergy Prediction

Drug synergy, characterized by the amplified combined effect of multiple drugs, is critically important for optimizing therapeutic outcomes. Limited data on drug synergy, arising from the vast number of possible drug combinations and testing costs, motivate the need for predictive methods. In this work, we introduce CongFu, a novel Conditional Graph Fusion Layer, designed to predict drug synergy. CongFu employs an attention mechanism and a bottleneck to extract local graph contexts and conditionally fuse graph data within a global context. Its modular architecture enables flexible replacement of layer modules, including readouts and graph encoders, facilitating customization for diverse applications. To evaluate the performance of CongFu, we conduct comprehensive experiments on four datasets, encompassing three distinct setups for drug synergy prediction. CongFu achieves state-of-the-art results on 11 out of 12 benchmark datasets, demonstrating its ability to capture intricate patterns of drug synergy. Through ablation studies, we validate the significance of individual layer components, affirming their contributions to overall predictive performance. Finally, we propose an explainability strategy for elucidating the effect of drugs on genes. By addressing the challenge of predicting drug synergy in untested drug pairs and utilizing our proposed explainability approach, CongFu opens new avenues for optimizing drug combinations and advancing personalized medicine.

cs.LG↗
arXiv2023

Spatially Resolved Gene Expression Prediction from H&E Histology Images via Bi-modal Contrastive Learning

Histology imaging is an important tool in medical diagnosis and research, enabling the examination of tissue structure and composition at the microscopic level. Understanding the underlying molecular mechanisms of tissue architecture is critical in uncovering disease mechanisms and developing effective treatments. Gene expression profiling provides insight into the molecular processes underlying tissue architecture, but the process can be time-consuming and expensive. We present BLEEP (Bi-modaL Embedding for Expression Prediction), a bi-modal embedding framework capable of generating spatially resolved gene expression profiles of whole-slide Hematoxylin and eosin (H&E) stained histology images. BLEEP uses contrastive learning to construct a low-dimensional joint embedding space from a reference dataset using paired image and expression profiles at micrometer resolution. With this approach, the gene expression of any query image patch can be imputed using the expression profiles from the reference dataset. We demonstrate BLEEP's effectiveness in gene expression prediction by benchmarking its performance on a human liver tissue dataset captured using the 10x Visium platform, where it achieves significant improvements over existing methods. Our results demonstrate the potential of BLEEP to provide insights into the molecular mechanisms underlying tissue architecture, with important implications in diagnosis and research of various diseases. The proposed approach can significantly reduce the time and cost associated with gene expression profiling, opening up new avenues for high-throughput analysis of histology images for both research and clinical applications.

cs.CV↗
arXiv2023

The backreaction effect of the sound speed resonance in DBI inflation

We examine the backreaction effect of the enhanced small-scale scalar perturbations from the sound speed resonance (SSR) mechanism for primordial black hole formation in Dirac-Born-Infeld (DBI) inflation on background as well as curvature perturbation, which can generate a considerable amount of primordial black holes (PBH) in the radiation-dominated epoch. Within the perturbative regime, the backreaction effect of perturbations on the background dynamics can be described by an effective action after integrating out the perturbation sector. Starting with the effective field theory of a specific DBI inflation model that incorporates SSR, we obtain the one-loop effective action by integrating out the scalar perturbations at the quadratic level. Using the effective Friedmann equations derived from this one-loop effective action, we solve the Hubble parameter with backreaction and the effective perturbation dynamics on this background as well. Our numerical findings reveal that, for a viable parameter space, the backreaction effect results in a relative correction to the Hubble parameter of approximately $10^{-7}$, whereas the relative correction to the slow-roll parameter can vary between $-0.3$ and $0.1$, before gradually converging to $10^{-7}$. Furthermore, our results show that the backreaction effect on SSR sound speed causes a slight reduction in the resonant peak of the curvature power spectrum, and the subsequent PBH formation predicted by the SSR mechanism remains almost unchanged.

astro-ph.CO↗
arXiv2023

Open Domain Multi-document Summarization: A Comprehensive Study of Model Brittleness under Retrieval

Multi-document summarization (MDS) assumes a set of topic-related documents are provided as input. In practice, this document set is not always available; it would need to be retrieved given an information need, i.e. a question or topic statement, a setting we dub "open-domain" MDS. We study this more challenging setting by formalizing the task and bootstrapping it using existing datasets, retrievers and summarizers. Via extensive automatic and human evaluation, we determine: (1) state-of-the-art summarizers suffer large reductions in performance when applied to open-domain MDS, (2) additional training in the open-domain setting can reduce this sensitivity to imperfect retrieval, and (3) summarizers are insensitive to the retrieval of duplicate documents and the order of retrieved documents, but highly sensitive to other errors, like the retrieval of irrelevant documents. Based on our results, we provide practical guidelines to enable future work on open-domain MDS, e.g. how to choose the number of retrieved documents to summarize. Our results suggest that new retrieval and summarization methods and annotated resources for training and evaluation are necessary for further progress in the open-domain setting.

cs.CL↗
arXiv2023

Supervised Domain Adaptation for Recognizing Retinal Diseases from Wide-Field Fundus Images

This paper addresses the emerging task of recognizing multiple retinal diseases from wide-field (WF) and ultra-wide-field (UWF) fundus images. For an effective use of existing large amount of labeled color fundus photo (CFP) data and the relatively small amount of WF and UWF data, we propose a supervised domain adaptation method named Cross-domain Collaborative Learning (CdCL). Inspired by the success of fixed-ratio based mixup in unsupervised domain adaptation, we re-purpose this strategy for the current task. Due to the intrinsic disparity between the field-of-view of CFP and WF/UWF images, a scale bias naturally exists in a mixup sample that the anatomic structure from a CFP image will be considerably larger than its WF/UWF counterpart. The CdCL method resolves the issue by Scale-bias Correction, which employs Transformers for producing scale-invariant features. As demonstrated by extensive experiments on multiple datasets covering both WF and UWF images, the proposed method compares favorably against a number of competitive baselines.

eess.IV↗
arXiv2023

On Spectral Bias Reduction of Multi-scale Neural Networks for Regression Problems

In this paper, we derive diffusion equation models in the spectral domain for the evolution of training errors of two-layer multi-scale deep neural networks (MscaleDNN) \cite{caixu2019,liu2020multi}, designed to reduce the spectral bias of fully connected deep neural networks in approximating oscillatory functions. The diffusion models are obtained from the spectral form of the error equation of the MscaleDNN, derived with a neural tangent kernel approach and gradient descent training and a sine activation function, assuming a vanishing learning rate and infinite network width and domain size. The involved diffusion coefficients are shown to have larger supports if more scales are used in the MscaleDNN, and thus, the proposed diffusion equation models in the frequency domain explain the MscaleDNN's spectral bias reduction capability. Numerical results of the diffusion models for a two-layer MscaleDNN training match with the error evolution of actual gradient descent training with a reasonably large network width, thus validating the effectiveness of the diffusion models. Meanwhile, the numerical results for MscaleDNN show error decay over a wide frequency range and confirm the advantage of using the MscaleDNN in approximating functions with a wide range of frequencies.

math.NA↗
arXiv2023

Jina Embeddings: A Novel Set of High-Performance Sentence Embedding Models

Jina Embeddings constitutes a set of high-performance sentence embedding models adept at translating textual inputs into numerical representations, capturing the semantics of the text. These models excel in applications like dense retrieval and semantic textual similarity. This paper details the development of Jina Embeddings, starting with the creation of high-quality pairwise and triplet datasets. It underlines the crucial role of data cleaning in dataset preparation, offers in-depth insights into the model training process, and concludes with a comprehensive performance evaluation using the Massive Text Embedding Benchmark (MTEB). Furthermore, to increase the model's awareness of grammatical negation, we construct a novel training and evaluation dataset of negated and non-negated statements, which we make publicly available to the community.

cs.CL↗
arXiv2023

Splicing Up Your Predictions with RNA Contrastive Learning

In the face of rapidly accumulating genomic data, our understanding of the RNA regulatory code remains incomplete. Recent self-supervised methods in other domains have demonstrated the ability to learn rules underlying the data-generating process such as sentence structure in language. Inspired by this, we extend contrastive learning techniques to genomic data by utilizing functional similarities between sequences generated through alternative splicing and gene duplication. Our novel dataset and contrastive objective enable the learning of generalized RNA isoform representations. We validate their utility on downstream tasks such as RNA half-life and mean ribosome load prediction. Our pre-training strategy yields competitive results using linear probing on both tasks, along with up to a two-fold increase in Pearson correlation in low-data conditions. Importantly, our exploration of the learned latent space reveals that our contrastive objective yields semantically meaningful representations, underscoring its potential as a valuable initialization technique for RNA property prediction.

cs.LG↗
arXiv2023

Simplicity of AdS Super Yang-Mills at One Loop

We perform a systematic bootstrap analysis of four-point one-loop Mellin amplitudes for super gluons in $\mathrm{AdS}_5\times\mathrm{S}^3$ with arbitrary Kaluza-Klein weights. The analysis produces the general expressions for these amplitudes at extremalities two and three, as well as analytic results for many other special cases. From these results we observe remarkable simplicity. We find that the Mellin amplitudes always contain only simultaneous poles in two Mellin-Mandelstam variables, extending a previous observation in the simplest case with the lowest Kaluza-Klein weights. Moreover, we discover a substantial extension of the implication of the eight-dimensional hidden conformal symmetry, which goes far beyond the Mellin poles associated with the leading logarithmic singularities. This leaves only a small finite set of poles which can be determined on a case-by-case basis from the contributions of protected operators in the OPE.

hep-th↗
arXiv2023

ORTexME: Occlusion-Robust Human Shape and Pose via Temporal Average Texture and Mesh Encoding

In 3D human shape and pose estimation from a monocular video, models trained with limited labeled data cannot generalize well to videos with occlusion, which is common in the wild videos. The recent human neural rendering approaches focusing on novel view synthesis initialized by the off-the-shelf human shape and pose methods have the potential to correct the initial human shape. However, the existing methods have some drawbacks such as, erroneous in handling occlusion, sensitive to inaccurate human segmentation, and ineffective loss computation due to the non-regularized opacity field. To address these problems, we introduce ORTexME, an occlusion-robust temporal method that utilizes temporal information from the input video to better regularize the occluded body parts. While our ORTexME is based on NeRF, to determine the reliable regions for the NeRF ray sampling, we utilize our novel average texture learning approach to learn the average appearance of a person, and to infer a mask based on the average texture. In addition, to guide the opacity-field updates in NeRF to suppress blur and noise, we propose the use of human body mesh. The quantitative evaluation demonstrates that our method achieves significant improvement on the challenging multi-person 3DPW dataset, where our method achieves 1.8 P-MPJPE error reduction. The SOTA rendering-based methods fail and enlarge the error up to 5.6 on the same dataset.

cs.CV↗
arXiv2023

Semantic reconstruction of continuous language from MEG signals

Decoding language from neural signals holds considerable theoretical and practical importance. Previous research has indicated the feasibility of decoding text or speech from invasive neural signals. However, when using non-invasive neural signals, significant challenges are encountered due to their low quality. In this study, we proposed a data-driven approach for decoding semantic of language from Magnetoencephalography (MEG) signals recorded while subjects were listening to continuous speech. First, a multi-subject decoding model was trained using contrastive learning to reconstruct continuous word embeddings from MEG data. Subsequently, a beam search algorithm was adopted to generate text sequences based on the reconstructed word embeddings. Given a candidate sentence in the beam, a language model was used to predict the subsequent words. The word embeddings of the subsequent words were correlated with the reconstructed word embedding. These correlations were then used as a measure of the probability for the next word. The results showed that the proposed continuous word embedding model can effectively leverage both subject-specific and subject-shared information. Additionally, the decoded text exhibited significant similarity to the target text, with an average BERTScore of 0.816, a score comparable to that in the previous fMRI study.

cs.HC↗
arXiv2023

Interlacing property of a family of generating polynomials over Dyck paths

In the study of a tantalizing symmetry on Catalan objects, Bóna et al. introduced a family of polynomials $\{W_{n,k}(x)\}_{n\geq k\geq 0}$ defined by \begin{align*} W_{n,k}(x)=\sum_{m=0}^{k}w_{n,k,m}x^{m}, \end{align*} where $w_{n,k,m}$ counts the number of Dyck paths of semilength $n$ with $k$ occurrences of $UD$ and $m$ occurrences of $UUD$. They proposed two conjectures on the interlacing property of these polynomials, one of which states that $\{W_{n,k}(x)\}_{n\geq k}$ is a Sturm sequence for any fixed $k\geq 1$, and the other states that $\{W_{n,k}(x)\}_{1\leq k\leq n}$ is a Sturm-unimodal sequence for any fixed $n\geq 1$. In this paper, we obtain certain recurrence relations for $W_{n,k}(x)$, and further confirm their conjectures.

math.CO↗
arXiv2023

An underground fiber cable discrimination method based on laser interferometer

The maintenance and repair of optical fiber networks often requires the discrimination of underground cables among a group of buried ones. Presently, the methods commonly used are either inefficient or harmful to the fiber cable. In this paper, we evaluate the effectiveness of fiber optic vibration sensing method on underground fiber cable discrimination. We find that the typical vibration sensing method-distributed acoustic sensing (DAS) is not suitable for fiber cable discrimination. Especially for long distance scenario, due to "frequency grafting" effect, the DAS method will give a false response on the spectrum of knock events during cable discrimination process. In response, we propose an underground fiber cable discrimination method based on laser interferometer, and demonstrate it on the 40-km urban fiber cable connecting Tsinghua University and Yongding Road. It can give an obvious and real response to the knock event, and can be used in practical applications.

physics.ins-det↗
arXiv2023

Type Ia supernovae in NS+He star systems and the isolated mildly recycled pulsars

Type Ia supernovae (SNe Ia) are successful cosmological distance indicators and important element factories in the chemical evolution of galaxies. They are generally thought to originate from thermonuclear explosions of carbon-oxygen white dwarfs in close binaries. However, the observed diversity among SNe Ia implies that they have different progenitor models. In this article, we performed the long-term evolution of NS+He star binaries with different initial He star masses ($M_{\rm He}^{\rm i}$) and orbital periods ($P_{\rm orb}^{\rm i}$) for the first time, in which the He star companions can explode as SNe Ia eventually. Our simulations indicate that after the He stars develop highly degenerate oxygen-neon (ONe) cores with masses near the Chandrasekhar limit, explosive oxygen burning can be triggered due to the convective Urca process. According to these calculations, we obtained an initial parameter space for the production of SNe Ia in the $\rm log\,$$P^{\rm i}_{\rm orb}-M^{\rm i}_{\rm He}$ plane. Meanwhile, we found that isolated mildly recycled pulsars can be formed after He stars explode as SNe Ia in NS+He star binaries, in which the isolated pulsars have minimum spin periods ($P_{\rm spin}^{\rm min}$) of $\sim 30-110\rm\,ms$ and final orbital velocities of $\sim \rm 60-360\,km\,s^{-1}$, corresponding to initial orbital periods of $0.07-10\rm\,d$. Our work suggests that the NS+He star channel may contribute to the formation of isolated mildly recycled pulsars with velocity $\rm \lesssim 360\,km\,s^{-1}$ in observations, and such isolated pulsars should locate in the region of pulsars with massive WD companions in the $P_{\rm spin}-\dot P_{\rm spin}$ diagram.

astro-ph.HE↗
arXiv2023

Mind vs. Mouth: On Measuring Re-judge Inconsistency of Social Bias in Large Language Models

Recent researches indicate that Pre-trained Large Language Models (LLMs) possess cognitive constructs similar to those observed in humans, prompting researchers to investigate the cognitive aspects of LLMs. This paper focuses on explicit and implicit social bias, a distinctive two-level cognitive construct in psychology. It posits that individuals' explicit social bias, which is their conscious expression of bias in the statements, may differ from their implicit social bias, which represents their unconscious bias. We propose a two-stage approach and discover a parallel phenomenon in LLMs known as "re-judge inconsistency" in social bias. In the initial stage, the LLM is tasked with automatically completing statements, potentially incorporating implicit social bias. However, in the subsequent stage, the same LLM re-judges the biased statement generated by itself but contradicts it. We propose that this re-judge inconsistency can be similar to the inconsistency between human's unaware implicit social bias and their aware explicit social bias. Experimental investigations on ChatGPT and GPT-4 concerning common gender biases examined in psychology corroborate the highly stable nature of the re-judge inconsistency. This finding may suggest that diverse cognitive constructs emerge as LLMs' capabilities strengthen. Consequently, leveraging psychological theories can provide enhanced insights into the underlying mechanisms governing the expressions of explicit and implicit constructs in LLMs.

cs.CL↗