Searcharxiv⌕ Search

arXiv subjects

Jing Bai

Publications and source records attributed to Jing Bai.

At least 37 records · Page 2Linked to original sources
arXiv2024

BSM: Small but Powerful Biological Sequence Model for Genes and Proteins

Modeling biological sequences such as DNA, RNA, and proteins is crucial for understanding complex processes like gene regulation and protein synthesis. However, most current models either focus on a single type or treat multiple types of data separately, limiting their ability to capture cross-modal relationships. We propose that by learning the relationships between these modalities, the model can enhance its understanding of each type. To address this, we introduce BSM, a small but powerful mixed-modal biological sequence foundation model, trained on three types of data: RefSeq, Gene Related Sequences, and interleaved biological sequences from the web. These datasets capture the genetic flow, gene-protein relationships, and the natural co-occurrence of diverse biological data, respectively. By training on mixed-modal data, BSM significantly enhances learning efficiency and cross-modal representation, outperforming models trained solely on unimodal data. With only 110M parameters, BSM achieves performance comparable to much larger models across both single-modal and mixed-modal tasks, and uniquely demonstrates in-context learning capability for mixed-modal tasks, which is absent in existing models. Further scaling to 270M parameters demonstrates even greater performance gains, highlighting the potential of BSM as a significant advancement in multimodal biological sequence modeling.

q-bio.GN↗
arXiv2024

CPL: Critical Plan Step Learning Boosts LLM Generalization in Reasoning Tasks

Post-training, particularly reinforcement learning (RL) using self-play-generated data, has become a new learning paradigm for large language models (LLMs). However, scaling RL to develop a general reasoner remains a research challenge, as existing methods focus on task-specific reasoning without adequately addressing generalization across a broader range of tasks. Moreover, unlike traditional RL with limited action space, LLMs operate in an infinite space, making it crucial to search for valuable and diverse strategies to solve problems effectively. To address this, we propose searching within the action space on high-level abstract plans to enhance model generalization and introduce Critical Plan Step Learning (CPL), comprising: 1) searching on plan, using Monte Carlo Tree Search (MCTS) to explore diverse plan steps in multi-step reasoning tasks, and 2) learning critical plan steps through Step-level Advantage Preference Optimization (Step-APO), which integrates advantage estimates for step preference obtained via MCTS into Direct Preference Optimization (DPO). This combination helps the model effectively learn critical plan steps, enhancing both reasoning capabilities and generalization. Experimental results demonstrate that our method, trained exclusively on GSM8K and MATH, not only significantly improves performance on GSM8K (+10.5%) and MATH (+6.5%), but also enhances out-of-domain reasoning benchmarks, such as HumanEval (+12.2%), GPQA (+8.6%), ARC-C (+4.0%), MMLU-STEM (+2.2%), and BBH (+1.8%).

cs.AI↗
arXiv2024

NutePrune: Efficient Progressive Pruning with Numerous Teachers for Large Language Models

The considerable size of Large Language Models (LLMs) presents notable deployment challenges, particularly on resource-constrained hardware. Structured pruning, offers an effective means to compress LLMs, thereby reducing storage costs and enhancing inference speed for more efficient utilization. In this work, we study data-efficient and resource-efficient structure pruning methods to obtain smaller yet still powerful models. Knowledge Distillation is well-suited for pruning, as the intact model can serve as an excellent teacher for pruned students. However, it becomes challenging in the context of LLMs due to memory constraints. To address this, we propose an efficient progressive Numerous-teacher pruning method (NutePrune). NutePrune mitigates excessive memory costs by loading only one intact model and integrating it with various masks and LoRA modules, enabling it to seamlessly switch between teacher and student roles. This approach allows us to leverage numerous teachers with varying capacities to progressively guide the pruned model, enhancing overall performance. Extensive experiments across various tasks demonstrate the effectiveness of NutePrune. In LLaMA-7B zero-shot experiments, NutePrune retains 97.17% of the performance of the original model at 20% sparsity and 95.07% at 25% sparsity. Our code is available at https://github.com/Lucius-lsr/NutePrune.

cs.CL↗
arXiv2024

Probing the non-Abelian fusion of a pair of Majorana zero modes

In this work, we perform real time simulations for probing the non-Abelian fusion of a pair of Majorana zero modes (MZMs). The nontrivial fusion outcomes can be either a vacuum, or an unpaired fermion, which reflect the underlying non-Abelian statistics. The two possible outcomes can cause different charge variations in the nearby probing quantum dot (QD), while the charge occupation in the dot is detected by a quantum point contact. In particular, we find that gradual fusion and gradual coupling of the MZMs to the QD (in nearly adiabatic switching-on limit) provide a simpler detection scheme than sudden coupling after fusion to infer the coexistence of two fusion outcomes, by measuring the occupation probability of the QD. For the scheme of sudden coupling (after fusion), we propose and analyze continuous weak measurement for the quantum oscillations of the QD occupancy. From the power spectrum of the measurement currents, one can identify the characteristic frequencies and infer thus the coexistence of the fusion outcomes.

cond-mat.mes-hall↗
arXiv2023

Structural dimerization and charge-orbital ordering in a ferromagnetic semiconductor LiV2S4 monolayer

With the rise of two-dimensional (2D) materials, unique properties that are completely distinct from bulk counterparts continue to emerge at low-dimensional scales, presenting numerous opportunities and challenges. It also provides a new perspective for the study of transition metal system. Here, based on density functional theory (DFT), the physical properties of 2D monolayer LiV2S4 have been studied. Remarkable changes have been observed, i.e., vanadium dimerization, ferromagnetism, charge distribution and metal-insulator transition (MIT). It is argued that the electronic instability leads to the V dimerization which further lifts the degeneracy of charge distribution and stabilizes the charge and spin ordering state.

cond-mat.mtrl-sci↗
arXiv2023

Metasurface-based Mueller Matrix Microscope

In conventional optical microscopes, image contrast of objects mainly results from the differences in light intensity and/or color. Muller matrix optical microscopes (MMMs), on the other hand, can provide significantly enhanced image contrast and rich information about objects by analyzing their interactions with polarized light. However, state-of-art MMMs are fundamentally limited by bulky and slow polarization state generators and analyzers. Here, we demonstrated the feasibility of applying metasurfaces to enable a fast and compact MMM, i.e., Meta-MMM. We developed a dual-color MMM, in both reflection and transmission modes, based on a chip-integrated high-speed (>20fps) metasurface polarization state analyzer (Meta-PSA) and realized high measurement accuracy for Muller matrix (MM) imaging. We then applied our Meta-MMM to nanostructure characterization, surface morphology analysis and discovered birefringent structures in honeybee wings. Our meta-MMMs hold the promise to revolutionize various applications from biological imaging, medical diagnosis, material characterization to industry inspection and space exploration.

physics.optics↗
arXiv2023

Maximal exponent of the Lorentz cones

We show that the maximal exponent (i.e., the minimum number of iterations required for a primitive map to become strictly positive) of the n-dimensional Lorentz cone is equal to n. As a byproduct, we show that the optimal exponent in the quantum Wielandt inequality for qubit channels is equal to 3.

math.MG↗
arXiv2023

AdapterGNN: Parameter-Efficient Fine-Tuning Improves Generalization in GNNs

Fine-tuning pre-trained models has recently yielded remarkable performance gains in graph neural networks (GNNs). In addition to pre-training techniques, inspired by the latest work in the natural language fields, more recent work has shifted towards applying effective fine-tuning approaches, such as parameter-efficient fine-tuning (PEFT). However, given the substantial differences between GNNs and transformer-based models, applying such approaches directly to GNNs proved to be less effective. In this paper, we present a comprehensive comparison of PEFT techniques for GNNs and propose a novel PEFT method specifically designed for GNNs, called AdapterGNN. AdapterGNN preserves the knowledge of the large pre-trained model and leverages highly expressive adapters for GNNs, which can adapt to downstream tasks effectively with only a few parameters, while also improving the model's generalization ability. Extensive experiments show that AdapterGNN achieves higher performance than other PEFT methods and is the only one consistently surpassing full fine-tuning (outperforming it by 1.6% and 5.7% in the chemistry and biology domains respectively, with only 5% and 4% of its parameters tuned) with lower generalization gaps. Moreover, we empirically show that a larger GNN model can have a worse generalization ability, which differs from the trend observed in large transformer-based models. Building upon this, we provide a theoretical justification for PEFT can improve generalization of GNNs by applying generalization bounds. Our code is available at https://github.com/Lucius-lsr/AdapterGNN.

cs.LG↗
arXiv2023

Parameter-efficient is not sufficient: Exploring Parameter, Memory, and Time Efficient Adapter Tuning for Dense Predictions

Pre-training & fine-tuning is a prevalent paradigm in computer vision (CV). Recently, parameter-efficient transfer learning (PETL) methods have shown promising performance in adapting to downstream tasks with only a few trainable parameters. Despite their success, the existing PETL methods in CV can be computationally expensive and require large amounts of memory and time cost during training, which limits low-resource users from conducting research and applications on large models. In this work, we propose Parameter, Memory, and Time Efficient Visual Adapter ($\mathrm{E^3VA}$) tuning to address this issue. We provide a gradient backpropagation highway for low-rank adapters which eliminates the need for expensive backpropagation through the frozen pre-trained model, resulting in substantial savings of training memory and training time. Furthermore, we optimise the $\mathrm{E^3VA}$ structure for CV tasks to promote model performance. Extensive experiments on COCO, ADE20K, and Pascal VOC benchmarks show that $\mathrm{E^3VA}$ can save up to 62.2% training memory and 26.2% training time on average, while achieving comparable performance to full fine-tuning and better performance than most PETL methods. Note that we can even train the Swin-Large-based Cascade Mask RCNN on GTX 1080Ti GPUs with less than 1.5% trainable parameters.

cs.CV↗
arXiv2023

On Metaverse Application Dependability Analysis

Metaverse as-a-Service (MaaS) enables Metaverse tenants to execute their APPlications (MetaAPP) by allocating Metaverse resources in the form of Metaverse service functions (MSF). Usually, each MSF is deployed in a virtual machine (VM) for better resiliency and security. However, these MSFs along with VMs and virtual machine monitors (VMM) running them may encounter software aging after prolonged continuous operation. Then, there is a decrease in MetaAPP dependability, namely, the dependability of the MSF chain (MSFC), consisting of MSFs allocated to MetaAPP. This paper aims to investigate the impact of both software aging and rejuvenation techniques on MetaAPP dependability in the scenarios, where both active components (MSF, VM and VMM) and their backup components are subject to software aging. We develop a hierarchical model to capture behaviors of aging, failure, and recovery by applying Semi-Markov process and reliability block diagram. Numerical analysis and simulation experiments are conducted to evaluate the approximation accuracy of the proposed model and dependability metrics. We then identify the key parameters for improving the MetaAPP/MSFC dependability through sensitivity analysis. The investigation is also made about the influence of various parameters on MetaAPP/MSFC dependability.

cs.SE↗
arXiv2023

Towards Semi-Markov Model-based Dependability Evaluation of VM-based Multi-Domain Service Function Chain

In NFV networks, service functions (SFs) can be deployed on virtual machines (VMs) across multiple domains and then form a service function chain (MSFC) for end-to-end network service provision. However, any software component in a VM-based MSFC must experience software aging issue after a long period of operation. This paper quantitatively investigates the capability of proactive rejuvenation techniques in reducing the damage of software aging on a VM-based MSFC. We develop a semi-Markov model to capture the behaviors of SFs, VMs and virtual machine monitors (VMMs) from software aging to recovery under the condition that failure times and recovery times follow general distributions. We derive the formulas for calculating the steady-state availability and reliability of the VM-based MSFC composed of multiple SFs running on VMs hosted by VMMs. Sensitivity analysis is also conducted to identify potential dependability bottlenecks.

cs.SE↗
arXiv2023

Understanding Container-based Services under Software Aging: Dependability and Performance Views

Container technology, as the key enabler behind microservice architectures, is widely applied in Cloud and Edge Computing. A long and continuous running of operating system (OS) host-ing container-based services can encounter software aging that leads to performance deterioration and even causes system fail-ures. OS rejuvenation techniques can mitigate the impact of software aging but the rejuvenation trigger interval needs to be carefully determined to reduce the downtime cost due to rejuve-nation. This paper proposes a comprehensive semi-Markov-based approach to quantitatively evaluate the effect of OS reju-venation on the dependability and the performance of a con-tainer-based service. In contrast to the existing studies, we nei-ther restrict the distributions of time intervals of events to be exponential nor assume that backup resources are always avail-able. Through the numerical study, we show the optimal con-tainer-migration trigger intervals that can maximize the de-pendability or minimize the performance of a container-based service.

cs.SE↗
arXiv2023

Time-aware Graph Structure Learning via Sequence Prediction on Temporal Graphs

Temporal Graph Learning, which aims to model the time-evolving nature of graphs, has gained increasing attention and achieved remarkable performance recently. However, in reality, graph structures are often incomplete and noisy, which hinders temporal graph networks (TGNs) from learning informative representations. Graph contrastive learning uses data augmentation to generate plausible variations of existing data and learn robust representations. However, rule-based augmentation approaches may be suboptimal as they lack learnability and fail to leverage rich information from downstream tasks. To address these issues, we propose a Time-aware Graph Structure Learning (TGSL) approach via sequence prediction on temporal graphs, which learns better graph structures for downstream tasks through adding potential temporal edges. In particular, it predicts time-aware context embedding based on previously observed interactions and uses the Gumble-Top-K to select the closest candidate edges to this context embedding. Additionally, several candidate sampling strategies are proposed to ensure both efficiency and diversity. Furthermore, we jointly learn the graph structure and TGNs in an end-to-end manner and perform inference on the refined graph. Extensive experiments on temporal link prediction benchmarks demonstrate that TGSL yields significant gains for the popular TGNs such as TGAT and GraphMixer, and it outperforms other contrastive learning methods on temporal graphs. We release the code at https://github.com/ViktorAxelsen/TGSL.

cs.LG↗
arXiv2023

Binary Star Evolution in Different Environments: Filamentary, Fractal, Halo and Tidal-tail Clusters

Using membership of 85 open clusters from previous studies (Pang et al. 2021a,b, 2022b; Li et al. 2021) based on Gaia DR3 data, we identify binary candidates in the color-magnitude diagram, for systems with mass ratio q > 0.4. The binary fraction is corrected for incompleteness at different distances due to the Gaia angular resolution limit. We find a decreasing binary fraction with increasing cluster age, with substantial scatter. For clusters with a total mass > 200$M_\odot$, the binary fraction is independent of cluster mass. The binary fraction depends strongly on stellar density. Among four types of cluster environments, the lowest-density filamentary and fractal stellar groups have the highest mean binary fraction: 23.6% and 23.2%, respectively. The mean binary fraction in tidal-tail clusters is 20.8%, and is lowest in the densest halo-type clusters: 14.8%. We find clear evidence of early disruptions of binary stars in the cluster sample. The radial binary fraction depends strongly on the cluster-centric distance across all four types of environments, with the smallest binary fraction within the half-mass radius $r_h$, and increasing towards a few $r_h$. Only hints of mass segregation is found in the target clusters. The observed amount of mass segregation is not significant to generate a global effect inside the target clusters. We evaluate the bias of unresolved binary systems (assuming a primary mass of 1$M_\odot$) in 1D tangential velocity, which is 0.1-1$\,\rm km\,s^{-1}$. Further studies are required to characterize the internal star cluster kinematics using Gaia proper motions.

astro-ph.GA↗
arXiv2023

Ultra-fast Graphene-Plasmonic Hybrid Metasurface Saturable Absorber with Low Saturation Fluence

Exploring novel materials with enhanced optical nonlinearities at low power levels with ultrafast response and small footprints is of great interests for information processing, communication, sensing and quantum systems. Recent progress on nonlinear metamaterials and metasurfaces suggests promising solutions to overcome the limitations of nonlinear materials in nature. Here we present a design concept for highly enhanced saturable absorption effect based on subwavelength-thick (<1/5$λ$0) hybrid graphene-plasmonic metasurface structures in infrared wavelengths. Our theoretical and experimental results demonstrated that, by exciting nonequilibrium carriers inside nanoscale hotspots, one could not only enhance the saturable absorption in graphene, but also reduce the saturation fluence by over 3 orders of magnitude (from ~ 1 mJ/cm2 to ~100 nJ/cm2). Our pump-probe measurement results suggested an ultrashort saturable absorption recovery time (<60 fs), which is ultimately determined by the relaxation dynamics of photoexcited carriers in graphene. We also observed pulse narrowing effects in our devices based on the autocorrelation measurement results. Such design concepts can be tailored via structure engineering to operate in broader wavelength ranges up to mid- and far- infrared spectral regions. These ultrafast low-saturation fluence saturable absorber designs can enable low-threshold, compact, self-starting mode-locked lasers, laser pulse shaping, and high-speed optical information processing.

physics.optics↗
arXiv2023

Dynamics simulation of braiding two Majorana-zero-modes via a quantum dot

In this work we perform real time simulations for the dynamics of braiding a pair of Majorana zero modes (MZMs) through a quantum dot in a minimal setup of pure 1D realization. We reveal the strong nonadiabatic effect when the dot energy level approaches to zero in order to achieve a geometric phase $π/4$ which is required for a full exchange between the MZMs. Rather than the strategies of nonuniformly manipulating the system according to adiabatic condition and shortcuts-to-adiabaticity, we propose and illustrate a more feasible scheme to suppress the nonadiabatic transition, meanwhile which allows for a full exchange between the Majorana modes.

cond-mat.mes-hall↗
arXiv2023

Chip-Integrated Metasurface Full-Stokes Polarimetric Imaging Sensor

Polarimetric imaging has a wide range of applications for uncovering features invisible to human eyes and conventional imaging sensors. Compact, fast, cost-effective and accurate full-Stokes polarimetric imaging sensors are highly desirable in many applications, which, however, remain elusive due to fundamental material limitations. Here we present a Metasurface-based Full-Stokes Polarimetric Imaging sensor (MetaPolarIm) realized by integrating an ultrathin (~600 nm) metasurface polarization filter array (MPFA) onto a visible imaging sensor with CMOS compatible fabrication processes. The MPFA is featured with broadband dielectric-metal hybrid chiral metasurfaces and double-layer nanograting polarizers. This chip-integrated polarimetric imaging sensor enables single-shot full-Stokes imaging (speed limited by the CMOS imager) with the most compact form factor, record high measurement accuracy, dual-color operation (green and red) and a full angle of view up to 40 degrees. MetaPolarIm holds great promise to enable transformative applications in autonomous vision, industry inspection, space exploration, medical imaging and diagnosis.

physics.optics↗
arXiv2022

Creating Training Sets via Weak Indirect Supervision

Creating labeled training sets has become one of the major roadblocks in machine learning. To address this, recent \emph{Weak Supervision (WS)} frameworks synthesize training labels from multiple potentially noisy supervision sources. However, existing frameworks are restricted to supervision sources that share the same output space as the target task. To extend the scope of usable sources, we formulate Weak Indirect Supervision (WIS), a new research problem for automatically synthesizing training labels based on indirect supervision sources that have different output label spaces. To overcome the challenge of mismatched output spaces, we develop a probabilistic modeling approach, PLRM, which uses user-provided label relations to model and leverage indirect supervision sources. Moreover, we provide a theoretically-principled test of the distinguishability of PLRM for unseen labels, along with a generalization bound. On both image and text classification tasks as well as an industrial advertising application, we demonstrate the advantages of PLRM by outperforming baselines by a margin of 2%-9%.

cs.LG↗