Akarsh Gupta | Searcharxiv

arXiv2026

Stacking-Based Weighting for Large Language Bayes in M-Open Settings

Large Language Bayes (LLB) transforms a natural language problem description into a collection of candidate probabilistic programs and performs inference within each model, combining their predictions using Bayesian model averaging. While this provides a flexible framework for probabilistic inference without manual model specification, the resulting candidate model set is often misspecified in practice, and likelihood-based aggregation can concentrate on a single model, under-representing uncertainty across the sampled models. We propose a stacking-based aggregation method for LLB that selects model weights by optimizing predictive performance rather than marginal likelihood, while preserving the original language-model-driven model generation and within-model inference pipeline. Because leave-one-out predictive quantities are intractable in latent-variable models, we derive a variational lower bound that yields a tractable ELBO-based stacking objective. Across real-world probabilistic inference tasks, our method achieves comparable predictive performance to Bayesian model averaging while yielding more dispersed model weights, reducing concentration on a single candidate model under misspecification.

cs.LG↗

arXiv2026

SCOPE: Siamese Contrastive Operon Pair Embeddings for Functional Sequence Representation and Classification

Identifying operons is a fundamental step in understanding prokaryotic gene regulation, as classifying genes into operons supports the reconstruction of regulatory networks, functional annotation of unannotated genes, and drug candidate development. Experimental approaches such as RT-PCR and RNA-seq provide precise evidence of operon structure, but are laborious and largely limited to well-studied model organisms, making scalable computational methods essential for genome-wide operon identification. Prior computational approaches have employed traditional classifiers such as logistic regression and decision trees, motivating our use of these as physicochemical baselines. The DGEB benchmark evaluates operonic pair classification by embedding each sequence independently with a pre-trained protein language model and computing pairwise cosine similarity. In contrast, our Siamese MLP learns a classifier over the fused embedding space, which is theoretically better motivated for binary classification, as cosine similarity can yield meaningless scores depending on the regularization of the embedding model. While protein language model embeddings substantially outperform physicochemical features in ROC-AUC, a learned Siamese MLP head does not significantly improve over unsupervised cosine similarity in Average Precision, suggesting that the geometry of the embedding space already captures the functional relationships needed for this task. Nonetheless, our Siamese MLP achieves a ROC-AUC of 0.71, competitive with state-of-the-art models on the DGEB leaderboard. These findings indicate that protein language model embeddings are a viable, scalable foundation for operonic pair classification across diverse microbial genomes, with implications for automated genome annotation, regulatory network reconstruction, and characterization of organisms lacking experimental operon annotations.

q-bio.GN↗