arXiv subjects

Soham Ghosh

Publications and source records attributed to Soham Ghosh.

At least 37 records · Page 2Linked to original sources

arXiv2025

Bayesian ICA with super-Gaussian Source Priors

Independent Component Analysis (ICA) plays a central role in modern machine learning as a flexible framework for feature extraction. We introduce a horseshoe-type prior with a latent Polya-Gamma scale mixture representation, yielding scalable algorithms for both point estimation via expectation-maximization (EM) and full posterior inference via Markov chain Monte Carlo (MCMC). This hierarchical formulation unifies several previously disparate estimation strategies within a single Bayesian framework. We also establish the first theoretical guarantees for hierarchical Bayesian ICA, including posterior contraction and local asymptotic normality results for the unmixing matrix. Comprehensive simulation studies demonstrate that our methods perform competitively with widely used ICA tools. We further discuss implementation of conditional posteriors, envelope-based optimization, and possible extensions to flow-based architectures for nonlinear feature extraction and deep learning. Finally, we outline several promising directions for future work.

stat.ME↗

arXiv2025

The Curious Problem of the Normal Inverse Mean: Robustness and Shrinkage

In astronomical observations, the estimation of distances from parallaxes is a challenging task due to the inherent measurement errors and the non-linear relationship between the parallax and the distance. This study leverages ideas from robust Bayesian inference to tackle these challenges, investigating a broad class of prior densities for estimating distances with a reduced bias and variance. Through theoretical analysis, simulation experiments, and the application to data from the Gaia Data Release 1 (GDR1), we demonstrate that heavy-tailed priors provide more reliable distance estimates, particularly in the presence of large fractional parallax errors. Theoretical results highlight the "curse of a single observation," where the likelihood dominates the posterior, limiting the impact of the prior. Nevertheless, heavy-tailed priors can delay the explosion of posterior risk, offering a more robust framework for distance estimation. The findings suggest that reciprocal invariant priors, with polynomial decay in their tails, such as the Half-Cauchy and Product Half-Cauchy, are particularly well-suited for this task, providing a balance between bias reduction and variance control.

stat.ME↗

arXiv2025

High-dimensional regression with outcomes of mixed-type using the multivariate spike-and-slab LASSO

We consider a high-dimensional multi-outcome regression in which $q,$ possibly dependent, binary and continuous outcomes are regressed onto $p$ covariates. We model the observed outcome vector as a partially observed latent realization from a multivariate linear regression model. Our goal is to estimate simultaneously a sparse matrix ($B$) of latent regression coefficients (i.e., partial covariate effects) and a sparse latent residual precision matrix ($Ω$), which induces partial correlations between the observed outcomes. To this end, we specify continuous spike-and-slab priors on all entries of $B$ and off-diagonal elements of $Ω$ and introduce a Monte Carlo Expectation-Conditional Maximization algorithm to compute the maximum a posterior estimate of the model parameters. Under a set of mild assumptions, we derive the posterior contraction rate for our model in the high-dimensional regimes where both $p$ and $q$ diverge with the sample size $n$ and establish a sure screening property, which implies that, as $n$ increases, we can recover all truly non-zero elements of $B$ with probability tending to one. We demonstrate the excellent finite-sample properties of our proposed method, which we call mixed-mSSL, using extensive simulation studies and three applications spanning medicine to ecology.

stat.ME↗

arXiv2025

Entanglement-assisted authenticated BB84 protocol

In this work, we present a novel authenticated Quantum Key Distribution (QKD) protocol employing maximally entangled qubit pairs. In the absence of noise, we securely authenticate the well-known BB84 QKD scheme under two assumptions: first, adversaries cannot simultaneously access preshared and non-pre-shared secret classical information, and second, adversaries cannot simultaneously access pre-shared secret classical information and quantum memories held by legitimate parties. The main strength of this noiseless result is that access to all secretly pre-shared classical information is insufficient for breaching our scheme. Additionally, our protocol desirably allows for pre-shared secrecy reusage, leading to secret key growing. In order to address noise, we simulate a photonic implementation of our scheme, together with a storage model that aims to replicate the performance of cavity-enhanced Atomic- Frequency Comb (AFC) memories. Two methods are used to distinguish authentic entities from forgery attempts: on the one hand, a statistical approach is used after calibration of its defining parameter $μ$. Alternatively, a Deep Neural Network (DNN) is designed and trained to learn the underlying different structure of that input data coming from adversaries in comparison to that one coming from legitimate parties. Both methods achieve a correct classification rate larger than 0.80 for memory storage time of 150 $μ$s and a 1 km distance between parties.

quant-ph↗

arXiv2025

Fitting sparse high-dimensional varying-coefficient models with Bayesian regression tree ensembles

By allowing the effects of $p$ covariates in a linear regression model to vary as functions of $R$ additional effect modifiers, varying-coefficient models (VCMs) strike a compelling balance between interpretable-but-rigid parametric models popular in classical statistics and flexible-but-opaque methods popular in machine learning. But in high-dimensional settings where $p$ and/or $R$ exceed the number of observations, existing approaches to fitting VCMs fail to identify which covariates have a non-zero effect and which effect modifiers drive these effects. We propose sparseVCBART, a fully Bayesian model that approximates each coefficient function in a VCM with a regression tree ensemble and encourages sparsity with a global--local shrinkage prior on the regression tree leaf outputs and a hierarchical prior on the splitting probabilities of each tree. We show that the sparseVCBART posterior contracts at a near-minimax optimal rate, automatically adapting to the unknown sparsity structure and smoothness of the true coefficient functions. Compared to existing state-of-the-art methods, sparseVCBART achieved competitive predictive accuracy and substantially narrower and better-calibrated uncertainty intervals, especially for null covariate effects. We use sparseVCBART to investigate how the effects of interpersonal conversations on prejudice could vary according to the political and demographic characteristics of the respondents.

stat.ME↗

arXiv2025

Devstral: Fine-tuning Language Models for Coding Agent Applications

We introduce Devstral-Small, a lightweight open source model for code agents with the best performance among models below 100B size. In this technical report, we give an overview of how we design and develop a model and craft specializations in agentic software development. The resulting model, Devstral-Small is a small 24B model, fast and easy to serve. Despite its size, Devstral-Small still attains competitive performance compared to models more than an order of magnitude larger.

cs.SE↗

arXiv2025

Scalable piecewise smoothing with BART

Although it is an extremely effective, easy-to-use, and increasingly popular tool for nonparametric regression, the Bayesian Additive Regression Trees (BART) model is limited by the fact that it can only produce discontinuous output. Initial attempts to overcome this limitation were based on regression trees that output Gaussian Processes instead of constants. Unfortunately, implementations of these extensions cannot scale to large datasets. We propose ridgeBART, an extension of BART built with trees that output linear combinations of ridge functions (i.e., a composition of an affine transformation of the inputs and non-linearity); that is, we build a Bayesian ensemble of localized neural networks with a single hidden layer. We develop a new MCMC sampler that updates trees in linear time and establish posterior contraction rates for estimating piecewise anisotropic Hölder functions and nearly minimax-optimal rates for estimating isotropic Hölder functions. We demonstrate ridgeBART's effectiveness on synthetic data and use it to estimate the probability that a professional basketball player makes a shot from any location on the court in a spatially smooth fashion.

stat.ME↗

arXiv2025

Voxtral

We present Voxtral Mini and Voxtral Small, two multimodal audio chat models. Voxtral is trained to comprehend both spoken audio and text documents, achieving state-of-the-art performance across a diverse range of audio benchmarks, while preserving strong text capabilities. Voxtral Small outperforms a number of closed-source models, while being small enough to run locally. A 32K context window enables the model to handle audio files up to 40 minutes in duration and long multi-turn conversations. We also contribute three benchmarks for evaluating speech understanding models on knowledge and trivia. Both Voxtral models are released under Apache 2.0 license.

cs.SD↗

arXiv2025

Magistral

We introduce Magistral, Mistral's first reasoning model and our own scalable reinforcement learning (RL) pipeline. Instead of relying on existing implementations and RL traces distilled from prior models, we follow a ground up approach, relying solely on our own models and infrastructure. Notably, we demonstrate a stack that enabled us to explore the limits of pure RL training of LLMs, present a simple method to force the reasoning language of the model, and show that RL on text data alone maintains most of the initial checkpoint's capabilities. We find that RL on text maintains or improves multimodal understanding, instruction following and function calling. We present Magistral Medium, trained for reasoning on top of Mistral Medium 3 with RL alone, and we open-source Magistral Small (Apache 2.0) which further includes cold-start data from Magistral Medium.

cs.CL↗

arXiv2025

Auto-Regressive Standard Precipitation Index: A Bayesian Approach for Drought Characterization

This study proposes Auto-Regressive Standardized Precipitation Index (ARSPI) as a novel alternative to the traditional Standardized Precipitation Index (SPI) for measuring drought by relaxing the assumption of independent and identical rainfall distribution over time. ARSPI utilizes an auto-regressive framework to tackle the auto-correlated characteristics of precipitation, providing a more precise depiction of drought dynamics. The proposed model integrates a spike-and-slab log-normal distribution for zero rainfall seasons. The Bayesian Monte Carlo Markov Chain (MCMC) approach simplifies the SPI computation using the non-parametric predictive density estimation of total rainfall across various time windows from simulated samples. The MCMC simulations further ensure robust estimation of severity, duration, peak and return period with greater precision. This study also provides a comparison between the performances of ARSPI and SPI using the precipitation data from the Colorado River Basin (1893-1991). ARSPI emerges to be more efficient than the benchmark SPI in terms of model fit. ARSPI shows enhanced sensitivity to climatic extremes, making it a valuable tool for hydrological research and water resource management.

stat.AP↗

arXiv2025

Impact of rainfall risk on rice production: realized volatility in mean model

Rural economies are largely dependent upon agriculture, which is greatly determined by climatic conditions such as rainfall. This study aims to forecast agricultural production in Maharashtra, India, which utilises annual data from the year 1962 to 2021. Since rainfall plays a major role with respect to the crop yield, we analyze the impact of rainfall on crop yield using four time series models that includes ARIMA, ARIMAX, GARCH-ARIMA and GARCH-ARIMAX. We take advantage of rainfall as an external regressor to examine if it contributes to the performance of the model. 1-step, 2-step, and 3-step ahead forecasts are obtained and the model performance is assessed using MAE and RMSE. The models are able to more accurately predict when using rainfall as a predictor compared to when solely dependant on historical production trends (more improved outcomes are seen in the ARIMAX and GARCH-ARIMAX models). As such, these findings underscore the need for climate-aware forecasting techniques that provide useful information to policymakers and farmers to aid in agricultural planning.

stat.AP↗

arXiv2025

Tropical Poincaré bundle, Fourier-Mukai transform, and a generalized Poincaré formula

We construct a tropical analogue of the Poincaré bundle and prove a (cohomological) Fourier-Mukai transform for real tori with integral structures. We then prove a tropical analogue of Beauville's generalized Poincaré formula for polarized abelian varieties. Some consequences include a geometric Riemann-Roch theorem for tropical abelian varieties, as well as a tropical Poincaré-Prym formula which was recently conjectured by Röhrle and Zakharov.

math.AG↗

arXiv2025

TIPS: Text-Image Pretraining with Spatial awareness

While image-text representation learning has become very popular in recent years, existing models tend to lack spatial awareness and have limited direct applicability for dense understanding tasks. For this reason, self-supervised image-only pretraining is still the go-to method for many dense vision applications (e.g. depth estimation, semantic segmentation), despite the lack of explicit supervisory signals. In this paper, we close this gap between image-text and self-supervised learning, by proposing a novel general-purpose image-text model, which can be effectively used off the shelf for dense and global vision tasks. Our method, which we refer to as Text-Image Pretraining with Spatial awareness (TIPS), leverages two simple and effective insights. First, on textual supervision: we reveal that replacing noisy web image captions by synthetically generated textual descriptions boosts dense understanding performance significantly, due to a much richer signal for learning spatially aware representations. We propose an adapted training method that combines noisy and synthetic captions, resulting in improvements across both dense and global understanding tasks. Second, on the learning technique: we propose to combine contrastive image-text learning with self-supervised masked image modeling, to encourage spatial coherence, unlocking substantial enhancements for downstream applications. Building on these two ideas, we scale our model using the transformer architecture, trained on a curated set of public images. Our experiments are conducted on 8 tasks involving 16 datasets in total, demonstrating strong off-the-shelf performance on both dense and global understanding, for several image-only and image-text tasks. Code and models are released at https://github.com/google-deepmind/tips.

cs.CV↗

arXiv2025

A finiteness result towards the Casas-Alvero Conjecture

The Casas-Alvero conjecture predicts that every univariate polynomial over an algebraically closed field of characteristic zero sharing a common factor with each of its Hasse-Schmidt derivatives is a power of a linear polynomial. The conjecture for polynomials of a fixed degree is equivalent to the projective variety of such polynomials being one-dimensional. In this paper, we show that for any algebraically closed field of arbitrary characteristic, this variety is at most two-dimensional for all positive degrees. Consequently, we show that the associated arithmetic Casas-Alvero scheme in any positive degree has finitely many rational points over any field. Along the way, we prove several rigidity results towards the conjecture. We also introduce intermediate arithmetic Casas-Alvero schemes and show that their $\mathbb{K}$ points form an almost complete intersection over any algebraically closed field $\mathbb{K}$. Furthermore, we consider the question of when they form a complete intersection.

math.AG↗

arXiv2024

Existential Unforgeability in Quantum Authentication From Quantum Physical Unclonable Functions Based on Random von Neumann Measurement

Physical Unclonable Functions (PUFs) leverage inherent, non-clonable physical randomness to generate unique input-output pairs, serving as secure fingerprints for cryptographic protocols like authentication. Quantum PUFs (QPUFs) extend this concept by using quantum states as input-output pairs, offering advantages over classical PUFs, such as challenge reusability via public channels and eliminating the need for trusted parties due to the no-cloning theorem. Recent work introduced a generalized mathematical framework for QPUFs. It was shown that random unitary QPUFs cannot achieve existential unforgeability against Quantum Polynomial Time (QPT) adversaries. Security was possible only with additional uniform randomness. To avoid the cost of external randomness, we propose a novel measurement-based scheme. Here, the randomness naturally arises from quantum measurements. Additionally, we introduce a second model where the QPUF functions as a nonunitary quantum channel, which guarantees existential unforgeability. These are the first models in the literature to demonstrate a high level of provable security. Finally, we show that the Quantum Phase Estimation (QPE) protocol, applied to a Haar random unitary, serves as an approximate implementation of the second type of QPUF by approximating a von Neumann measurement on the unitary's eigenbasis.

quant-ph↗

arXiv2024

Entanglement-assisted Quantum Error Correcting Code Saturating The Classical Singleton Bound

We introduce a construction for entanglement-assisted quantum error-correcting codes (EAQECCs) that saturates the classical Singleton bound with less shared entanglement than any known method for code rates below $ \frac{k}{n} = \frac{1}{3} $. For higher rates, our EAQECC also meets the Singleton bound, although with increased entanglement requirements. Additionally, we demonstrate that any classical $[n,k,d]_q$ code can be transformed into an EAQECC with parameters $[[n,k,d;2k]]_q$ using $2k$ pre-shared maximally entangled pairs. The complexity of our encoding protocol for $k$-qudits with $q$ levels is $\mathcal{O}(k \log_{\frac{q}{q-1}}(k))$, excluding the complexity of encoding and decoding the classical MDS code. While this complexity remains linear in $k$ for systems of reasonable size, it increases significantly for larger-levelled systems, highlighting the need for further research into complexity reduction.

quant-ph↗

arXiv2024

Pixtral 12B

We introduce Pixtral-12B, a 12--billion-parameter multimodal language model. Pixtral-12B is trained to understand both natural images and documents, achieving leading performance on various multimodal benchmarks, surpassing a number of larger models. Unlike many open-source models, Pixtral is also a cutting-edge text model for its size, and does not compromise on natural language performance to excel in multimodal tasks. Pixtral uses a new vision encoder trained from scratch, which allows it to ingest images at their natural resolution and aspect ratio. This gives users flexibility on the number of tokens used to process an image. Pixtral is also able to process any number of images in its long context window of 128K tokens. Pixtral 12B substanially outperforms other open models of similar sizes (Llama-3.2 11B \& Qwen-2-VL 7B). It also outperforms much larger open models like Llama-3.2 90B while being 7x smaller. We further contribute an open-source benchmark, MM-MT-Bench, for evaluating vision-language models in practical scenarios, and provide detailed analysis and code for standardized evaluation protocols for multimodal LLMs. Pixtral-12B is released under Apache 2.0 license.

cs.CV↗

arXiv2024

ConDense: Consistent 2D/3D Pre-training for Dense and Sparse Features from Multi-View Images

To advance the state of the art in the creation of 3D foundation models, this paper introduces the ConDense framework for 3D pre-training utilizing existing pre-trained 2D networks and large-scale multi-view datasets. We propose a novel 2D-3D joint training scheme to extract co-embedded 2D and 3D features in an end-to-end pipeline, where 2D-3D feature consistency is enforced through a volume rendering NeRF-like ray marching process. Using dense per pixel features we are able to 1) directly distill the learned priors from 2D models to 3D models and create useful 3D backbones, 2) extract more consistent and less noisy 2D features, 3) formulate a consistent embedding space where 2D, 3D, and other modalities of data (e.g., natural language prompts) can be jointly queried. Furthermore, besides dense features, ConDense can be trained to extract sparse features (e.g., key points), also with 2D-3D consistency -- condensing 3D NeRF representations into compact sets of decorated key points. We demonstrate that our pre-trained model provides good initialization for various 3D tasks including 3D classification and segmentation, outperforming other 3D pre-training methods by a significant margin. It also enables, by exploiting our sparse features, additional useful downstream tasks, such as matching 2D images to 3D scenes, detecting duplicate 3D scenes, and querying a repository of 3D scenes through natural language -- all quite efficiently and without any per-scene fine-tuning.

cs.CV↗