arXiv subjects

Masahiro Tanaka

Publications and source records attributed to Masahiro Tanaka.

At least 19 recordsLinked to original sources

arXiv2026

Generalized Posterior Calibration via Infinitesimal-Jackknife Variance Matching

Generalized posteriors update a prior through an exponential transformation of empirical loss rather than a conventional likelihood, with a learning rate controlling posterior concentration. Their credible intervals may have incorrect frequentist coverage because posterior dispersion need not reflect sampling variability. We propose infinitesimal-jackknife variance matching (IJVM), a computationally efficient method for selecting the learning rate for a prespecified scalar functional. IJVM matches the functional's posterior variance to an infinitesimal-jackknife estimate of the sampling variance of its posterior mean, computed from posterior covariances with observation-level losses. A multiplicative one-step update requires one pilot posterior fit and a second fit for inference, avoiding explicit estimation of loss Hessians or gradient covariance matrices and repeated bootstrap posterior fitting. Under suitable regularity and Monte Carlo accuracy conditions, we establish consistency of the selected rate for the functional-specific first-order calibration target. With additional sampling and posterior-quantile assumptions, the resulting credible intervals achieve asymptotically nominal frequentist coverage. Simulations involving homoscedastic and heteroscedastic regression and hinge-loss classification show that one-step IJVM achieves coverage near the nominal 95% level, with interval lengths comparable to bootstrap-based generalized posterior calibration but substantially fewer posterior fits. Additional updates offer little improvement.

stat.ME↗

arXiv2026

Efficient Bootstrap Calibration of Generalized Posteriors with Importance Reweighting

The learning rate of a generalized posterior governs posterior uncertainty but typically has no canonical value. Generalized posterior calibration (GPC) selects this rate by matching bootstrap-estimated credible-interval coverage to its nominal level, often requiring repeated posterior simulation. We propose GPC via importance reweighting (GPC-IRW), which reuses draws from a reference posterior fitted to the original data across bootstrap resamples and candidate learning rates. For quantile-based intervals of a scalar functional, we express the coverage criterion in terms of bootstrap posterior cumulative distribution functions, avoiding computation of individual bootstrap credible intervals. Safeguards based on importance-weight effective sample size and proximity to coverage-decision boundaries invoke reweighting from cached bootstrap-specific draws or direct posterior simulation when necessary. Under the stated regularity conditions, with numerical error tending to zero, we show that median-centered calibration preserves the first-order learning-rate target and achieves asymptotically nominal coverage. The approach extends to independent clusters through cluster-level resampling. Simulations yield coverage and interval lengths comparable to those of existing GPC algorithms at substantially lower computational cost. In an exoplanet mass-radius analysis, GPC-IRW achieves 6.7- and 9.6-fold speedups over MCMC-based calibration in the single-planet and full-sample analyses, respectively, while illustrating the effects of within-host dependence.

stat.ME↗

arXiv2026

Learning Bidirectional Causal Interactions with Heteroscedastic Neural Networks

Estimating contemporaneous bidirectional interactions from observational data is difficult because each outcome is endogenous to the other, while flexible regressions may capture only reduced-form dependence. This paper proposes SEM-DNN, a heteroscedastic neural simultaneous-equation estimator that learns reciprocal structural interactions without external instruments. Identification exploits conditional covariance diagonalization: when structural shocks have zero conditional means, are conditionally uncorrelated given predetermined covariates, and exhibit nonproportional conditional variances, only the true interaction coefficients diagonalize the conditional residual covariance across the feature space. The method jointly approximates nonlinear structural mean functions and feature-dependent variances using a diagonal Gaussian quasi-likelihood that incorporates the simultaneous-system Jacobian. We establish unique identification and positive-definite local curvature of the profiled population criterion and show that, under neural-profile compatibility conditions, the implemented neural criterion inherits this curvature despite nonunique network parameterizations. The coefficients admit a causal interpretation when the structural equations represent autonomous mechanisms that remain invariant under the relevant interventions. Monte Carlo experiments with nonlinear, high-dimensional nuisance functions and non-Gaussian shocks show that SEM-DNN recovers structural effects more reliably than parametric, kernel-based, and separate-equation neural alternatives as information increases, although at greater computational cost. An application to ready-to-eat cereal scanner data illustrates how the method can study contemporaneous price-sales feedback and assess identification strength, residual diagonalization, variance calibration, and optimization sensitivity.

stat.ML↗

arXiv2026

ZenFlow: Enabling Stall-Free Offloading for LLM Training

Fine-tuning large language models (LLMs) often exceeds GPU memory limits, prompting systems to offload model states to CPU memory. However, existing offloaded training frameworks like ZeRO-Offload treat all parameters equally and update the full model on the CPU, causing severe GPU stalls, where fast, expensive GPUs sit idle, waiting for slow CPU updates and limited-bandwidth PCIe transfers. We present ZenFlow, a new offloaded training framework that decouples updates between GPU and CPU by prioritizing important gradients. ZenFlow performs in-place updates of important gradients on GPU while offloading and accumulating less important ones on CPU without blocking training iterations, fully overlapping CPU work with GPU computation. To make this importance-aware design scalable, ZenFlow leverages a novel spatio-temporal locality of important gradients, concentrated in specific positions that remain stable across iterations, to make important gradient identification efficient and lightweight while preserving fidelity. Evaluation shows that ZenFlow achieves up to 5x end-to-end speedup, 2x lower PCIe traffic, 85% less GPU stalls, and up to 72% lower cloud GPU cost, all while preserving accuracy.

cs.DC↗

arXiv2026

PEEK: Predictive Queue-Informed KV Cache Management for LLM Serving

We present PEEK, a lightweight scheduling and eviction framework for both online (streaming) and offline (batch) LLM serving; this paper focuses on the online regime. PEEK maintains an incremental radix tree over the pending queue, exposing prefix-sharing clusters no existing engine surfaces. A low-overhead dual-walk matches the tree against the engine's prefix cache to yield longest-prefix-match for every waiting request; PEEK then admits cluster pioneers first so siblings inherit the freshly cached prefix, a co-designed eviction hook protects blocks ancestral to queued demand, and a multi-lane stride scheduler bounds starvation. On SGLang and vLLM across five workloads up to 4$\times$H100 (DP=2 over TP=2), PEEK delivers up to 3.0$\times$/2.6$\times$ cache hit, 7.9$\times$/7.1$\times$ TTFT, 6.7$\times$/5.5$\times$ E2E, and 3.6$\times$/4.5$\times$ throughput gains over each engine's strongest stock baseline (SGLang/vLLM), while matching baselines within noise on workloads with no exploitable prefix structure. Wins hold as KV-cache pressure and inference parallelism scale.

cs.DC↗

arXiv2026

CIERA: Cross-Iteration Exponent Reuse for Lossless Allgather in Sharded MoE Training

In training Mixture-of-Experts (MoE) models, sharded data parallelism partitions each expert's parameters across GPUs, requiring an Allgather operation to reconstruct the full weight matrix before each layer executes. This communication often dominates iteration time. Prior work often reduces this overhead using lossy compression methods that sacrifice numerical fidelity, while existing lossless methods do not exploit cross-iteration exponent stability. In this paper, we propose Cross-Iteration Exponent Reuse Allgather (CIERA), a lossless, system-aware communication method for sharded MoE training. We observe that after a brief warmup phase, the exponent values of most weights remain unchanged across iterations. Based on this, we cache exponents locally and transmit only the sign and mantissa when exponents are unchanged. The receiver reconstructs the original weights exactly by combining the cached exponents with the received data. Since parameter matrices vary in shape across layers, compression is applied only when it yields net time savings. Moreover, the compression operation is overlapped with both Allgather communication and computation. Our real experiments and large-scale trace-driven simulator show that on OLMoE-1B-7B at 16 GPUs, CIERA achieves a 3.70x speedup over the lossless baseline and 3.68x over the lossy baseline, projected to reach 4.28x and 4.42x respectively at 128 GPUs, while preserving bitwise-exact parameter reconstruction in all evaluated runs.

cs.DC↗

arXiv2026

Curvature-Calibrated Quasi-Bayesian Updating for Moment-Restricted Models

Moment restrictions provide a flexible basis for quasi-Bayesian inference when a full likelihood is unavailable, but the weighting matrix in a quadratic moment criterion determines both the relative importance of the moments and the information scale of posterior updating. We propose curvature-calibrated quasi-Bayesian updating, which uses the inverse of the covariance (or long-run covariance) of the moment conditions evaluated at a self-consistent quasi-posterior center. The resulting fixed-point procedure alternates between covariance estimation and simulation from a fixed-weight quasi-posterior, thereby avoiding parameter-dependent weighting during each simulation run. Under a Bernstein-von Mises condition for the fixed-weight quasi-posterior at the efficient population weight, we show that the calibration map is locally contractive, that its fixed point is consistent at the standard parametric rate, and that the Gaussian approximation continues to hold under the calibrated data-dependent weight, with covariance given by the inverse Godambe information matrix. Under a uniform fourth-moment condition, the scaled quasi-posterior covariance converges to the same matrix, so quasi-posterior and repeated-sampling uncertainty agree to first order. Simulations show improved covariance calibration and interval coverage after a few updates. An application to longitudinal binary-response data illustrates the method with within-subject dependence and overidentified residual moments.

stat.ME↗

arXiv2026

Shirakami: A Hybrid Concurrency Control Protocol for Tsurugi Relational Database System

Bill-of-materials and telecommunications billing applications need to process both short and long read-write transactions simultaneously. Recent work rarely addresses such evolving workloads. To deal with these workloads, we propose a new concurrency control protocol, Shirakami. Shirakami is a hybrid protocol. The first protocol, Shirakami-LTX, is for long read-write transactions based on multiversion view serializability. The second protocol, Shirakami-OCC, is for short transactions based on Silo. Shirakami naturally integrates them with write-preservation and epoch-based synchronization. It does not require dynamic protocol switching and provides stable performance. We implemented Shirakami as the transaction processing module of the Tsurugi system, which is a production-grade relational database system. The experimental results demonstrated that Tsurugi was at least 7.9x faster than PostgreSQL, and Shirakami-LTX exhibited 680x higher throughput than Shirakami-OCC.

cs.DB↗

arXiv2026

Quantization Inflates Reasoning: Token Inflation as a Hidden Cost of Low-Bit Reasoning Models

Quantization is widely used to reduce the inference cost of large language models, but its effect on reasoning models is not fully captured by final-answer accuracy or per-token latency. We show that low-bit post-training quantization can introduce a hidden test-time compute cost: quantized reasoning models often generate longer chains of thought even when they still answer correctly. Across mathematical reasoning, code generation, scientific question answering, and agentic tool-use benchmarks, we find that INT4/INT3 quantization can preserve accuracy but increase reasoning-token usage, offsetting the expected per-token speedup. To measure this effect, we introduce the CoT Token Inflation Ratio, which compares reasoning length between quantized and full-precision models averaged across all evaluation benchmarks. We further show that token inflation is accompanied by behavioral changes in the reasoning trace, including more intermediate steps and greater semantic repetition. These changes translate into measurable end-to-end real-world serving penalties. Finally, we evaluate mitigation strategies and find that prompting and decoding-time sampling offer inconsistent accuracy-length trade-offs, while quantization-aware training shows more promise in reducing both accuracy degradation and token inflation. Our results suggest that reasoning-token usage should be reported alongside accuracy when evaluating quantized reasoning models.

cs.AI↗

arXiv2026

A Theory of Bootstrap Coverage Calibration for Generalized Posterior Credible Sets

Generalized posteriors replace the likelihood by an exponentiated empirical criterion, but their credible sets generally lack asymptotic justification for frequentist coverage. General posterior calibration selects a scalar learning rate by estimating coverage with the bootstrap. Using Edgeworth expansions under regular fixed-dimensional asymptotics, we derive higher-order coverage expansions and analyze the stochastic approximation step used in the implemented algorithm. For a fixed nominal level, the root of the bootstrap coverage equation is consistent under a uniform coverage approximation and local identification. The higher-order expansions separate two sources of coverage error: the sampling Edgeworth correction for the estimator and the posterior Edgeworth correction for credible set boundaries, centres, and shapes. A scalar learning rate can calibrate all nominal levels in the Gaussian limit only when the posterior covariance and the sampling covariance are proportional. Hence, bootstrap calibration is a level-specific scale correction, not a remedy for general shape misspecification.

stat.ME↗

arXiv2026

Delayed Acceptance Markov Chain Monte Carlo for Robust Bayesian Analysis

This study introduces a computationally efficient algorithm, delayed acceptance Markov chain Monte Carlo (DA-MCMC), designed to improve posterior simulation in quasi-Bayesian inference. Quasi-Bayesian methods, which do not require fully specifying a probabilistic model, are often computationally expensive owing to the need to evaluate the inverse and determinant of large covariance matrices. DA-MCMC addresses this challenge by employing a two-stage process: In the first stage, proposals are screened using an approximate posterior, whereas a final acceptance or rejection decision is made in the second stage based on the exact target posterior. This reduces the need for costly matrix computations, thereby improving efficiency without sacrificing accuracy. We demonstrate the effectiveness of DA-MCMC through applications to both synthetic and real data. The results demonstrate that, although DA-MCMC slightly reduces the effective sample size per iteration compared with the standard MCMC, it achieves substantial improvement in terms of effective sample size per second, approximately doubling the efficiency. This makes DA-MCMC particularly useful for cases where posterior simulation is computationally intensive. Thus, the DA-MCMC algorithm offers a significant advancement in computational efficiency for quasi-Bayesian inference, making it a valuable tool for robust Bayesian analysis.

stat.CO↗

arXiv2026

Estimating Bidirectional Causal Effects with Large Scale Online Kernel Learning

In this study, a scalable online kernel learning framework is proposed for estimating bidirectional causal effects in systems characterized by mutual dependence and heteroskedasticity. Traditional causal inference often focuses on unidirectional effects, overlooking the common bidirectional relationships in real-world phenomena. Building on heteroskedasticity-based identification, the proposed method integrates a quasi-maximum likelihood estimator for simultaneous equation models with large scale online kernel learning. It employs random Fourier feature approximations to flexibly model nonlinear conditional means and variances, while an adaptive online gradient descent algorithm ensures computational efficiency for streaming and high-dimensional data. Results from extensive simulations demonstrate that the proposed method achieves superior accuracy and stability than single equation and polynomial approximation baselines, exhibiting lower bias and root mean squared error across various data-generating processes. These results confirm that the proposed approach effectively captures complex bidirectional causal effects with near-linear computational scaling. By combining econometric identification with modern machine learning techniques, the proposed framework offers a practical, scalable, and theoretically grounded solution for large scale causal inference in natural/social science, policy making, business, and industrial applications.

stat.ML↗

arXiv2026

Generalized Posterior Calibration via Sequential Monte Carlo Sampler

As the amount and complexity of available data increases, the need for robust statistical learning becomes more pressing. To enhance resilience against model misspecification, the generalized posterior inference method adjusts the likelihood term by exponentiating it with a learning rate, thereby fine-tuning the dispersion of the posterior distribution. This study proposes a computationally efficient strategy for selecting an appropriate learning rate. The proposed approach builds upon the generalized posterior calibration (GPC) algorithm, which is designed to select a learning rate that ensures nominal frequentist coverage. This algorithm, which evaluates the coverage probability using bootstrap samples, has high computational costs because of the repeated posterior simulations needed for bootstrap samples. To address this limitation, the study proposes an algorithm that combines elements of the GPC algorithm with the sequential Monte Carlo (SMC) sampler. By leveraging the similarity between the learning rate in generalized posterior inference and the inverse temperature in SMC sampling, the proposed algorithm efficiently calibrates the posterior distribution with a reduced computational cost. For demonstration, the proposed algorithm was applied to several statistical learning models and shown to be significantly faster than the original GPC.

stat.CO↗

arXiv2026

Quasi-Bayesian Local Projections: Simultaneous Inference and Extension to the Instrumental Variable Method

Local projections (LPs) are widely used for impulse response analysis, but Bayesian methods face challenges due to the absence of a likelihood function. Existing approaches rely on pseudo-likelihoods, which often result in poorly calibrated posteriors. We propose a quasi-Bayesian method based on the Laplace-type estimator, where a quasi-likelihood is constructed using a generalized method of moments criterion. This approach avoids strict distributional assumptions, ensures well-calibrated inferences, and supports simultaneous credible bands. Additionally, it can be naturally extended to the instrumental variable method. We validate our approach through Monte Carlo simulations.

econ.EM↗

arXiv2026

Quasi-Bayesian Local Projection Instrumental-Variables Method: Application to Renewable Energy and Electricity Prices

This paper introduces a quasi-Bayesian approach for local projection instrumental-variables (LP-IV) estimation. It builds a moment-based quasi-posterior using the generalized method of moments (GMM) objective and applies a roughness-penalty prior to smooth impulse responses over different horizons. The approach maintains the key first-order features of traditional LP-IV methods, while enhancing stability in finite samples and allowing for joint inference through simultaneous bands. Simulations indicate that this regularization decreases root mean squared error compared to standard GMM, especially at medium and longer horizons. An application to Danish electricity markets highlights the method's practical usefulness.

econ.EM↗

arXiv2026

AutoSP: Unlocking Long-Context LLM Training Via Compiler-Based Sequence Parallelism

Large-language-models (LLMs) demonstrate enormous utility in long-context tasks which require processing prompts that consist of tens to hundreds of thousands of tokens. However, existing LLM training libraries do not provide easy to use abstractions to optimize for long-context training, instead focusing on optimizations for models with large parameter counts through ZeRO-3/FSDP, Tensor and Pipeline parallelism. This forces users to rewrite LLM training libraries to incorporate compositions of various complex long-context optimizations, such as sequence-parallelism, to training pipelines; a process that requires in-depth expertise, reducing developer productivity. To tackle these challenges, we introduce AutoSP: the first automated solution to automatically optimize LLM training for longer-contexts. AutoSP compiles models and applies a targeted set of optimizations: automated sequence parallelism, and long-context aware activation-checkpointing, to drastically enhance LLM trainability at negligible cost to throughput. Our evaluation demonstrates AutoSP's capability on both NVIDIA and AMD hardware, increasing training contexts by upto 2.7$\times$ and 2.5$\times$ respectively over competitive hand-written baseline at negligible cost to runtime performance.

cs.LG↗

arXiv2026

MAC-Attention: a Match-Amend-Complete Scheme for Fast and Accurate Attention Computation

Long-context decoding in LLMs is IO-bound: each token re-reads an ever-growing KV cache. Prior accelerations cut bytes via compression, which lowers fidelity, or selection/eviction, which restricts what remains accessible, and both can degrade delayed recall and long-form generation. We introduce MAC-Attention, a fidelity- and access-preserving alternative that accelerates decoding by reusing prior attention computations for semantically similar recent queries. It starts with a match stage that performs pre-RoPE L2 matching over a short local window; an amend stage rectifies the reused attention by recomputing a small band near the match boundary; and a complete stage fuses the rectified results with fresh attention computed on the KV tail through a numerically stable merge. On a match hit, the compute and bandwidth complexity is constant regardless of context length. The method is model-agnostic and composes with IO-aware kernels, paged-KV managers, and MQA/GQA. Across LongBench v2 (120K), RULER (120K), and LongGenBench (16K continuous generation), compared to the latest FlashInfer library, MAC-Attention reduces KV accesses by up to 99%, cuts token generation latency by over 60% at 128K, and achieves over 14.3x attention-phase speedups, up to 2.6x end-to-end, while maintaining full-attention quality. By reusing computation, MAC-Attention delivers long-context inference that is both fast and faithful. Code is available here: https://github.com/YJHMITWEB/MAC-Attention.git

cs.LG↗

arXiv2026

DeepCompile: A Compiler-Driven Approach to Optimizing Distributed Deep Learning Training

The rapid growth of deep learning models has increased the demand for efficient distributed training strategies. Fully sharded approaches like ZeRO-3 and FSDP partition model parameters across GPUs and apply optimizations such as prefetching and unsharding to reduce communication overhead. However, these systems lack fine-grained control over memory and communication scheduling, making it difficult to balance computation--communication overlap with memory requirements. Coordinating multiple optimizations such as prefetching and unsharding is also difficult, since their effects on memory usage can influence each other. To tackle these challenges, we propose DeepCompile, a compiler-based optimization framework for distributed training. DeepCompile transforms user-defined models into computation graphs and applies a series of profiling-guided optimization passes, each modifying the graph based on profiling information such as execution time and memory usage. This design allows each pass to flexibly insert, reorder, or remove operations such as all-gather and memory allocation, improving communication--computation overlap and reducing memory pressure. Each pass can access updated profiling feedback from earlier passes, enabling coordinated optimizations. We further enhance DeepCompile by three additional optimizations: proactive prefetching, selective unsharding, and adaptive offloading. Our evaluation shows that DeepCompile achieves up to 1.28$\times$ and 1.54$\times$ speedups over ZeRO-3 and FSDP baselines, respectively, and up to a 7.01$\times$ throughput increase in settings with limited GPU resources using offloading.

cs.DC↗