arXiv subjects

Ashutosh Kumar

Publications and source records attributed to Ashutosh Kumar.

At least 19 recordsLinked to original sources

arXiv2026

Sonar-MoE: In-Backward Expert Updates for Memory-Efficient Mixture-of-Experts Training

Most parameters of a Mixture-of-Experts (MoE) model belong to its experts, and standard training keeps every expert gradient in GPU memory until the optimizer step. This memory grows with the number and size of the experts, so it rises as MoE models scale. We introduce Sonar-MoE, which updates each expert tensor during the backward pass and frees its gradient right away. It brings optimizer-in-backward from dense models to batched MoE expert tensors, so the GPU never holds more than one layer of expert gradients. When we fine-tune OLMoE-1B-7B and Granite-3.1-3B-A800M in bf16 with two expert-kernel backends, Sonar-MoE lowers peak memory by 19% to 22%. Its updates are bitwise identical to standard training without expert gradient clipping, and held-out loss stays within 0.003 of all baselines. Each step also takes 17% to 23% less time than standard training with global-norm clipping. Updating whole batched tensors rather than individual experts is what keeps Sonar-MoE fast, as it avoids a 20% to 72% slowdown while using the same memory. Sonar-MoE matches optimizer-in-backward over all parameters in memory and speed while still clipping the dense parameters.

cs.LG↗

arXiv2026

Thermodynamic origin of entropy-assisted stabilization in multicomponent rock-salt oxides

High entropy oxides are commonly associated with high configurational entropy ($ΔS_{conf}\geq$ 1.61R) corresponding to five equimolar cations occupying a crystallographic sublattice. However, recent experimental observations indicate that medium-entropy compositions may also exhibit entropy-stabilized rock-salt phases, raising an important question regarding the minimum entropy required for phase stabilization. In this work, we employ a first-principles thermodynamic framework to investigate the stability of rock-salt oxides containing two to five principal cations components analogous to (Ni$_{0.8}$Cu$_{0.2}$)O, (Ni$_{0.6}$Cu$_{0.2}$Zn$_{0.2}$)O, (Ni$_{0.4}$Cu$_{0.2}$Zn$_{0.2}$Co$_{0.2}$)O, (Ni$_{0.2}$Cu$_{0.2}$Zn$_{0.2}$Co$_{0.2}$Mg$_{0.2}$)O. Density functional theory, MCSQS-based structural modeling, and finite-temperature Gibbs free-energy analysis are combined to quantify the roles of enthalpy mixing ($ΔH_{mix}$), configurational ($ΔS_{conf}$), vibrational ($ΔS_{vib}$), electronic and magnetic ($ΔS_{mag}$) contributions towards ($ΔS_{elec}$) entropy change in governing phase stability. The results show that $ΔS_{conf}$ alone is not a universal descriptor of phase stability. While the two-cation system is enthalpy-stabilized but three-, four- and five-cation systems become thermodynamically stable at high-temperature due to entropy-driven reduction of the Gibbs free energy. These findings demonstrate that single-phase rock-salt oxides are not restricted to the conventional high-entropy limit and that medium-entropy compositions can also be stabilized under suitable thermodynamic conditions.

cond-mat.mtrl-sci↗

arXiv2026

TrajTok: Learning Trajectory Tokens enables better Video Understanding

Tokenization in video models, typically through patchification, generates an excessive and redundant number of tokens. This severely limits video efficiency and scalability. While recent trajectory-based tokenizers offer a promising solution by decoupling video duration from token count, they rely on complex external segmentation and tracking pipelines that are slow and task-agnostic. We propose TrajTok, an end-to-end video tokenizer module that is fully integrated and co-trained with video models for a downstream objective, dynamically adapting its token granularity to semantic complexity, independent of video duration. TrajTok contains a unified segmenter that performs implicit clustering over pixels in both space and time to directly produce object trajectories in a single forward pass. By prioritizing downstream adaptability over pixel-perfect segmentation fidelity, TrajTok is lightweight and efficient, yet empirically improves video understanding performance. With TrajTok, we implement a video CLIP model trained from scratch (TrajViT2). It achieves the best accuracy at scale across both classification and retrieval benchmarks, while maintaining efficiency comparable to the best token-merging methods. TrajTok also proves to be a versatile component beyond its role as a tokenizer. We show that it can be seamlessly integrated as either a probing head for pretrained visual features (TrajAdapter) or an alignment connector in vision-language models (TrajVLM) with especially strong performance in long-video reasoning.

cs.CV↗

arXiv2026

NutriVision: Ingredient-Conditioned Fusion and Prediction for Single-Image Food Nutrition Estimation

Nutrition estimation is a fundamental task in consumer diet tracking, clinical dietetics, chronic disease management, sports and hospital nutrition, and broader food computing systems. The existing approaches have progressed along two largely separate axes, vision models that rely on calibrated RGB-depth captures and ingredient-aware methods that use textual cues but use limited multimodal fusion. We introduce NutriVision, an end-to-end framework that leverages visual geometry and ingredient semantics to estimate calories, mass, fat content, carbohydrates, and protein from a single RGB image and an optional ingredient list. It obtains the unavailable depth modality using DepthAnything-V3 and encodes ingredient descriptions using CLIP. It integrates three complementary mechanisms: (1) an \emph{Ingredient-Conditioned Frequency-Aligned Fusion Module (IC-FAFM)}, which uses textual guidance to reweight and align RGB-depth frequency components; (2) an \emph{Ingredient-Aware Mask-based Prediction Head (IA-MPH)}, whose gating and channel masks are conditioned on food identity; and (3) modality-specific \emph{Internal Semantic Modeling (ISM)} blocks. On the Nutrition5k dataset, NutriVision achieves a mean PMAE of $\mathbf{13.60\pm0.10\%}$, outperforming our IGSMNet implementation by $0.89$ percentage points and OmniFood8k by $2.90$ percentage points (both $p<0.001$). The module-level ablations identify the ingredient-aware prediction head as the primary architectural contributor, improving mean PMAE by $1.50\pm0.17$ percentage points ($p<0.001$). These results demonstrate that ingredient-conditioned prediction and frequency-aware RGB-depth fusion provide measurable gains for single-image nutrient estimation. More broadly, NutriVision offers a practical route toward nutrition-assessment systems that exploit geometric and semantic cues without requiring specialized depth-sensing hardware

cs.CV↗

arXiv2026

dKFD: Phase-Structured Evidence Allocation for Fixed-Budget Localized Event Understanding

Sparse video understanding often requires selecting a small set of visual evidence under a fixed frame budget. Most sparse selectors allocate this budget globally, allowing all frames to compete with one another. For temporally localized events, this can be a poor inductive bias: useful evidence is often distributed across pre-event context, the event itself, and post-event consequences. We study fixed-budget evidence allocation for localized event videos and show that globally competitive Top-$K$ selectors can preserve recognition and grounding while producing unstable event evidence. On DoTA Video Anomaly Recognition, Global Top-$K$ obtains competitive recognition and temporal grounding, but low selector-event alignment at $K=12$ (Frame AUC $51.5 \pm 10.8$). We propose dKFD, a phase-structured differentiable selector that reserves evidence capacity across pre-event, event, and post-event phases after full-sequence temporal encoding. Under matched-budget multi-seed evaluation, dKFD improves Frame AUC by $+30.97$ over a matched Global Top-$K$ selector at $K=12$ ($p<0.01$), while yielding modest but statistically significant recognition gains and comparable temporal grounding. Mechanism ablations show that phase supervision is load-bearing: removing it reduces Frame AUC to $41.1 \pm 12.0$ even when phase-partitioned budgets are retained. Downstream diagnostics on VRU-Accident show consistent gains over learned Global Top-$K$ across VLM families, while dense captioning reveals a boundary condition where uniform sampling remains competitive. These results support phase-structured allocation as a controlled fixed-budget approach for event-centric sparse evidence selection, not as a universal video summarization strategy.

cs.CV↗

arXiv2026

LLMs Anchor on Chief Complaint and Fail to Integrate Evidence in Sequential Clinical Triage

Triage in the emergency department (ED) is a sequential decision process that unfolds turn by turn. Existing evaluations of large language models (LLMs) for triage use completed retrospective records and report performance close to that of physicians. We implement a methodology for evaluating LLMs on sequential triage, the task of predicting a triage acuity label from a growing prefix of a nurse-patient conversation. We evaluate six LLMs at five sequential checkpoints on two corpora: 425 LLM-generated (SIMULATED) and 50 physician-authored (CLINICIAN) conversations, both labelled under the Emergency Severity Index (ESI). Every model, measured by quadratic weighted kappa (QWK), degrades from moderate-to-substantial agreement on completed records to fair-to-moderate agreement at every sequential checkpoint. Controlled perturbations show that the label at every checkpoint is anchored on the chief complaint exchanges, and prompting interventions fail to lift this plateau. Models extract clinically relevant content from later turns, yet the surprisal of the true label rises across the checkpoints. So the model fails to integrate the evidence. Three expert clinicians on the same conversations reach a QWK of 0.887-0.929, while the best model reaches 0.295. Predictions concentrate at ESI-2 and ESI-3, and models agree with each other more than with the ground truth, so ensembling worsens the failure. Deploying LLMs for ED triage based on offline benchmarks alone misses this sequential failure.

cs.CL↗

arXiv2026

Compositionally Engineered Non-Equimolar LaCoO$_3$-Based High-Entropy Perovskites with Enhanced Thermoelectric Performance

The decoupling of phonon and electron transport remains a central challenge in the development of high-performance TE materials. Configurational-entropy maximization is widely invoked as a design principle for decoupling phonon and electron transport. This study investigates compositionally engineered LaCoO$_3$-based high-entropy perovskites to determine whether thermoelectric transport can be improved by tuning cation identity and concentration rather than maximizing configurational entropy. La$_{1-x}$Sr$_x$(CoFeMnCrNi)O$_3$ ($x=0.0$--$0.2$) and selected non-equimolar A- and B-site perovskite compositions were prepared by solid-state reaction. The obtained samples are predominantly single-phase, as confirmed by X-ray diffraction and Rietveld refinement, consistent with the calculated size-disorder parameters. Multication disorder introduces substantial mass and strain-field fluctuations that promote phonon scattering. All samples exhibit p-type, thermally activated electrical transport consistent with adiabatic SPH. Sr substitution progressively reduces the hopping barrier and $ρ$, whereas non-equimolar B-site engineering partially recovers electrical transport while retaining low k. La$_{0.9}$Sr$_{0.1}$Co$_{0.4}$Cr$_{0.3}$Ni$_{0.1}$Fe$_{0.1}$Mn$_{0.1}$O$_3$, featuring a Co-rich and Cr-rich B-site composition, combines Co-associated mixed valence and spin-state degeneracy that sustain a large alpha with Cr-mediated control of carrier concentration and a reduced polaron hopping barrier of 0.19eV. This composition achieves a power factor of 40--43 $μ$W/(m K$^2$) and a $zT\sim0.072$ at 1100K, approximately 2.7 times that of its equimolar analogue. These results demonstrate that targeted cation chemistry and mass contrast, rather than configurational-entropy maximization alone, provide an effective strategy for balancing electronic and phonon transport in multicomponent oxide TE.

cond-mat.mtrl-sci↗

arXiv2026

The 10th AI City Challenge

The 10th AI City Challenge, held with ECCV 2026, marks a decade of community benchmarking for intelligent transportation, smart cities, and physical AI. Since its 2017 start with vehicle detection, classification, and tracking, the challenge has grown into a broad benchmark suite for multi-camera perception, multimodal reasoning, synthetic-to-real learning, generative forecasting, and privacy-preserving evaluation. The 2026 edition continued this growth with 325 registered teams, up from 245 in 2025, and participation from 26 countries and regions, up from 15. Its six primary tracks cover multi-camera 3D perception, transportation safety captioning and VQA, traffic anomaly reasoning, text-based person anomaly search, generative traffic video forecasting, and cross-city object detection. Track 3 further includes two out-of-domain leaderboards, submitted as Tracks 7 and 8, for fisheye traffic-violation understanding and pedestrian situated-intent VQA. This paper summarizes the challenge setup, datasets, evaluation protocols, leaderboard results, and workshop papers. Across tracks, successful systems combine foundation models with geometric grounding, retrieval or reranking, synthetic-data design, domain adaptation, and controlled inference.

cs.CV↗

arXiv2026

The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modes

Reasoning has become central to how Large Language Models (LLMs) are evaluated and interpreted, spanning Chain-of-Thought (CoT), mathematical problem-solving, multi-hop question answering, code generation, retrieval-augmented reasoning, tool use, and multimodal decision-making. In this survey, we introduce the Periodic Table of LLM Reasoning, a framework organizing 300+ recent papers by reasoning paradigm, methodological mechanism, evaluation setting, and failure mode. We classify LLM reasoning into nine paradigms: Chain-of-Thought, Multi-Hop, Mathematical, Commonsense, Visual and Temporal, Code and Algorithmic, Retrieval-Augmented, Tool-Augmented or Agentic, and Reinforcement Learning-based reasoning. For each, we review approaches, including prompting, architectural interventions, supervised fine-tuning, verifier-guided inference, reward modeling, retrieval, tool interfaces, agentic workflows, and benchmark design. We argue that LLM reasoning is not a single emergent capability but a family of scaffolded behaviors shaped by model scale, task structure, external memory, supervision, and evaluation protocols. We synthesize recurring failure modes, including hallucinated reasoning, brittle multi-step inference, spurious rationales, weak causal grounding, poor out-of-distribution generalization, benchmark contamination, and unreliable self-verification. Progress is difficult to compare across paradigms because gains may arise from prompting, retrieval, verifier design, or benchmark structure rather than general reasoning ability. The survey connects methods to their assumptions, strengths, and failure modes, providing a reference map of the field and a diagnostic framework for future work. We conclude that robust LLM reasoning will require meta-reasoning, multimodal and temporal grounding, adaptive tool use, and principled evaluation under distribution shift.

cs.CL↗

arXiv2026

Growth and characterization of GaN/Ga2O3 Nanowire Heterostructures for Ultraviolet Optoelectronics

Ultraviolet-range GaN/b-Ga2O3 heterostructures were fabricated and investigated in both planar and nanowire geometries using pulsed laser deposition and reactive magnetron sputtering from a liquid gallium target for b-Ga2O3 deposition, while both GaN nanowire arrays and planar p-type Mg-doped GaN layers were grown by metal-organic chemical vapor deposition. Precise control of film uniformity and thickness was achieved as confirmed by structural and morphology studies using X-ray diffraction, X-ray photoelectron spectroscopy, atomic force microscopy and scanning electron microscopy. Planar n-Ga2O3/p-GaN heterojunction diodes were electrically and photoelectronically characterized, exhibiting pronounced rectifying behavior, high forward current and a visible-blind ultraviolet photoresponse under zero external bias, demonstrating intrinsic self-powered operation. Furthermore, GaN/b-Ga2O3 core/shell nanowire heterostructures were developed and systematically studied with a focus on morphology control and process optimization. The influence of deposition parameters on shell thickness, uniformity, and tapering was investigated, enabling improved conformality of the b-Ga2O3 coating on the M-plane facets of GaN nanowires. The results highlight the viability of physical vapor deposition techniques for forming GaN/b-Ga2O3 heterostructures and establish a pathway toward nanowire-based ultraviolet optoelectronic devices.

cond-mat.mtrl-sci↗

arXiv2026

TextNCA: Neural Cellular Automata for Language Modeling via Hierarchical Local Attention

Can a strictly local, iterated, weight-shared computation primitive support language modelling, and which of those three properties actually drives the model's behaviour? We define \textsc{TextNCA}, a 1D causal windowed-attention realisation of the Neural Cellular Automaton primitive, and study a hierarchical variant that cascades three stages with windows $w \in \{8, 32, 128\}$ and $T_s$ shared-weight iterations per stage, all on WikiText-103 at roughly 30M parameters and 60k training steps. The model does not match a parameter-matched Transformer at this scale (Hier-TextNCA $60.3$ vs.\ Transformer-6L $52.8$ and Transformer-12L $44.7$ PPL), so we treat it as an analytical probe rather than a proposed alternative. The behaviour we observe is largely explained by the staged narrow-to-wide schedule: a non-iterating sliding-window Transformer that reuses the same schedule comes within $+4.1$ PPL of the iterated model, while reversing, flattening, or breaking the monotonic ordering of the schedule costs between $+16.7$ and $+70.8$ PPL. Iteration adds a smaller bounded benefit on top of the schedule, with a clear optimum at $T_s{=}4$ and a U-shaped degradation beyond it. The GRU gate and learned per-step embeddings are required for that benefit to appear, and training with random $T_s$ yields an inference-time iteration-count knob at the cost of substantially higher absolute PPL. We position the work as a controlled reading of which parts of NCA-style computation carry the weight in language modelling.

cs.CL↗

arXiv2026

Spinel Ferrite-Based Materials for Electrochemical Applications: Synthesis, Applications, and Future Perspectives

Spinel ferrites have shown wide range of applications in various fields, including supercapacitors, Li-ion batteries, water splitting, chemical sensors, catalytic activity, high-frequency magnetic devices, and biomedical, hyperthermia, drug delivery applications., etc. This review focuses on the latest progress and trends on design of spinel ferrites (MFe2O4; M = Divalent transition metal ion) based materials for electrochemical applications. Spinel ferrites exhibit good chemical stability, high surface area and excellent electrochemical behaviour with their multiple oxidation states, making them suitable for the detection of a wide range of gaseous analytes including volatile organic compounds, heavy metal ions, biomolecules, and environmental pollutants. It also makes spinel ferrites a great choice for efficient energy storage and utility in supercapacitors. The electrochemical performance of spinel ferrite-based electrode materials can be effectively tuned via morphology control, incorporation of carbon-based materials, compositional substitution, doping and forming composition systems, especially in nanostructured form. This review will serve as a comprehensive resource for researchers interested in the synthesis and various enhancement techniques for electrode material composition of electrochemical device applications specifically, gas sensors and supercapacitors, which are two of the highest emerging functional applications for the new sustainability directed world, utilizing these advanced materials.

cond-mat.mtrl-sci↗

arXiv2026

Mutually Exclusive Multiclass Lesion Segmentation in Neuroimaging: Binary-Guided Weak Supervision with Inter-Class Orthogonality

Weakly supervised segmentation of co-occurring neuroimaging lesion subclasses remains challenging due to overlapping activations, noisy pseudo-labels, and the absence of explicit inter-class exclusivity constraints. We propose BiMEx-MS (Binary-guided Mutually Exclusive Multiclass Segmentation), a framework that decomposes multiclass segmentation into whole-lesion localization and exclusive class assignment: a binary localization module provides a class-frequency-agnostic structural prior confining multiclass predictions within the detected lesion domain, while a multi-exit classification architecture with supervised contrastive pretraining produces multi-scale class-discriminative activation maps aggregated via a class-specific attention network. Inter-class exclusivity is enforced through a tri-partite loss comprising per-class separation, inter-class orthogonality, and binary-multiclass spatial consensus, followed by hierarchical morphological pseudo-label refinement. Evaluated across brain tumor MRI (BraTS 2020, BraTS 2023 SSA) and intracranial hemorrhage CT (RSNA-ICH to BHSD) against sixteen weakly supervised baselines, BiMEx-MS achieves Edema HD95 of 29.56 mm (the only method below 40 mm) and subdural hemorrhage Dice of 0.704, with gains consistently largest on boundary metrics and rare subtypes. Cross-dataset generalization, backbone ablations across six architectures, and uncertainty quantification confirm that structural guidance rather than model capacity drives performance. Code: https://github.com/ashutoshkr45/BiMEx-MS-Neuro.

eess.IV↗

arXiv2026

CaST-Bench: Benchmarking Causal Chain-Grounded Spatio-Temporal Reasoning for Video Question Answering

Cause-and-effect reasoning in video is a significant challenge for Vision-Language Models (VLMs), as it requires going beyond surface-level perception to a deeper understanding of causal mechanisms. However, existing benchmarks rarely provide the fine-grained, grounded evidence needed to rigorously evaluate this capability. To address this gap, we introduce CaST-Bench, a benchmark for Causal Chain-Grounded Spatio-Temporal Video Reasoning. CaST-Bench presents complex causal questions that require models to identify and localize a chain of multiple spatio-temporal evidences. Through a human-AI collaborative pipeline, we construct a high-quality dataset of 2,066 questions over 1,015 videos, with causal chains annotated by temporal segments and bounding-box tracks. Furthermore, we design a comprehensive evaluation suite with novel metrics that assess not only answer correctness but also the capability for visual evidence grounded reasoning. This grounding is crucial for improving accuracy by mitigating spurious correlations and for enhancing user trust by making models more transparent. Our experiments show that current VLMs struggle with causal questions, largely due to their limited ability to construct precise and grounded causal chains. This highlights an important direction for improving future VLMs. Homepage: https://woven-by-toyota.github.io/CaST-Bench.

cs.CV↗

arXiv2026

Self-Driving Negotiator: An interactive, verifiable benchmark for social negotiation and theory of mind under hidden intent

Autonomous driving is full of tiny social negotiations: a driver presses forward, another yields, a pedestrian fakes toward the curb, or a lane vehicle chooses whether to open a merge gap. Such interactions require inferring hidden intent from behavior under partial observability and then acting safely and efficiently. Existing autonomous-driving language benchmarks mostly focus on perception, visual question answering, or open-loop planning, while existing language-agent negotiation benchmarks typically make the negotiation explicit in text. Self-Driving Negotiator bridges the gap between the two: a text-only, multi-turn, procedurally generated environment for measuring implicit social coordination in driving. Agents generate specific driving actions. Reward and diagnostics are computed from the privileged simulator state, not from the explanation of the model. This report covers task design, reward and anti-gaming invariants, validated scenarios, non-LLM baselines, and a six-model inference leaderboard. Current models are far removed from the scripted expert. The best average success rate across three scenarios is 0.68; contested merge is statistically flat across models; and difficulty tiers separate cue-following from true wait-for-commitment behavior.

cs.GT↗

arXiv2026

Flexible PDMS/La$_{0.7}$Sr$_{0.3}$MnO$_3$/MWCNT Composite Thin Films for Multifunctional Temperature and Magnetic Sensing Electronic Skin

The development of multifunctional electronic skin (e-skin) requires materials that combine mechanical flexibility with responsiveness to multiple stimuli. In this work, a flexible PDMS/La0.7Sr0.3MnO3 (LSMO)/MWCNT composite thin film was fabricated via solution casting, using LSMO powder synthesized by a solid-state reaction method. Structural and spectroscopic analyses confirm the formation of single-phase rhombohedral LSMO and successful incorporation of PDMS, LSMO, and MWCNT components. The composite exhibits a smooth and uniform surface morphology, along with significantly enhanced thermal stability, retaining ~70% mass at elevated temperatures. Electrical measurements reveal thermally activated resistivity behavior, enabling temperature sensing functionality. Additionally, the composite shows a notable decrease in resistance under an applied magnetic field, exhibiting magnetoresistance due to spin-dependent transport in the LSMO phase. Mechanical testing indicates elastomeric behavior with a maximum load of ~0.49 N and stretchability of ~26%, along with ductile deformation characteristics. The multifunctional sensing properties arise from the synergistic interaction between the conductive MWCNT network and magnetically active LSMO within the flexible PDMS matrix. Overall, the composite demonstrates a unique combination of thermal stability, mechanical flexibility, and dual sensing capability, making it a promising material for next-generation e-skin applications.

cond-mat.mtrl-sci↗

arXiv2026

When do complex-valued neural networks help? A study of representation, geometry, and optimization

Complex-valued Neural Networks (CVNNs) are often motivated by domains where information is naturally encoded in magnitude and phase. Yet complex-valued inputs alone do not determine when complex arithmetic improves learning: the label signal may lie in amplitude, phase, their coupling, or a symmetry that real-valued models can also represent under suitable coordinates. We study this through a representation-first evaluation of CVNNs against Cartesian real, polar, phase-only, magnitude-only, parameter-matched real, and FLOP-matched real baselines. Across synthetic RF tasks, complex representations are useful but not universally superior. PSK-only tasks favor phase-aware and complex-valued models, QAM-only tasks favor magnitude-based models, mixed PSK+QAM gives only a small complex-valued advantage, and unseen carrier-phase rotations break coordinate-dependent models without augmentation. Similar patterns appear beyond RF: in quantum-wavefunction prediction, momentum is invisible to $|ψ|$ but recoverable from phase, while EEG analytic-signal experiments show that phase locking, amplitude bursts, and phase-amplitude coupling each favor different coordinate views. We also identify a benchmarking artifact on RadioML 2018.01A. Under matched-shared-trial selection, a CReLU complex model exceeds the best real baseline by 22.94 PP; under independent per-family tuning on the same data and 16-trial search space, the gap collapses to 2.46 PP. Gradient analysis traces the inflated gap to high-learning-rate first-step instability in real baselines, while complex parameter coupling distributes the loss signal more robustly. A learning-rate $\times$ activation factorial confirms the failure is primarily hyperparameter-driven. Overall, CVNNs are best viewed as structured inductive biases whose gains depend on representation, symmetry, and optimization, not as universally superior architectures.

cs.LG↗

arXiv2026

InstAP: Instance-Aware Vision-Language Pre-Train for Spatial-Temporal Understanding

Current vision-language pre-training (VLP) paradigms excel at global scene understanding but struggle with instance-level reasoning due to global-only supervision. We introduce InstAP, an Instance-Aware Pre-training framework that jointly optimizes global vision-text alignment and fine-grained, instance-level contrastive alignment by grounding textual mentions to specific spatial-temporal regions. To support this, we present InstVL, a large-scale dataset (2 million images, 50,000 videos) with dual-granularity annotations: holistic scene captions and dense, grounded instance descriptions. On the InstVL benchmark, InstAP substantially outperforms existing VLP models on instance-level retrieval, and also surpasses a strong VLP baseline trained on the exact same data corpus, isolating the benefit of our instance-aware objective. Moreover, instance-centric pre-training improves global understanding: InstAP achieves competitive zero-shot performance on multiple video benchmarks, including MSR-VTT and DiDeMo. Qualitative visualizations further show that InstAP localizes textual mentions to the correct instances, while global-only models exhibit more diffuse, scene-level attention.

cs.CV↗