arXiv subjects

Mustafa Khan

Publications and source records attributed to Mustafa Khan.

7 recordsLinked to original sources

arXiv2026

Modern Solid Electrolytes for All-Solid-State Batteries: Materials Chemistry, Structure, and Transport

In this review, from crystallographic symmetry to amorphous local polyhedra arrangement and combinations, we examine inorganic solid state electrolytes through the lens of structure property relationships, with oxides, sulfides, and halides representing three major framework chemistries. Halide solid electrolytes and their derivatives, including mixed anion halides and antiperovskite related materials, have expanded this landscape further by introducing new ways to regulate local coordination chemistry, defect populations, and transport active frameworks. Across these families, fast ion conduction depends not simply on composition or crystallographic diffusion pathways, but on the coupled effects of framework topology, site energy distribution, defect chemistry, bottleneck response, and local anion flexibility. Oxides illustrate transport within chemically robust but geometrically constrained frameworks. Sulfides demonstrate that a soft, easily polarizable lattice can broaden the array of low energy migration pathways. Halides occupy an intermediate state, in which the closely packed anion sublattices, an approximately degenerate lithium environment, and mixed anion coordination enable effective transport while simultaneously enhancing oxidation stability and compatibility with cathodes. Building on these comparisons, we argue that long range ion transport is increasingly understood not as motion along a single idealized pathway, but as the macroscopic outcome of statistically connected low barrier local migration events distributed across the structure. We further discuss the experimental and computational approaches required to establish such multiscale structure property relationships and outline future strategies for designing transport active frameworks in which conductivity, stability, and processability are optimized together.

cond-mat.mtrl-sci↗

arXiv2025

Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddings

Large language models (LLMs) often concentrate their attention on a few specific tokens referred to as attention sinks. Common examples include the first token, a prompt-independent sink, and punctuation tokens, which are prompt-dependent. While the tokens causing the sinks often lack direct semantic meaning, the presence of the sinks is critical for model performance, particularly under model compression and KV-caching. Despite their ubiquity, the function, semantic role, and origin of attention sinks -- especially those beyond the first token -- remain poorly understood. In this work, we conduct a comprehensive investigation demonstrating that attention sinks: catch a sequence of tokens, tag them using a common direction in embedding space, and release them back into the residual stream, where tokens are later retrieved based on the tags they have acquired. Probing experiments reveal these tags carry semantically meaningful information, such as the truth of a statement. These findings extend to reasoning models, where the mechanism spans more heads and explains greater variance in embeddings, or recent models with query-key normalization, where sinks remain just as prevalent. To encourage future theoretical analysis, we introduce a minimal problem which can be solved through the 'catch, tag, release' mechanism, and where it emerges through training.

cs.CL↗

arXiv2024

Enhancing Knowledge Distillation for LLMs with Response-Priming Prompting

Large language models (LLMs) have demonstrated remarkable performance across a wide range of natural language processing (NLP) tasks. However, these models are often difficult to deploy due to significant computational requirements and resource constraints. Knowledge distillation (KD) is an effective technique for transferring the performance of larger LLMs to smaller models. Traditional KD methods primarily focus on the direct output of the teacher model, with little emphasis on the role of prompting during knowledge transfer. In this paper, we propose a set of novel response-priming prompting strategies applied in the knowledge distillation pipeline to enhance the performance of student models. Our approach fine-tunes a smaller Llama 3.1 8B Instruct model by distilling knowledge from a quantized Llama 3.1 405B Instruct teacher model. We apply LoRA optimization and evaluate on the GSM8K benchmark. Experimental results demonstrate that integrating reasoning-eliciting prompting into the proposed KD pipeline significantly improves student model performance, offering an efficient way to deploy powerful models in resource-constrained environments. We find that Ground Truth prompting results in a 55\% performance increase on GSM8K for a distilled Llama 3.1 8B Instruct compared to the same model distilled without prompting. A thorough investigation into the self-attention layers of the student models indicates that the more successful prompted models tend to exhibit certain positive behaviors inside their attention heads which can be tied to their increased accuracy. Our implementation can be found at https://github.com/alonso130r/knowledge-distillation.

cs.CL↗

arXiv2024

Multi-Aspect Reviewed-Item Retrieval via LLM Query Decomposition and Aspect Fusion

While user-generated product reviews often contain large quantities of information, their utility in addressing natural language product queries has been limited, with a key challenge being the need to aggregate information from multiple low-level sources (reviews) to a higher item level during retrieval. Existing methods for reviewed-item retrieval (RIR) typically take a late fusion (LF) approach which computes query-item scores by simply averaging the top-K query-review similarity scores for an item. However, we demonstrate that for multi-aspect queries and multi-aspect items, LF is highly sensitive to the distribution of aspects covered by reviews in terms of aspect frequency and the degree of aspect separation across reviews. To address these LF failures, we propose several novel aspect fusion (AF) strategies which include Large Language Model (LLM) query extraction and generative reranking. Our experiments show that for imbalanced review corpora, AF can improve over LF by a MAP@10 increase from 0.36 to 0.52, while achieving equivalent performance for balanced review corpora.

cs.IR↗

arXiv2024

AutoSplat: Constrained Gaussian Splatting for Autonomous Driving Scene Reconstruction

Realistic scene reconstruction and view synthesis are essential for advancing autonomous driving systems by simulating safety-critical scenarios. 3D Gaussian Splatting excels in real-time rendering and static scene reconstructions but struggles with modeling driving scenarios due to complex backgrounds, dynamic objects, and sparse views. We propose AutoSplat, a framework employing Gaussian splatting to achieve highly realistic reconstructions of autonomous driving scenes. By imposing geometric constraints on Gaussians representing the road and sky regions, our method enables multi-view consistent simulation of challenging scenarios including lane changes. Leveraging 3D templates, we introduce a reflected Gaussian consistency constraint to supervise both the visible and unseen side of foreground objects. Moreover, to model the dynamic appearance of foreground objects, we estimate residual spherical harmonics for each foreground Gaussian. Extensive experiments on Pandaset and KITTI demonstrate that AutoSplat outperforms state-of-the-art methods in scene reconstruction and novel view synthesis across diverse driving scenarios. Visit our project page at https://autosplat.github.io/.

cs.CV↗

arXiv2023

Uncovering and experimental realization of multimodal 3D topological metamaterials for low-frequency and multiband elastic wave control

Topological metamaterials unlock confined and robust elastic wave control in mechanical structures. Recent breakthroughs have precipitated the development of 3D topological mechanical metamaterials, which extend beyond the conventional 1D and 2D metamaterials to facilitate extraordinary wave manipulation along 2D planar and layer-dependent elastic waveguides. While promising, significant research gaps exist that impede the practical implementation of 3D topological metamaterials. The 3D topological metamaterials studied thus far are constrained to function in single frequency bandwidths that are typically in a high-frequency regime, and a comprehensive experimental investigation remains elusive. In this paper, we address these research gaps and advance the state of the art through the synthesis and experimental realization of a 3D topological metamaterial that exploits multimodal local resonance to enable low-frequency elastic wave control over multiple distinct frequency bands. The proposed metamaterial is geometrically configured to create multimodal local resonators whose frequency characteristics govern the emergence of four unique low-frequency topological states. Numerical simulations uncover how these topological states can be employed to achieve polarization-, frequency-, and layer-dependent wave manipulation in 3D structures. An experimental study results in the attainment of complete wave fields that unambiguously illustrate 2D topological waveguides and multi-polarized wave control in a physical testbed. The outcomes from this work open the door for future research with 3D topological mechanical metamaterials and reveal the applicability of the proposed metamaterial for various wave control applications.

physics.app-ph↗

arXiv2020

Spatio-temporal of Dirac and Klein Gordon particles in a one-dimensional box

Present work is devoted to studying the spatio-temporal of the Dirac and the Klein Gordon (KG) particles confined in a one-dimensional box. We discuss the quantum carpets and revivals time for each particle. Moreover, we explain that the relativistic phenomena not only decrease the density of the corresponding carpets but also affect the revivals time of each particle within a box. Further, we impose different limiting conditions on the obtained Dirac energy solution and explain the wellknown slight-relativistic and non-relativistic quantum carpets, respectively. In addition, we extend our study to explain the quantum carpets of the KG particle at different energy states and compare its revivals time with the Dirac particle.

quant-ph↗