arXiv subjects

Zicheng Yan

Publications and source records attributed to Zicheng Yan.

6 recordsLinked to original sources

arXiv2026

PrismServe: Efficient LLM Serving on Heterogeneous Accelerator Clusters

Serving large language models across heterogeneous accelerators is challenging because partitioning a replica across device types introduces communication overhead during autoregressive decoding, while deploying complete quantized replicas can degrade model quality. We present PrismServe, a serving system that combines request-level collaboration across heterogeneous hardware with a mixture of quantized experts derived from the same foundation model. Each expert executes within a group of same-type accelerators in a single node, while a two-level router selects an expert based on predicted response quality and then selects a replica according to load and prefix-cache locality. Implemented on unmodified vLLM, PrismServe achieves 2.0--2.6$\times$ the throughput of cross-node or cross-vendor full-precision pipelined deployments and reduces P99 time per output token by 55--65\% when serving Qwen3-14B on 48 accelerators from three vendors. Across Qwen3-8B, Qwen3-14B, and Qwen3-32B on five downstream tasks, PrismServe outperforms the best single quantization method by 2.0--4.3 points while remaining within 1.4 points of full-precision accuracy. Our code is available at https://anonymous.4open.science/r/LLM-Inference-System-B000/

cs.LG↗

arXiv2026

Is Human-Readable Text Necessary for Effective LLM Fine-Tuning?

Is human readability necessary for effective fine-tuning of large language models? We investigate whether model-conditioned training representations can preserve or improve adaptation utility without requiring a human-readable textual form. We propose Desired-Update-Aligned Synthetic Data (DASA), which uses activation-gradient feedback from a frozen reference model to guide the optimization of continuous synthetic input embeddings. Inspired by the role of activation gradients in local risk reduction, DASA targets useful adaptation updates rather than source-text reconstruction or linguistic fluency. The resulting embeddings are used directly for downstream fine-tuning; discrete token projections are employed only for qualitative inspection. Experiments on six models from the Llama and Qwen families, ranging from 1B to 32B parameters, cover six benchmarks spanning knowledge, mathematical reasoning, code generation, and commonsense reasoning. Under matched LoRA adaptation settings, DASA achieves performance comparable to the source natural-language data and surpasses it in multiple configurations, while outperforming GRADMM in most comparisons. Further experiments cover general-domain and task-specialized source data. Under the evaluated synthesis settings, DASA provides a $3.6$--$4.9\times$ speedup over GRADMM with comparable peak GPU memory.

cs.AI↗

arXiv2026

ProFocus: Interpreting Affective Experience in Artistic Images with Progressive Visual Focusing

Interpreting the emotional responses triggered by images is central to achieving emotional intelligence. Compared with natural images, visual art is intentionally created to elicit emotional responses from its viewers through abstract concepts and visual metaphors, making affective interpretation particularly challenging. However, most existing methods rely on general-purpose visual embeddings (e.g., CLIP), failing to capture the nuanced cues underlying artistic emotion. To address this gap, we propose \textbf{ProFocus}, a novel framework that models affective experience in artistic images via progressive visual focusing. The key idea is to model visual representation learning inspired by a hierarchical cognitive theory of human aesthetic appreciation. Technically, ProFocus contains two core components: a Hierarchical Art Critic (HAC) and a Progressive Hint Fusion (PHF) module. HAC leverages multimodal large language models to generate structured linguistic priors at three cognitive levels--atmospheric style, narrative subjects, and concrete details--thereby translating artistic perception into coherent semantic guidance. Building upon these priors, PHF departs from conventional cross-modal fusion by sequentially injecting the hierarchical hints into visual features, enabling a progressive focusing process that mirrors human perception. This design allows the model to capture subtle affective cues and produce more faithful explanations. Extensive experiments on the ArtEmis v1.0 and v2.0 datasets demonstrate that ProFocus consistently outperforms state-of-the-art methods in both emotion recognition and affective explanation. Project page: https://github.com/Zhang-Zhiyan/ProFocus.

cs.CV↗

arXiv2026

Mechanisms of Width Scaling in Normalized Residual Networks: The Effective Alignment Dimension

Existing theories of neural-network width characterize asymptotic limits, but provide limited guidance on whether an expansion direction identified from finite training data remains beneficial on unseen data. We study this problem for function-preserving residual expansion and introduce the effective alignment dimension, a measurable quantity describing the signal-noise geometry of activation gradients. By deriving the exact mean and variance of the inner product between independently estimated training and test gradients, we obtain a finite-sample upper bound on misalignment probability. The bound depends only on the effective alignment dimension and an effective sample size, requiring finite second moments and a nonzero population gradient, without covariance spectral assumptions or prescribed width-growth rates. We integrate this certificate into the train-test residual-expansion framework, yielding a high-probability condition for test-risk improvement. Experiments across width-controlled LLaMA-style Transformers, Pythia, and ResNet-20 show that wider models exhibit larger effective alignment dimensions and lower empirical misalignment. Direct residual interventions confirm that the alignment statistic predicts the sign and magnitude of held-out loss changes.

cs.LG↗

arXiv2026

CALM: A CKA-Guided Adaptive Layer-Wise Modularization Framework for LLM Quantization

Current mainstream post-training quantization methods for large language models typically apply a uniform quantization strategy across all network layers, overlooking the substantial differences in algorithmic suitability among layers. To address this limitation, we propose CALM (A CKA-guided Adaptive Layer-wise Modularization)a fine-tuning-free, plug-and-play framework for algorithmic heterogeneous quantization. CALM independently evaluates multiple PTQ algorithms on each layer and employs Linear Centered Kernel Alignment (CKA) as a metric to automatically select the optimal quantization strategy per layer. The individually optimized strategies are then integrated to construct a hybrid quantized model. Experiments demonstrate that our approach consistently outperforms both uniform quantization baselines and state-of-the-art mixed-precision methods across mainstream LLMsincluding LLaMA and Qwenin terms of perplexity (PPL) and downstream task performance.

cs.LG↗

arXiv2025

A Virtual Fields Method-Genetic Algorithm (VFM-GA) calibration framework for isotropic hyperelastic constitutive models with application to an elastomeric foam material

This work introduces a calibration framework for material parameter identification in isotropic hyperelastic constitutive models. The framework synergizes the Virtual Fields Method (VFM) to define an objective function with a Genetic Algorithm (GA) as the optimization method to facilitate automated calibration. The formulation of the objective function uses experimental displacement fields measured from Digital Image Correlation (DIC) synchronized with load cell data and can accommodate data from experiments involving homogeneous or inhomogeneous deformation fields. The framework places no restrictions on the target isotropic hyperelastic constitutive model, accommodating models with coupled dependencies on deformation invariants and specialized functional forms with a number of material parameters, and assesses material stability, eliminating sets of material parameters that potentially lead to non-physical behavior for the target hyperelastic constitutive model. To minimize the objective function, a GA is deployed as the optimization tool due to its ability to navigate the intricate landscape of material parameter space. The VFM-GA framework is evaluated by applying it to a hyperelastic constitutive model for compressible elastomeric foams. The evaluation process entails a number of tests that employ both homogeneous and inhomogeneous displacement fields collected from DIC experiments on open-cell foam specimens. The results outperform manual fitting, demonstrating the framework's robust and efficient capability to handle material parameter identification for complex hyperelastic constitutive models.

physics.comp-ph↗