arXiv subjects

Zhihao Song

Publications and source records attributed to Zhihao Song.

4 recordsLinked to original sources

arXiv2026

YouZhi: Towards High-Concurrency Financial LLMs via Adaptive GQA-to-MLA Transition

Large language models (LLMs) drive significant financial innovations, yet their high-concurrency deployment is severely bottlenecked by KV cache memory overhead, which inflates infrastructure costs and throttles scalability. To address this, we propose YouZhi-LLM, a highly efficient financial LLM empowered by a comprehensive structural transition and training pipeline natively built on the Huawei Ascend ecosystem. At its algorithmic core, YouZhi-LLM features a layer-adaptive GQA-to-MLA transition framework that dynamically assigns per-layer FreqFold sizes, maximizing KV-cache compression while minimizing perplexity degradation. To recover representation capacity and inject domain expertise, the Ascend-based training pipeline seamlessly integrates generalized knowledge distillation with financial-specific supervised fine-tuning. Evaluations demonstrate the superiority of this systematic approach, with the adaptive transition reducing perplexity degradation by up to 35% over uniform baselines. Crucially, when evaluated on Ascend NPUs via vLLM-Ascend, the massive KV-cache reduction translates directly into deployment efficiency. Compared to their respective base models, YouZhi-7B yields a 12.3% improvement in average financial benchmark score alongside a 2.69$\times$ increase in maximum concurrency; similarly, YouZhi-14B achieves a 7.0% accuracy gain and a 2.43$\times$ concurrency boost, establishing a new paradigm for cost-effective, high-throughput financial inference.

cs.CL↗

arXiv2024

Diamond of Thought: A Design Thinking-Based Framework for LLMs in Wearable Design

Wearable design is an interdisciplinary field that balances technological innovation, human factors, and human-computer interactions. Despite contributions from various disciplines, many projects lack stable interdisciplinary teams, which often leads to design failures. Large language models (LLMs) integrate diverse information and generate innovative solutions, making them a valuable tool for enhancing design processes. Thus, we have explored the use of LLMs in wearable design by combining design-thinking principles with LLM capabilities. We have developed the "Diamond of Thought" framework and analysed 1,603 prototypes and 1,129 products from a body-centric perspective to create a comprehensive database. We employed retrieval-augmented generation to input database details into the LLMs, ensuring applicability to wearable design challenges and integration of embodied cognition into the process. Our LLM-based methodology for wearables has been experimentally validated, demonstrating the potential of LLMs for the advancement of design practices. This study offers new tools and methods for future wearable designs.

cs.HC↗

arXiv2024

INViT: A Generalizable Routing Problem Solver with Invariant Nested View Transformer

Recently, deep reinforcement learning has shown promising results for learning fast heuristics to solve routing problems. Meanwhile, most of the solvers suffer from generalizing to an unseen distribution or distributions with different scales. To address this issue, we propose a novel architecture, called Invariant Nested View Transformer (INViT), which is designed to enforce a nested design together with invariant views inside the encoders to promote the generalizability of the learned solver. It applies a modified policy gradient algorithm enhanced with data augmentations. We demonstrate that the proposed INViT achieves a dominant generalization performance on both TSP and CVRP problems with various distributions and different problem scales.

cs.LG↗

arXiv2023

Learning to Reduce Information Bottleneck for Object Detection in Aerial Images

Object detection in aerial images is a fundamental research topic in the geoscience and remote sensing domain. However, the advanced approaches on this topic mainly focus on designing the elaborate backbones or head networks but ignore neck networks. In this letter, we first underline the importance of the neck network in object detection from the perspective of information bottleneck. Then, to alleviate the information deficiency problem in the current approaches, we propose a global semantic network (GSNet), which acts as a bridge from the backbone network to the head network in a bidirectional global pattern. Compared to the existing approaches, our model can capture the rich and enhanced image features with less computational costs. Besides, we further propose a feature fusion refinement module (FRM) for different levels of features, which are suffering from the problem of semantic gap in feature fusion. To demonstrate the effectiveness and efficiency of our approach, experiments are carried out on two challenging and representative aerial image datasets (i.e., DOTA and HRSC2016). Experimental results in terms of accuracy and complexity validate the superiority of our method. The code has been open-sourced at GSNet.

cs.CV↗