arXiv subjects

Kan Wu

Publications and source records attributed to Kan Wu.

At least 19 recordsLinked to original sources

arXiv2026

OBASE: Object-Based Address-Space Engineering to Improve Memory Tiering

Hardware and OS mechanisms for memory tiering are widely deployed, yet datacenters still overprovision DRAM. The root cause is hotness fragmentation: allocators place objects by size rather than access pattern, so hot and cold objects become interleaved within the same pages. A single hot object marks its page as active, trapping surrounding cold data in expensive DRAM. Our analysis of Google production workloads shows that up to 97% of the bytes in active pages are cold and unreclaimable. We propose address-space engineering: dynamically reorganizing virtual memory so that hot objects cluster into uniformly hot pages and cold objects into uniformly cold pages. We present OBASE, a compiler-runtime system for unmanaged languages that serves as an object-aware frontend for page-aware OS backends. OBASE tracks accesses via lightweight pointer instrumentation and migrates objects at runtime using a lock-free protocol that is safe under concurrency. By reorganizing the address space, OBASE enables unmodified backends (kswapd, TMO, TPP, Memtis) to tier memory effectively. Across ten concurrent data structures, six backends, and production traces from Meta and Twitter, OBASE improves page utilization by 2-4x and reduces memory footprint by up to 70%, with only 2-5% overhead.

cs.OS↗

arXiv2025

Getting the MOST out of your Storage Hierarchy with Mirror-Optimized Storage Tiering

We present Mirror-Optimized Storage Tiering (MOST), a novel tiering-based approach optimized for modern storage hierarchies. The key idea of MOST is to combine the load balancing advantages of mirroring with the space-efficiency advantages of tiering. Specifically, MOST dynamically mirrors a small amount of hot data across storage tiers to efficiently balance load, avoiding costly migrations. As a result, MOST is as space-efficient as classic tiering while achieving better bandwidth utilization under I/O-intensive workloads. We implement MOST in Cerberus, a user-level storage management layer based on CacheLib. We show the efficacy of Cerberus through a comprehensive empirical study: across a range of static and dynamic workloads, Cerberus achieves better throughput than competing approaches on modern storage hierarchies especially under I/O-intensive and dynamic workloads.

cs.OS↗

arXiv2025

HunyuanOCR Technical Report

This paper presents HunyuanOCR, a commercial-grade, open-source, and lightweight (1B parameters) Vision-Language Model (VLM) dedicated to OCR tasks. The architecture comprises a Native Vision Transformer (ViT) and a lightweight LLM connected via an MLP adapter. HunyuanOCR demonstrates superior performance, outperforming commercial APIs, traditional pipelines, and larger models (e.g., Qwen3-VL-4B). Specifically, it surpasses current public solutions in perception tasks (Text Spotting, Parsing) and excels in semantic tasks (IE, Text Image Translation), securing first place in the ICDAR 2025 DIMT Challenge (Small Model Track). Furthermore, it achieves state-of-the-art (SOTA) results on OCRBench among VLMs with fewer than 3B parameters. HunyuanOCR achieves breakthroughs in three key aspects: 1) Unifying Versatility and Efficiency: We implement comprehensive support for core capabilities including spotting, parsing, IE, VQA, and translation within a lightweight framework. This addresses the limitations of narrow "OCR expert models" and inefficient "General VLMs". 2) Streamlined End-to-End Architecture: Adopting a pure end-to-end paradigm eliminates dependencies on pre-processing modules (e.g., layout analysis). This fundamentally resolves error propagation common in traditional pipelines and simplifies system deployment. 3) Data-Driven and RL Strategies: We confirm the critical role of high-quality data and, for the first time in the industry, demonstrate that Reinforcement Learning (RL) strategies yield significant performance gains in OCR tasks. HunyuanOCR is officially open-sourced on HuggingFace. We also provide a high-performance deployment solution based on vLLM, placing its production efficiency in the top tier. We hope this model will advance frontier research and provide a solid foundation for industrial applications.

cs.CV↗

arXiv2025

Tidying Up the Address Space

Memory tiering in datacenters does not achieve its full potential due to hotness fragmentation -- the intermingling of hot and cold objects within memory pages. This fragmentation prevents page-based reclamation systems from distinguishing truly hot pages from pages containing mostly cold objects, fundamentally limiting memory efficiency despite highly skewed accesses. We introduce address-space engineering: dynamically reorganizing application virtual address spaces to create uniformly hot and cold regions that any page-level tiering backend can manage effectively. HADES demonstrates this frontend/backend approach through a compiler-runtime system that tracks and migrates objects based on access patterns, requiring minimal developer intervention. Evaluations across ten data structures achieve up to 70% memory reduction with 3% performance overhead, showing that address space engineering enables existing reclamation systems to reclaim memory aggressively without performance degradation.

cs.OS↗

arXiv2025

Spark Transformer: Reactivating Sparsity in FFN and Attention

The discovery of the lazy neuron phenomenon in trained Transformers, where the vast majority of neurons in their feed-forward networks (FFN) are inactive for each token, has spurred tremendous interests in activation sparsity for enhancing large model efficiency. While notable progress has been made in translating such sparsity to wall-time benefits, modern Transformers have moved away from the ReLU activation function crucial to this phenomenon. Existing efforts on re-introducing activation sparsity often degrade model quality, increase parameter count, complicate or slow down training. Sparse attention, the application of sparse activation to the attention mechanism, often faces similar challenges. This paper introduces the Spark Transformer, a novel architecture that achieves a high level of activation sparsity in both FFN and the attention mechanism while maintaining model quality, parameter count, and standard training procedures. Our method realizes sparsity via top-k masking for explicit control over sparsity level. Crucially, we introduce statistical top-k, a hardware-accelerator-friendly, linear-time approximate algorithm that avoids costly sorting and mitigates significant training slowdown from standard top-$k$ operators. Furthermore, Spark Transformer reallocates existing FFN parameters and attention key embeddings to form a low-cost predictor for identifying activated entries. This design not only mitigates quality loss from enforced sparsity, but also enhances wall-time benefit. Pretrained with the Gemma-2 recipe, Spark Transformer demonstrates competitive performance on standard benchmarks while exhibiting significant sparsity: only 8% of FFN neurons are activated, and each token attends to a maximum of 256 tokens. This sparsity translates to a 2.5x reduction in FLOPs, leading to decoding wall-time speedups of up to 1.79x on CPU and 1.40x on GPU.

cs.LG↗

arXiv2025

VecTrans: Enhancing Compiler Auto-Vectorization through LLM-Assisted Code Transformations

Auto-vectorization is a fundamental optimization for modern compilers to exploit SIMD parallelism. However, state-of-the-art approaches still struggle to handle intricate code patterns, often requiring manual hints or domain-specific expertise. Large language models (LLMs), with their ability to capture intricate patterns, provide a promising solution, yet their effective application in compiler optimizations remains an open challenge due to issues such as hallucinations and a lack of domain-specific reasoning. In this paper, we present VecTrans, a novel framework that leverages LLMs to enhance compiler-based code vectorization. VecTrans first employs compiler analysis to identify potentially vectorizable code regions. It then utilizes an LLM to refactor these regions into patterns that are more amenable to the compilers auto-vectorization. To ensure semantic correctness, VecTrans further integrates a hybrid validation mechanism at the intermediate representation (IR) level. With the above efforts, VecTrans combines the adaptability of LLMs with the precision of compiler vectorization, thereby effectively opening up the vectorization opportunities. experimental results show that among all TSVC functions unvectorizable by GCC, ICC, Clang, and BiSheng Compiler, VecTrans achieves an geomean speedup of 1.77x and successfully vectorizes 24 of 51 test cases. This marks a significant advancement over state-of-the-art approaches while maintaining a cost efficiency of $0.012 per function optimization for LLM API usage.

cs.SE↗

arXiv2025

Scaling Laws for Floating Point Quantization Training

Low-precision training is considered an effective strategy for reducing both training and downstream inference costs. Previous scaling laws for precision mainly focus on integer quantization, which pay less attention to the constituents in floating-point (FP) quantization, and thus cannot well fit the LLM losses in this scenario. In contrast, while FP quantization training is more commonly implemented in production, it's research has been relatively superficial. In this paper, we thoroughly explore the effects of FP quantization targets, exponent bits, mantissa bits, and the calculation granularity of the scaling factor in FP quantization training performance of LLM models. In addition to an accurate FP quantization unified scaling law, we also provide valuable suggestions for the community: (1) Exponent bits contribute slightly more to the model performance than mantissa bits. We provide the optimal exponent-mantissa bit ratio for different bit numbers, which is available for future reference by hardware manufacturers; (2) We discover the formation of the critical data size in low-precision LLM training. Too much training data exceeding the critical data size will inversely bring in degradation of LLM performance; (3) The optimal FP quantization precision is directly proportional to the computational power, but within a wide computational power range. We estimate that the best cost-performance precision should lie between 4-8 bits.

cs.LG↗

arXiv2024

Scalable Analysis of Urban Scaling Laws: Leveraging Cloud Computing to Analyze 21,280 Global Cities

Cities play a pivotal role in human development and sustainability, yet studying them presents significant challenges due to the vast scale and complexity of spatial-temporal data. One such challenge is the need to uncover universal urban patterns, such as the urban scaling law, across thousands of cities worldwide. In this study, we propose a novel large-scale geospatial data processing system that enables city analysis on an unprecedented scale. We demonstrate the system's capabilities by revisiting the urban scaling law across 21,280 cities globally, using a range of open-source datasets including road networks, nighttime light intensity, built-up areas, and population statistics. Analyzing the characteristics of 21,280 cities involves querying over half a billion geospatial data points, a task that traditional Geographic Information Systems (GIS) would take several days to process. In contrast, our cloud-based system accelerates the analysis, reducing processing time to just minutes while significantly lowering resource consumption. Our findings reveal that the urban scaling law varies across cities in under-developed, developing, and developed regions, extending the insights gained from previous studies focused on hundreds of cities. This underscores the critical importance of cloud-based big data processing for efficient, large-scale geospatial analysis. As the availability of satellite imagery and other global datasets continues to grow, the potential for scientific discovery expands exponentially. Our approach not only demonstrates how such large-scale tasks can be executed efficiently but also offers a powerful solution for data scientists and researchers working in the fields of city and geospatial science.

cs.DC↗

arXiv2024

Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent

In this paper, we introduce Hunyuan-Large, which is currently the largest open-source Transformer-based mixture of experts model, with a total of 389 billion parameters and 52 billion activation parameters, capable of handling up to 256K tokens. We conduct a thorough evaluation of Hunyuan-Large's superior performance across various benchmarks including language understanding and generation, logical reasoning, mathematical problem-solving, coding, long-context, and aggregated tasks, where it outperforms LLama3.1-70B and exhibits comparable performance when compared to the significantly larger LLama3.1-405B model. Key practice of Hunyuan-Large include large-scale synthetic data that is orders larger than in previous literature, a mixed expert routing strategy, a key-value cache compression technique, and an expert-specific learning rate strategy. Additionally, we also investigate the scaling laws and learning rate schedule of mixture of experts models, providing valuable insights and guidances for future model development and optimization. The code and checkpoints of Hunyuan-Large are released to facilitate future innovations and applications. Codes: https://github.com/Tencent/Hunyuan-Large Models: https://huggingface.co/tencent/Tencent-Hunyuan-Large

cs.CL↗

arXiv2024

Lossless KV Cache Compression to 2%

Large language models have revolutionized data processing in numerous domains, with their ability to handle extended context reasoning receiving notable recognition. To speed up inference, maintaining a key-value (KV) cache memory is essential. Nonetheless, the growing demands for KV cache memory create significant hurdles for efficient implementation. This work introduces a novel architecture, Cross-Layer Latent Attention (CLLA), aimed at compressing the KV cache to less than 2% of its original size while maintaining comparable performance levels. CLLA integrates multiple aspects of KV cache compression, including attention head/dimension reduction, layer sharing, and quantization techniques, into a cohesive framework. Our extensive experiments demonstrate that CLLA achieves lossless performance on most tasks while utilizing minimal KV cache, marking a significant advancement in practical KV cache compression.

cs.CL↗

arXiv2024

Stochastic Multi-objective Multi-trip AMR Routing Problem with Time Windows

In recent years, with the rapidly aging population, alleviating the pressure on medical staff has become a critical issue. To improve the work efficiency of medical staff and reduce the risk of infection, we consider the multi-trip autonomous mobile robot (AMR) routing problem with the stochastic environment to find the solution to minimizing the total expected operating cost and maximizing the total service quality of patients so that each route violates the vehicle capacity and the time window with only a very small probability. The travel time of AMRs is stochastic affected by the surrounding environment, the demand for each ward is unknown until the AMR reaches the ward, and the service time is linearly related to the actual demand. We develop a population-based tabu search algorithm (PTS) that combines the genetic algorithm with the tabu search algorithm to solve the problem. Extensive numerical experiments were conducted on the modified Solomon instances to show that the PTS algorithm the efficient and reveals the impacts of the confidence level on the optimal solution, providing insights for the decision-maker to devise delivery schemes that trade-off the operating cost for patient satisfaction.

math.OC↗

arXiv2024

Human-machine cooperation: optimization of drug retrieval sequencing in automated drug dispensing systems

Automated drug dispensing systems (ADDSs) are increasingly in demand in today's pharmacies, primarily driven by the growing ageing population. Recognizing the practical challenges faced by pharmacies implementing ADDSs, this study aims to optimize the layout design and sequencing issues within a human-machine cooperation environment to enhance the system throughput of ADDSs. Specifically, we develop models for drug retrieval sequencing under different system layout designs, taking into account the stochastic sorting time of pharmacists. The prescription order arrival pattern follows a successive arrival mode. To assess the efficiency of ADDSs with one input/output point and two input/output points, we propose dual command retrieval sequencing models that optimize the retrieval sequence of drugs in adjacent prescription orders. Notably, our models incorporate the stochastic sorting time of pharmacists to analyze its impact on ADDS performance. Through experimental comparisons of average picking times for prescription orders under various operational conditions, we demonstrate that a system layout design incorporating two input/output points significantly enhances the efficiency of prescription order fulfilment within a human-machine cooperation environment. Furthermore, our proposed retrieval sequencing method outperforms dynamic programming, greedy, and random strategies in terms of improving prescription order-picking efficiency. By addressing the layout design and sequencing challenges, our research contributes to the field of intelligent warehousing, particularly in smart pharmacies. The findings provide valuable insights for healthcare facilities and organizations seeking to optimize ADDS performance and enhance drug dispensing efficiency.

eess.SY↗

arXiv2023

FP8-LM: Training FP8 Large Language Models

In this paper, we explore FP8 low-bit data formats for efficient training of large language models (LLMs). Our key insight is that most variables, such as gradients and optimizer states, in LLM training can employ low-precision data formats without compromising model accuracy and requiring no changes to hyper-parameters. Specifically, we propose a new FP8 automatic mixed-precision framework for training LLMs. This framework offers three levels of FP8 utilization to streamline mixed-precision and distributed parallel training for LLMs. It gradually incorporates 8-bit gradients, optimizer states, and distributed learning in an incremental manner. Experiment results show that, during the training of GPT-175B model on H100 GPU platform, our FP8 mixed-precision training framework not only achieved a remarkable 39% reduction in real memory usage but also ran 75% faster than the widely adopted BF16 framework (i.e., Megatron-LM), surpassing the speed of Nvidia Transformer Engine by 37%. This largely reduces the training costs for large foundation models. Furthermore, our FP8 mixed-precision training methodology is generic. It can be seamlessly applied to other tasks such as LLM instruction tuning and reinforcement learning with human feedback, offering savings in fine-tuning expenses. Our FP8 low-precision training framework is open-sourced at {https://github.com/Azure/MS-AMP}{aka.ms/MS.AMP}.

cs.LG↗

arXiv2023

Order picking efficiency: A scattered storage and clustered allocation strategy in automated drug dispensing systems

In the smart hospital, optimizing prescription order fulfilment processes in outpatient pharmacies is crucial. A promising device, automated drug dispensing systems (ADDSs), has emerged to streamline these processes. These systems involve human order pickers who are assisted by ADDSs. The ADDS's robotic arm transports bins from storage locations to the input/output (I/O) points, while the pharmacist sorts the requested drugs from the bins at the I/O points. This paper focuses on coordinating the ADDS and the pharmacists to optimize the order-picking strategy. Another critical aspect of order-picking systems is the storage location assignment problem (SLAP), which determines the allocation of drugs to storage locations. In this study, we consider the ADDS as a smart warehouse and propose a two-stage scattered storage and clustered allocation (SSCA) strategy to optimize the SLAP for ADDSs. The first stage primarily adopts a scattered storage approach, and we develop a mathematical programming model to group drugs accordingly. In the second stage, we introduce a sequential alternating (SA) heuristic algorithm that takes into account the drug demand frequency and the correlation between drugs to cluster and locate them effectively. To evaluate the proposed SSCA strategy, we develop a double objective integer programming model for the order-picking problem in ADDSs to minimize the number of machines visited in prescription orders while maintaining the shortest average picking time of orders. The numerical results demonstrate that the proposed strategy can optimize the SLAP in ADDSs and improve significantly the order-picking efficiency of ADDSs in a human-robot cooperation environment.

stat.AP↗

arXiv2023

A Quick Response Algorithm for Dynamic Autonomous Mobile Robot Routing Problem with Time Windows

This paper investigates the optimization problem of scheduling autonomous mobile robots (AMRs) in hospital settings, considering dynamic requests with different priorities. The primary objective is to minimize the daily service cost by dynamically planning routes for the limited number of available AMRs. The total cost consists of AMR's purchase cost, transportation cost, delay penalty cost, and loss of denial of service. To address this problem, we have established a two-stage mathematical programming model. In the first stage, a tabu search algorithm is employed to plan prior routes for all known medical requests. The second stage involves planning for real-time received dynamic requests using the efficient insertion algorithm with decision rules, which enables quick response based on the time window and demand constraints of the dynamic requests. One of the main contributions of this study is to make resource allocation decisions based on the present number of service AMRs for dynamic requests with different priorities. Computational experiments using Lackner instances demonstrate the efficient insertion algorithm with decision rules is very fast and robust in solving the dynamic AMR routing problem with time windows and request priority. Additionally, we provide managerial insights concerning the AMR's safety stock settings, which can aid in decision-making processes.

math.OC↗

arXiv2023

The Multi-Trip Autonomous Mobile Robot Scheduling Problem with Time Windows in a Stochastic Environment at Smart Hospitals

Autonomous mobile robots (AMRs) play a crucial role in transportation and service tasks at hospitals, contributing to enhanced efficiency and meeting medical demands. This paper investigates the optimization problem of scheduling strategies for AMRs at smart hospitals, where the service and travel times of AMRs are stochastic. A stochastic mixed-integer programming model is formulated to minimize the total cost of the hospital by reducing the number of AMRs and travel distance while satisfying constraints such as AMR battery state of charge, AMR capacity, and time windows for medical requests. To address this objective, some properties of the solutions with time window constraints are identified. The variable neighborhood search (VNS) algorithm is adjusted by incorporating the properties of the AMR scheduling problem to solve the model. Experimental results demonstrate that VNS generates high-quality solutions. Both enhanced efficiency and the meeting of medical demands are achieved through intelligently arranging the driving routes of AMRs for both charging and service requests, resulting in substantial cost reductions for hospitals and enhanced utilization of medical resources.

cs.RO↗

arXiv2023

Stochastic scheduling of autonomous mobile robots at hospitals

This paper studies the scheduling of autonomous mobile robots (AMRs) at hospitals where the stochastic travel times and service times of AMRs are affected by the surrounding environment. The routes of AMRs are planned to minimize the daily cost of the hospital (including the AMR fixed cost, penalty cost of violating the time window, and transportation cost). To efficiently generate high-quality solutions, some properties are identified and incorporated into an improved tabu search (I-TS) algorithm for problem-solving. Experimental evaluations demonstrate that the I-TS algorithm outperforms existing methods by producing high-quality solutions. Based on the characteristics of healthcare requests and the AMR working environment, scheduling AMRs reasonably can effectively provide medical services, improve the utilization of medical resources, and reduce hospital costs.

cs.RO↗

arXiv2023

TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance

In this paper, we propose a novel cross-modal distillation method, called TinyCLIP, for large-scale language-image pre-trained models. The method introduces two core techniques: affinity mimicking and weight inheritance. Affinity mimicking explores the interaction between modalities during distillation, enabling student models to mimic teachers' behavior of learning cross-modal feature alignment in a visual-linguistic affinity space. Weight inheritance transmits the pre-trained weights from the teacher models to their student counterparts to improve distillation efficiency. Moreover, we extend the method into a multi-stage progressive distillation to mitigate the loss of informative weights during extreme compression. Comprehensive experiments demonstrate the efficacy of TinyCLIP, showing that it can reduce the size of the pre-trained CLIP ViT-B/32 by 50%, while maintaining comparable zero-shot performance. While aiming for comparable performance, distillation with weight inheritance can speed up the training by 1.4 - 7.8 $\times$ compared to training from scratch. Moreover, our TinyCLIP ViT-8M/16, trained on YFCC-15M, achieves an impressive zero-shot top-1 accuracy of 41.1% on ImageNet, surpassing the original CLIP ViT-B/16 by 3.5% while utilizing only 8.9% parameters. Finally, we demonstrate the good transferability of TinyCLIP in various downstream tasks. Code and models will be open-sourced at https://aka.ms/tinyclip.

cs.CV↗