arXiv subjects

Yi Hu

Publications and source records attributed to Yi Hu.

At least 73 records · Page 4Linked to original sources

arXiv2025

Tin-Tin: Towards Tiny Learning on Tiny Devices with Integer-based Neural Network Training

Recent advancements in machine learning (ML) have enabled its deployment on resource-constrained edge devices, fostering innovative applications such as intelligent environmental sensing. However, these devices, particularly microcontrollers (MCUs), face substantial challenges due to limited memory, computing capabilities, and the absence of dedicated floating-point units (FPUs). These constraints hinder the deployment of complex ML models, especially those requiring lifelong learning capabilities. To address these challenges, we propose Tin-Tin, an integer-based on-device training framework designed specifically for low-power MCUs. Tin-Tin introduces novel integer rescaling techniques to efficiently manage dynamic ranges and facilitate efficient weight updates using integer data types. Unlike existing methods optimized for devices with FPUs, GPUs, or FPGAs, Tin-Tin addresses the unique demands of tiny MCUs, prioritizing energy efficiency and optimized memory utilization. We validate the effectiveness of Tin-Tin through end-to-end application examples on real-world tiny devices, demonstrating its potential to support energy-efficient and sustainable ML applications on edge platforms.

cs.LG↗

arXiv2025

The Mini-SiTian Array: Evaluation Camera System

The Mini-SiTian project, which is the pathfinder for the SiTian project, utilizes three 30 cm telescopes equipped with commercial CMOS cameras (ZWO ASI6200MM Pro) to simulate large-area time-domain survey. Due to the avoidance of the traditional mechanical shutter, the CMOS camera is favorable in time-domain survey projects. In the future, the SiTian telescope array will employ a two-by-two scientific-grade mosaic CMOS camera to survey a 10,000-degree square area every 30 minutes. Therefore, the performance of CMOS directly determines the detection capability of SiTian telescopes for transient sources, and a comprehensive understanding of the performance of CMOS cameras is crucial. In this research, laboratory testing was conducted to thoroughly evaluate three cameras by assessing several critical parameters, including bias stability, dark current, pixel anomalies, linearity, gain, and read noise. We find exceptional short-term bias stability with standard deviations below 0.02 ADU, negligible dark current of approximately 0.002 e$^{-}$ pixel$^{-1}$ s$^{-1}$ at $0^\circ\text{C}$, and excellent linearity with nonlinearity consistently below $\pm$ 0.5\%, and a small proportion (0.06\% to 0.08\%) of pixels with anomalous responses. Furthermore, our analysis demonstrates uniform gain values across all cameras, ranging from 0.252 to 0.255 e$^{-}$ ADU$^{-1}$, with low readout noise, measured to be below 1.6 e$^{-}$ using conventional methods. We also propose a novel method for pixel-level gain and read noise calculation for CMOS sensors, which revealed a narrow gain distribution and a low median read noise of 1.028 e$^-$ for one of the cameras. The laboratory testing of the ZWO ASI6200MM Pro cameras indicates their potential to meet the requirements of time-domain surveys for the Mini-SiTian project.

astro-ph.IM↗

arXiv2025

The Mini-SiTian Array: Design and application of Master Control System

The SiTian Project represents a groundbreaking initiative in astronomy, aiming to deploy a global network of telescopes, each with a 1-meter aperture, for comprehensive time-domain sky surveys. The network's innovative architecture features multiple observational nodes, each comprising three strategically aligned telescopes equipped with filters. This design enables three-color (g, r, i) channel imaging within each node, facilitating precise and coordinated observations. As a pathfinder to the full-scale project, the Mini-SiTian Project serves as the scientific and technological validation platform, utilizing three 30-centimeter aperture telescopes to validate the methodologies and technologies planned for the broader SiTian network. This paper focuses on the development and implementation of the Master Control System (MCS),and the central command hub for the Mini-SiTian array. The MCS is designed to facilitate seamless communication with the SiTian Brain, the project's central processing and decision-making unit, while ensuring accurate task allocation, real-time status monitoring, and optimized observational workflows. The system adopts a robust architecture that separates front-end and back-end functionalities.A key innovation of the MCS is its ability to dynamically adjust observation plans in response to transient source alerts, enabling rapid and coordinated scans of target sky regions...(abridged)

astro-ph.IM↗

arXiv2025

Machine Learning Analysis of Anomalous Diffusion

The rapid advancements in machine learning have made its application to anomalous diffusion analysis both essential and inevitable. This review systematically introduces the integration of machine learning techniques for enhanced analysis of anomalous diffusion, focusing on two pivotal aspects: single trajectory characterization via machine learning and representation learning of anomalous diffusion. We extensively compare various machine learning methods, including both classical machine learning and deep learning, used for the inference of diffusion parameters and trajectory segmentation. Additionally, platforms such as the Anomalous Diffusion Challenge that serve as benchmarks for evaluating these methods are highlighted. On the other hand, we outline three primary strategies for representing anomalous diffusion: the combination of predefined features, the feature vector from the penultimate layer of neural network, and the latent representation from the autoencoder, analyzing their applicability across various scenarios. This investigation paves the way for future research, offering valuable perspectives that can further enrich the study of anomalous diffusion and advance the application of artificial intelligence in statistical physics and biophysics.

cs.LG↗

arXiv2025

Number Cookbook: Number Understanding of Language Models and How to Improve It

Large language models (LLMs) can solve an increasing number of complex reasoning tasks while making surprising mistakes in basic numerical understanding and processing (such as 9.11 > 9.9). The latter ability is essential for tackling complex arithmetic and mathematical problems and serves as a foundation for most reasoning tasks, but previous work paid little attention to it or only discussed several restricted tasks (like integer addition). In this paper, we comprehensively investigate the numerical understanding and processing ability (NUPA) of LLMs. Firstly, we introduce a benchmark covering four common numerical representations and 17 distinct numerical tasks in four major categories, resulting in 41 meaningful combinations in total. These tasks are derived from primary and secondary education curricula, encompassing nearly all everyday numerical understanding and processing scenarios, and the rules of these tasks are very simple and clear. Through the benchmark, we find that current LLMs fail frequently in many of the tasks. To study the problem, we train small models with existing and potential techniques for enhancing NUPA (such as tokenizers, PEs, and number formats), comprehensively evaluating their effectiveness using our testbed. We also finetune practical-scale LLMs on our proposed NUPA tasks and find that 1) naive finetuning can improve NUPA a lot on many but not all tasks, and 2) surprisingly, techniques designed to enhance NUPA prove ineffective for finetuning pretrained models. We further explore the impact of chain-of-thought techniques on NUPA. Our work provides a more detailed and comprehensive understanding of NUPA in LLMs. Our benchmark and code are released at https://github.com/GraphPKU/number_cookbook.

cs.CL↗

arXiv2025

RedStar: Does Scaling Long-CoT Data Unlock Better Slow-Reasoning Systems?

Can scaling transform reasoning? In this work, we explore the untapped potential of scaling Long Chain-of-Thought (Long-CoT) data to 1000k samples, pioneering the development of a slow-thinking model, RedStar. Through extensive experiments with various LLMs and different sizes, we uncover the ingredients for specialization and scale for Long-CoT training. Surprisingly, even smaller models show significant performance gains with limited data, revealing the sample efficiency of Long-CoT and the critical role of sample difficulty in the learning process. Our findings demonstrate that Long-CoT reasoning can be effectively triggered with just a few thousand examples, while larger models achieve unparalleled improvements. We also introduce reinforcement learning (RL)-scale training as a promising direction for advancing slow-thinking systems. RedStar shines across domains: on the MATH-Hard benchmark, RedStar-code-math boosts performance from 66.2\% to 81.6\%, and on the USA Math Olympiad (AIME), it solves 46.7\% of problems using only 21k mixed-code-math datasets. In multimodal tasks like GeoQA and MathVista-GEO, RedStar-Geo achieves competitive results with minimal Long-CoT data, outperforming other slow-thinking systems like QvQ-Preview. Compared to QwQ, RedStar strikes the perfect balance between reasoning and generalizability. Our work highlights that, with careful tuning, scaling Long-CoT can unlock extraordinary reasoning capabilities-even with limited dataset and set a new standard for slow-thinking models across diverse challenges. Our data and models are released at https://huggingface.co/RedStar-Reasoning.

cs.LG↗

arXiv2024

Secondary Grain Boundary Dislocations Alter Segregation Energy Spectra

Grain boundaries (GBs) trigger structure-specific chemical segregation of solute atoms. According to the three-dimensional (3D) topology of grains, GBs - although defined as planar defects - cannot be free of curvature. This implies formation of topologically-necessary arrays of secondary GB dislocations. We report here that these pattern-forming secondary GB dislocations can have an additional and, in some cases, even a much stronger effect on GB segregation than defect-free GBs. Using nanoscale correlative tomography combining crystallography and chemical analysis, we quantified the relationship between secondary GB dislocations and their segregation energy spectra for a model Fe-W alloy. This discovery unlocks new design opportunities for advanced materials, leveraging the additional degrees of freedom provided by topologically-necessary secondary GB dislocations to modulate segregation.

cond-mat.mtrl-sci↗

arXiv2024

GRRIS: a real-time intra-site observation scheduling scheme for distributed survey telescope arrays

The distributed telescope array offers promise for conducting large-sky-area, high-frequency time domain surveys. Multiple telescopes can be deployed at each observation site, so intra-site observation task scheduling is crucial for enhancing observation efficiency and quality. Efficient use of observable time and rapid response to special situations are critical to maximize scientific discovery in time domain surveys. Besides, the competing scientific priorities, time-varying observation conditions, and capabilities of observation equipment, lead to a vast search space of the scheduling. So with the increasing number of telescopes and observation fields, balancing computational time with solution quality in observation scheduling poses a significant challenge. Informed by the seminal contributions of earlier studies on a multilevel scheduling model and global scheduler for time domain telescope array, this study is devoted to further exploring the site scheduler. Formulating the observation scheduling of multiple telescopes at the site as a cooperative decision-making problem, this paper proposes GRRIS, a real-time intra-site observation scheduling scheme for telescope array using graph and reinforcement learning. It employs a graph neural network to learn node features that can embed the spatial structure of the observation scheduling. An algorithm based on multi-agent reinforcement learning is designed to efficiently learn the optimum allocation policy of telescope agents to field nodes. Through numerical simulations with real-world scenarios, GRRIS can achieve up to a 22% solution improvement over the most competitive scheme. It offers better scalability and sub-second decision speed, meeting the needs of observation scheduling control for future distributed telescope arrays.

astro-ph.IM↗

arXiv2024

EXAONEPath 1.0 Patch-level Foundation Model for Pathology

Recent advancements in digital pathology have led to the development of numerous foundational models that utilize self-supervised learning on patches extracted from gigapixel whole slide images (WSIs). While this approach leverages vast amounts of unlabeled data, we have discovered a significant issue: features extracted from these self-supervised models tend to cluster by individual WSIs, a phenomenon we term WSI-specific feature collapse. This problem can potentially limit the model's generalization ability and performance on various downstream tasks. To address this issue, we introduce EXAONEPath, a novel foundational model trained on patches that have undergone stain normalization. Stain normalization helps reduce color variability arising from different laboratories and scanners, enabling the model to learn more consistent features. EXAONEPath is trained using 285,153,903 patches extracted from a total of 34,795 WSIs. Our experiments demonstrate that EXAONEPath significantly mitigates the feature collapse problem, indicating that the model has learned more generalized features rather than overfitting to individual WSI characteristics. We compared EXAONEPath with state-of-the-art models across six downstream task datasets, and our results show that EXAONEPath achieves superior performance relative to the number of WSIs used and the model's parameter count. This suggests that the application of stain normalization has substantially improved the model's efficiency and generalization capabilities.

cs.LG↗

arXiv2024

Simple fluctuations in simple glass formers

Critical single-particle fluctuations associated with particle displacements are inherent to simple glass-forming liquids in the limit of large dimensions and leave a pseudo-critical trace across all finite dimensions. This characteristic could serve as a crucial test for distinguishing between theories of glass formation. We here examine these critical fluctuations, as captured by the well-established non-Gaussian parameter, within both mode-coupling theory (MCT) and dynamical mean-field theory (DMFT) across dimensions for hard sphere liquids and for the minimally structured Mari--Kurchan model. We establish general scaling laws relevant to any liquid dynamics theory in large dimensions and show that the dimensional scalings predicted by MCT are inconsistent with those from DMFT. Simulation results for hard spheres in moderately high dimensions align with the DMFT scenario, reinforcing the relevance of mean-field theory for capturing glass physics in finite dimensions. We identify potential adjustments to MCT to account for certain mean-field physics. Our findings also highlight that local structure and spatial dimensionality can affect single-particle critical fluctuations in non-trivial ways.

cond-mat.dis-nn↗

arXiv2024

Case-Based or Rule-Based: How Do Transformers Do the Math?

Despite the impressive performance in a variety of complex tasks, modern large language models (LLMs) still have trouble dealing with some math problems that are simple and intuitive for humans, such as addition. While we can easily learn basic rules of addition and apply them to new problems of any length, LLMs struggle to do the same. Instead, they may rely on similar cases seen in the training corpus for help. We define these two different reasoning mechanisms as "rule-based reasoning" and "case-based reasoning". Since rule-based reasoning is essential for acquiring systematic generalization ability, we aim to explore exactly whether transformers use rule-based or case-based reasoning for math problems. Through carefully designed intervention experiments on five math tasks, we confirm that transformers are performing case-based reasoning, no matter whether scratchpad is used, which aligns with the previous observations that transformers use subgraph matching/shortcut learning to reason. To mitigate such problems, we propose a Rule-Following Fine-Tuning (RFFT) technique to teach transformers to perform rule-based reasoning. Specifically, we provide explicit rules in the input and then instruct transformers to recite and follow the rules step by step. Through RFFT, we successfully enable LLMs fine-tuned on 1-5 digit addition to generalize to up to 12-digit addition with over 95% accuracy, which is over 40% higher than scratchpad. The significant improvement demonstrates that teaching LLMs to use rules explicitly helps them learn rule-based reasoning and generalize better in length.

cs.AI↗

arXiv2024

Applying Fine-Tuned LLMs for Reducing Data Needs in Load Profile Analysis

This paper presents a novel method for utilizing fine-tuned Large Language Models (LLMs) to minimize data requirements in load profile analysis, demonstrated through the restoration of missing data in power system load profiles. A two-stage fine-tuning strategy is proposed to adapt a pre-trained LLMs, i.e., GPT-3.5, for missing data restoration tasks. Through empirical evaluation, we demonstrate the effectiveness of the fine-tuned model in accurately restoring missing data, achieving comparable performance to state-of-the-art specifically designed models such as BERT-PIN. Key findings include the importance of prompt engineering and the optimal utilization of fine-tuning samples, highlighting the efficiency of few-shot learning in transferring knowledge from general user cases to specific target users. Furthermore, the proposed approach demonstrates notable cost-effectiveness and time efficiency compared to training models from scratch, making it a practical solution for scenarios with limited data availability and computing resources. This research has significant potential for application to other power system load profile analysis tasks. Consequently, it advances the use of LLMs in power system analytics, offering promising implications for enhancing the resilience and efficiency of power distribution systems.

cs.LG↗

arXiv2024

Possible spin-polarized Cooper pairing in high temperature FeSe superconductor

Superconductivity and long-range ferromagnetism hardly coexist in a uniform manner. The counter-example has been observed, in uranium-based superconductors for instance, with a coexisting temperature limited to about 1 K. Here, we report the coexistence of high temperature superconductivity and itinerant ferromagnetism in lithium intercalated FeSe flakes. In superconducting samples with transition temperature around 40 K, we observe the anomalous Hall effect with a hysteresis loop in transverse resistivity and a butterfly-like pattern of magneto-resistance. Intriguingly, such ferromagnetism persists down to a temperature at which the zero-field resistance fully vanishes. Furthermore, the superconductivity is enhanced under an in-plane magnetic field, suggestive of the participation of spin-polarized Cooper pairs. The surprising finding underscores a uniform coexistence of the two antagonistic phenomena on a record-high energy scale.

cond-mat.supr-con↗

arXiv2024

InstructEdit: Instruction-based Knowledge Editing for Large Language Models

Knowledge editing for large language models can offer an efficient solution to alter a model's behavior without negatively impacting the overall performance. However, the current approaches encounter issues with limited generalizability across tasks, necessitating one distinct editor for each task, significantly hindering the broader applications. To address this, we take the first step to analyze the multi-task generalization issue in knowledge editing. Specifically, we develop an instruction-based editing technique, termed InstructEdit, which facilitates the editor's adaptation to various task performances simultaneously using simple instructions. With only one unified editor for each LLM, we empirically demonstrate that InstructEdit can improve the editor's control, leading to an average 14.86% increase in Reliability in multi-task editing setting. Furthermore, experiments involving holdout unseen task illustrate that InstructEdit consistently surpass previous strong baselines. To further investigate the underlying mechanisms of instruction-based knowledge editing, we analyze the principal components of the editing gradient directions, which unveils that instructions can help control optimization direction with stronger OOD generalization. Code and datasets are available in https://github.com/zjunlp/EasyEdit.

cs.CL↗

arXiv2024

An Element-Wise Weights Aggregation Method for Federated Learning

Federated learning (FL) is a powerful Machine Learning (ML) paradigm that enables distributed clients to collaboratively learn a shared global model while keeping the data on the original device, thereby preserving privacy. A central challenge in FL is the effective aggregation of local model weights from disparate and potentially unbalanced participating clients. Existing methods often treat each client indiscriminately, applying a single proportion to the entire local model. However, it is empirically advantageous for each weight to be assigned a specific proportion. This paper introduces an innovative Element-Wise Weights Aggregation Method for Federated Learning (EWWA-FL) aimed at optimizing learning performance and accelerating convergence speed. Unlike traditional FL approaches, EWWA-FL aggregates local weights to the global model at the level of individual elements, thereby allowing each participating client to make element-wise contributions to the learning process. By taking into account the unique dataset characteristics of each client, EWWA-FL enhances the robustness of the global model to different datasets while also achieving rapid convergence. The method is flexible enough to employ various weighting strategies. Through comprehensive experiments, we demonstrate the advanced capabilities of EWWA-FL, showing significant improvements in both accuracy and convergence speed across a range of backbones and benchmarks.

cs.LG↗

arXiv2024

A Novel Vision Transformer based Load Profile Analysis using Load Images as Inputs

This paper introduces ViT4LPA, an innovative Vision Transformer (ViT) based approach for Load Profile Analysis (LPA). We transform time-series load profiles into load images. This allows us to leverage the ViT architecture, originally designed for image processing, as a pre-trained image encoder to uncover latent patterns within load data. ViT is pre-trained using an extensive load image dataset, comprising 1M load images derived from smart meter data collected over a two-year period from 2,000 residential users. The training methodology is self-supervised, masked image modeling, wherein masked load images are restored to reveal hidden relationships among image patches. The pre-trained ViT encoder is then applied to various downstream tasks, including the identification of electric vehicle (EV) charging loads and behind-the-meter solar photovoltaic (PV) systems and load disaggregation. Simulation results illustrate ViT4LPA's superior performance compared to existing neural network models in downstream tasks. Additionally, we conduct an in-depth analysis of the attention weights within the ViT4LPA model to gain insights into its information flow mechanisms.

eess.SY↗

arXiv2024

CoRAST: Towards Foundation Model-Powered Correlated Data Analysis in Resource-Constrained CPS and IoT

Foundation models (FMs) emerge as a promising solution to harness distributed and diverse environmental data by leveraging prior knowledge to understand the complicated temporal and spatial correlations within heterogeneous datasets. Unlike distributed learning frameworks such as federated learning, which often struggle with multimodal data, FMs can transform diverse inputs into embeddings. This process facilitates the integration of information from various modalities and the application of prior learning to new domains. However, deploying FMs in resource-constrained edge systems poses significant challenges. To this end, we introduce CoRAST, a novel learning framework that utilizes FMs for enhanced analysis of distributed, correlated heterogeneous data. Utilizing a server-based FM, CoRAST can exploit existing environment information to extract temporal, spatial, and cross-modal correlations among sensor data. This enables CoRAST to offer context-aware insights for localized client tasks through FM-powered global representation learning. Our evaluation on real-world weather dataset demonstrates CoRAST's ability to exploit correlated heterogeneous data through environmental representation learning to reduce the forecast errors by up to 50.3% compared to the baselines.

cs.LG↗

arXiv2023

Is ChatGPT a Good Personality Recognizer? A Preliminary Study

In recent years, personality has been regarded as a valuable personal factor being incorporated into numerous tasks such as sentiment analysis and product recommendation. This has led to widespread attention to text-based personality recognition task, which aims to identify an individual's personality based on given text. Considering that ChatGPT has recently exhibited remarkable abilities on various natural language processing tasks, we provide a preliminary evaluation of ChatGPT on text-based personality recognition task for generating effective personality data. Concretely, we employ a variety of prompting strategies to explore ChatGPT's ability in recognizing personality from given text, especially the level-oriented prompting strategy we designed for guiding ChatGPT in analyzing given text at a specified level. The experimental results on two representative real-world datasets reveal that ChatGPT with zero-shot chain-of-thought prompting exhibits impressive personality recognition ability and is capable to provide natural language explanations through text-based logical reasoning. Furthermore, by employing the level-oriented prompting strategy to optimize zero-shot chain-of-thought prompting, the performance gap between ChatGPT and corresponding state-of-the-art model has been narrowed even more. However, we observe that ChatGPT shows unfairness towards certain sensitive demographic attributes such as gender and age. Additionally, we discover that eliciting the personality recognition ability of ChatGPT helps improve its performance on personality-related downstream tasks such as sentiment classification and stress prediction.

cs.CL↗