arXiv subjects

Yu Qi

Publications and source records attributed to Yu Qi.

35 records · Page 2Linked to original sources

arXiv2025

Human-like Cognitive Generalization for Large Models via Brain-in-the-loop Supervision

Recent advancements in deep neural networks (DNNs), particularly large-scale language models, have demonstrated remarkable capabilities in image and natural language understanding. Although scaling up model parameters with increasing volume of training data has progressively improved DNN capabilities, achieving complex cognitive abilities - such as understanding abstract concepts, reasoning, and adapting to novel scenarios, which are intrinsic to human cognition - remains a major challenge. In this study, we show that brain-in-the-loop supervised learning, utilizing a small set of brain signals, can effectively transfer human conceptual structures to DNNs, significantly enhancing their comprehension of abstract and even unseen concepts. Experimental results further indicate that the enhanced cognitive capabilities lead to substantial performance gains in challenging tasks, including few-shot/zero-shot learning and out-of-distribution recognition, while also yielding highly interpretable concept representations. These findings highlight that human-in-the-loop supervision can effectively augment the complex cognitive abilities of large models, offering a promising pathway toward developing more human-like cognitive abilities in artificial systems.

cs.LG↗

arXiv2025

MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency

Answering questions with Chain-of-Thought (CoT) has significantly enhanced the reasoning capabilities of Large Language Models (LLMs), yet its impact on Large Multimodal Models (LMMs) still lacks a systematic assessment and in-depth investigation. In this paper, we introduce MME-CoT, a specialized benchmark evaluating the CoT reasoning performance of LMMs, spanning six domains: math, science, OCR, logic, space-time, and general scenes. As the first comprehensive study in this area, we propose a thorough evaluation suite incorporating three novel metrics that assess the reasoning quality, robustness, and efficiency at a fine-grained level. Leveraging curated high-quality data and a unique evaluation strategy, we conduct an in-depth analysis of state-of-the-art LMMs, uncovering several key insights: 1) Models with reflection mechanism demonstrate a superior CoT quality, with Kimi k1.5 outperforming GPT-4o and demonstrating the highest quality results; 2) CoT prompting often degrades LMM performance on perception-heavy tasks, suggesting a potentially harmful overthinking behavior; and 3) Although the CoT quality is high, LMMs with reflection exhibit significant inefficiency in both normal response and self-correction phases. We hope MME-CoT serves as a foundation for advancing multimodal reasoning in LMMs. Project Page: https://mmecot.github.io/

cs.CV↗

arXiv2023

MindGPT: Interpreting What You See with Non-invasive Brain Recordings

Decoding of seen visual contents with non-invasive brain recordings has important scientific and practical values. Efforts have been made to recover the seen images from brain signals. However, most existing approaches cannot faithfully reflect the visual contents due to insufficient image quality or semantic mismatches. Compared with reconstructing pixel-level visual images, speaking is a more efficient and effective way to explain visual information. Here we introduce a non-invasive neural decoder, termed as MindGPT, which interprets perceived visual stimuli into natural languages from fMRI signals. Specifically, our model builds upon a visually guided neural encoder with a cross-attention mechanism, which permits us to guide latent neural representations towards a desired language semantic direction in an end-to-end manner by the collaborative use of the large language model GPT. By doing so, we found that the neural representations of the MindGPT are explainable, which can be used to evaluate the contributions of visual properties to language semantics. Our experiments show that the generated word sequences truthfully represented the visual information (with essential details) conveyed in the seen stimuli. The results also suggested that with respect to language decoding tasks, the higher visual cortex (HVC) is more semantically informative than the lower visual cortex (LVC), and using only the HVC can recover most of the semantic information. The code of the MindGPT model will be publicly available at https://github.com/JxuanC/MindGPT.

cs.CV↗

arXiv2023

NeuSort: An Automatic Adaptive Spike Sorting Approach with Neuromorphic Models

Objective. Spike sorting, a critical step in neural data processing, aims to classify spiking events from single electrode recordings based on different waveforms. This study aims to develop a novel online spike sorter, NeuSort, using neuromorphic models, with the ability to adaptively adjust to changes in neural signals, including waveform deformations and the appearance of new neurons. Approach. NeuSort leverages a neuromorphic model to emulate template-matching processes. This model incorporates plasticity learning mechanisms inspired by biological neural systems, facilitating real-time adjustments to online parameters. Results. Experimental findings demonstrate NeuSort's ability to track neuron activities amidst waveform deformations and identify new neurons in real-time. NeuSort excels in handling non-stationary neural signals, significantly enhancing its applicability for long-term spike sorting tasks. Moreover, its implementation on neuromorphic chips guarantees ultra-low energy consumption during computation. Significance. NeuSort caters to the demand for real-time spike sorting in brain-machine interfaces through a neuromorphic approach. Its unsupervised, automated spike sorting process makes it a plug-and-play solution for online spike sorting.

cs.NE↗

arXiv2023

STGIN: Spatial-Temporal Graph Interaction Network for Large-scale POI Recommendation

In Location-Based Services, Point-Of-Interest(POI) recommendation plays a crucial role in both user experience and business opportunities. Graph neural networks have been proven effective in providing personalized POI recommendation services. However, there are still two critical challenges. First, existing graph models attempt to capture users' diversified interests through a unified graph, which limits their ability to express interests in various spatial-temporal contexts. Second, the efficiency limitations of graph construction and graph sampling in large-scale systems make it difficult to adapt quickly to new real-time interests. To tackle the above challenges, we propose a novel Spatial-Temporal Graph Interaction Network. Specifically, we construct subgraphs of spatial, temporal, spatial-temporal, and global views respectively to precisely characterize the user's interests in various contexts. In addition, we design an industry-friendly framework to track the user's latest interests. Extensive experiments on the real-world dataset show that our method outperforms state-of-the-art models. This work has been successfully deployed in a large e-commerce platform, delivering a 1.1% CTR and 6.3% RPM improvement.

cs.IR↗

arXiv2023

A Human-Machine Joint Learning Framework to Boost Endogenous BCI Training

Brain-computer interfaces (BCIs) provide a direct pathway from the brain to external devices and have demonstrated great potential for assistive and rehabilitation technologies. Endogenous BCIs based on electroencephalogram (EEG) signals, such as motor imagery (MI) BCIs, can provide some level of control. However, mastering spontaneous BCI control requires the users to generate discriminative and stable brain signal patterns by imagery, which is challenging and is usually achieved over a very long training time (weeks/months). Here, we propose a human-machine joint learning framework to boost the learning process in endogenous BCIs, by guiding the user to generate brain signals towards an optimal distribution estimated by the decoder, given the historical brain signals of the user. To this end, we firstly model the human-machine joint learning process in a uniform formulation. Then a human-machine joint learning framework is proposed: 1) for the human side, we model the learning process in a sequential trial-and-error scenario and propose a novel ``copy/new'' feedback paradigm to help shape the signal generation of the subject toward the optimal distribution; 2) for the machine side, we propose a novel adaptive learning algorithm to learn an optimal signal distribution along with the subject's learning process. Specifically, the decoder reweighs the brain signals generated by the subject to focus more on ``good'' samples to cope with the learning process of the subject. Online and psuedo-online BCI experiments with 18 healthy subjects demonstrated the advantages of the proposed joint learning process over co-adaptive approaches in both learning efficiency and effectiveness.

cs.HC↗

arXiv2023

Decoding Chinese phonemes from intracortical brain signals with hyperbolic-space neural representations

Speech brain-computer interfaces (BCIs), which translate brain signals into spoken words or sentences, have shown significant potential for high-performance BCI communication. Phonemes are the fundamental units of pronunciation in most languages. While existing speech BCIs have largely focused on English, where words contain diverse compositions of phonemes, Chinese Mandarin is a monosyllabic language, with words typically consisting of a consonant and a vowel. This feature makes it feasible to develop high-performance Mandarin speech BCIs by decoding phonemes directly from neural signals. This study aimed to decode spoken Mandarin phonemes using intracortical neural signals. We observed that phonemes with similar pronunciations were often represented by inseparable neural patterns, leading to confusion in phoneme decoding. This finding suggests that the neural representation of spoken phonemes has a hierarchical structure. To account for this, we proposed learning the neural representation of phoneme pronunciation in a hyperbolic space, where the hierarchical structure could be more naturally optimized. Experiments with intracortical neural signals from a Chinese participant showed that the proposed model learned discriminative and interpretable hierarchical phoneme representations from neural signals, significantly improving Chinese phoneme decoding performance and achieving state-of-the-art. The findings demonstrate the feasibility of constructing high-performance Chinese speech BCIs based on phoneme decoding.

cs.HC↗

arXiv2023

LaSNN: Layer-wise ANN-to-SNN Distillation for Effective and Efficient Training in Deep Spiking Neural Networks

Spiking Neural Networks (SNNs) are biologically realistic and practically promising in low-power computation because of their event-driven mechanism. Usually, the training of SNNs suffers accuracy loss on various tasks, yielding an inferior performance compared with ANNs. A conversion scheme is proposed to obtain competitive accuracy by mapping trained ANNs' parameters to SNNs with the same structures. However, an enormous number of time steps are required for these converted SNNs, thus losing the energy-efficient benefit. Utilizing both the accuracy advantages of ANNs and the computing efficiency of SNNs, a novel SNN training framework is proposed, namely layer-wise ANN-to-SNN knowledge distillation (LaSNN). In order to achieve competitive accuracy and reduced inference latency, LaSNN transfers the learning from a well-trained ANN to a small SNN by distilling the knowledge other than converting the parameters of ANN. The information gap between heterogeneous ANN and SNN is bridged by introducing the attention scheme, the knowledge in an ANN is effectively compressed and then efficiently transferred by utilizing our layer-wise distillation paradigm. We conduct detailed experiments to demonstrate the effectiveness, efficacy, and scalability of LaSNN on three benchmark data sets (CIFAR-10, CIFAR-100, and Tiny ImageNet). We achieve competitive top-1 accuracy compared to ANNs and 20x faster inference than converted SNNs with similar performance. More importantly, LaSNN is dexterous and extensible that can be effortlessly developed for SNNs with different architectures/depths and input encoding methods, contributing to their potential development.

cs.NE↗

arXiv2022

Exploring Stochastic Autoregressive Image Modeling for Visual Representation

Autoregressive language modeling (ALM) have been successfully used in self-supervised pre-training in Natural language processing (NLP). However, this paradigm has not achieved comparable results with other self-supervised approach in computer vision (e.g., contrastive learning, mask image modeling). In this paper, we try to find the reason why autoregressive modeling does not work well on vision tasks. To tackle this problem, we fully analyze the limitation of visual autoregressive methods and proposed a novel stochastic autoregressive image modeling (named SAIM) by the two simple designs. First, we employ stochastic permutation strategy to generate effective and robust image context which is critical for vision tasks. Second, we create a parallel encoder-decoder training process in which the encoder serves a similar role to the standard vision transformer focus on learning the whole contextual information, and meanwhile the decoder predicts the content of the current position, so that the encoder and decoder can reinforce each other. By introducing stochastic prediction and the parallel encoder-decoder, SAIM significantly improve the performance of autoregressive image modeling. Our method achieves the best accuracy (83.9%) on the vanilla ViT-Base model among methods using only ImageNet-1K data. Transfer performance in downstream tasks also show that our model achieves competitive performance.

cs.CV↗

arXiv2022

Benchmarking Unsupervised Anomaly Detection and Localization

Unsupervised anomaly detection and localization, as of one the most practical and challenging problems in computer vision, has received great attention in recent years. From the time the MVTec AD dataset was proposed to the present, new research methods that are constantly being proposed push its precision to saturation. It is the time to conduct a comprehensive comparison of existing methods to inspire further research. This paper extensively compares 13 papers in terms of the performance in unsupervised anomaly detection and localization tasks, and adds a comparison of inference efficiency previously ignored by the community. Meanwhile, analysis of the MVTec AD dataset are also given, especially the label ambiguity that affects the model fails to achieve full marks. Moreover, considering the proposal of the new MVTec 3D-AD dataset, this paper also conducts experiments using the existing state-of-the-art 2D methods on this new dataset, and reports the corresponding results with analysis.

cs.CV↗

arXiv2022

Dynamic Ensemble Bayesian Filter for Robust Control of a Human Brain-machine Interface

Objective: Brain-machine interfaces (BMIs) aim to provide direct brain control of devices such as prostheses and computer cursors, which have demonstrated great potential for mobility restoration. One major limitation of current BMIs lies in the unstable performance in online control due to the variability of neural signals, which seriously hinders the clinical availability of BMIs. Method: To deal with the neural variability in online BMI control, we propose a dynamic ensemble Bayesian filter (DyEnsemble). DyEnsemble extends Bayesian filters with a dynamic measurement model, which adjusts its parameters in time adaptively with neural changes. This is achieved by learning a pool of candidate functions and dynamically weighting and assembling them according to neural signals. In this way, DyEnsemble copes with variability in signals and improves the robustness of online control. Results: Online BMI experiments with a human participant demonstrate that, compared with the velocity Kalman filter, DyEnsemble significantly improves the control accuracy (increases the success rate by 13.9% and reduces the reach time by 13.5% in the random target pursuit task) and robustness (performs more stably over different experiment days). Conclusion: Our results demonstrate the superiority of DyEnsemble in online BMI control. Significance: DyEnsemble frames a novel and flexible framework for robust neural decoding, which is beneficial to different neural decoding applications.

cs.LG↗

arXiv2021

Study on the simulation control of neural network algorithm in thermally coupled distillation

Thermally coupled distillation is a new energy-saving method, but the traditional thermally coupled distillation simulation calculation process is complicated, and the optimization method based on the traditional simulation process is difficult to obtain a good feasible solution. The neural network algorithm has the advantages of fast learning and can approach nonlinear functions arbitrarily. For the problems in complex process control systems, neural network control does not require cumbersome control structures or precise mathematical models. When training the network, only the input and output samples it needs are given, so that the dynamics of the system can be controlled. Performance is approaching. This method can effectively solve the mathematical model of the thermally coupled distillation process, and quickly obtain the solution of the optimized variables and the objective function. This article summarizes the research progress of artificial neural network and the optimization control of thermally coupled distillation and the application of neural network in thermally coupled distillation.

physics.chem-ph↗

arXiv2021

Reconstructing Perceptive Images from Brain Activity by Shape-Semantic GAN

Reconstructing seeing images from fMRI recordings is an absorbing research area in neuroscience and provides a potential brain-reading technology. The challenge lies in that visual encoding in brain is highly complex and not fully revealed. Inspired by the theory that visual features are hierarchically represented in cortex, we propose to break the complex visual signals into multi-level components and decode each component separately. Specifically, we decode shape and semantic representations from the lower and higher visual cortex respectively, and merge the shape and semantic information to images by a generative adversarial network (Shape-Semantic GAN). This 'divide and conquer' strategy captures visual information more accurately. Experiments demonstrate that Shape-Semantic GAN improves the reconstruction similarity and image quality, and achieves the state-of-the-art image reconstruction performance.

cs.NE↗

arXiv2020

A review of neural network algorithms and their applications in supercritical extraction

Neural network realizes multi-parameter optimization and control by simulating certain mechanisms of the human brain. It can be used in many fields such as signal processing, intelligent driving, optimal combination, vehicle abnormality detection, and chemical process optimization control. Supercritical extraction is a new type of high-efficiency chemical separation process, which is mainly used in the separation and purification of natural substances. There are many influencing factors. The neural network model can quickly optimize the process parameters and predict the experimental results under different process conditions. It is helpful to understand the inner law of the experiment and determine the optimal experimental conditions. This paper briefly describes the basic concepts and research progress of neural networks and supercritical extraction, and summarizes the application of neural network algorithms in supercritical extraction, aiming to provide reference for the development and innovation of industry technology.

cs.NE↗

arXiv2020

Sequential online prediction in the presence of outliers and change points: an instant temporal structure learning approach

In this paper, we consider sequential online prediction (SOP) for streaming data in the presence of outliers and change points. We propose an INstant TEmporal structure Learning (INTEL) algorithm to address this problem. Our INTEL algorithm is developed based on a full consideration of the duality between online prediction and anomaly detection. We first employ a mixture of weighted Gaussian process models (WGPs) to cover the expected possible temporal structures of the data. Then, based on the rich modeling capacity of this WGP mixture, we develop an efficient technique to instantly learn (capture) the temporal structure of the data that follows a regime shift. This instant learning is achieved only by adjusting one hyper-parameter value of the mixture model. A weighted generalization of the product of experts (POE) model is used for fusing predictions yielded from multiple GP models. An outlier is declared once a real observation seriously deviates from the fused prediction. If a certain number of outliers are consecutively declared, then a change point is declared. Extensive experiments are performed using a diverse of real datasets. Results show that the proposed algorithm is significantly better than benchmark methods for SOP in the presence of outliers and change points.

cs.LG↗

arXiv2019

Dynamic Ensemble Modeling Approach to Nonstationary Neural Decoding in Brain-Computer Interfaces

Brain-computer interfaces (BCIs) have enabled prosthetic device control by decoding motor movements from neural activities. Neural signals recorded from cortex exhibit nonstationary property due to abrupt noises and neuroplastic changes in brain activities during motor control. Current state-of-the-art neural signal decoders such as Kalman filter assume fixed relationship between neural activities and motor movements, thus will fail if this assumption is not satisfied. We propose a dynamic ensemble modeling (DyEnsemble) approach that is capable of adapting to changes in neural signals by employing a proper combination of decoding functions. The DyEnsemble method firstly learns a set of diverse candidate models. Then, it dynamically selects and combines these models online according to Bayesian updating mechanism. Our method can mitigate the effect of noises and cope with different task behaviors by automatic model switching, thus gives more accurate predictions. Experiments with neural data demonstrate that the DyEnsemble method outperforms Kalman filters remarkably, and its advantage is more obvious with noisy signals.

eess.SP↗

arXiv2010

Signatures of magnetic reconnection at boundaries of interplanetary small-scale magnetic flux ropes

The interaction between interplanetary small-scale magnetic flux ropes and the magnetic field in the ambient solar wind is an important topic to understand- ing the evolution of magnetic structures in the heliosphere. Through a survey of 125 previously reported small flux ropes from 1995 to 2005, we find that 44 of them reveal clear signatures of Alfvenic fluctuations, and thus classify them into Alfven wave trains rather than flux ropes. Signatures of magnetic reconnection, generally including a plasma jet of ~30 km/s within a magnetic field rotational region, are clearly present at boundaries of about 42% of the flux ropes and 14% of the wave trains. The reconnection exhausts are often observed to show a local increase in the proton temperature, density and plasma beta. About 66% of the reconnection events at flux rope boundaries are associated with a magnetic field shear angle larger than 90 deg and 73% of them reveal a decrease by 20% or more in the magnetic field magnitude, suggesting a dominance of anti-parallel reconnec- tion at flux rope boundaries. The occurrence rate of magnetic reconnection at flux rope boundaries through the year of 1995 to 2005 is also investigated and we find that it is relatively low around solar maximum and much higher when ap- proaching solar minima. The average magnetic field depression and shear angle for reconnection events at flux rope boundaries also reveal a similar trend from 1995 to 2005. Our results demonstrate for the first time that boundaries of a substantial fraction of small-scale flux ropes have properties similar to those of magnetic clouds, in the sense that both of them exhibit signatures of magnetic reconnection. The observed reconnection signatures could be related either to the formation of small flux ropes, or to the interaction between flux ropes and the interplanetary magnetic fields.

astro-ph.SR↗