arXiv subjects

Yunji Wang

Publications and source records attributed to Yunji Wang.

3 recordsLinked to original sources

arXiv2024

Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanism

Large language models have consistently struggled with complex reasoning tasks, such as mathematical problem-solving. Investigating the internal reasoning mechanisms of these models can help us design better model architectures and training strategies, ultimately enhancing their reasoning capability. In this study, we constructed a symbolic multi-step reasoning task to investigate the information propagation mechanisms in Transformer models when solving the task through direct answering and Chain-of-Thought (CoT) reasoning. We introduced the concept of buffer mechanism: the model stores various information in distinct buffers and selectively extracts it through the query-key matrix. We proposed a random matrix-based algorithm to enhance the model's reasoning ability. This algorithm introduces only 132 trainable parameters, yet leads to significant performance improvements on 7 multi-step reasoning datasets, including PrOntoQA, LogicAsker, and LogicInference. These findings provide new insights into understanding the large language models.

cs.AI↗

arXiv2015

Regularized maximum correntropy machine

In this paper we investigate the usage of regularized correntropy framework for learning of classifiers from noisy labels. The class label predictors learned by minimizing transitional loss functions are sensitive to the noisy and outlying labels of training samples, because the transitional loss functions are equally applied to all the samples. To solve this problem, we propose to learn the class label predictors by maximizing the correntropy between the predicted labels and the true labels of the training samples, under the regularized Maximum Correntropy Criteria (MCC) framework. Moreover, we regularize the predictor parameter to control the complexity of the predictor. The learning problem is formulated by an objective function considering the parameter regularization and MCC simultaneously. By optimizing the objective function alternately, we develop a novel predictor learning algorithm. The experiments on two chal- lenging pattern classification tasks show that it significantly outperforms the machines with transitional loss functions.

cs.LG↗

arXiv2014

A novel energy-efficient resource allocation algorithm based on immune clonal optimization for green cloud computing

Cloud computing is a style of computing in which dynamically scalable and other virtualized resources are provided as a service over the Internet. The energy consumption and makespan associated with the resources allocated should be taken into account. This paper proposes an improved clonal selection algorithm based on time cost and energy consumption models in cloud computing environment. We have analyzed the performance of our approach using the CloudSim toolkit. The experimental results show that our approach has immense potential as it offers significant improvement in the aspects of response time and makespan, demonstrates high potential for the improvement in energy efficiency of the data center, and can effectively meet the service level agreement requested by the users.

cs.DC↗