arXiv subjects

Shanyan Lai

Publications and source records attributed to Shanyan Lai.

3 recordsLinked to original sources

arXiv2025

Is attention truly all we need? An empirical study of asset pricing in pretrained RNN sparse and global attention models

This study investigates the pre-trained RNN attention models with the mainstream attention mechanisms, such as additive attention, Luong's three attentions, global self-attention and sliding window sparse attention, for the empirical asset pricing research on the top 420 large-cap US stocks. This is the first paper on the large-scale state-of-the-art (SOTA) attention mechanisms applied in the asset pricing context. They overcome the limitations of the traditional machine learning-based asset pricing, such as mis-capturing the temporal dependency and short memory. Moreover, the enforced causal masks in the attention mechanisms address the future data leaking issue ignored by the more advanced attention-based models, such as the classic Transformer. The proposed attention models also consider the temporal sparsity characteristic of asset pricing data and mitigate potential overfitting issues by deploying the simplified model structures. This provides some insights for future empirical economic research. All models are examined in three periods, which cover pre-COVID-19, COVID-19 and one year post-COVID-19, for testing the stability of these models under extreme market conditions. The study finds that in value-weighted portfolio back testing, the global self-attention model and the sliding window sparse attention model exhibit excellent capabilities in deriving the absolute returns and hedging downside risks, while they achieve an annualized Sortino ratio of 2.0 and 1.80 respectively in the period with COVID-19 in the static transaction cost scenario. Moreover, the sliding window sparse attention model performs more stably than the global self-attention model from the perspective of absolute portfolio returns with respect to the size of stocks' market capitalization.

q-fin.PR↗

arXiv2025

Multilayer Perceptron Neural Network Models in Asset Pricing: An Empirical Study on Large-Cap US Stocks

In this study, MLP models with dynamic structure are applied to factor models for asset pricing tasks. Concretely, the MLP pyramid model structure was employed on firm characteristic-sorted portfolio factors for modelling the large-cap US stocks. It was further developed as a practical factor investing strategy based on the predictions. The main findings were evaluated from 2 angles: model predictive power and backtesting performance, which were compared for the periods with and without COVID-19. The empirical results indicated that, given the constraints of the data size, the MLP models no longer perform 'deeper, better' in terms of predictive power, whereas the proposed MLP models with 2 and 3 hidden layers have greater flexibility in modelling the factors in this case. This study also verified the idea from previous work that MLP models for factor investing are more meaningful for downside risk control than for pursuing absolute annual returns.

q-fin.PR↗

arXiv2025

Asset Pricing in Pre-trained Transformer

This paper proposes an innovative Transformer model, Single-directional representative from Transformer (SERT), for US large capital stock pricing. It also innovatively applies the pre-trained Transformer models under the stock pricing and factor investment context. They are compared with standard Transformer models and encoder-only Transformer models in three periods covering the entire COVID-19 pandemic to examine the model adaptivity and suitability during the extreme market fluctuations. Namely, pre-COVID-19 period, COVID-19 period and 1-year post-COVID-19. The best proposed SERT model achieves the highest out-of-sample $R^2$, 11.94\% and 11.47\% respectively, when extreme market fluctuation takes place, followed by pre-trained Transformer models (11.13\% and 9.72\%). Their Trend-following-based strategy's performance also proves their excellent capability for hedging downside risks during market shocks. The proposed SERT model achieves a Sortino ratio 47\% higher than the buy-and-hold benchmark in the equal-weighted portfolio and 28\% higher in the value-weighted portfolio in the static transaction cost scenario when the pandemic period is considered. It proves that Transformer models have a strong ability to capture patterns of temporal sparsity in asset pricing factor models, especially with high volatility. I also find the softmax signal filter as the common configuration of Transformer models in alternative contexts, which only eliminates differences between models, but does not improve strategy-wise performance, while increasing attention heads improves the model performance insignificantly and applying the 'layer normalization first' method does not boost the model performance in our case.

q-fin.CP↗