Short-Term Power Load Forecasting Algorithm Based on Hybrid Transformer-LSTM and Attention Mechanism

Minjing Yang, Binghong Su, Qinwei Duan, Yashan Zhong, Jiaxin Zhuo, Xuanli Lan
Article
2026 / Volume 9 / Pages 6068-6094
Published 15 June 2026

Abstract

For short-term forecasting in the textile industry, a single model is inherently limited because it struggles to balance local temporal dynamics (such as rapidly changing seasonal styles) with global dependencies (such as fluctuations in raw material costs), and often fails to capture the time-varying characteristics of feature importance. This paper proposes a hybrid Transformer-LSTM architecture incorporating a multi-head temporal-feature attention mechanism. The input sequence is segmented into sliding windows, with features and variables uniformly encoded via an embedding layer. The Transformer's multi-head self-attention models global temporal dependencies and long-term patterns. Its output feeds into the LSTM, which uses a gating structure to capture local dynamic changes and enhance sequence evolution modeling. Finally, an attention mechanism on the bidirectional LSTM's hidden state adaptively weights key time steps to generate a context-aware feature vector. Finally, the attention output and Transformer features are concatenated, and a fully connected layer performs regression to predict the load value, achieving multi-scale feature optimization. Experiments confirm this method's significant advantage in short-term forecasting, showing an MAE of (0.75±0.04) MW and RMSE of (0.90±0.06) MW. In sudden change scenarios, the mean MAPE is ≤5.12% and mean R2 is ≥0.928, effectively capturing dynamic correlations under social temporal changes. This study tackles the complexity of the textile industry by explicitly accounting for both local temporal dynamics, like fast-fashion cycles, and global dependencies, such as supply chain risks, alongside the dynamic changes in feature importance.

Keywords

short-term power load forecasting, hybrid architecture, long short-term memory, attention mechanism, textile industry