A survey of transformer networks for time series forecasting

• Introduction to Transformers, their framework, hyperparameters, and learning paradigms. • Comprehensive assessment of tailored Transformers for advanced time series forecasting. • In-depth analysis of applications in finance, healthcare, energy, and transportation. • Discussion of challenges such as computational cost, datasets, overfitting, and interpretability. • Insights into integrating knowledge, multimodal, efficiency and AI-driven approaches. Time-series data support critical functions in forecasting, anomaly detection, resource optimization, and real-time decision making across finance, healthcare, energy systems, and networked computing. Classical statistical approaches and early deep-learning architectures (RNNs, LSTMs, CNNs) have achieved notable progress, yet they exhibit structural limitations: recurrent models struggle with long-range dependencies, convolutional models require deep stacking to enlarge receptive fields, and both scale suboptimally to high-dimensional and high-volume data. Transformer architectures—characterized by global attention mechanisms, flexible temporal receptive fields, and growing adoption within foundation-model paradigms—have consequently gained increasing prominence for modern time-series analysis. Drawing on a systematic review of IEEE Xplore, ACM Digital Library, and Scopus (2020–2025), this survey offers a unified, theoretically grounded synthesis of Transformer-based methods for time-series learning. The survey summarizes defining characteristics of time-series data and analyzes core architectural elements, including attention formulations, encoder–decoder structures, hyperparameter design, and domain-specific adaptations. An architecture-centered and task-aware taxonomy is presented to organize recent advances across forecasting, representation learning, anomaly detection, and multimodal fusion. Persistent challenges—spanning computational scalability, data-efficiency constraints, distributional heterogeneity, overfitting risks, hyperparameter instability, interpretability, and reproducibility—are examined in depth. A forward-looking research agenda is outlined, highlighting opportunities in physics-informed architectural design, hybrid neural–mechanistic modeling, resource-efficient real-time inference, multi-resolution spatiotemporal learning, and emerging human–AI collaborative paradigms. By consolidating these methodological developments, this survey aims to provide a structured reference point for ongoing research on Transformer models for time-series machine learning.