학술논문

EGFormer: An Enhanced Transformer Model with Efficient Attention Mechanism for Traffic Flow Forecasting
Document Type
article
Source
Vehicles, Vol 6, Iss 1, Pp 120-139 (2024)
Subject
traffic flow forecasting
Transformer
Multi-Head Efficient Self-Attention mechanism
Generative Decoding mechanism
Mechanical engineering and machinery
TJ1-1570
Machine design and drawing
TJ227-240
Motor vehicles. Aeronautics. Astronautics
TL1-4050
Language
English
ISSN
2624-8921
Abstract
Due to the regular influence of human activities, traffic flow data usually exhibit significant periodicity, which provides a foundation for further research on traffic flow data. However, the temporal dependencies in traffic flow data are often obscured by entangled temporal regularities, making it challenging for general models to capture the intrinsic functional relationships within the data accurately. In recent years, a plethora of methods based on statistics, machine learning, and deep learning have been proposed to tackle these problems of traffic flow forecasting. In this paper, the Transformer is improved from two aspects: (1) an Efficient Attention mechanism is proposed, which reduces the time and memory complexity of the Scaled Dot Product Attention; (2) a Generative Decoding mechanism instead of a Dynamic Decoding operation, which accelerates the inference speed of the model. The model is named EGFormer in this paper. Through a lot of experiments and comparative analysis, the authors found that the EGFormer has better ability in the traffic flow forecasting task. The new model has higher prediction accuracy and shorter running time compared with the traditional model.