Evaluation of Stock Closing Prices using Transformer Learning
Received: 7 May 2023 | Revised: 2 June 2023, 12 July 2023, and 18 July 2023 | Accepted: 19 July 2023 | Online: 13 October 2023
Corresponding author: Tariq Saeed Mian
Abstract
Predicting stock markets remains a critical and challenging task due to many factors, such as the enormous volume of generated price data, instant price data changes, and sensitivity to human sentiments, wars, and natural disasters. Since the previous three years of the COVID-19 pandemic, forecasting stock markets is more difficult, complex, and problematic for stock market analysts. However, technical analysts of the stock market and academic researchers are continuously trying to develop innovative and modern methods for forecasting stock market prices, using statistical techniques, machine learning, and deep learning-based algorithms. This study investigated a Transformer sequential-based approach to forecast the closing price for the next day. Ten sliding window timesteps were used to forecast next-day stock closing prices. This study aimed to investigate reliable techniques based on stock input features. The proposed Transformer-based method was compared with ARIMA, Long-Short Term Memory (LSTM), and Random Forest (RF) algorithms, showing its outstanding results on Yahoo Finance data, Facebook Intra data, and JPMorgan's Intra data. Each model was evaluated using Mean Absolute Error (MSE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE).
Keywords:
stock prediction, ARIMA, SARIMA, LSTM, transformer, stock volatility, stock market, stock market prediction, machine learning, deep learningDownloads
References
A. Thakkar and K. Chaudhari, “Fusion in stock market prediction: A decade survey on the necessity, recent developments, and potential future directions," Information Fusion, vol. 65, pp. 95–107, Jan. 2021.
S. M. Idrees, M. A. Alam, and P. Agarwal, “A Prediction Approach for Stock Market Volatility Based on Time Series Data," IEEE Access, vol. 7, pp. 17287–17298, 2019.
K. C. Rasekhschaffe and R. C. Jones, “Machine Learning for Stock Selection," Financial Analysts Journal, vol. 75, no. 3, pp. 70–88, Jul. 2019.
C. S. Wong and W. K. Li, “On a Mixture Autoregressive Model," Journal of the Royal Statistical Society Series B: Statistical Methodology, vol. 62, no. 1, pp. 95–115, Jan. 2000.
M. R. Hassan and B. Nath, “Stock market forecasting using hidden Markov model: a new approach," in 5th International Conference on Intelligent Systems Design and Applications (ISDA’05), Warsaw, Poland, Sep. 2005, pp. 192–196.
E. K. Ampomah, Z. Qin, and G. Nyame, “Evaluation of Tree-Based Ensemble Machine Learning Models in Predicting Stock Price Direction of Movement," Information, vol. 11, no. 6, Jun. 2020, Art. no. 332.
X. Yu and D. Li, “Important Trading Point Prediction Using a Hybrid Convolutional Recurrent Neural Network," Applied Sciences, vol. 11, no. 9, Jan. 2021, Art. no. 3984.
A. P. Ruiz, A. A. Gila, U. Irusta, and J. E. Huguet, “Why Deep Learning Performs Better than Classical Learning?," DYNA Ingeneria e Industria, vol. 95, no. 2, pp. 119–122, Mar. 2020.
J. Liu, X. Guo, B. Li, and Y. Yuan, “COINet: Adaptive Segmentation with Co-Interactive Network for Autonomous Driving," in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, Sep. 2021, pp. 4800–4806.
N. M. H. Masoud, “The Impact of Stock Market Performance upon Economic Growth," International Journal of Economics and Financial Issues, vol. 3, no. 4, pp. 788–798, Dec. 2013.
A. Murkute and T. Sarode, “Forecasting Market Price of Stock using Artificial Neural Network," International Journal of Computer Applications, vol. 124, no. 12, pp. 11–15, Aug. 2015.
L. J. Kao, C. C. Chiu, C. J. Lu, and J. L. Yang, “Integration of nonlinear independent component analysis and support vector regression for stock price forecasting," Neurocomputing, vol. 99, pp. 534–542, Jan. 2013.
W. Khan, M. ali Ghazanfar, M. Assam, S. Ahmad, and J. Khan, “Predicting Trend in Stock Market Exchange Using Machine Learning Classifiers," Science International, vol. 28, no. 2, pp. 1363–1367, May 2016.
R. Gupta, N. Garg, and S. Singh, “Stock Market Prediction Accuracy Analysis Using Kappa Measure," in 2013 International Conference on Communication Systems and Network Technologies, Gwalior, India, Apr. 2013, pp. 635–639.
A. E. Ezugwu et al., “A comprehensive survey of clustering algorithms: State-of-the-art machine learning applications, taxonomy, challenges, and future research prospects," Engineering Applications of Artificial Intelligence, vol. 110, Apr. 2022, Art. no. 104743.
L. Li, Y. Wu, Y. Ou, Q. Li, Y. Zhou, and D. Chen, “Research on machine learning algorithms and feature extraction for time series," in 2017 IEEE 28th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC), Montreal, QC, Canada, Jul. 2017, pp. 1–5.
J. Zhao, N. Sun, and W. Cheng, “Logistics forum based prediction on stock index using intelligent data analysis and processing of online web posts," Journal of Ambient Intelligence and Humanized Computing, vol. 11, no. 9, pp. 3575–3584, Sep. 2020.
A. S. Saud and S. Shakya, “Analysis of look back period for stock price prediction with RNN variants: A case study on banking sector of NEPSE," Procedia Computer Science, vol. 167, pp. 788–798, Jan. 2020.
S. Mohan, S. Mullapudi, S. Sammeta, P. Vijayvergia, and D. C. Anastasiu, “Stock Price Prediction Using News Sentiment Analysis," presented at the 2019 IEEE Fifth International Conference on Big Data Computing Service and Applications (BigDataService), Newark, CA, USA, Apr. 2019, pp. 205–208.
A. Ghosh, S. Bose, G. Maji, N. Debnath, and S. Sen, “Stock Price Prediction Using LSTM on Indian Share Market," in Proceedings of 32nd International Conference on Computer Applications in Industry and Engineering, 2019, pp. 101–110.
“Neural Network Model vs. SARIMA Model In Forecasting Korean Stock Price Index (KOSPI)," Issues In Information Systems, vol. VIII, no. 2, 2007.
A. A. Ariyo, A. O. Adewumi, and C. K. Ayo, “Stock Price Prediction Using the ARIMA Model," in 2014 UKSim-AMSS 16th International Conference on Computer Modelling and Simulation, Cambridge, UK, Mar. 2014, pp. 106–112.
M. Almasarweh and S. A. Wadi, “ARIMA Model in Predicting Banking Stock Market Data," Modern Applied Science, vol. 12, no. 11, 2018.
I. R. Parray, S. S. Khurana, M. Kumar, and A. A. Altalbe, “Time series data analysis of stock price movement using machine learning techniques," Soft Computing, vol. 24, no. 21, pp. 16509–16517, Nov. 2020.
S. Joseph, N. Mduma, and D. Nyambo, “A Deep Learning Model for Predicting Stock Prices in Tanzania," Engineering, Technology & Applied Science Research, vol. 13, no. 2, pp. 10517–10522, Apr. 2023.
U. P. Gurav and S. Kotrappa, “Sentiment Aware Stock Price Forecasting using an SA-RNN-LBL Learning Model," Engineering, Technology & Applied Science Research, vol. 10, no. 5, pp. 6356–6361, Oct. 2020.
Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning," Nature, vol. 521, no. 7553, pp. 436–444, May 2015.
D. Salinas, V. Flunkert, J. Gasthaus, and T. Januschowski, “DeepAR: Probabilistic forecasting with autoregressive recurrent networks," International Journal of Forecasting, vol. 36, no. 3, pp. 1181–1191, Jul. 2020.
D. T. Mirikitani and N. Nikolaev, “Recursive Bayesian Recurrent Neural Networks for Time-Series Modeling," IEEE Transactions on Neural Networks, vol. 21, no. 2, pp. 262–274, Oct. 2010.
H. Wang, G. Li, G. Wang, J. Peng, H. Jiang, and Y. Liu, “Deep learning based ensemble approach for probabilistic wind power forecasting," Applied Energy, vol. 188, pp. 56–70, Feb. 2017.
X. Shi, Z. Chen, H. Wang, D.-Y. Yeung, W. Wong, and W. Woo, “Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting," in Advances in Neural Information Processing Systems, 2015, vol. 28.
K. Amarasinghe, D. L. Marino, and M. Manic, “Deep neural networks for energy load forecasting," in 2017 IEEE 26th International Symposium on Industrial Electronics (ISIE), Edinburgh, UK, Jun. 2017, pp. 1483–1488.
C. J. Huang and P. H. Kuo, “A Deep CNN-LSTM Model for Particulate Matter (PM2.5) Forecasting in Smart Cities," Sensors, vol. 18, no. 7, 2018.
Y. Sudriani, I. Ridwansyah, and H. A. Rustini, “Long short term memory (LSTM) recurrent neural network (RNN) for discharge level prediction and forecast in Cimandiri river, Indonesia," IOP Conference Series: Earth and Environmental Science, vol. 299, no. 1, Apr. 2019, Art. no. 012037.
X. Ding, Y. Zhang, T. Liu, and J. Duan, “Deep learning for event-driven stock prediction," in Proceedings of the 24th International Conference on Artificial Intelligence, Buenos Aires, Argentina, Apr. 2015, pp. 2327–2333.
D. M. Q. Nelson, A. C. M. Pereira, and R. A. de Oliveira, “Stock market’s price movement prediction with LSTM neural networks," in 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA, Feb. 2017, pp. 1419–1426.
S. Al-Janabi, A. Alkaim, E. Al-Janabi, A. Aljeboree, and M. Mustafa, “Intelligent forecaster of concentrations (PM2.5, PM10, NO2, CO, O3, SO2) caused air pollution (IFCsAP)," Neural Computing and Applications, vol. 33, no. 21, pp. 14199–14229, Nov. 2021.
Y. Hu, “Stock market timing model based on convolutional neural network–a case study of Shanghai composite index," Finance& Economy, vol. 4, pp. 71–74, 2018.
W. Long, Z. Lu, and L. Cui, “Deep learning-based feature engineering for stock price movement prediction," Knowledge-Based Systems, vol. 164, pp. 163–173, Jan. 2019.
X. Pang, Y. Zhou, P. Wang, W. Lin, and V. Chang, “An innovative neural network approach for stock market prediction," The Journal of Supercomputing, vol. 76, no. 3, pp. 2098–2118, Mar. 2020.
A. Zhang, Z. C. Lipton, M. Li, and A. J. Smola, “Dive into Deep Learning." arXiv, Feb. 10, 2023.
S. K. Filipova-Petrakieva and V. Dochev, “Short-Term Forecasting of Hourly Electricity Power Demand: Reggresion and Cluster Methods for Short-Term Prognosis," Engineering, Technology & Applied Science Research, vol. 12, no. 2, pp. 8374–8381, Apr. 2022.
“Yahoo Finance - Stock Market Live, Quotes, Business & Finance News." https://finance.yahoo.com/.
“Meta - Financials." https://investor.fb.com/financials/default.aspx.
“J.P. Morgan Data and Analytics." https://www.jpmorgan.com/
securities-services/data-analytics.
E. Haugsdal, E. Aune, and M. Ruocco, “Persistence Initialization: A novel adaptation of the Transformer architecture for Time Series Forecasting." arXiv, Aug. 30, 2022.
S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory," Neural Computation, vol. 9, no. 8, pp. 1735–1780, Nov. 1997.
S. Fernández, A. Graves, and J. Schmidhuber, “An Application of Recurrent Neural Networks to Discriminative Keyword Spotting," in Artificial Neural Networks – ICANN 2007, 2007, pp. 220–229.
J. Schmidhuber, “Deep learning in neural networks: An overview," Neural Networks, vol. 61, pp. 85–117, Jan. 2015.
Downloads
How to Cite
License
Copyright (c) 2023 Tariq Saeed Mian
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
