Prediction of Agricultural Commodity Prices using Big Data Framework


  • Humaira Rana Department of Computer Science & Information Technology, NED University of Engineering and Technology, Pakistan
  • Muhammad Umer Farooq Department of Computer Science & Information Technology, NED University of Engineering and Technology, Pakistan
  • Abdul Karim Kazi Department of Computer Science & Information Technology, NED University of Engineering and Technology, Pakistan
  • Mirza Adnan Baig Department of Computer Science, IQRA University, Pakistan
  • Muhammad Ali Akhtar Department of Computer and Information Systems Engineering, NED University of Engineering and Technology, Pakistan
Volume: 14 | Issue: 1 | Pages: 12652-12658 | February 2024 |


The agriculture sector plays a crucial role in the economy of Pakistan, contributing significantly to the Gross Domestic Product (GDP) and the employment rate. However, this sector faces challenges such as climate change, water scarcity, and low productivity, which have a direct impact on agricultural commodity prices. Accurate forecasting of commodity prices is essential for farmers, traders, and policymakers to make informed decisions and improve economic outcomes. This paper explores the use of a big data framework for agricultural commodity price forecasting in Pakistan, using a historical dataset on commodity prices in various Pakistani cities from 2007 to 2022 and Apache Spark to preprocess and clean the data. Based on historical spinach prices in Vehari City, the machine learning models Auto-Regressive Moving Average (ARIMA), Random Forest, and Long-Short-Term Memory (LSTM) were applied to price trends, and their performance was compared using Mean Absolute Error (MAE), Mean Squared Error (MSE), Root Mean Squared Error (RMSE) and squared correlation coefficient (R2). LSTM outperformed ARIMA and Random Forest with a higher R2 value of 0.8 and the lowest MAE of 125.29. Such predictions can help farmers to effectively plan crop cultivation and traders to make well-informed decisions.


Pyspark, agricultural commodity, price forecasting, big data analytics, Apache Spark framework


Download data is not yet available.


Y. H. Gu, D. Jin, H. Yin, R. Zheng, X. Piao, and S. J. Yoo, "Forecasting Agricultural Commodity Prices Using Dual Input Attention LSTM," Agriculture, vol. 12, no. 2, Feb. 2022, Art. no. 256.

"The impact of higher agricultural commodity prices on emerging and low-income countries," CaixaBank Research, Dec. 22, 2022.

"Agriculture Statistics", Pakistan Bureau of Statistics,

"Pakistan Arable Land 1961-2023."

S. Škrbić et al., "Analysis of Plant-Production-Obtained Biomass in Function of Sustainable Energy," Sustainability, vol. 12, no. 13, Jan. 2020, Art. no. 5486.

U. Ali et al., "Climate change impacts on agriculture sector: A case study of Pakistan," Ciência Rural, vol. 51, Apr. 2021, Art. no. e20200110.

N. C. Eli-Chukwu, "Applications of Artificial Intelligence in Agriculture: A Review," Engineering, Technology & Applied Science Research, vol. 9, no. 4, pp. 4377–4383, Aug. 2019.

A. Rasheed, M. S. Younis, F. Ahmad, J. Qadir, and M. Kashif, "District Wise Price Forecasting of Wheat in Pakistan using Deep Learning." arXiv, Mar. 05, 2021.

A. Qaiser, M. U. Farooq, S. M. N. Mustafa, and N. Abrar, "Comparative Analysis of ETL Tools in Big Data Analytics," Pakistan Journal of Engineering and Technology, vol. 6, no. 1, pp. 7–12, Jan. 2023.

S. M. Nabeel Mustafa, M. Umer Farooque, M. Tahir, S. M. Khan, and R. Qamar, "Frameworks, Applications and Challenges in Streaming Big Data Analytics: A Review," in 2022 3rd International Conference on Innovations in Computer Science & Software Engineering (ICONICS), Karachi, Pakistan, Sep. 2022, pp. 1–6.

"Apache SparkTM - Unified Engine for large-scale data analytics."

A. Mishra, "Machine learning classification models for detection of the fracture location in dissimilar friction stir welded joint," Applied Engineering Letters, 2020.

A. Rehman, Z. Deyuan, I. Hussain, M. S. Iqbal, Y. Yang, and L. Jingdong, "Prediction of Major Agricultural Fruits Production in Pakistan by Using an Econometric Analysis and Machine Learning Technique," International Journal of Fruit Science, vol. 18, no. 4, pp. 445–461, Oct. 2018.

N. Khan et al., "Prediction of Oil Palm Yield Using Machine Learning in the Perspective of Fluctuating Weather and Soil Moisture Conditions: Evaluation of a Generic Workflow," Plants, vol. 11, no. 13, Jan. 2022, Art. no. 1697.

B. Wang et al., "Research on Hybrid Model of Garlic Short-term Price Forecasting based on Big Data," Computers, Materials & Continua, vol. 57, no. 2, pp. 283–296, 2018.

S. Akshay Prassanna et al., "Crop value forecasting using decision tree regressor and models," European Journal of Molecular & Clinical Medicine, vol. 7, no. 2, 2020.

A. Vohra, N. Pandey, and S. K. Khatri, "Decision Making Support System for Prediction of Prices in Agricultural Commodity," in 2019 Amity International Conference on Artificial Intelligence (AICAI), Dubai, United Arab Emirates, Oct. 2019, pp. 345–348.

S. Bayona-Oré, R. Cerna, and E. Tirado Hinojoza, "Machine Learning for Price Prediction for Agricultural Products," WSEAS Transactions on Business and Economics, vol. 18, pp. 969–977, Jun. 2021.

S. A. Haider et al., "LSTM Neural Network Based Forecasting Model for Wheat Production in Pakistan," Agronomy, vol. 9, no. 2, Feb. 2019, Art. no. 72.

Y. Su and X. Wang, "Innovation of agricultural economic management in the process of constructing smart agriculture by big data," Sustainable Computing: Informatics and Systems, vol. 31, Sep. 2021, Art. no. 100579.

"AMIS Agriculture Marketing Wing Punjab."

S. K. Filipova-Petrakieva and V. Dochev, "Short-Term Forecasting of Hourly Electricity Power Demand: Reggresion and Cluster Methods for Short-Term Prognosis," Engineering, Technology & Applied Science Research, vol. 12, no. 2, pp. 8374–8381, Apr. 2022.

S. Joseph, N. Mduma, and D. Nyambo, "A Deep Learning Model for Predicting Stock Prices in Tanzania," Engineering, Technology & Applied Science Research, vol. 13, no. 2, pp. 10517–10522, Apr. 2023.


How to Cite

H. Rana, M. U. Farooq, A. K. Kazi, M. A. Baig, and M. A. Akhtar, “Prediction of Agricultural Commodity Prices using Big Data Framework”, Eng. Technol. Appl. Sci. Res., vol. 14, no. 1, pp. 12652–12658, Feb. 2024.


Abstract Views: 168
PDF Downloads: 77

Metrics Information