Distributed Streaming Storage Performance Benchmarking: Pravega and Pulsar
Received: 10 June 2024 | Revised: 27 June 2024 | Accepted: 5 July 2024 | Online: 9 October 2024
Corresponding author: Ramesh Kadaba Vasudevamurthy
Abstract
Massive data shoving can reach the greatest throughput, which is necessary for distributed streaming storage to function at its best. The comparison of the distributed streaming storage systems Pulsar and Pravega for a given number of producers and data packet size is covered in detail in this study. This analysis' benchmark tool accommodates several producers and consumers. When connection pooling is enabled and 0.5 million records are thrust at a 10 Mbps data rate, both streaming storages are assessed for latency percentile comparison. A novel idea called sbk-charts is introduced in the current study, which can create practical charts from CSV files. Multiple CSV files can be joined by sbk-charts to construct a single combined xlsx file with helpful charts. The outcomes of the experiment are then evaluated for performance comparison in a number of dimensions.
Keywords:
connection pooling, benchmarking, throughput, storage benchmarking kit, latency, Pravega, PulsarDownloads
References
"Pravega – A Reliable Stream Storage System." https://cncf.pravega.io/.
"pravega/pravega," https://github.com/pravega/pravega.
"Apache Pulsar." https://pulsar.apache.org/.
N. V. Sanjay Kumar and K. Munegowda, "Distributed Streaming Storage Performance Benchmarking: Kafka and Pravega," International Journal of Innovative Technology and Exploring Engineering, vol. 9, no. 2S, pp. 1–8, Dec. 2019.
"Release Storage Benchmark Kit Version 5.0 · kmgowda/SBK," GitHub. https://github.com/kmgowda/SBK/releases/tag/5.0.
"Dashboards | Grafana documentation," Grafana Labs. https://grafana.com/docs/grafana/latest/dashboards/.
"Apache BookKeeper." https://bookkeeper.apache.org/.
"Apache ZooKeeper." https://zookeeper.apache.org/.
F. Junqueira and B. Reed, ZooKeeper: Distributed Process Coordination. Sebastopol, CA, USA: O’Reilly, 2013.
"apache/flink," https://github.com/apache/flink.
"apache/samza." https://github.com/apache/samza.
N. Sajitha and S. P. Priya, "Optimal Artificial Neural Network-based Fabric Defect Detection and Classification," Engineering, Technology & Applied Science Research, vol. 14, no. 2, pp. 13148–13152, Apr. 2024.
T. Alshammari, "Using Artificial Neural Networks with GridSearchCV for Predicting Indoor Temperature in a Smart Home," Engineering, Technology & Applied Science Research, vol. 14, no. 2, pp. 13437–13443, Apr. 2024.
H. T. S. Alrikabi, I. A. Aljazaery, and A. H. M. Alaidi, "Using a Chaotic Digital System to Generate Random Numbers for Secure Communication on 5G Networks," Engineering, Technology & Applied Science Research, vol. 14, no. 2, pp. 13598–13603, Apr. 2024.
K. V. Ramesh and G. T. Raju, "Pravega: Performance impact analysis with Connection Pooling’," in 2nd IEEE International Conference on Knowledge Engineering and Communication Systems (ICKECS 2024), Karnataka, India, Apr. 2024.
Downloads
How to Cite
License
Copyright (c) 2024 K. V. Ramesh, G. T. Raju
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
