Improving Amazon EC2 Spot Instances Price Prediction using Machine Learning Algorithm

M. Prasanthi; G.Chishma; P. Padmavathi; K. Reethika; A. Chandra Sekhar

doi:10.32628/CSEIT24102102

Authors

M. Prasanthi Assistant Professor, Department of CSE, Sri Vasavi Institute of Engineering and Technology, Nandamuru, Andhra Pradesh, India Author
G.Chishma UG Student, Department of CSE, Sri Vasavi Institute of Engineering and Technology, Nandamuru, Andhra Pradesh, India Author
P. Padmavathi UG Student, Department of CSE, Sri Vasavi Institute of Engineering and Technology, Nandamuru, Andhra Pradesh, India Author
K. Reethika UG Student, Department of CSE, Sri Vasavi Institute of Engineering and Technology, Nandamuru, Andhra Pradesh, India Author
A. Chandra Sekhar UG Student, Department of CSE, Sri Vasavi Institute of Engineering and Technology, Nandamuru, Andhra Pradesh, India Author

DOI:

https://doi.org/10.32628/CSEIT24102102

Keywords:

Amazon EC2, Compute instances, One-day-ahead prediction, One-week-ahead prediction, Regression Random Forests, Spot instances, Spot price prediction

Abstract

Spot instances were introduced by Amazon EC2 in December 2009 to sell its spare capacity through auction based market mechanism. Despite its extremely low prices, cloud spot market has low utilization. Spot pricing being dynamic, spot instances are prone to out-of bid failure. Bidding complexity is another reason why users today still fear using spot instances. This work aims to present Regression Random Forests (RRFs) model to predict one-week-ahead and one-day-ahead spot prices. The prediction would assist cloud users to plan in advance when to acquire spot instances, estimate execution costs, and also assist them in bid decision making to minimize execution costs and out-of-bid failure probability. Simulations with 12 months real Amazon EC2 spot history traces to forecast future spot prices show the effectiveness of the proposed technique. Comparison of RRFs based spot price forecasts with existing non-parametric machine learning models reveal that RRFs based forecast accuracy outperforms other models. We measure predictive accuracy using MAPE, MCPE, OOBError and speed. Evaluation results show that

Downloads

Download data is not yet available.

References

Agmon Ben-Yehuda, O., Ben-Yehuda, M., Schuster, A., and Tsafrir, D. Deconstructing amazon ec2 spot instancepricing. ACM Transactions on Economics and Computation 1, 3 (2013), 16. DOI: https://doi.org/10.1145/2509413.2509416

Amazon Web Services. AWS EC2 Spot Instance Price Histories. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances-history.html Accessed Jul-2018.

Amazon Web Services. New AWS EC2 Spot Instance Pricing. https://aws.amazon.com/about-aws/whats-new/2017/11/amazon-ec2-spot-introduces-new-pricing-model-and-the-ability-to-launch-new-spot-instances-via-runinstances-api/Accessed Jul-2018.

Amazon Web Services. Amazon EC2. https://aws.amazon.com/ec2/, 2018. [Online; accessed July-2018].

Amazon Web Services. Amazon ec2 service level agreement, 2018. https://aws.amazon.com/ec2/sla/ accessed July2018.

Amazon Web Services. Amazon ec2 spot instances, 2018. https://aws.amazon.com/ec2/spot/ accessed July 2018.

Amazon Web Services. Amazon instance pricing, 2018. https://aws.amazon.com/ec2/pricing/on-demand/ accessedJuly 2018.

Amazon Web Services. Amazon simple storage service, 2018. https://aws.amazon.com/s3 accessed July 2018.

Amazon Web Services. How spot instances work, July 2018. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/how-spot-instances-work.html accessed July 2018.

Amazon Web Services. New amazon ec2 spot pricing model, July 2018. https://aws.amazon.com/blogs/compute/new-amazon-ec2-spot-pricing/ accessed July 2018.

Amazon Web Services. New amazon ec2 spot pricing model: Simplified purchasing without bidding and fewer interrup tions, 2018. https://aws.amazon.com/blogs/compute/new-amazon-ec2-spot-pricing/ accessed August 2018.

Amazon Web Services. New amazon ec2 spot pricing model: Simplified purchas ing without bidding and fewer interruptions, 2018. https://aws.amazon.com/blogs/aws/amazon-ec2-update-streamlined-access-to-spot-capacity-smooth-price-changes-instance-hibernation/accessedAugust 2018.

Amazon Web Services. New ec2 spot instance termination notices, July 2018. https://aws.amazon.com/blogs/aws/new-ec2-spot-instance-termination-notices/ accessed July 2018.

Aristotle Cloud Federation. https://federatedcloud.org [Online; accessed Aug-2018].

Chohan, N., Castillo, C., Spreitzer, M., Steinder, M., Tantawi, A., and Krintz, C. See Spot Run: Using SpotInstances for MapReduce Workflows. In Usenix HotCloud (2010).

Google Cloud Platform. Google preemptable virtual machines, July 2018. https://cloud.google.com/preemptible-vms/ accessed July 2018.

He, X., Shenoy, P., Sitaraman, R., and Irwin, D. Cutting the cost of hosting online services using cloud spot markets.In Proceedings of the 24th International Symposium on High-Performance Parallel and Distributed Computing (New York,NY, USA, 2015), HPDC ’15, ACM, pp. 207–218. DOI: https://doi.org/10.1145/2749246.2749275

Javadi, B., Thulasiram, R. K., and Buyya, R. Characterizing spot price dynamics in public cloud environments. FutureGeneration Computer Systems 29, 4 (2013), 988–999. DOI: https://doi.org/10.1016/j.future.2012.06.012

Khodak, M., Zheng, L., Lan, A. S., Joe-Wong, C., and Chiang, M. Learning cloud dynamics to optimize spot instancebidding strategies.

Nurmi, D., Brevik, J., and Wolski, R. Qbets: Queue bounds estimation from time series. In Job Scheduling Strategiesfor Parallel Processing (2008), Springer, pp. 76–101. DOI: https://doi.org/10.1007/978-3-540-78699-3_5

Nurmi, D., Wolski, R., and Brevik, J. Probabilistic advanced reservations for batch-scheduled parallel machines. InProceedings of the 13th ACM SIGPLAN symposium on principles and practice of parallel programming (2008), ACM,pp. 289–290. DOI: https://doi.org/10.1145/1345206.1345260

Schwarz, G. Estimating the dimension of a model. Annals of Statistics 6, 2 (1978). DOI: https://doi.org/10.1214/aos/1176344136

Song, J., and Guerin, R. Pricing and bidding strategies for cloud computing spot instances. In 2017 IEEE Conferenceon Computer Communications Workshops (INFOCOM WKSHPS) (May 2017), pp. 647–653. DOI: https://doi.org/10.1109/INFCOMW.2017.8116453

Steve Fox. New aws spot pricing model: The good, the bad, and the ugly, July 2018. http://autoscalr.com/2018/01/04/new-aws-spot-pricing-model-good-bad-ugly/ accessed July 2018.

Subramanya, S., Guo, T., Sharma, P., Irwin, D., and Shenoy, P. Spoton: A batch computing service for the spotmarket. In Proceedings of the Sixth ACM Symposium on Cloud Computing (New York, NY, USA, 2015), SoCC ’15, ACM,pp. 329–341. DOI: https://doi.org/10.1145/2806777.2806851

Wallace, R. M., Turchenko, V., Sheikhalishahi, M., Turchenko, I., Shults, V., Vazquez-Poletti, J. L., andGrandinetti, L. Applications of neural-based spot market prediction for cloud computing. In Intelligent Data Acquisitionand Advanced Computing Systems (IDAACS), 2013 IEEE 7th International Conference on (2013), vol. 2, IEEE, pp. 710–716. DOI: https://doi.org/10.1109/IDAACS.2013.6663017

Willsky, A. S., and Jones, H. L. A generalized likelihood ratio approach to the detection and estimation of jumps inlinear systems. IEEE Transactions on Automatic Control 21, 1 (1976). DOI: https://doi.org/10.1109/TAC.1976.1101146

Wolski, R., Brevik, J., Chard, R., and Chard, K. Probabilistic guarantees of execution duration for amazon spotinstances. In Proceedings of the International Conference for High Performance Computing, Networking, Storage andAnalysis (2017), ACM, p. 18. DOI: https://doi.org/10.1145/3126908.3126953

Yi, S., Kondo, D., and Andrzejak, A. Reducing costs of spot instances via checkpointing in the amazon elastic computecloud. In 2010 IEEE 3rd International Conference on Cloud Computing (July 2010), pp. 236–243. DOI: https://doi.org/10.1109/CLOUD.2010.35

Zheng, L., Joe-Wong, C., Tan, C. W., Chiang, M., and Wang, X. How to bid the cloud. In Proceedings of the 2015ACM Conference on Special Interest Group on Data Communication (New York, NY, USA, 2015), SIGCOMM ’15, ACM,pp. 71–84. DOI: https://doi.org/10.1145/2785956.2787473