Data Mining Algorithm in Cloud Computing Using Map Reduce Framework

Abstract

Today's Cloud computing technology has been emerged to manage large data sets efficiently and due to rapid growth of data, large-scale data processing is becoming a major point of information technique. The Hadoop Distributed File System (HDFS) is designed for reliable storage of very large data sets and to stream those data sets at high bandwidth to user applications. In a large cluster, hundreds of servers both host directly attached storage and execute user application tasks. By distributing storage and computation across many servers, the resource can grow on demand while remaining economical at every size. Map Reduce has been widely used for large-scale data analysis in the Cloud. Hadoop is an open source implementation of Map Reduce which can achieve better performance with the allocation of more compute nodes from the cloud to speed up computation; however, this approach of 'renting more nodes' isn't cost effective in a pay-as-you-go environment.