An Efficient Management for Map Reduce Using Partition and Aggregation in Software Application

Abstract

In this paper, we study to reduce data traffic and to avoid duplication using a MapReduce technique and data partition scheme. Aggregator problem and large-scale optimization problem of duplication and data traffic were made in online or offline. In these problem we use map reduce technique to clustering the data and use k-nearest algorithm is used to reduce the time and cluster the nearest data to avoid conflict. Then we also use ProMiSH based on random projections and hashing for partition process to avoid data traffic in the search engine. In this, we also using k-nearest algorithm for aggregation to clustering the nearest neighbor data and using ProMiSH based on random projections and hashing for partition process to avoid data traffic in the search engine. Then these we use algorithms to decrease the traffic and conflict while processing the data and improve the speed to access the data faster.