Sorted Positional Indexing Based Computation for Large Data

Abstract

The performance of Hadoop Map Reduce mainly depends on its configuration parameters. Tuning the job configuration parameters is an effective way to improve performance so that we can reduce the execution time and the disk utilization. The performance of tuning is mainly based on CPU usage, disk I/O rate, memory usage, network traffic components. In this work we are discussing about the tuning techniques to upgrade the execution of Map Reduce occupations. It is found that the current calculations can't prepare the skyline on huge information productively. So, here we are using a novel skyline algorithm Skyline Sorted Positional Index List (SSPL) on huge data like social data. SSPL utilizes sorted positional index lists which require low space overhead to reduce I/O cost significantly. The experimental results on synthetic and real data sets show that SSPL has a significant advantage over the existing skyline algorithms.