Challenges in Handling Imbalanced Big Data: A Survey

Abstract

Big Data describes enormous sets that have more divergent and intricate structure like weblogs, social media, email, sensors, and photographs. These unstructured data and peculiar characteristics from traditional databases typically associated with extra complications in storing, analyzing and applying further procedures or extracting results. Big Data analytics is the process of auditing gigantic amounts of complex data to find out unseen patterns or recognizing hidden correlations. Big Data applications are rising during the last years, and researchers from many disciplines are aware of the advantages related to the knowledge extraction from this type of problem. However traditional learning approaches cannot be enforced due to the scalability issues. Being still a recent discipline, handful research has been conducted on imbalanced data classification for Big Data. The apprehension behind this is mainly the difficulties in adapting standard techniques to the Map-Reduce programming style. Additionally, inner problems of imbalanced data, namely lack of data for training, the overlap between classes, the presence of noise and small disjuncts, are emphasized during the data partitioning to fit the Map-Reduce programming style. A literature survey on classification problem in Big Data has been done and existing methodologies were discussed with their pros and cons in this paper. This study suggests that there is a great need for finding a new method of classification when it comes to Big Data which addresses several issues like multi-class problems, class imbalance etc.,