Implementation of Map Reduce using Fuzzy C-Medoids Clustering on Time-Series Stock Market Big Data for Prediction

Abstract

Stock market data prediction is a problem of time series clustering and prediction. Clustering time series data can extract important information as well as additional features which help in efficient data classification. Several algorithms have been presented by researchers over the years in this domain. Most of the efficient clustering algorithms suffer from the drawback of higher complexity thereby increase in the computational time. This paper presents a weighted fuzzy C – Medoids algorithm with Dynamic Time Warping (DTW) modified according to parallel map reduce algorithm. The traditional fuzzy C - Medoids clustering is a sequential algorithm which clusters the input data into groups. As the historical data available for the stock market is huge, the time taken for clustering such data for further processing is time consuming. Modifying the existing algorithms based on the parallel map reduce would improve the efficiency of the overall method. The novel algorithm presented in the paper divides the data to be clustered into chunks and processes each chunk in parallel. The experimental results present how the proposed method has a reduced complexity of O(nk/m), which is m times less than the conventional weighted fuzzy C - Medoids. The experimental results prove that the accuracy of the proposed algorithm has increased compared to the existing approaches.