A Big Data Processing Framework Extended with Data Sets Management

Authors

  • Tae-Hyung Kim
  • Seo-Young Noh

Abstract

Background/Objectives: Big data deals with massive, compound, diverse data sets. its characteristics are usually denoted using multiple words starting with a letter “V” in industrial fields and academic comminutes. The V characteristics make it extremely difficult for a conventional software system and traditional databases to effectively process and manage big data.
Methods/Statistical analysis: This paper proposes the data sets management approach for dealing with the critical data sets, and presents a generic processing framework for big data and discusses how its internal stages are related to the six V characteristics of big data and quality attributes.
Findings: The purpose of processing big data is to extract the information or generate deliverables valuable to stakeholder and customers using the specific data sets that need to be regularly monitored and updated. For this purpose, the maintenance method for reserve and revise those important data sets used on big data processing are integrated in order to help them slowly aged and keep pace with the rapid and frequent changes of big data.
Improvements/Applications: The big data processing framework is extended with the data sets management methods, which contributes to increase understandability and maintainability of big data itself as well as the design and development of big data processing systems.

Downloads

Published

2020-03-26

Issue

Section

Articles