학술논문

MapReduce Preprocess of Big Graphs for Rapid Connected Components Detection
Document Type
Conference
Source
2022 IEEE 12th Annual Computing and Communication Workshop and Conference (CCWC) Computing and Communication Workshop and Conference (CCWC), 2022 IEEE 12th Annual. :0112-0118 Jan, 2022
Subject
Communication, Networking and Broadcast Technologies
Components, Circuits, Devices and Systems
Computing and Processing
General Topics for Engineers
Photonics and Electrooptics
Power, Energy and Industry Applications
Robotics and Control Systems
Signal Processing and Analysis
Costs
Social networking (online)
Conferences
Exponential distribution
Data preprocessing
Clustering algorithms
Transportation
big data
big graph
connected components
MapReduce
preprocess
Language
Abstract
Paramount and vast applications such as social networks deal with big graphs. For this reason, big graph analysis and processing is currently a necessity. Detection of connected components is one of the analysis of graphs which is utilized as a sub-part in many graph algorithms, such as clustering. The goal of this paper is to propose a parallel preprocess algorithm with MapReduce to decrease graph volume for rapid detection of connected components. Suggested method is able to lessen the volume up to more than 99% quickly by just two rounds of MapReduce. Our evaluation shows that the combination of the preprocess with detection of connected components has a significant impact on: reduction of execution time up to 7 times, decrease in data transmission of processing nodes in network and MapReduce rounds.