학술논문
Optimization Techniques for Data Consistency and Throughput Using Kafka Stateful Stream Processing
Document Type
Conference
Source
2023 6th International Seminar on Research of Information Technology and Intelligent Systems (ISRITI) Research of Information Technology and Intelligent Systems (ISRITI), 2023 6th International Seminar on. :480-485 Dec, 2023
Subject
Language
ISSN
2832-1456
Abstract
Nowadays, some organizations may have various application systems, databases, and services. Sometimes, each of them is designed using different technologies. Data integration has an important role in modern complex software architectures. In this context, data integration enables data to be spread across an organization's entire technology ecosystem. It requires a guarantee that the data being transmitted is valid and consistent. Therefore, a strategy is needed to ensure the validity of data exchange between applications. This research takes an in-depth look at the analysis and implementation of data integration using Apache Kafka. It discusses optimization strategies and techniques to improve data consistency and throughput in a Kafka environment. This research provides important insights into how Kafka Stream and Kafka Connect can improve data integration compared to Kafka's conventional producer-consumer model.