학술논문
Data Infrastructure at LinkedIn
Document Type
Conference
Author
Auradkar, Aditya; Botev, Chavdar; Das, Shirshanka; De Maagd, Dave; Feinberg, Alex; Ganti, Phanindra; Gao, Lei; Ghosh, Bhaskar; Gopalakrishna, Kishore; Harris, Brendan; Koshy, Joel; Krawez, Kevin; Kreps, Jay; Lu, Shi; Nagaraj, Sunil; Narkhede, Neha; Pachev, Sasha; Perisic, Igor; Qiao, Lin; Quiggle, Tom; Rao, Jun; Schulman, Bob; Sebastian, Abraham; Seeliger, Oliver; Silberstein, Adam; Shkolnik, BBoris; Soman, Chinmay; Sumbaly, Roshan; Surlaker, Kapil; Topiwala, Sajid; Tran, Cuong; Varadarajan, Balaji; Westerman, Jemiah; White, Zach; Zhang, David; Zhang, Jason
Source
2012 IEEE 28th International Conference on Data Engineering Data Engineering (ICDE), 2012 IEEE 28th International Conference on. :1370-1381 Apr, 2012
Subject
Language
ISSN
1063-6382
2375-026X
2375-026X
Abstract
Linked In is among the largest social networking sites in the world. As the company has grown, our core data sets and request processing requirements have grown as well. In this paper, we describe a few selected data infrastructure projects at Linked In that have helped us accommodate this increasing scale. Most of those projects build on existing open source projects and are themselves available as open source. The projects covered in this paper include: (1) Voldemort: a scalable and fault tolerant key-value store, (2) Data bus: a framework for delivering database changes to downstream applications, (3) Espresso: a distributed data store that supports flexible schemas and secondary indexing, (4) Kafka: a scalable and efficient messaging system for collecting various user activity events and log data.