학술논문
Serving DNNs in Real Time at Datacenter Scale with Project Brainwave
Document Type
Periodical
Author
Chung, E.; Fowers, J.; Ovtcharov, K.; Papamichael, M.; Caulfield, A.; Massengill, T.; Liu, M.; Lo, D.; Alkalay, S.; Haselman, M.; Abeydeera, M.; Adams, L.; Angepat, H.; Boehn, C.; Chiou, D.; Firestein, O.; Forin, A.; Gatlin, K.S.; Ghandi, M.; Heil, S.; Holohan, K.; El Husseini, A.; Juhasz, T.; Kagi, K.; Kovvuri, R.K.; Lanka, S.; Van Megen, F.; Mukhortov, D.; Patel, P.; Perez, B.; Rapsang, A.; Reinhardt, S.; Rouhani, B.; Sapek, A.; Seera, R.; Shekar, S.; Sridharan, B.; Weisz, G.; Woods, L.; Yi Xiao, P.; Zhang, D.; Zhao, R.; Burger, D.
Source
IEEE Micro Micro, IEEE. 38(2):8-20 Apr, 2018
Subject
Language
ISSN
0272-1732
1937-4143
1937-4143
Abstract
To meet the computational demands required of deep learning, cloud operators are turning toward specialized hardware for improved efficiency and performance. Project Brainwave, Microsofts principal infrastructure for AI serving in real time, accelerates deep neural network (DNN) inferencing in major services such as Bings intelligent search features and Azure. Exploiting distributed model parallelism and pinning over low-latency hardware microservices, Project Brainwave serves state-of-the-art, pre-trained DNN models with high efficiencies at low batch sizes. A high-performance, precision-adaptable FPGA soft processor is at the heart of the system, achieving up to 39.5 teraflops (Tflops) of effective performance at Batch 1 on a state-of-the-art Intel Stratix 10 FPGA.