e-Article
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Document Type
Working Paper
Author
Shen, Jonathan; Nguyen, Patrick; Wu, Yonghui; Chen, Zhifeng; Chen, Mia X.; Jia, Ye; Kannan, Anjuli; Sainath, Tara; Cao, Yuan; Chiu, Chung-Cheng; He, Yanzhang; Chorowski, Jan; Hinsu, Smit; Laurenzo, Stella; Qin, James; Firat, Orhan; Macherey, Wolfgang; Gupta, Suyog; Bapna, Ankur; Zhang, Shuyuan; Pang, Ruoming; Weiss, Ron J.; Prabhavalkar, Rohit; Liang, Qiao; Jacob, Benoit; Liang, Bowen; Lee, HyoukJoong; Chelba, Ciprian; Jean, Sébastien; Li, Bo; Johnson, Melvin; Anil, Rohan; Tibrewal, Rajat; Liu, Xiaobing; Eriguchi, Akiko; Jaitly, Navdeep; Ari, Naveen; Cherry, Colin; Haghani, Parisa; Good, Otavio; Cheng, Youlong; Alvarez, Raziel; Caswell, Isaac; Hsu, Wei-Ning; Yang, Zongheng; Wang, Kuan-Chieh; Gonina, Ekaterina; Tomanek, Katrin; Vanik, Ben; Wu, Zelin; Jones, Llion; Schuster, Mike; Huang, Yanping; Chen, Dehao; Irie, Kazuki; Foster, George; Richardson, John; Macherey, Klaus; Bruguier, Antoine; Zen, Heiga; Raffel, Colin; Kumar, Shankar; Rao, Kanishka; Rybach, David; Murray, Matthew; Peddinti, Vijayaditya; Krikun, Maxim; Bacchiani, Michiel A. U.; Jablin, Thomas B.; Suderman, Rob; Williams, Ian; Lee, Benjamin; Bhatia, Deepti; Carlson, Justin; Yavuz, Semih; Zhang, Yu; McGraw, Ian; Galkin, Max; Ge, Qi; Pundak, Golan; Whipkey, Chad; Wang, Todd; Alon, Uri; Lepikhin, Dmitry; Tian, Ye; Sabour, Sara; Chan, William; Toshniwal, Shubham; Liao, Baohua; Nirschl, Michael; Rondon, Pat
Source
Subject
Language
Abstract
Lingvo is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models. Lingvo models are composed of modular building blocks that are flexible and easily extensible, and experiment configurations are centralized and highly customizable. Distributed training and quantized inference are supported directly within the framework, and it contains existing implementations of a large number of utilities, helper functions, and the newest research ideas. Lingvo has been used in collaboration by dozens of researchers in more than 20 papers over the last two years. This document outlines the underlying design of Lingvo and serves as an introduction to the various pieces of the framework, while also offering examples of advanced features that showcase the capabilities of the framework.