Léon Bottou (born 1965) is a researcher best known for his work in machine learning and data compression. His work presents stochastic gradient descent as a fundamental learning algorithm.[1] He is also one of the main creators of the DjVu image compression technology (together with Yann LeCun and Patrick Haffner), and the maintainer of DjVuLibre, the open source implementation of DjVu. He is the original developer of the Lush programming language.
Léon Bottou | |
|---|---|
| Born | 1965 (age 58–59) |
| Nationality | French |
| Alma mater | École Polytechnique École Normale Supérieure Université Paris-Sud |
| Known for | DjVu |
| Awards | Blavatnik Award for Young Scientists |
| Scientific career | |
| Fields | Machine learning |
| Institutions | Facebook Research |
Life
editLéon Bottou was born in France in 1965. He obtained the Diplôme d'Ingénieur from École Polytechnique in 1987, a Magistère de Mathématiques Fondamentales et Appliquées et d’Informatique from École Normale Supérieure in 1988 and a PhD from Université Paris-Sud in 1991. He then joined the Adaptive Systems Research Department at AT&T Bell Laboratories in Holmdel, New Jersey, where he collaborated with Vladimir Vapnik on local learning algorithms.[2] in 1992, he returned to France and founded Neuristique S.A., a company that produced machine learning tools and one of the first data mining software packages. In 1995, he returned to Bell Laboratories, where he developed a number of new machine learning methods, such as Graph Transformer Networks (similar to conditional random field), and applied them to handwriting recognition and OCR.[3] The bank check recognition system that he helped develop was widely deployed by NCR and other companies, reading over 10% of all the checks in the US in the late 1990s and early 2000s.
In 1996, he joined AT&T Labs and worked primarily on the DjVu image compression technology,[4] that is used by some websites, notably the Internet Archive, to distribute scanned documents. Between 2002 and 2010, he was a research scientist at NEC Laboratories in Princeton, New Jersey, where he focused on the theory and practice of machine learning with large-scale datasets,[5] on-line learning, and stochastic optimization methods.[6] He developed the open source software LaSVM for fast large-scale support vector machine, and stochastic gradient descent software for training linear SVM and Conditional Random Fields. In 2010 he joined the Microsoft adCenter in Redmond, Washington, and in 2012 became a Principal Researcher at Microsoft Research in New York City. In March 2015 he joined Facebook Artificial Intelligence Research, also in New York City, as a research lead.
His work in gradient descent argued that both stochastic gradient descent and batch gradient descent reach similar levels of loss with the same number of training samples, but SGD is faster when running on large datasets. He also argued that second-order gradient descent methods, such as quasi-Newton methods, can be beneficial compared to plain SGD. See (Bottou et al 2018)[7] for a review.
He was program chair of the 2013 Conference on Neural Information Processing Systems and the 2009 International Conference on Machine Learning. He is an associate editor of the IEEE's Transactions on Pattern Analysis and Machine Intelligence, the IAPR's Pattern Recognition Letters and the independently published Journal of Machine Learning Research.[citation needed] In 2007, he was received one of the first Blavatnik Awards for Young Scientists from the Blavatnik Family Foundation and the New York Academy of Sciences.
References
edit- ^ Bottou, Léon. "Learning with Stochastic Gradient Descent". leon.bottou.org. Retrieved 14 May 2024.
- ^ Vapnik, Vladimir N.; Bottou, Léon (1993). "Local Algorithms for Pattern Recognition and Dependencies Estimation". Neural Computation. 5 (6): 893–909. doi:10.1162/neco.1993.5.6.893. S2CID 2327934.
- ^ LeCun, Yann; Bottou, Léon; Bengio, Yoshua; Haffner, Patrick (1998). "Gradient Based Learning Applied to Document Recognition". Proceedings of the IEEE. 86 (11): 2278–2324. doi:10.1109/5.726791. S2CID 14542261.
- ^ Bottou, Léon; et al. (1998). "High Quality Document Image Compression with DjVu". Journal of Electronic Imaging. 7 (3): 410–425. Bibcode:1998JEI.....7..410B. CiteSeerX 10.1.1.38.4518. doi:10.1117/1.482609.
- ^ Bottou, Léon; Chapelle, Olivier; DeCoste, Dennis; Weston, Jason, eds. (2007). Large Scale Kernel Machines. Neural Information Processing Series. Cambridge, MA: MIT Press. ISBN 978-0-262-02625-3.
- ^ Bottou, Léon (2004). "Stochastic Learning". In Bousquet, Olivier; von Luxburg, Ulrike (eds.). Advanced Lectures on Machine Learning. Lecture Notes in Artificial Intelligence. Vol. 3176. Berlin: Springer Verlag. pp. 146–168. ISBN 978-3-540-23122-6.
- ^ Bottou, Léon; Curtis, Frank E.; Nocedal, Jorge (January 2018). "Optimization Methods for Large-Scale Machine Learning". SIAM Review. 60 (2): 223–311. arXiv:1606.04838. doi:10.1137/16M1080173. ISSN 0036-1445.
External links
edit- Léon Bottou's personal website
- Léon Bottou publications indexed by Google Scholar
