Mohammad Norouzi
mnorouzi[at]google[.]com

I am a senior research scientist at Google Brain in Toronto. I am interested in developing simple and efficient machine learning algorithms that are broadly applicable across a range of domains including natural language processing and computer vision. I joined the brain team in Mountain View in January, 2016 and moved to Toronto in January, 2018. I completed my PhD in computer science at the University of Toronto in December, 2015. My advisor was Prof. David Fleet, and I was supported by a Google PhD fellowship in machine learning. My PhD thesis focused on scalable similarity search for web-scale data collections. I am from Iran and I finished my undergraduate studies at Sharif University of Technology.

Currently, I am highlighting:

My research lies at the intersection of machine learning, reinforcement learning, computer vision, and natural language processing with a focus on neural networks. My current research concerns:

  1. Building statistical models of sequential data, sentences, programs, images, and other structured objects.
  2. Advancing reinforcement learning algorithms and applications.
  3. Unifying 1 & 2.

Google scholar profile
Old Curriculum Vitae
GitHub page

Papers

Trust-PCL: An Off-Policy Trust Region Method for Continuous Control
Ofir Nachum, Mohammad Norouzi, Kelvin Xu, Dale Schuurmans
ArXiv preprint, 2017. [pdf]

Filtering Variational Objectives
Chris J Maddison, Dieterich Lawson, George Tucker, Nicolas Heess, Mohammad Norouzi, Andriy Mnih, Arnaud Doucet, Yee Whye Teh
NIPS, 2017. [pdf]

Bridging the Gap Between Value and Policy Based Reinforcement Learning
Ofir Nachum, Mohammad Norouzi, Kelvin Xu, Dale Schuurmans
NIPS, 2017. [pdf]

PixColor: Pixel Recursive Colorization
Sergio Guadarrama, Ryan Dahl, David Bieber, Mohammad Norouzi, Jonathon Shlens, Kevin Murphy
BMVC, 2017. [pdf]

Pixel Recursive Super Resolution
Ryan Dahl, Mohammad Norouzi, Jonathon Shlens
ICCV, 2017. [pdf]

Deep Value Networks Learn to Evaluate and Iteratively Refine Structured Outputs
Michael Gygli, Mohammad Norouzi, Anelia Angelova
ICML, 2017. [pdf] [image segmentation process -- gif files] [slides] [code]

Device Placement Optimization with Reinforcement Learning
Azalia Mirhoseini, Hieu Pham, Quoc V Le, Benoit Steiner, Rasmus Larsen, Yuefeng Zhou, Naveen Kumar, Mohammad Norouzi, Samy Bengio, Jeff Dean
ICML, 2017. [pdf]

Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders
Jesse Engel, Cinjon Resnick, Adam Roberts, Sander Dieleman, Douglas Eck, Karen Simonyan, Mohammad Norouzi
ICML, 2017. [pdf]

Detecting Cancer Metastases on Gigapixel Pathology Images
Yun Liu, Krishna Gadepalli, Mohammad Norouzi, George E. Dahl, Timo Kohlberger, Aleksey Boyko, Subhashini Venugopalan, Aleksei Timofeev, Philip Q. Nelson, Greg S. Corrado, Jason D. Hipp, Lily Peng, Martin C. Stumpe
ArXiv preprint, 2017. [pdf]

Neural Combinatorial Optimization with Reinforcement Learning
Irwan Bello, Hieu Pham, Quoc V. Le, Mohammad Norouzi, Samy Bengio
ICLR workshop, 2017. [pdf]

Improving Policy Gradient by Exploring Under-appreciated Rewards
Ofir Nachum, Mohammad Norouzi, Dale Schuurmans
ICLR, 2017. [pdf] [poster]

Google's neural machine translation system: Bridging the Gap between Human and Machine Translation
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Łukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, Jeffrey Dean
ArXiv preprint, 2016. [pdf]

Reward Augmented Maximum Likelihood for Neural Structured Prediction
Mohammad Norouzi, Samy Bengio, Zhifeng Chen, Navdeep Jaitly, Mike Schuster, Yonghui Wu, Dale Schuurmans
Neural Information Processing Systems (NIPS), 2016. [pdf] [poster]

Compact Discrete Representations for Scalable Similarity Search
Mohammad Norouzi, PhD thesis, 2016. [pdf]

Efficient Non-greedy Optimization of Decision Trees
Mohammad Norouzi, Maxwell D. Collins, Matthew Johnson, David J. Fleet, Pushmeet Kohl,
Neural Information Processing Systems (NIPS), 2015. [pdf]

CO2 Forest: Improved Random Forest by Continuous Optimization of Oblique Splits
Mohammad Norouzi, Maxwell D. Collins, David J. Fleet, Pushmeet Kohli,
ArXiv preprint, 2015. [pdf]

Zero-Shot Learning by Convex Combination of Semantic Embeddings
Mohammad Norouzi, Tomas Mikolov, Samy Bengio, Yoram Singer, Jonathon Shlens, Andrea Frome, Greg S. Corrado, Jeffrey Dean,
International Conference on Learning Representations (ICLR) 2014. [pdf] [slides:pptx] [dataset: 2-hop, 3-hop]

Fast Exact Search in Hamming Space with Multi-Index Hashing,
Mohammad Norouzi, Ali Punjani, David J. Fleet,
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 36, no. 6, 2014. [pdf] [code]

Cartesian k-means,
Mohammad Norouzi, David J. Fleet,
IEEE Computer Vision and Pattern Recognition (CVPR), 2013. [pdf] [code] [slides:ppt/pptx]

Hamming Distance Metric Learning,
Mohammad Norouzi, David J. Fleet, Ruslan Salakhutdinov,
Neural Information Processing Systems (NIPS), 2012. [pdf] [code] [supplemental] [poster]

Fast Search in Hamming Space with Multi-Index Hashing,
Mohammad Norouzi, Ali Punjani, David J. Fleet,
IEEE Computer Vision and Pattern Recognition (CVPR), 2012. [pdf] [code] [poster]

Minimal Loss Hashing for Compact Binary Codes,
Mohammad Norouzi, David J. Fleet,
International Conference in Machine Learning (ICML), 2011. [pdf] [code] [slides:ppt]

Stacks of Convolutional Restricted Boltzmann Machines for Shift-Invariant Feature Learning,
Mohammad Norouzi, Mani Ranjbar, Greg Mori,
IEEE Computer Vision and Pattern Recognition (CVPR), 2009. [pdf]
Extended version: Master's thesis at Simon Fraser University, 2009. [pdf] [slides:ppt]


Recorded talks

[81 mins] Lecture on "Towards a unified view of supervised learning and reinforcement learning" at UC Berkeley, 2017

[18 mins] Zero-Shot Learning by Convex Combination of Semantic Embeddings at ICLR 2014

[13 mins] Cartesian k-means at CVPR 2013

[20 mins] Minimal Loss Hashing for Compact Binary Codes at ICML 2011

Past

PhD student, Computer Science, University of Toronto
Advisor: David Fleet, Sep 2010 - Dec 2015
Teaching assistant for
CSC2503: Foundations of Computer Vision (Grad),
CSC411: Machine Learning and Data Mining,
CSC373: Algorithm Design, Analysis and Complexity,
CSC263: Data Structures and Analysis,
CSC236: Intro to Theory of Computation.

Research Intern, Google, Mountain View, CA USA.
Mentors: Samy Bengio, Yoram Singer - Summer 2013.

Research Intern, Microsoft Research, Cambridge UK.
Mentor: Pushmeet Kohli - Spring 2013.