Mohammad Norouzi

Senior Research Scientist
Google Brain in Mountain View
mnorouzi[at]google[.]com

Currently, I am highlighting:

My research lies at the intersection of machine learning, computer vision, and natural language processing with an emphasis on neural networks and reinforcement learning. My current research focuses on

  1. Learning to produce images, sentences, and other structured objects.
  2. Advancing reinforcement learning and its applications.
  3. Connecting 1 & 2.

I graduated with a PhD in computer science from the University of Toronto in Dec 2015. My advisor was Prof. David Fleet, and my PhD thesis concerned scalable and efficient algorithms for processing, indexing, and searching digital media, to ease the use of web-scale datasets in machine learning. My PhD research was supported by a Google PhD fellowship.

Google scholar profile
Curriculum Vitae
GitHub page

Papers

Trust-PCL: An Off-Policy Trust Region Method for Continuous Control
Ofir Nachum, Mohammad Norouzi, Kelvin Xu, Dale Schuurmans
ArXiv preprint, 2017. [pdf]

Filtering Variational Objectives
Chris J Maddison, Dieterich Lawson, George Tucker, Nicolas Heess, Mohammad Norouzi, Andriy Mnih, Arnaud Doucet, Yee Whye Teh
NIPS, 2017. [pdf]

Bridging the Gap Between Value and Policy Based Reinforcement Learning
Ofir Nachum, Mohammad Norouzi, Kelvin Xu, Dale Schuurmans
NIPS, 2017. [pdf]

PixColor: Pixel Recursive Colorization
Sergio Guadarrama, Ryan Dahl, David Bieber, Mohammad Norouzi, Jonathon Shlens, Kevin Murphy
BMVC, 2017. [pdf]

Pixel Recursive Super Resolution
Ryan Dahl, Mohammad Norouzi, Jonathon Shlens
ICCV, 2017. [pdf]

Deep Value Networks Learn to Evaluate and Iteratively Refine Structured Outputs
Michael Gygli, Mohammad Norouzi, Anelia Angelova
ICML, 2017. [pdf] [image segmentation process -- gif files] [slides] [code]

Device Placement Optimization with Reinforcement Learning
Azalia Mirhoseini, Hieu Pham, Quoc V Le, Benoit Steiner, Rasmus Larsen, Yuefeng Zhou, Naveen Kumar, Mohammad Norouzi, Samy Bengio, Jeff Dean
ICML, 2017. [pdf]

Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders
Jesse Engel, Cinjon Resnick, Adam Roberts, Sander Dieleman, Douglas Eck, Karen Simonyan, Mohammad Norouzi
ICML, 2017. [pdf]

Detecting Cancer Metastases on Gigapixel Pathology Images
Yun Liu, Krishna Gadepalli, Mohammad Norouzi, George E. Dahl, Timo Kohlberger, Aleksey Boyko, Subhashini Venugopalan, Aleksei Timofeev, Philip Q. Nelson, Greg S. Corrado, Jason D. Hipp, Lily Peng, Martin C. Stumpe
ArXiv preprint, 2017. [pdf]

Neural Combinatorial Optimization with Reinforcement Learning
Irwan Bello, Hieu Pham, Quoc V. Le, Mohammad Norouzi, Samy Bengio
ICLR workshop, 2017. [pdf]

Improving Policy Gradient by Exploring Under-appreciated Rewards
Ofir Nachum, Mohammad Norouzi, Dale Schuurmans
ICLR, 2017. [pdf] [poster]

Google's neural machine translation system: Bridging the Gap between Human and Machine Translation
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Łukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, Jeffrey Dean
ArXiv preprint, 2016. [pdf]

Reward Augmented Maximum Likelihood for Neural Structured Prediction
Mohammad Norouzi, Samy Bengio, Zhifeng Chen, Navdeep Jaitly, Mike Schuster, Yonghui Wu, Dale Schuurmans
Neural Information Processing Systems (NIPS), 2016. [pdf] [poster]

Compact Discrete Representations for Scalable Similarity Search
Mohammad Norouzi, PhD thesis, 2016. [pdf]

Efficient Non-greedy Optimization of Decision Trees
Mohammad Norouzi, Maxwell D. Collins, Matthew Johnson, David J. Fleet, Pushmeet Kohl,
Neural Information Processing Systems (NIPS), 2015. [pdf]

CO2 Forest: Improved Random Forest by Continuous Optimization of Oblique Splits
Mohammad Norouzi, Maxwell D. Collins, David J. Fleet, Pushmeet Kohli,
ArXiv preprint, 2015. [pdf]

Zero-Shot Learning by Convex Combination of Semantic Embeddings
Mohammad Norouzi, Tomas Mikolov, Samy Bengio, Yoram Singer, Jonathon Shlens, Andrea Frome, Greg S. Corrado, Jeffrey Dean,
International Conference on Learning Representations (ICLR) 2014. [pdf] [slides:pptx] [dataset: 2-hop, 3-hop]

Fast Exact Search in Hamming Space with Multi-Index Hashing,
Mohammad Norouzi, Ali Punjani, David J. Fleet,
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 36, no. 6, 2014. [pdf] [code]

Cartesian k-means,
Mohammad Norouzi, David J. Fleet,
IEEE Computer Vision and Pattern Recognition (CVPR), 2013. [pdf] [code] [slides:ppt/pptx]

Hamming Distance Metric Learning,
Mohammad Norouzi, David J. Fleet, Ruslan Salakhutdinov,
Neural Information Processing Systems (NIPS), 2012. [pdf] [code] [supplemental] [poster]

Fast Search in Hamming Space with Multi-Index Hashing,
Mohammad Norouzi, Ali Punjani, David J. Fleet,
IEEE Computer Vision and Pattern Recognition (CVPR), 2012. [pdf] [code] [poster]

Minimal Loss Hashing for Compact Binary Codes,
Mohammad Norouzi, David J. Fleet,
International Conference in Machine Learning (ICML), 2011. [pdf] [code] [slides:ppt]

Stacks of Convolutional Restricted Boltzmann Machines for Shift-Invariant Feature Learning,
Mohammad Norouzi, Mani Ranjbar, Greg Mori,
IEEE Computer Vision and Pattern Recognition (CVPR), 2009. [pdf]
Extended version: Master's thesis at Simon Fraser University, 2009. [pdf] [slides:ppt]


Recorded talks

[81 mins] Lecture on "Towards a unified view of supervised learning and reinforcement learning" at UC Berkeley, 2017

[18 mins] Zero-Shot Learning by Convex Combination of Semantic Embeddings at ICLR 2014

[13 mins] Cartesian k-means at CVPR 2013

[20 mins] Minimal Loss Hashing for Compact Binary Codes at ICML 2011

Past

PhD student, Computer Science, University of Toronto
Advisor: David Fleet, Sep 2010 - Dec 2015
Teaching assistant for
CSC2503: Foundations of Computer Vision (Grad),
CSC411: Machine Learning and Data Mining,
CSC373: Algorithm Design, Analysis and Complexity,
CSC263: Data Structures and Analysis,
CSC236: Intro to Theory of Computation.

Research Intern, Google, Mountain View, CA USA.
Mentors: Samy Bengio, Yoram Singer - Summer 2013.

Research Intern, Microsoft Research, Cambridge UK.
Mentor: Pushmeet Kohli - Spring 2013.