Mohammad Norouzi
mnorouzi[at]google[.]com

I am a senior research scientist at Google Brain in Toronto. I am interested in developing simple and efficient machine learning algorithms that are broadly applicable across a range of domains including natural language processing and computer vision.

Currently, I am highlighting:

My current research focuses on:

  1. Building statistical models of sequential data, sentences, programs, images, and other structured objects.
  2. Advancing reinforcement learning algorithms and applications.
  3. Unifying 1 & 2.

I joined the brain team in Mountain View in January, 2016 and moved to Toronto in January, 2018. I completed my PhD in computer science at the University of Toronto in December, 2015. My advisor was Prof. David Fleet, and I was supported by a Google PhD fellowship in machine learning. My PhD thesis focused on scalable similarity search for web-scale data collections. I am from Iran and I finished my undergraduate studies at Sharif University of Technology.

Google scholar profile
Curriculum Vitae
GitHub page

Publications
Preprints

Optimal Completion Distillation for Sequence Learning
Sara Sabour, William Chan, Mohammad Norouzi
ArXiv, 2018. [pdf]

Contingency-Aware Exploration in Reinforcement Learning
Jongwook Choi, Yijie Guo, Marcin Moczulski, Junhyuk Oh, Neal Wu, Mohammad Norouzi, Honglak Lee
ArXiv, 2018. [pdf] [web]

2018

Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning
Supasorn Suwajanakorn, Noah Snavely, Jonathan Tompson, Mohammad Norouzi
NIPS, 2018 (Oral). [pdf] [code] [web]

Memory Augmented Policy Optimization for Program Synthesis with Generalization
Chen Liang, Mohammad Norouzi, Jonathan Berant, Quoc Le, Ni Lao
NIPS, 2018 (Spotlight). [pdf] [code]

Sequence to Sequence Mixture Model for Diverse Machine Translation
Xuanli He, Gholamreza Haffari, Mohammad Norouzi
CoNLL, 2018. [pdf]

The Importance of Generation Order in Language Modeling
Nicolas Ford, Daniel Duckworth, Mohammad Norouzi, George E Dahl
EMNLP, 2018. [pdf]

Parallel Architecture and Hyperparameter Search via Successive Halving and Classification
Manoj Kumar, George E Dahl, Vijay Vasudevan, Mohammad Norouzi
ArXiv, 2018. [pdf]

Embedding Text in Hyperbolic Spaces
Bhuwan Dhingra, Christopher J Shallue, Mohammad Norouzi, Andrew M Dai, George E Dahl
TextGraphs Workshop, 2018. [pdf]

Artificial Intelligence–Based Breast Cancer Nodal Metastasis Detection: Insights Into the Black Box for Pathologists
Yun Liu, Timo Kohlberger, Mohammad Norouzi, George E Dahl, Jenny L Smith, Arash Mohtashamian, Niels Olson, Lily H Peng, Jason D Hipp, Martin C Stumpe
Archives of Pathology & Laboratory Medicine, 2018. [pdf]

Smoothed Action Value Functions for Learning Gaussian Policies
Ofir Nachum, Mohammad Norouzi, George Tucker, Dale Schuurmans
ICML, 2018. [pdf]

Trust-PCL: An Off-Policy Trust Region Method for Continuous Control
Ofir Nachum, Mohammad Norouzi, Kelvin Xu, Dale Schuurmans
ICLR, 2018. [pdf]

QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
Adams Wei Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, Quoc V Le
ICLR, 2018. [pdf] [code]

Neural Program Synthesis with Priority Queue Training
Daniel A Abolafia, Mohammad Norouzi, Jonathan Shen, Rui Zhao, Quoc V Le
ArXiv, 2010. [pdf] [code]

2017

Bridging the Gap Between Value and Policy Based Reinforcement Learning
Ofir Nachum, Mohammad Norouzi, Kelvin Xu, Dale Schuurmans
NIPS, 2017. [pdf]

Filtering Variational Objectives
Chris J Maddison, Dieterich Lawson, George Tucker, Nicolas Heess, Mohammad Norouzi, Andriy Mnih, Arnaud Doucet, Yee Whye Teh
NIPS, 2017. [pdf]

PixColor: Pixel Recursive Colorization
Sergio Guadarrama, Ryan Dahl, David Bieber, Mohammad Norouzi, Jonathon Shlens, Kevin Murphy
BMVC, 2017. [pdf]

Pixel Recursive Super Resolution
Ryan Dahl, Mohammad Norouzi, Jonathon Shlens
ICCV, 2017. [pdf]

Deep Value Networks Learn to Evaluate and Iteratively Refine Structured Outputs
Michael Gygli, Mohammad Norouzi, Anelia Angelova
ICML, 2017. [pdf] [image segmentation process -- gif files] [slides] [code]

Device Placement Optimization with Reinforcement Learning
Azalia Mirhoseini, Hieu Pham, Quoc V Le, Benoit Steiner, Rasmus Larsen, Yuefeng Zhou, Naveen Kumar, Mohammad Norouzi, Samy Bengio, Jeff Dean
ICML, 2017. [pdf]

Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders
Jesse Engel, Cinjon Resnick, Adam Roberts, Sander Dieleman, Douglas Eck, Karen Simonyan, Mohammad Norouzi
ICML, 2017. [pdf]

Detecting Cancer Metastases on Gigapixel Pathology Images
Yun Liu, Krishna Gadepalli, Mohammad Norouzi, George E. Dahl, Timo Kohlberger, Aleksey Boyko, Subhashini Venugopalan, Aleksei Timofeev, Philip Q. Nelson, Greg S. Corrado, Jason D. Hipp, Lily Peng, Martin C. Stumpe
ArXiv, 2017. [pdf]

Neural Combinatorial Optimization with Reinforcement Learning
Irwan Bello, Hieu Pham, Quoc V Le, Mohammad Norouzi, Samy Bengio
ICLR workshop, 2017. [pdf]

Improving Policy Gradient by Exploring Under-appreciated Rewards
Ofir Nachum, Mohammad Norouzi, Dale Schuurmans
ICLR, 2017. [pdf] [poster]

2016

Google's neural machine translation system: Bridging the Gap between Human and Machine Translation
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Łukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, Jeffrey Dean
ArXiv, 2016. [pdf]

Reward Augmented Maximum Likelihood for Neural Structured Prediction
Mohammad Norouzi, Samy Bengio, Zhifeng Chen, Navdeep Jaitly, Mike Schuster, Yonghui Wu, Dale Schuurmans
NIPS, 2016. [pdf] [poster]

Compact Discrete Representations for Scalable Similarity Search
Mohammad Norouzi, PhD thesis, 2016. [pdf]

2015 and before

Efficient Non-greedy Optimization of Decision Trees
Mohammad Norouzi, Maxwell D. Collins, Matthew Johnson, David J. Fleet, Pushmeet Kohl,
NIPS, 2015. [pdf]

CO2 Forest: Improved Random Forest by Continuous Optimization of Oblique Splits
Mohammad Norouzi, Maxwell D. Collins, David J. Fleet, Pushmeet Kohli,
ArXiv, 2015. [pdf]

Zero-Shot Learning by Convex Combination of Semantic Embeddings
Mohammad Norouzi, Tomas Mikolov, Samy Bengio, Yoram Singer, Jonathon Shlens, Andrea Frome, Greg S. Corrado, Jeffrey Dean,
ICLR, 2014. [pdf] [slides:pptx] [dataset: 2-hop, 3-hop]

Fast Exact Search in Hamming Space with Multi-Index Hashing,
Mohammad Norouzi, Ali Punjani, David J. Fleet,
TPAMI, vol. 36, no. 6, 2014. [pdf] [code]

Cartesian k-means,
Mohammad Norouzi, David J. Fleet,
CVPR, 2013. [pdf] [code] [slides:ppt/pptx]

Hamming Distance Metric Learning,
Mohammad Norouzi, David J. Fleet, Ruslan Salakhutdinov,
NIPS, 2012. [pdf] [code] [supplemental] [poster]

Fast Search in Hamming Space with Multi-Index Hashing,
Mohammad Norouzi, Ali Punjani, David J. Fleet,
CVPR, 2012. [pdf] [code] [poster]

Minimal Loss Hashing for Compact Binary Codes,
Mohammad Norouzi, David J. Fleet,
ICML, 2011. [pdf] [code] [slides:ppt]

Stacks of Convolutional Restricted Boltzmann Machines for Shift-Invariant Feature Learning,
Mohammad Norouzi, Mani Ranjbar, Greg Mori,
CVPR, 2009. [pdf]
Extended version: Master's thesis at Simon Fraser University, 2009. [pdf] [slides:ppt]


Recorded talks

[18 mins] Deep Value Networks Learn to Evaluate and Iteratively Refine Structured Outputs at ICML 2017

[81 mins] Lecture on "Towards a unified view of supervised learning and reinforcement learning" at UC Berkeley, 2017

[18 mins] Zero-Shot Learning by Convex Combination of Semantic Embeddings at ICLR 2014

[13 mins] Cartesian k-means at CVPR 2013

[20 mins] Minimal Loss Hashing for Compact Binary Codes at ICML 2011

Past

PhD student, Computer Science, University of Toronto
Advisor: David Fleet, Sep 2010 - Dec 2015

Research Intern, Google, Mountain View, CA USA.
Mentors: Samy Bengio, Yoram Singer - Summer 2013.

Research Intern, Microsoft Research, Cambridge UK.
Mentor: Pushmeet Kohli - Spring 2013.

Teaching assistant for
CSC2503: Foundations of Computer Vision (Grad),
CSC411: Machine Learning and Data Mining,
CSC373: Algorithm Design, Analysis and Complexity,
CSC263: Data Structures and Analysis,
CSC236: Intro to Theory of Computation.