Mohammad Norouzi

I am a machine learning engineer and innovator. Currently, I am working at a new startup called Ideogram AI. Our mission is to help people become more creative through Generative AI. We're hiring!

Here are some of my recent projects:

My work focuses on generative models. Previously, I was a senior staff research scientist at Google Brain in Toronto. I joined the Google brain team in Mountain View in January 2016, and moved to Toronto in January 2018. Before that, I completed my PhD in computer science at the University of Toronto in December 2015, under the supervision of David Fleet. I was supported by a Google PhD fellowship in machine learning. My PhD thesis focused on scalable similarity search. I am from Iran, where I finished my undergraduate studies at Sharif University of Technology.

Google scholar
LinkedIn
Twitter

Publications

Big Self-Supervised Models Advance Medical Image Classification
Shekoofeh Azizi, Basil Mustafa, Fiona Ryan, Zachary Beaver, Jan Freyberg, Jonathan Deaton, Aaron Loh, Alan Karthikesalingam, Simon Kornblith, Ting Chen, Vivek Natarajan, Mohammad Norouzi
[pdf]

What's in a Loss Function for Image Classification?
Simon Kornblith, Honglak Lee, Ting Chen, Mohammad Norouzi
[pdf]

2021

Autoregressive Dynamics Models for Offline Policy Evaluation and Optimization
Michael R Zhang, Thomas Paine, Ofir Nachum, Cosmin Paduraru, George Tucker, Ziyu Wang, Mohammad Norouzi
ICLR 2021. [pdf]

Benchmarks for Deep Off-Policy Evaluation
Justin Fu, Mohammad Norouzi, Ofir Nachum, George Tucker, Ziyu Wang, Alexander Novikov, Mengjiao Yang, Michael R Zhang, Yutian Chen, Aviral Kumar, Cosmin Paduraru, Sergey Levine, Thomas Paine
ICLR 2021. [pdf]

WaveGrad: Estimating gradients for waveform generation
Nanxin Chen, Yu Zhang, Heiga Zen, Ron J Weiss, Mohammad Norouzi, William Chan
ICLR 2021. [pdf]

Mastering Atari with discrete world models
Danijar Hafner, Timothy Lillicrap, Mohammad Norouzi, Jimmy Ba
ICLR 2021. [pdf]

No MCMC for me: Amortized sampling for fast and stable training of energy-based models
Will Grathwohl, Jacob Kelly, Milad Hashemi, Mohammad Norouzi, Kevin Swersky, David Duvenaud
ICLR 2021. [pdf]

2020

Big self-supervised models are strong semi-supervised learners
Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, Geoffrey Hinton
NeurIPS 2020. [pdf]

Exemplar VAE: Linking Generative Models, Nearest Neighbor Retrieval, and Data Augmentation
Sajad Norouzi, David J Fleet, Mohammad Norouzi
NeurIPS 2020. [pdf]

Memory Based Trajectory-conditioned Policies for Learning from Sparse Rewards
Yijie Guo, Jongwook Choi, Marcin Moczulski, Shengyu Feng, Samy Bengio, Mohammad Norouzi, Honglak Lee
NeurIPS 2020. [pdf]

RL Unplugged: A Collection of Benchmarks for Offline Reinforcement Learning
Caglar Gulcehre, Ziyu Wang, Alexander Novikov, Thomas Paine, Sergio Gómez, Konrad Zolna, Rishabh Agarwal, Josh S. Merel, Daniel J. Mankowitz, Cosmin Paduraru, Gabriel Dulac-Arnold, Jerry Li, Mohammad Norouzi, Matthew Hoffman, Nicolas Heess, Nando de Freitas
NeurIPS 2020. [pdf]

Non-autoregressive machine translation with latent alignments
Chitwan Saharia, William Chan, Saurabh Saxena, Mohammad Norouzi
EMNLP 2020. [pdf]

Dynamic Programming Encoding for Subword Segmentation in Neural Machine Translation
Xuanli He, Gholamreza Haffari, Mohammad Norouzi
ACL 2020. [pdf]

NASA: Neural articulated shape approximation
Timothy Jeruzalski, Boyang Deng, Mohammad Norouzi, John P Lewis, Geoffrey Hinton, Andrea Tagliasacchi
ECCV 2020. [pdf]

Imputer: Sequence modelling via imputation and dynamic programming
William Chan, Chitwan Saharia, Geoffrey Hinton, Mohammad Norouzi, Navdeep Jaitly
ICML 2020. [pdf]

A Simple Framework for Contrastive Learning of Visual Representations
Ting Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey Hinton
ICML 2020. [pdf]

SUMO: Unbiased Estimation of Log Marginal Probability for Latent Variable Models
Yucen Luo, Alex Beatson, Mohammad Norouzi, Jun Zhu, David Duvenaud, Ryan P Adams, Ricky TQ Chen
ICLR 2020. [pdf]

Dream to control: Learning behaviors by latent imagination
Danijar Hafner, Timothy Lillicrap, Jimmy Ba, Mohammad Norouzi
ICLR 2020. [pdf]

2019

Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapse
James Lucas, George Tucker, Roger B Grosse, Mohammad Norouzi
NeurIPS 2010. [pdf]

Learning to Generalize from Sparse and Underspecified Rewards
Rishabh Agarwal, Chen Liang, Dale Schuurmans, Mohammad Norouzi
ICML, 2019. [pdf]

Similarity of Neural Network Representations Revisited
Simon Kornblith, Mohammad Norouzi, Honglak Lee, Geoffrey Hinton
ICML, 2019. [pdf]

Understanding the Impact of Entropy on Policy Optimization
Zafarali Ahmed, Nicolas Le Roux, Mohammad Norouzi, Dale Schuurmans
ICML, 2019. [pdf]

Optimal Completion Distillation for Sequence Learning
Sara Sabour, William Chan, Mohammad Norouzi
ICLR, 2019. [pdf]

Contingency-Aware Exploration in Reinforcement Learning
Jongwook Choi, Yijie Guo, Marcin Moczulski, Junhyuk Oh, Neal Wu, Mohammad Norouzi, Honglak Lee
ICLR, 2019. [pdf] [web]

Artificial intelligence–based breast cancer nodal metastasis detection: Insights into the black box for pathologists
Yun Liu, Timo Kohlberger, Mohammad Norouzi, George E Dahl, Jenny L Smith, Arash Mohtashamian, Niels Olson, Lily H Peng, Jason D Hipp, Martin C Stumpe
Archives of pathology & laboratory medicine, 2019. [pdf]

2018

Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning
Supasorn Suwajanakorn, Noah Snavely, Jonathan Tompson, Mohammad Norouzi
NeurIPS, 2018 (Oral). [pdf] [code] [web]

Memory Augmented Policy Optimization for Program Synthesis with Generalization
Chen Liang, Mohammad Norouzi, Jonathan Berant, Quoc Le, Ni Lao
NeurIPS, 2018 (Spotlight). [pdf] [code]

Sequence to Sequence Mixture Model for Diverse Machine Translation
Xuanli He, Gholamreza Haffari, Mohammad Norouzi
CoNLL, 2018. [pdf]

The Importance of Generation Order in Language Modeling
Nicolas Ford, Daniel Duckworth, Mohammad Norouzi, George E Dahl
EMNLP, 2018. [pdf]

Parallel Architecture and Hyperparameter Search via Successive Halving and Classification
Manoj Kumar, George E Dahl, Vijay Vasudevan, Mohammad Norouzi
Technical Report, 2018. [pdf]

Embedding Text in Hyperbolic Spaces
Bhuwan Dhingra, Christopher J Shallue, Mohammad Norouzi, Andrew M Dai, George E Dahl
TextGraphs Workshop, 2018. [pdf]

Smoothed Action Value Functions for Learning Gaussian Policies
Ofir Nachum, Mohammad Norouzi, George Tucker, Dale Schuurmans
ICML, 2018. [pdf]

Trust-PCL: An Off-Policy Trust Region Method for Continuous Control
Ofir Nachum, Mohammad Norouzi, Kelvin Xu, Dale Schuurmans
ICLR, 2018. [pdf]

QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
Adams Wei Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, Quoc V Le
ICLR, 2018. [pdf] [code]

Neural Program Synthesis with Priority Queue Training
Daniel A Abolafia, Mohammad Norouzi, Jonathan Shen, Rui Zhao, Quoc V Le
ArXiv, 2010. [pdf] [code]

2017

Bridging the Gap Between Value and Policy Based Reinforcement Learning
Ofir Nachum, Mohammad Norouzi, Kelvin Xu, Dale Schuurmans
NIPS, 2017. [pdf]

Filtering Variational Objectives
Chris J Maddison, Dieterich Lawson, George Tucker, Nicolas Heess, Mohammad Norouzi, Andriy Mnih, Arnaud Doucet, Yee Whye Teh
NIPS, 2017. [pdf]

PixColor: Pixel Recursive Colorization
Sergio Guadarrama, Ryan Dahl, David Bieber, Mohammad Norouzi, Jonathon Shlens, Kevin Murphy
BMVC, 2017. [pdf]

Pixel Recursive Super Resolution
Ryan Dahl, Mohammad Norouzi, Jonathon Shlens
ICCV, 2017. [pdf]

Deep Value Networks Learn to Evaluate and Iteratively Refine Structured Outputs
Michael Gygli, Mohammad Norouzi, Anelia Angelova
ICML, 2017. [pdf] [image segmentation process -- gif files] [slides] [code]

Device Placement Optimization with Reinforcement Learning
Azalia Mirhoseini, Hieu Pham, Quoc V Le, Benoit Steiner, Rasmus Larsen, Yuefeng Zhou, Naveen Kumar, Mohammad Norouzi, Samy Bengio, Jeff Dean
ICML, 2017. [pdf]

Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders
Jesse Engel, Cinjon Resnick, Adam Roberts, Sander Dieleman, Douglas Eck, Karen Simonyan, Mohammad Norouzi
ICML, 2017. [pdf]

Detecting Cancer Metastases on Gigapixel Pathology Images
Yun Liu, Krishna Gadepalli, Mohammad Norouzi, George E. Dahl, Timo Kohlberger, Aleksey Boyko, Subhashini Venugopalan, Aleksei Timofeev, Philip Q. Nelson, Greg S. Corrado, Jason D. Hipp, Lily Peng, Martin C. Stumpe
ArXiv, 2017. [pdf]

Neural Combinatorial Optimization with Reinforcement Learning
Irwan Bello, Hieu Pham, Quoc V Le, Mohammad Norouzi, Samy Bengio
ICLR workshop, 2017. [pdf]

Improving Policy Gradient by Exploring Under-appreciated Rewards
Ofir Nachum, Mohammad Norouzi, Dale Schuurmans
ICLR, 2017. [pdf] [poster]

2016

Google's neural machine translation system: Bridging the Gap between Human and Machine Translation
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Łukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, Jeffrey Dean
ArXiv, 2016. [pdf]

Reward Augmented Maximum Likelihood for Neural Structured Prediction
Mohammad Norouzi, Samy Bengio, Zhifeng Chen, Navdeep Jaitly, Mike Schuster, Yonghui Wu, Dale Schuurmans
NIPS, 2016. [pdf] [poster]

Compact Discrete Representations for Scalable Similarity Search
Mohammad Norouzi, PhD thesis, 2016. [pdf]

≤ 2015

Efficient Non-greedy Optimization of Decision Trees
Mohammad Norouzi, Maxwell D. Collins, Matthew Johnson, David J. Fleet, Pushmeet Kohl,
NIPS, 2015. [pdf]

CO2 Forest: Improved Random Forest by Continuous Optimization of Oblique Splits
Mohammad Norouzi, Maxwell D. Collins, David J. Fleet, Pushmeet Kohli,
ArXiv, 2015. [pdf]

Zero-Shot Learning by Convex Combination of Semantic Embeddings
Mohammad Norouzi, Tomas Mikolov, Samy Bengio, Yoram Singer, Jonathon Shlens, Andrea Frome, Greg S. Corrado, Jeffrey Dean,
ICLR, 2014. [pdf] [slides:pptx] [dataset: 2-hop, 3-hop]

Fast Exact Search in Hamming Space with Multi-Index Hashing,
Mohammad Norouzi, Ali Punjani, David J. Fleet,
TPAMI, vol. 36, no. 6, 2014. [pdf] [code]

Cartesian k-means,
Mohammad Norouzi, David J. Fleet,
CVPR, 2013. [pdf] [code] [slides:ppt/pptx]

Hamming Distance Metric Learning,
Mohammad Norouzi, David J. Fleet, Ruslan Salakhutdinov,
NIPS, 2012. [pdf] [code] [supplemental] [poster]

Fast Search in Hamming Space with Multi-Index Hashing,
Mohammad Norouzi, Ali Punjani, David J. Fleet,
CVPR, 2012. [pdf] [code] [poster]

Minimal Loss Hashing for Compact Binary Codes,
Mohammad Norouzi, David J. Fleet,
ICML, 2011. [pdf] [code] [slides:ppt]

Stacks of Convolutional Restricted Boltzmann Machines for Shift-Invariant Feature Learning,
Mohammad Norouzi, Mani Ranjbar, Greg Mori,
CVPR, 2009. [pdf]
Extended version: Master's thesis at Simon Fraser University, 2009. [pdf] [slides:ppt]