Emtiyaz Khan

Research

Humans, animals, and other living beings have a natural ability to autonomously learn throughout their lives and quickly adapt to their surroundings, but computers lack such abilities. My goal is to bridge such gaps between the learning of living-beings and computers. My current focus is on AI that can autonomously learn to perceive, act, and reason throughout their lives.

I work on problems in several areas of machine learning, such as approximate inference, deep learning, reinforcement learning, active learning, online learning, and reasoning in computer vision. In my recent works, I have worked on ideas from a wide range of fields, such as, optimization, Bayesian statistics, information geometry, signal processing, and control systems.

For details of my research activities, see the following pages,

• Approximate Bayesian Inference team at RIKEN-AIP
• The Bayes-Duality Project (funded by CREST)
• Summary of research for the year [2023] [2022] [2021] [2019] [2018] [2017]

We are thankful for the following funding (amount is approximate),

• (2021-2026, USD 2.23 Million) JST-CREST and French-ANR's grant, The Bayes-Duality Project
• (2020-2023, USD 167,000) KAKENHI Grant-in-Aid for scientific Research (B), Life-Long Deep Learning using Bayesian Principles
• (2020-2023, USD 11,000) KAKENHI Grant-in-Aid for Chellenging Research (Exloratory), Linear algebra for continuous learning of large neural networks, PI: Rio Yokota, Total Budget: JPY 19,710,000
• (2019-2022, USD 237,000) External funding through companies for several Bayes related projects

Research Highlights

• The IVON optimizer (for training GPT-2 from scratch)
• Memory Perturbation
• SAM as Bayes
• The Bayesian Learning Rule
• Knowledge-Adaptation Priors
• Continual Deep Learning by Functional Regularisation of Memorable Past
• NeurIPS 2019 Tutorial on "Deep learning with Bayesian principles" [Slides] [SlidesLive] [YouTube]
• Practical Deep Learning with Bayesian Principles
• Bayesian Deep Learning using Weight-Perturbation in Adam (code here)
• Conjugate-Computation Variational Inference (CVI).