Continual Learning in Practice

In the real world, we are often faced with situations where data distributions are shifting over time, and we would like to make predictions about things we have never seen before. These situations fall under the umbrella of “Continual Learning”, which has gained popularity in recent years in the AI/ML community. Researchers have focused methods that can adapt to new tasks while preventing “catastrophic forgetting” (e.g. in neural networks) through regularization, generative replay, approximate Bayesian approaches etc. I argue, however, that these are far away from the reality of production environments. In this talk I will present efforts we have been making to bridge this gap, tackling the problem from both research and engineering perspectives.