This is a machine-learning-focused Podcast, where we interview people in the field of Artificial Intelligence and discuss interesting technical topics of Machine Learning. In the episodes, we focus on business-related use-cases (especially with Deep Learning ) and we also try to bring some technical white papers to the ground, not forgetting on the way that there are always some people behind the technology, so we try to understand their motivation and drive.
http://mlcafe.libsyn.com/website
episode 4: Cutting-Edge Gradient Descent Variants with Levente Szabados - 004
In this episode, we are focusing on Deep Learning Optimizers, the different Gradient Descent Variants from vanilla GD to RAdam and Ranger. Levente tells us the story of GD from the simplest ones to the newest ones. This is part 1.
Levente's Linkedin URL:
https://www.linkedin.com/in/levente-szabados-76334728/
An overview of gradient descent optimization
algorithms:
https://arxiv.org/pdf/1609.04747.pdf
Lookahead optimizer algorithm:
https://arxiv.org/pdf/1907.08610.pdf
Ranger optimizer by Less Wright:
https://medium.com/@lessw/new-deep-learning-optimizer-ranger-synergistic-combination-of-radam-lookahead-for-the-best-of-2dc83f79a48d
---Copyright Info---
Music is from https://filmmusic.io, intro first part is by Miklos Toth and some free garage band loops. :) intro second part: "Aces High" by Kevin MacLeod, outro "Acid Trumpet" by Kevin MacLeod (https://incompetech.com), License: CC BY (http://creativecommons.org/licenses/by/4.0/)