News

Gradient Descent with Momentum

Gradient Descent with Momentum considers the past gradients to smooth out the update. It computes an exponentially weighted average of your gradients, and then use that gradient to update your weights instead. This article shall explain the algorithm in details.
Want to leave a comment?