I am currently doing a Deep Learning project, and I am trying to decrease my learning rate by a factor of 5.
If my Learning Rate is 0.0004, then would 0.0004 * 0.05 decrease it by a factor of 5? My dyslexic brain and poor English language does not let me understand this.
My model has an initial Learning Rate which is 0.0004. I want to reduce it by a factor of N if the accuracy of the model does not increase after a set amount of epochs. Is it division or multiplication?
Just for reference, I am trying to implement this in my project. The factor parameter says:
factor: factor by which the learning rate will be reduced. new_lr = lr * factor.
Sorry if this is a truly silly question, but I am stuck! Thank you :)
To reduce a quantity, $q$, by a factor of $n$, simply divide by $n$.
$\text{Reduced factor} = q * n^{-1} = q / n$