There are gradient approximating methods like central difference approximation, forward approximation and backward approximation. However, I would like to understand in what situations such approximations are necessary and are helpful over the analytical method of computing gradients. I have heard that analytical gradients cannot be computed for all functions, but would like to know an example for such case so that I can understand better.
Thank you in advance!