I have some questions about fraction calculus.
In Caputo-Fabrizio fractional derivative
$$ ^{CF}_aD_t^\alpha f(t)=\frac{M(\alpha)}{1-\alpha}\int_a^t f'(\tau)exp(-\alpha\frac{t-\tau}{1-\alpha})d\tau $$
And Atangana-Baleanu derivative
$$ ^{AB}_aD^\alpha_t f(t)=\frac{AB(\alpha)}{1-\alpha}\int^t_a f'(\tau) E_\alpha(-\alpha\frac{(t-\tau)^\alpha}{1-\alpha})d\tau $$
I searched related papers, they all say $M(\alpha)$ and $AB(\alpha)$ are normalization function, but I still don't understand what they really are and how can I calculate them.
Hopefully I can revive a discussion on this question as I am quite interested too. My understanding of a "normalisation function" in this sense is a function that reacts to changes in an equation to keep it inside a certain range. In a statistical sense, we often form a normalisation to set an integral to value $1$ over an entire domain. However it is obvious in this sense we do not want our integral always being of value $1$! Therefore I have drawn the conclusion we are instead "normalising" the input $\alpha$ to be valid for the entire range $\alpha \in [0,1]$.
With this definition for the Caputo-Fabrizio fractional derivative, the normalisation function aims to achieve a value such that: $$M(0)=M(1)=1$$ As seen in the original 2015 paper here.
If $\alpha=0$ we are, going off the correct definition in your question, going to see the value of the equation go to zero. The normalisation function $M$ prevents this.
So, what exactly does this achieve? I will try to work through an example with $\alpha = 0$ to show how this prevents the function setting to zero, when a zero order derivative should be the original function. Here I am just assessing the zero order derivative of a plain variable $t$: $$ ^{CF}_0D_t^0 t=\frac{M(0)}{1-0}\int_0^t f'(\tau)exp(-0\frac{t-\tau}{1-0})d\tau $$ $$ ={M(0)}\int_0^t exp(0)d\tau $$ $$ ={M(0)}\left[\tau\right]^t_0 = M(0)t $$ From here we can see how it is essential for $M(0)$ to equal 1, otherwise the zero-order derivative would not work!
So after all of that we can reach the conclusion this function is to prevent everything going to zero when $\alpha$ is zero. From this paper, it looks like the Atangana-Baleanu derivative works in the exact same way, as expected as they are related.
The final part of your question in terms of "calculating" them has hopefully been answered, as it should just be the value of $\alpha$ (but $\alpha = 0$ goes to $1$).