The answer scheme tells that C is the critical point and the the inequality $Reaction>0$ should be used over C.
How is this point C decided? Why not B or midpoint of BC? Is there a way to formula a function whose stationary points will indicate that C is the critical point?
Please provide (if possible) both intuitive (using physics) explanation and mathematical explanation.
A particle in free fall follows a parabolic path. The particle losing contact from the circular track in some point between $B$ and $C$, say $D$, means two things 1) that the trajectory is now parabolic and that 2) the parabola the particle is following is tangent to the circular track(*) in $D$, thus, it will never get contact again with the track. So, you have to assume that the particle is in $C$ and right then loses contact, so is, the particle is in $C$ and the force from the track is zero there.
At $C$, the particle is still in a circular path, so, the acceleration in the direction perpendicular to the track is $v^2/R$, with $v$ the particle's speed and $R$ the track radius. By other side, the force along that direction is the difference between the component of gravitational force along that direction and the normal reaction of the track: $mg\cos40º-N$ The Newton's second law says:
$mg\cos\alpha-N=mv^2/R$ and, with the condition of $N=0$, $v_{max}=\sqrt{gR\cos\alpha}$
You can get $v$ from the energy conservation theorem and with some trigonometry to calculte the heigh $h_{max}$ at wich the particle has to be initially.
(*) I am not sure whether or not you need a proof about this "no return" of a circle tangent to a parabola. As conviction, draw a circle tangent to a parabola and see.