When we solve the Laplace equation in spherical polar coordinate, we get the radial part whose solution is:
$$R=Ar^l+Br^{-(l+1)}$$
Now, some solutions keep this two terms, but when we derive the Hydrogen Atom Wave Equation, we usually kill $Br^{-(l+1)}$ since $l+1$ is not well Normalized so we kill it. The interesting part is, this solution turns out to predict the behavior of Hydrogen atom to a high degree of accuracy! Math is truly great! :)
Now, if its not well normalized why do some DE solutions keep it?
Or in a more general way, what is the right circumstance to keep or kill either $r^l$ or $r^{-(l+1)}$?
This all depends on the desired behaviour of the solution as $r \to 0$. In physical applications, it is often reasonable to demand that the solution you're looking for is bounded as $r \to 0$. In your case, the differential equation gives two linearly independent solutions, which is equivalent to the fact that you can choose $A$ and $B$ 'freely'. That is, this is the moment when you're going to apply the given boundary conditions. Usually, these conditions give the value of the solution at a certain point; however, boundary conditions may also come in the form of a limit -- in your case, the condition that the limit of the solution for $r \to 0$ actually exists.
In other problems, such a 'limit condition' may take other forms. For example, it may very well possible that the boundary condition for a certain solution $\phi$ is \begin{equation} \lim_{r \to 0} r^2 \phi(r) \quad \text{exists}. \end{equation} In your case, this condition is satisfied for any choice of $A$; it is also satisfied for any choice of $B$ as long as $l \leq 1$. So, to conclude: the choice of 'free' constants $A$ and $B$ depends on the boundary conditions, which are very much related to the physical interpretation of the problem at hand.