in the formula for the calulation of the angle between 2 vectors
$$\cos \theta \overset{\text{def}}= \dfrac{\vec\alpha \cdot \vec\beta}{|\vec \alpha|\cdot |\vec \beta|}$$
is the output angle is always the smaller of the two angles which are between the vectors? what happens if I get a larger than 90 degrees angle?
The angle you get most straightforwardly (by applying arcos to the expression on the right hand side) will be an angle between 0 and $\pi $ radians (0 and 180 degrees), so it may well be larger than 90 degrees. This is because the expression may be negative, which comes from the cosine of an obtuse angle.
With this formula, you get the angle between the vectors, the one made by joining the two vectors at their tail ends and looking at the angle from the head of one vector to the intersection of their tails to the head of the other vector. This may be acute, right, or obtuse. If you look at the orientation of the vectors you may even get a negative angle, but this formula does not distinguish between positive and negative angle, since cosine is an even function. If orientation matters, you would use something such as the cross product, rather than the dot product, to find the orientation.