The graph of the function $f:x \mapsto x^{1/3}$ has a 'vertical tangent' at $x=0$:
Although this idea is certainly geometrically sound, from what I understand the tangent line is defined by the derivative, not vice versa. In other words, the tangent line to a function at the point $(a,f(a))$ is simply the line given by the equation $$ y - f(a) = f'(a)(x-a) \, , $$ where $f'(a)$ is of course defined as a limit. Since $f'(0)$ does not exist in this case, I'm unsure if we can truly say that the graph has a vertical tangent. The intuitive idea of a tangent 'just touching' the curve breaks down when we consider, for instance, the graph of a linear function, where the tangent touches the graph of the function itself at infinitely many points. Nevertheless, I have heard people say that the tangent line is fundamentally a geometric concept. Although the slope of the tangent line 'agrees' with the derivative if the derivative exists, there are instances where the tangent line is a meaningful concept even when the derivative doesn't exist. If this be the case, then what is the formal definition of a tangent?

Your geometric intuition is sound. The problem about taking the derivative is an artifact of the coordinate system used to define it. This is solved in differential geometry where a space is covered by an atlas of compatible coordinate charts. This way, we can choose the right coordinate chart for the problem at hand. For example, to find the 'vertical tangent', we swap the coordinate axes, and find that this tangent is now 'horizontal'. Of course, the terms 'horizontal' and 'vertical' relate to our choice of chart. In more invariant geometric language, what we want to understand is how the curve actually curves in the plane. And what we see is that the location of your 'vertical tangent' is better described as the location of an inflection point
In differential geometry, you will see that both the geometric and derivational notion of a tangent dovetail exactly. In fact, this occurs so naturally, that in some treatments of differential geometry, it's the derivations - aka the derivative - that takes the starring role - as the development is simpler.
(In more sophisticated treatments of differential geometry for singular spaces or infinite-dimensional spaces it's seen that they diverge again, though of course there remains an intimate relationship. For example, in Michors & Kriegl conveniant calculus for infinite dimensional spaces it is the geometric tangent along curves that takes the starring role and the derivational is dispensed with. The same goes for Neeb's development of calculus in locally convex spaces (again it's the infinite-dimensional context that is important). In contrast, in Souriau's diffeology and developed by Isglesias-Zemmour (important for both infinite-dimensional and singular spaces and vastly more general than the foregoing), a notion of smoothness is introduced that is inspired from topology. Here, Christensen after correcting a definition by Hector and Laubinger, shows that there are distinct geometric and derivational tangent vectors. They are not the same here, but of course they are in many natural cases).