I know that a function is defined as something where very input has a unique output, but does "undefined" or "infinity" count as a unique output?
In other words, would a reciprocal function count as a formally defined function? Or would it not because of the asymptote?
When defining a function, it is crucial to consider the set that we are inputting (the domain) and the set that we are outputting (the codomain). The notion of "undefined" is not a unique output because it is not a point in any sort of codomain. In certain special cases, we can define a specific output of a function to be infinity (or a point "at infinity"), but we do not regard a function as well defined if the function makes no sense for some point in the domain.
To answer your other question, let us just consider functions which input real numbers and output real numbers. The reciprocal function is certainly a well defined function, but only in domains which do not include the point zero. Around the asymptote, the function behaves nicely, and we can say it is well defined there because the points approaching the asymptote is the output of numbers approaching zero. It is only at the point zero where we have a problem.
So the answer to your question is that it can be both. If zero is in our domain, then we do not have a well defined function. If zero is not in our domain, then we have a well defined function.