Consider $E$ a vector space over any field and $U,V$ two subspaces of it, such that $U$ is a subspace of $V$, too.
If I want to refer to the subspace $\bar{U}\space\colon\space U \oplus \bar{U}=V$, my only choice is to say that explicitly, there is no operation between subspaces that yields $\bar{U}$ nor a shorter way to express it symbolically. To me, it is shocking that there exists no notation for this, because I have encountered more than one a situation where it would've come in handy. I would've used it more than, say, Knuth's up-arrow notation.
I thought of defining a "direct subtraction" of sorts, so $$U \ominus V \space\colon=\bar{U}, \space\space\space\space\space \text{where} \space U\oplus \bar{U}=V.$$
My questions are: does this make sense?, is this notation not well-defined?, is there any situation where it can lead to confusion?
I have tried, but failed to find any problems with it, so far. In fact, if I'm not wrong, the subspace $U\ominus V$ would be naturally isomorphic to $U/V$. But, then again, the fact that I'm not able to find any notation similar to this one makes me believe that I'm going wrong somewhere.
When you say the subspace $\bar{U}$, you are making an error. Unless there is extra structure on $V$, there is no canonical choice of such a subspace.
Consider $\Bbb{R}^2 = \text{$x$-axis}\oplus \text{$y$-axis}$, but also $\Bbb{R}^2 = \text{$x$-axis}\oplus \text{the line $x=y$}$.
On the other hand, if there is extra structure, this sometimes exists, but we usually call it something different. For example if $V$ is an inner product space, we call your "direct subtraction" the orthogonal complement, $U^\perp$.