In computers we use two's-complement to subtract and it works well, but how does it work for subtraction without two's-complement when subtracting a larger number from a smaller number?
0x0F
- 0x13
------
-1C
Doesn't seem to work like subtracting decimal numbers would.
01
- 11
----
-10
It works if I flip the numbers.
0x13
- 0x0F
------
-4
What am I missing?
Your example for base 10 is poorly chosen. Try $3-12$. What you get is $3-2=1$ and $0-1=-1$ So the answer is $-1\cdot 10+1=-9$. You should apply the same to the hex base. $-10_{16}+B_{16}=-5_{16}$.