Is this okay to prove $\frac{1}{x}$ uniformly continuous on $(2,\infty)$
Let $\epsilon>0$ choose $\delta=4\epsilon.$
Then $|x-y|<\delta \implies |\frac{1}{x}-\frac{1}{y}|<\epsilon.$
Is it okay to choose $4$ even though $2$ is not in the domain of the function?
It is okay. It doesn't matter whether $\delta$ itself is or is not an element of the domain.
In the definition of uniform continuity, the numerical value of $\delta$ does not represent an element of the domain. Instead it represents distance between elements of the domain. For example, when you read the inequality "$|x-y|<\delta$" out loud to yourself, you should be saying "the distance between $x$ and $y$ is less than $\delta$".
Notice in particular: You can find a pair of elements of the domain $(2,\infty)$ which have distance $1$; you can find a pair which have distance $.000001$; and in fact if someone hands you any positive number whatsoever, no matter how small, you can find a pair of elements of the domain $(2,\infty)$ which have that distance.