This question occurred to me while rewording the title of that one. I hope that an educated answer to the present question can perhaps help figure out an answer to that other one which would not be coordinate-dependent.
My favourite interpretation of the concept of conjugation in geometry and algebra consists of thinking that you have a group acting on a set $X$, and you want to trick this group into acting on another set $Y$. For this, you dress up all elements of $Y$ as elements of $X$ (via $f:Y\to X)$, you have your group act on $X$, and then you de-make-up (via $f^{-1}:X\to Y)$.
The most common requirement is for $f$ to have an inverse, but at the very least you need $\operatorname{Im}f$ to be stable under the action.
Now imagine that $f$ is a linear map $\Bbb R^d\to \Bbb R^n$ with $d<n$, which clearly cannot be surjective (in the original question $f$ was an isometric embedding, that is, represented by an $n\times d$ matrix $U$ such that $U^TU$ is the $d\times d$ identity.
You cannot use $f$ to conjugate just any old action on $\Bbb R^n$, since $\operatorname{Im}f$ will not necessarily be stable. But, absent $U^{-1}$, you can still travel back to $\Bbb R^d$ by using $U^T$. This is as close as you can get to an inverse to $U$, since $U^TU=\operatorname{Id}_d$. Let's call this pseudo-conjugation of $R$ by $U$: $$U^TRU$$
Does this operation have a name? Are there any contexts in mathematics where it is commonly used, where actual conjugation cannot be afforded?
I would say that if $U: \mathbb{R}^n \to V$ is an isometric embedding, then the conjugation $R \mapsto U^T R U$ can be thought of as taking a certain block of a block matrix representation of $R$. Let me explain.
Any time you have a direct sum decomposition $V = A \oplus B$ of a vector space, the direct sum defines inclusions $$ i_A: A \to V, \quad i_B: B \to V,$$ and projections $$ p_A: V \to A, \quad p_B: V \to B,$$ such that various equations hold: $$ \begin{aligned} p_A \circ i_A &= \operatorname{id}_A & p_B \circ i_A = 0 \\ p_B \circ i_B &= \operatorname{id}_B & p_A \circ i_B = 0 \\ i_A \circ p_A + i_B \circ p_B &= \operatorname{id}_V. \end{aligned}$$ These inclusions and projections allow us to decompose an operator $R: V \to V$ into a block matrix along the direct sum decomposition $V = A \oplus B$, as follows: $$ R = \begin{pmatrix} R_{AA} & R_{BA} \\ R_{AB} & R_{BB} \end{pmatrix} = \begin{pmatrix} p_A \circ R \circ i_A & p_A \circ R \circ i_B \\ p_B \circ R \circ i_A & p_B \circ R \circ i_B \end{pmatrix}, $$ where $R_{AB}$ is a map $A \to B$, and so on. You should check that this is the exact same block decomposition that you would get by choosing bases for $A$ and $B$, and writing out the matrix of $A$ in those bases.
Now, if $U: \mathbb{R}^n \to V$ is an isometric embedding, then it defines a direct sum decomposition of $\operatorname{im}(U)$ and $\operatorname{im}(U)^\perp = \ker(U^\perp)$. So writing $A = \operatorname{im}(U)$ and $B = \ker U^\perp$, we find that $U$ is the inclusion $i_A$ from before, and $U^\perp$ is the projection $p_A$ from before. So the operation $R \mapsto U^T R U = p_A R i_A$ is exactly taking the top-left block of the block matrix of $R$, corresponding to the part of $R$ which acts only on $\operatorname{im}(U)$.