Let $X$ and $Y$ be two $n$-dimensional finite random vectors. They are distributed on $n$-dimensional surfaces. Their probability density functions are not known but are unlikely to fit any of the standard ones. There exists an affine transformation that maps $X$ to $Y$ (i.e. $Y = AX + b$, where $A$ is $n \times n$-matrix and $b$ is $n$-vector).
Given $k_X$ samples from $X$ and $k_Y$ samples from $Y$, how does one find transformation (i.e. $A$ and $b$), that fits $X$ to $Y$ as closely as possible, for some reasonable definition of closeness.
One can estimate translation encoded in $b$ as difference of their means, i.e $b = E\{Y\} - E\{X\}$, where means are estimated from given samples. But how does one estimate rotation and scaling within $A$?