recently I am stuck in solving a geometric problem. I hope someone could give me some tips, thanks for all in advance!!!
Question 1: given a constant polygon $M1$ with 4 vertices: $(1,0),(1,1),(0,1),(0,0)$, a variable point $p1: ({ x_{0} , y_{0} })$ and a variable matrix: $H\in \mathbb{R^{2\times 2}}$.
If a linear transformation using $H$ is conducted to polygon $M1$ and point $p1$, we will get a transformed polygon $M2$ together with point $p2$.
e.g. in the picture below, the red constant polygon $M1$ with a random generated red point $p1= (0.58,0.22)$, a random generated $H=\begin{bmatrix}0.12& 0.96\\0.5&0.34\end{bmatrix}$, after transformation $ \begin{bmatrix}0 & 1 & 1& 0& 0.58\\0 & 0 & 1&1& 0.22 \end{bmatrix}^T \cdot \begin{bmatrix}0.12& 0.96\\0.5&0.34\end{bmatrix} $, we will get the blue transformed polygon $M2$ and the blue point $p2$.

The question is, is it possible to establish a function, so that for an arbitrary given point $p1$ and an arbitrary given matrix $H$, the proper vertex in polygon $M1$, which after transformation has the minimal Euclidean distance to point $p2$, could be directly found. Here vertex $(1 , 0)$ in $M1$ corresponds to the vertex in $M2$ that is nearest to point $p2$. It can be formed as an equation as below:
$min \quad d(v\cdot H,p2) = f(H,p1), \quad v\in M1, \quad p2 =p1 \cdot H$
I just want to build an explicit function or look-up table of above implicit constraints.
PS: distance is considered only in polygon $M2$ , not in $M1$, exact distance value is not required, an optimal vertex in $M1$ is already enough, e.g. vertex (1,0) in the image
Question 2: above is just 2d situation, if we extend polygon $M1$ to polyhedron $MP1$ in n dimensions, how would the function look like? Each dimension of the vertex contains always 0 or 1.
e.g. the vertex would be like (1,0,1) in 3d and (1,0,1,1) in 4d. For n dimensions there would be in total $2^n$ vertices in polyhedron $MP1$.
Point $p1$ is still a variable in n dimensions and the transformation matrix $H \in \mathbb{R^{n\times n}}$
UPDATE: A small example to demonstrate that the closest vertex to $p1$ in $M1$ may not be the closest one to $p2$ in $M2$.
Consider $p1 = (0.6,0.4)$ and $H = \begin{bmatrix}0.8 & 1 \\-0.8 & 0 \end{bmatrix} $, then the plot shows as follows:

I am not sure if this problem can be solved efficiently, one guess is that it may be related to the Singular Value Decomposition.
If $H$ is just an upper or lower triangular matrix, would it be a little easier to get the function?
Any idea or guess will be appreciated. Thanks!