we have analytically calculated distance between the centers of a big circle and a small circle in mm.
We also have coordinates of the centers of the mentioned circles - yellow and blue - identified from an the image with image processing algorithm in pixels. Now we have to calculate the size of a pixel in mm in $x$-direction and $y$-direction based on the following information;
- The analytical distance between center of the two circles in mm - $RDistance$
- The center of the two circles identified with image processing in pixels, i.e. $(x_1, y_1)$ and $(x_2, y_2)$
I can calculate a single pixel size with the formula $$\frac{RDistance}{\sqrt{(x_2-x_1)^2 + (y_2-y_1)^2}}.$$
However, the requirement is to calculate the pixel size in both $x$- and $y$-direction differently. How can we calculate the pixel size $x$-direction and pixel size $y$-direction separately?

If the aspect ratio, which @MattiP mentioned in his comment above, is equal to 1, you have, $$\mathrm{pixel\space size\space in}\space y\mathrm{-direction} \space =\mathrm{pixel\space size\space in}\space x\mathrm{-direction} \space=\frac{RDistance}{\sqrt{2\left(\left(x_2-x_1\right)^2 + \left(y_2-y_1\right)^2\right)}}.$$
If the aspect ratio is not equal to 1, then you have to use your image processing algorithm to find out,
$(A).\space$ the coordinates of the center of the $\mathrm{\color{blue}{blue\space circle}}$ directly above the $\mathrm{\color{brown}{yellow\space circle}}$ and
$(B).\space$ the coordinates of the center of the $\mathrm{\color{blue}{blue\space circle}}$ on the left hand side of the $\mathrm{\color{brown}{yellow\space circle}}.$
Denote the coordinates of the yellow circle and that of the blue circle above it as $\left(x_{yel}, y_{yel}\right)$ and $\left(x_{bluA}, y_{bluA}\right)$ respectively. Your image processing algorithm must give $x_{yel}=x_{bluA}$. We also assume that the analytical distance between these two circles is $RDistance_A$. Then, you shall write, $$\mathrm{pixel\space size\space in}\space y\mathrm{-direction}=\frac{RDistance_A}{\left|y_{bluA}-y_{yel}\right|}.$$
Now, denote the coordinates of the blue circle on the left hand side of the yellow circle as $\left(x_{bluB}, y_{bluB}\right)$. In this case, the image processing algorithm must give $y_{yel}=y_{bluB}$. We also assume that the analytical distance between these two circles is $RDistance_B$. Then, you shall write, $$\mathrm{pixel\space size\space in}\space x\mathrm{-direction}=\frac{RDistance_B}{\left|x_{bluB}-x_{yel}\right|}.$$