I have a pretty straightforward regression problem I need to solve within the .NET framework in C#. I have a nonlinear transfer function $$ Z = A + \frac{1}{\frac{1}{\frac{X}{256}*B+3*C}+\frac{1}{\frac{Y}{1024}*D+2*E}} $$ with two independent variables $$ X, Y $$ and five parameters $$ A, B, C, D, E $$ for which I need to optimize.
I've been able to successfully model and solve this problem with MATLAB's fit and Python Scipy's curve_fit with very accurate results. However, I have not been able to find a solution available to C#. There are a couple MIT licensed Nuget packages available but they either only support a single independent variable/no custom function (Math.NET) or have poor results (Accord.NET - which may very well be my fault as evidenced here).
I was hoping someone here would have some ideas for alternative solutions or could point me in the right direction in writing my own custom method to solve this. MATLAB used the Trust Region Reflective algorithm and Scipy used either Levenberg-Marquardt or Trust Region Reflective according to their documentation. The data I am working with is
$$ \begin{gather} X = \{0, 128, 255, 0, 128, 255, 0, 128, 255\}\\ Y = \{0, 0, 0, 512, 512, 512, 1023, 1023, 1023\}\\ Z = \{89.66623397, 122.9866434, 123.8610312, 197.274736, 4255.419371, 7129.346848, 197.8635428, 4655.314692, 8335.298909\} \end{gather} $$
with a best guess of
$$ \begin{gather} A = 20\\ B = 10000\\ C = 50\\ D = 50000\\ E = 60 \end{gather} $$
and I would expect to get
$$ \begin{gather} A = 27.85\\ B = 9886.98\\ C = 56.87\\ D = 48581\\ E = 48.47 \end{gather} $$
Any direction or ideas are greatly appreciated.
$$ Z = A + \frac{1}{\frac{1}{\frac{X}{256}*B+3*C}+\frac{1}{\frac{Y}{1024}*D+2*E}} \tag 1 $$ Simplification of writing with $\begin{cases} a=A \\ b=\frac{B}{256}\\ c=3C \\ d=\frac{D}{1024}\\ e=2E \end{cases}$ $$ Z = a + \frac{1}{\frac{1}{b\,X+c}+\frac{1}{d\,Y+e}} $$
$(Z-a)(b\,X+c+d\,Y+e) = (b\,X+c)(d\,Y+e)$
$bdXY-bXZ-dYZ+b(a+e)X+d(a+c)Y-(e+c)Z+ac+ae+ce=0$
This is the equation of a quadratic surface : $a_{110}XY+a_{101}XZ+a_{011}YZ+a_{100}X+a_{010}Y+a_{001}Z=1 \tag 2$
$a_{110}=-\frac{bd}{ac+ae+ce}\quad;\quad a_{101}=\frac{b}{ac+ae+ce}\quad;\quad a_{011}=\frac{d}{ac+ae+ce}\quad;\quad a_{100}=-\frac{b(a+e)}{ac+ae+ce}\quad;\quad a_{010}=-\frac{d(a+c)}{ac+ae+ce}\quad;\quad a_{001}=\frac{e+c}{ac+ae+ce}$
So, this is a problem of regression to fit a quadratic surface. But not a general quadratic surface since the equation doesn't includes all coefficients of the general quadratic surface. http://mathworld.wolfram.com/QuadraticSurface.html
This kind of regression is roughly considered in the paper https://fr.scribd.com/doc/14819165/Regressions-coniques-quadriques-circulaire-spherique page 17. All the cases are not detailed but the method is the same. This is applied in the numerical example below, with the data from the OP.
One can see that the result is quite exactly the result expected by the OP : $$ \begin{gather} A = 27.85\\ B = 9886.98\\ C = 56.87\\ D = 48581\\ E = 48.47 \end{gather} $$
DISCUSSION :
This result is unusually excellent for a regression problem. This is certainly because the data doesn't come from experiment but comes from numerical simulation without scatter.
If the data was scattered the result would be not so accurate due to several causes :
Of course the scatter it-self.
The number of independent parameters in the equation $(2)$ of the particular quadratic surface is six while the number of parameters in the original equation $(1)$ is five. It doesn't matter if the scatter is low. But the more the scatter is important the more the optimization differs from the cases $(1)$ and $(2)$. Also the presence of an extra-parameter eventually makes the calculus less robust.
The criteria of fitting for the above method is not the least mean square with respect to the data $Z$, but is a least mean square with respect to a the right term of Eq.$(2)$. Again this doesn't matter if the scatter is low. But if the scatter is not negligible, a criteria of fitting has to be specified and taken into account.
As a conclusion, in case of data coming from real measurements, such above oversimplified method is not sufficient. The above method can produce a rough first fit. The obtained values of the parameters are probably more accurate and more reliable than "guessed" values. They could be used for starting an iterative process of non-linear regression which anyways cannot be avoid to take into account of a particular criteria of fitting.