Linear or non-linear regression

Question

Given an equation, say, $y^{1/n} = x^{1/n} + z^{1/n}$ and a bunch of 3-dimensional sample points, what is the best way to find the optimal value for $n$ that best fits the sample points? I suppose least-squares can be a metric, but is the regression non-linear? Can the equation somehow be made linear?

Answer 1

You have $k$ data points $(x_i,z_i,y_i)$ and the model you want to fit is in fact $$y=\left(x^{\frac{1}{n}}+z^{\frac{1}{n}}\right)^n$$ since what is measured is $y$ and not any of its possible transform.

So, what you want to minimize is $$SSQ(n)=\sum_{i=1}^k \Big[\left(x_i^{\frac{1}{n}}+z_i^{\frac{1}{n}}\right)^n-y_i \Big]^2$$ which is extremely nonlinear with respect to the parameter. This means that you need at least a reasonable guess.

What I should do is to try a few values of $n$ until you see more or less a minimum. When this is done, you are ready for the nonlinear regression.

Suppose that we have the following data set $$\left( \begin{array}{ccc} x & z & y\\ 12 & 11 & 100 \\ 13 & 13 & 110 \\ 14 & 15 & 130 \\ 15 & 17 & 140 \\ 16 & 19 & 150 \\ 17 & 21 & 170 \\ 18 & 23 & 180 \\ 19 & 25 & 190 \\ 20 & 27 & 210 \end{array} \right)$$

Trying with a fixed step size (we could do better), you will have $$\left( \begin{array}{cc} n & SSQ(n) \\ 1.0 & 132705 \\ 1.5 & 102443 \\ 2.0 & 66281 \\ 2.5 & 28528 \\ 3.0 & 1956 \\ 3.5 & 18027 \\ 4.0 & 148044 \end{array} \right)$$ So, $n=3$ seems to be quite good.

Now, the nonlinear regression would give $(R^2=0.99964)$ $$\begin{array}{clclclclc} \text{} & \text{Estimate} & \text{Standard Error} & \text{Confidence Interval} \\ a & 3.1391 & 0.0104 & \{3.1146,3.1636\} \\ \end{array}$$ corresponding to $SSQ=80.2602$.

The predicted values would be $$\{101.2,114.5,127.7,140.8,153.8,166.7,179.7,192.6,205.5\}$$

If you do not access a nonlinear regression software, you could continue zooming more and more around the minimum. There is also an algebraic way to do the job; if you are concerned, let me know.