I stumbled upon a question from an old university exam and I cannot bring myself to find an argument that explains the solution. Consider the following scenario:
Given the function from the picture where $\forall i:x_i, w_i \in \mathbb{R}$. The question is now "Is the function a linear method?".
The options are:
- True
- False
- depends on f
- depends on g
- depends on f and g
My initial guess would have been answer 5 - it depends on the choice of f and g. The solutions says that it only depends on the choice of f. So choosing a linear f and nonlinear g should result in a function which is linear in w. Writing the function in the following form would imply for me that a nonlinear g would make the resulting function nonlinear:
$g(w^T f(Wx)) = y$ with $w = \begin{pmatrix}w_1^{(2)}\\w_2^{(2)}\end{pmatrix}^T$ and $W = \begin{pmatrix}w_{11}^{(1)} w_{12}^{(1)}\\w_{11}^{(1)} w_{22}^{(1)}\end{pmatrix}$
Where is the error in my train of thought?
Maybe because you can apply directly the inverse of the nonlinear transformation $g$ to the output values $z=g^{-1}(y)$ so you have just a transformed set of values under the form: $$\omega^T f(Wx)=z$$ So the problem is only related to the linearity of $f(x)$... I am thinking in the example of doing a linear regression of exponential distributed data.... I first apply an affine log transformation to my outputs, and then I can use a simple linear reagression.