The function f(x)=theta·x where theta is a row vector and x is a column vector, is a linear function. How can I obtain a non-linear function g(x) using a multi layer network, that also takes in x as input and theta as the parameter of the first layer?
If I have a 2 layer network, with a first dense layer of parameter theta and second dense layer with a non-linear activation function, would that suffice to say that I now have a non-linear function?
Mathematically speaking, your argument is insufficient. After all, if the second layer have weights of 0, then $g(x)=0$, which is linear.
That being said, it is highly unlikely that a neural network that was not designed for that purpose ends up being linear, so depending on your needs, that may be sufficient.