I need help with this two part question. I am rather confused by it.
let $f(x, y, z)$, $g(x, y, z)$ be smooth on $U \subset \mathbb{R}^3$ with $f^2 + g^2 > 0$ on $U$. Define the differential form $$\omega = f(x, y, z)\,dx + g(x, y, z)\,dy$$ on $U$. Establish necessary and sufficient conditions on $f$, $g$ for the plane field defined by $\omega$ on $U$ to be integrable. Explain the condition you find geometrically. What do the integral surfaces look like?
Let $a(z)$, $b(x)$, $c(y)$ be smooth functions on $U \subset \mathbb{R}^3$, not all vanishing. Repeat the first part for $$\omega = a(z)\,dx + b(x)\,dy + c(y)\,dz.$$
I understand that for the first part the necessary and sufficient condition for integrability is $\omega \wedge d\omega = 0$, and that the right thing to do for the second part is to forget about $U$ and assume that the functions are smooth on all of $\mathbb{R}^3$.
1.
The first thing to observe is that we switch to $\tilde{\omega} = \phi(x, y, z)\omega$ where $\phi \neq 0$ then the plane field defined by $\omega$ is the same as that defined by $\tilde{\omega}$. So the integrability condition cannot depend on such a rescaling of $\omega$. Sure enough,$$d\tilde{\omega} \wedge \tilde{\omega} = (d\phi\omega + \phi{d}\omega) \wedge \phi\omega = \phi^2d\omega \wedge \omega.$$So whatever condition we find on $f$, $g$, it will be up to such a rescaling. By plugging in we find that $f_zg - fg_z = 0$. If $g \neq 0$ near some point, then this is the same as $(f/g)_z = 0$ or $f(x, y, z) = g(x, y, z)\psi(x, y)$. So we see that$$\omega = g(x, y, z)(dx + \psi(x, y)dy).$$So the plane field is the same as that defined by $dx + \psi(x, y)\,dy$ which does not depend on $z$. If $f \neq 0$, we find in the same way that the form reduces to $\eta(x, y)\,dx + dy$. In either way, we arrive at the conclusion that up to a smooth rescaling, $\omega$ does not depend on $z$. Try and picture the plane field and understand geometrically why it can only be integrable if it does not turn with $z$.
2.
With regards to what you put down for the right thing to do for the second part, otherwise, we of course not that $U$ needs to be a box (a product of three open intervals on the line) since each $a$, $b$, $c$ is a function of one variable only. First, we look at special cases. If $a$, $b$, $c$ are constant, then $d\omega = 0$ so clearly integrable. In fact, $d\omega = 0$ if and only if the coefficients are constant (why?). So next, suppose $b$, $c$ are constant but $a$ is not. Then $$d\omega = a'(z)\,dz \wedge dx,\text{ }d\omega \wedge \omega = a'(z)b(y)\,dz \wedge dy \wedge dz = 0$$if and only if $a'(z)b(y) = 0$. Since $a$ is not constant, $a' \neq 0$ whence $b = 0$. So we have a "degenerate case" of the form$$\omega = a(z)\,dz + c\,dz,\text{ }c = \text{const}$$which has no $dy$ in it. But we exclude this degenerate case by requiring all coefficients to be different from zero.
So we will still have some cases to consider. We of course have to start from$$d\omega \wedge \omega = (a'(z)b(x) + b'(x)c(y) + c'(y)a(z))\,dx \wedge dy \wedge dz = 0.$$In other words,$$a'(z)b(x) + b'(x)c(y) + c'(y)a(z) = 0.$$Clearly, $a$, $b$, $c$ all constant is a solution. We assume now that they are not all constant. We differentiate in $x$, $y$, $z$ to conclude that$$a''(z)b'(x) = b''(x)c'(y) = c''(y)a'(z) = 0.$$Suppose there exist points $x_0, y_0, z_0 \in \mathbb{R}$ with $a'(z_0) \neq 0$, $b'(x_0) \neq 0$, $c'(y_0) \neq 0$. Then this nonvanishing needs to be locally near these points. But this implies that $a'' = 0$, $b'' = 0$, $c'' = 0$ locally, too. In other words, $a(z) = a_1z + a_2$, $b(x) = b_1x + b_2$, $c(y) = c_1y + c_2$. Plugging this back into the main equation yields $$a_1(b_1x + b_2) + b_1(c_1y + c_2) + c_1(a_1z + a_2) = 0$$which is a contradiction, since $a_1 \neq 0$, $b_1 \neq 0$, $c_1 \neq 0$.
Now suppose $a$ is not constant. Then there exists a point $z_0$ with $a'(z_0) \neq 0$. From what we have just done, it follows that not both $b$, $c$ can be nonconstant. Suppose $b$, $c$ are both constant. Then we find that $a'(z)b = 0$ with a constant $b \neq 0$. So then $a'(z) = 0$ which is a contradiction. So at least one of $b$, $c$ is not constant. Say $b$ is not constant, which requires $c \neq 0$ to be constant. Then $a'(z)b(x) + b'(x)c = 0$. Differentiate in $z$ to get $a''(z)b(x) = 0$. We have that $b \neq 0$ whence $a'' = 0$. So $a'$ is constant, but not vanishing since $a'(z_0) \neq 0$. We obtain$$a_1b(x) + b'(x)c = 0, \text{ }a_1 \neq 0, \text{ }c \neq 0.$$This has a solution of the form $b(x) = e^{kx}$ where $k$ satisfies $a_1 + kc = 0$. So the solution is$$\omega = (a_1z + a_2)\,dx + b_1e^{kx}dy + c\,dz.$$All other solutions are obtained by cyclic permutations $x \to y \to z \to x$. In other words, there are a total of six of these, for example$$\omega = a\,dx + b_1e^{kx}dy + (c_1y + c_2)\,dz$$ $$\omega = (a_1z + a_2)\,dx + b\,dy + ce^{ky}dz$$where $k$ satisfies the same type of linear relations as above. Note that the constant case is included here, set $k=0$ and $a_1 = 0$ etc.