I know this is true if F is infinite but don't know how to prove it. And is this still true if F is finite?
I think The Primitive Element Theorem is the keypoint to prove this statement. Primitive Element Theorem: Let K/F be a finite extension of fields, if F(K/F) is finite, then there exists q in K such that K=F(q).
One way is to use Galois Theory. I'm assuming $a$ and $b$ are separable algebraic over $F$. Suppose $N$ is the normal closure of $F(a, b)$ over $F$. The subextensions of $N$ over $F$ are then in one to one correspondence with the quotients of the finite Galois group of $N/F$. Hence, there are only finitely many subextensions of $F(a, b)$ over $F$.
Now, consider subextensions of the form $F(a + cb)$, with $c \in F$. Since $F$ is infinite, there are infinitely many of these. Hence, for some $c \not = c'$, $$F(a + cb) = F(a + c'b).$$ Hence, $a + cb - (a + c'b) \in F(a + cb)$, which implies that $(c - c')b \in F(a + cb)$, and as $c- c'$ is nonzero and in $F$, we have $b \in F(a + cb).$ But this also implies that $a \in F(a + cb)$. Hence, we have $F(a + cb) \supseteq F(a, b)$ and the reverse inclusion is trivial.