In investigating approaches to Fermat's Last Theorem I came across the following and I can't figure out where I am going wrong. Any input would be greatly appreciated.
We want to show that $a^n + b^n = c^n$ cannot hold for odd $n>1$ and pairwise relatively prime $a$, $b$, and $c$. Assuming by way of contradiction that we have $a^n + b^n = c^n$ we must have $a$, $b$, and $c$ forming the sides of a triangle since $(a+b)^n > c^n$ so $a+b>c$. Therefore the law of cosines can apply and we can write:
$$c^2 = a^2+b^2 - 2ab{\cos{C}}$$
where $C$ is the angle opposite to side $c$. If we add and subtract $2ab$ on the right-hand side we get
$$c^2 = {(a+b)}^2 -2ab(\cos{C}+1)$$
Now, $a+b$ and $c$ share a common factor since $(a+b) | (a^n+b^n)$ for odd $n$ and $c^n = a^n+b^n$. (Here $x | y$ means as usual, "$x$ divides $y$").Therefore, they share the same factor with $2ab(\cos{C}+1)$. Now, $\cos{C} + 1$ must be a rational number since $a$, $b$, and $c$ are all integers. So let $\cos{C} +1 = \frac{r}{s}$ where $r$ and $s$ are integers and $(r,s)=1$. (i.e. $\frac{r}{s}$ is a reduced fraction). (Here, $(r,s)$ means as usual the greatest common divisor of $r$ and $s$.)
Now assuming $a$, $b$, and $c$ are relatively prime we must have $(ab) |s$ for otherwise $c$ and $2ab$ would share a common factor. Even moreso we must have $ab=s$ since otherwise $\frac{2abr}{s}$ would not be an integer. (Since $c - a - b$ is even, we don't need $2 | s$). So we can write:
$$\cos{C}+1 = \frac{r}{ab}$$ or equivalently $$\cos{C} = \frac{r - ab}{ab}$$
Now we had from the law of cosines:
$$c^2 = a^2+b^2 - 2ab{\cos{C}}$$
so making the substitution $\cos{C} = \frac{r - ab}{ab}$ we get
$$c^2 = a^2 + b^2 - 2r + 2ab$$
If we subtract $a^2$ to both sides and factor out the $b$ on the right-hand side, we get:
$$c^2 - a^2 = b(b + 2a) - 2r$$
Now, $(c - a) | (c^2 - a^2)$ and also $(c-a) | (c^n - a^n)$. Then we must have $((c-a),b) >1$ since $b^n = c^n - a^n$. From the equation above, we must therefore also have $(b,2r) > 1$. Similarly we can show that we must have $(a,2r) > 1$.
However, both of these conclusions are problematic since $r$ was initially assumed to be relatively prime to $s = ab$. The only other option is that $a$ and $b$ are both even, but this is also problematic since $a$ and $b$ are assumed to be relatively prime.
Thus we cannot have $a^n + b^n = c^n$ for odd $n>1$ and pairwise relatively prime $a$, $b$, and $c$.
I'm sure someone has thought of this approach before so where am I going wrong?
How do you get to the conclusion that $ab\vert s$? I honestly can't see it. The way I see it you have:
$$\dfrac{r}{s}=\dfrac{(a+b)^2-c^2}{2ab}. $$
Now $(a+b)^2-c^2$ is even. You can check this case by case, when $a,b$ are odd then $c$ has to be even and so forth. So at least one of them is even but by your assumption maximal one is even and therefore $(a+b)^2-c^2$ is even. Therefore
$$ \dfrac{r}{s}=\dfrac{\dfrac{(a+b)^2-c^2}{2}}{ab}. $$
But there is no apparent (at least not to me) reason why this shouldn't reduce further. If it does your argument breaks down at this point.
Here is an actual counter example: of course I can't give an example of $a,b,c$ with $a^n+b^n=c^n$ but your argument that $ab\vert s$ only uses that $a,b,c$ are coprime. So let $a=13, b=15$ and $c=22$ than you have that $a,b,c$ are relatively prime and furthermore:
$$\dfrac{r}{s}=\dfrac{(a+b)^2-c^2}{2ab}=\dfrac{10}{13}, $$
therefore $s\neq ab=195$.