I have an ellipse $\frac{x^2}{64} + \frac{y^2}{32}=1$. Furthermore, I have three points ($P,Q,R$) on the ellipse. How can I find the points that maximize the area of the triangle by using the Lagrange multiplier? I realize it can also be solved without using Lagrange using the fact that an ellipse is in fact a scaled version of a circle, but I am looking to solve it in a different way.
How can I get an expression of a triangle to use in the Lagrange multiplier rule? The area of such triangle is equal to $\frac{1}{2}\det(u \times v)$, but I am not sure how that can be used in the formula: $\text{area triangle}-\lambda\left(\frac{x^2}{64}+\frac{y^2}{32}-1\right)$.
This is not really an answer, but the observations made here are too long for a comment. What is described here is how we should approach the probem.
Let us "cheat" by peeking at what the scaling method shows. We imagine that we are viewing the ellipse obliquely so that it looks like a circle. In this view the largest inscribed triangle is any equilateral triangle, regardless of orientation. Now remove the foreshortening, and of course the triangles are no longer equilateral. In general the maximum area triangles are now geometrically different, depending on how they are oriented relative to the axes of the ellipse. But the foreshortening (or removing it) will have the same impact on all areas. So all the noncongruent, differently oriented triangles that appear equilateral in the oblique view are equally optimized.
This implies that we cannot expect to get a unique maximum area triangle for all possible orientations. We need to consider separate orientations and compile the results for the different orientations.
So, for a Lagrange multiplier based solution we should first consider fixing point $P$. Now we can get unique locations for $Q$ and $R$ using the suggested Lagrange multiplier method. Then we need to prove that for different choices of $P$ the largest triangle always has the same area.