Let $$\Sigma = \{x\in R^3: x_1x_2 +x_1x_3 +x_2x_3=1 \}$$
and $$f(x) = x_1^2 + x_2^2 + \frac{9}{2} x_3^2$$
a) Show that $\Sigma$ is a smooth surface in $R^3$.
b) Show that $\inf_{x\in\Sigma}$ f(x) is achieved. Find $\inf_{x\in\Sigma} f(x)$.
For part (a), since this is an old advanced calculus (introductory, non-measure -theoretic analysis) exam question, I am not sure how much proof is required, other than to make the observation that the set is a level set of a smooth function, g, in the variables $x_1$, $x_2$ and $x_3$. I've read up a bit on how to prove surfaces are "smooth", but that material uses a bit of differential topology, namely, the "constant rank theorem". I think that would be unnecessary, but feel free to comment and offer a suggestion -- or even answer part(a) with a proof, if you'd like to give one.
For part(b), to both show that the infimum is achieved and to find it, I considered a set of equations, using the method of Lagrange multipliers:
$$2x_1 = \lambda (x_2+x_3)$$ $$2x_2 = \lambda (x_1+x_3)$$ $$9x_3 = \lambda (x_1+x_2)$$ $$g=x_1x_2 +x_1x_3 +x_2x_3=1$$
I am stuck at this point. I have tried subtracting equations, which didn't lead to much. My other attempt was to multiply the first equation by $x_2$, the second equation by $x_3$, and the third equation by $x_1$, and then adding these three new equations gives a single equation, where I can use the constraint $x_1x_2 +x_1x_3 +x_2x_3=1$. This method didn't turn out very well either.
Any hints or solutions are greatly appreciated.
Thanks,
For (a), $1$ is a regular value of $f:\mathbb{R}^3\to\mathbb{R}$ defined by $\left(x_1,x_2,x_3\right)\mapsto x_2x_3+x_3x_1+x_1x_2$. That is, at every point $p$ in $f^{-1}(1)$, $f$ is a submersion at $p$. In other words, $\text{d}_pf:\text{T}_p\mathbb{R}^3\to\text{T}_1\mathbb{R}$ is surjective. This can be easily seen since the matrix representing $\text{d}_pf$ (you probably call it $Df(p)$) is $$\left[x_2+x_3\;\;\;x_3+x_1\;\;x_1+x_2\right]\Big|_{\left(x_1,x_2,x_3\right)=p}\,.$$ This matrix certainly has a nonzero entry. Now, by the Regular Value Theorem (see, for example, here), $f^{-1}(1)$ is a smooth submanifold of $\mathbb{R}^3$ (of dimension $2$).
For (b), note that $$\left(x_1^2+x_2^2+\frac{9}{2}x_3^2\right)-\frac{3}{2}\left(x_2x_3+x_3x_1+x_1x_2\right)=\frac{3}{4}\left(x_1-x_2\right)^2+\frac{1}{4}\left(x_1-3x_3\right)^2+\frac{1}{4}\left(x_2-3x_3\right)^2\geq0\,.$$ Hence, if $x_2x_3+x_3x_1+x_1x_2=1$, then $$x_1^2+x_2^2+\frac{9}{2}x_3^2\geq \frac{3}{2}\,.$$ The equality holds if and only if $\left(x_1,x_2,x_3\right)=\pm\left(\frac{3}{\sqrt{15}},\frac{3}{\sqrt{15}},\frac{1}{\sqrt{15}}\right)$.
You can continue with your attempt with Lagrange multipliers as follows. If $\lambda=0$, then all the $x_i$'s are zero, which is clearly not a feasible point. Hence, $\lambda\neq 0$. The equations $2x_1=\lambda\left(x_2+x_3\right)$ and $2x_2=\lambda\left(x_1+x_3\right)$ imply that $$2x_1\left(x_1+x_3\right)=2x_2\left(x_2+x_3\right)\,.$$ That is, $$\left(x_1-x_2\right)\left(x_1+x_2+x_3\right)=0\,.$$
Case I: $x_1+x_2+x_3=0$. Then, $9x_3=\lambda\left(x_1+x_2\right)=-\lambda x_3$ leads to $\lambda=-9$ or $x_3=0$, whilst $2x_1=\lambda\left(x_2+x_3\right)=-\lambda x_1$ and $2x_2=\lambda\left(x_1+x_3\right)=-\lambda x_2$ mean that either $\lambda=-2$ or both $x_1$ and $x_2$ equal $0$. However, if two of the variables are zero, then $x_2x_3+x_3x_1+x_1x_2=0$, making the point infeasible. Hence, $\lambda=-2$ and $x_3=0$, which means $x_1+x_2=x_1+x_2+x_3=0$. That is, $1=x_2x_3+x_3x_1+x_1x_2=-x_1^2\leq 0$, which is absurd.
Case II: $x_1+x_2+x_3\neq 0$. Then, $x_1=x_2$. Write $a$ for the common value of $x_1$ and $x_2$. We get $9x_3=\lambda\left(x_1+x_2\right)=2a\lambda$, or $x_3=\frac{2a\lambda}{9}$. Trivially, $a\neq 0$ (otherwise, we would get a contradiction with $x_1=x_2=x_3=0$). Also, $2a=2x_1=\lambda\left(x_1+x_3\right)=\lambda\left(a+\frac{2a\lambda}{9}\right)$ yields $$2=\lambda\left(1+\frac{2\lambda}{9}\right)\text{ or }\lambda\in\left\{-6,\frac{3}{2}\right\}\,.$$ If $\lambda=-6$, then $\left(x_1,x_2,x_3\right)=\left(a,a,-\frac{4}{3}a\right)$, so that $1=x_2x_3+x_3x_1+x_1x_2=-\frac{5}{3}a^2\leq 0$, which is absurd. Hence, $\lambda=\frac{3}{2}$, or $\left(x_1,x_2,x_3\right)=\left(a,a,\frac{1}{3}a\right)$. As $1=x_2x_3+x_3x_1+x_1x_2=\frac{5}{3}a^2$, we get $a=\pm\sqrt{\frac{3}{5}}=\pm\frac{3}{\sqrt{15}}$.
In summary, we have that $\lambda=\frac{3}{2}$ and that $\left(x_1,x_2,x_3\right)=\pm\left(\frac{3}{\sqrt{15}},\frac{3}{\sqrt{15}},\frac{1}{\sqrt{15}}\right)$ are critical points of $x_1^2+x_2^2+\frac{9}{2}x_3^2$ subject to $x_2x_3+x_3x_1+x_1x_2=1$. Since $x_1^2+x_2^2+\frac{9}{2}x_3^2$ is bounded below, and the existence of these critical points implies that either of them must be a minimizing point. It turns out that both points correspond to the same value, whence they are both minimizing points.