"Nice" applications of Lagrange multipliers

Question

What are some 'nice' real-world applications of Lagrange multipliers that could be used to motivate the concept to students in an introductory optimization course?

I like the pressure as a Lagrange multiplier and Dido's problem examples, but they maybe a bit too advanced. This application is nice but not very real-world.

Answer 1

I'm not sure that this is the "nicest" application, but if the students were exposed to some statistics, you can show that for a linear model of a kind $$ Y = X\beta + \epsilon, $$ by using Lagrange multipliers one can find the BLU estimators of $\beta$. This isn't much of a "surprising" fact, but it still interesting as usually in statistics courses the $\hat{\beta}$ are derived by minimizing the least squares (or the empirical MSE) or (e..g., for Gaussian errors) maximizing the Likelihood function. Hence, it could be nice to see that for appropriate assumptions on $\epsilon$, you get the same result independently of which method you've used (including Lagrange multipliers).

---

Let us look at the simplest although non-trivial example. Assume the following model $$ Y_i = \beta x_i + \epsilon_i, \quad i=1,...,n $$ where $E\epsilon = 0$, $E\epsilon_i \epsilon_j =0$ and $E\epsilon_i = \sigma^2$, thus if we are looking at the class of all linear unbiased estimators then we have to look at estimators of the form $$ \alpha = \sum_{i=1}^nw_iY_i, $$ that are satisfy $E(\alpha) = \beta \sum w_ix_i = \beta$, i.e., $\sum w_ix_i = 1$. Thus, you minimization problem (the optimality, "best", criterion) is $$ \arg\min_{w\in \mathbb{R}^n} \left( \sigma^2\sum_{i=1}^nw_i^2 - \lambda (\sum_{i=1}^n w_ix_i-1) \right), $$ where $\sigma^2\sum w_i^2=Var(\sum w_iY_i)$. Now, note that $$ \mathcal{L}'_{w_k} = 2\sigma^2w_k - \lambda x_k = 0\to w_k =\frac{\lambda x_k}{2\sigma^2}, \forall k . $$ By using the restriction you get that $$ \sum w_ix_i = \lambda\frac{\sum x_i^2}{2\sigma^2} =1 \to\lambda=\frac{2\sigma^2}{\sum x_i^2} \to w_k = \frac{x_k}{\sum x_i^2}. $$ As such the estimator is given by $$ \alpha = \frac{\sum x_i Y_i}{\sum x_i^2}. $$ You can check that the Hessian matrix is indeed positive definite. This is, actually, called the Gauss - Markov theorem.

1. This is the same as the OLS estimator (hence the OLS is BLUE).
2. If you assume that the errors are also Gaussian, then this result is also the MLE.
3. For more then $2$ $\beta$-s you'll need linear algebra to make the derivations as the direct calculations become too messy. See here for example.
4. Another interesting application of Lagrange multipliers is in LASSO that can be appropriate for Machine Learning or Stats majors.

Answer 2

I shall mention as a nice example not really as an application. Almost all Lagrangians for $n$ variables and functions with $n-1$ Lagrange multipliers. The classical Dido's problem may not be a so nice an example from geometry but easily understood.

For example if Lagrange Multiplier is denoted by $\rho$ then the functional connecting Area and perimeters $ (A,p) $ of the patch as object and constraints

$$ F= A - \rho \, p $$

directly results in $\rho $ as the curve (circle) property with a constant radius of curvature $=\rho.$

It should be emphasized to the student a priori, right at the outset, that the

Constant Lagrange multiplier is a constant property of the solution

that we are about to seek.

We thus have an opportunity to synthesize a curve using Lagrange multiplier

when we put $F=0$

$$ \dfrac {Area}{perimeter} = constant = \rho $$

But it should not be a late realization by the student after obtaining solution in a particular case he is dealing in but a general variation.

The concept runs in all optimization e.g., optimal control theory and financial mathematics.

I do not know if it is appropriate to mention here something from history of mathematics. The importance of the powerful differential concept over particular geometric diagrammatic representations was repeatedly emphasized (many of his time said he was boasting !) by Lagrange.

EDIT1:

Whether for finding optimum area for given rectangle area (square solution, invariant unit Aspect Ratio=1), a volume of surface enclosed for given area ( DeLaunay unduloid, Mean curvature = H =constant ) or Navier-Stokes ( invariant pressure ) in recognizing pressure parameter $p$, the same differential view of the Lagrangian is generally valid and so is developed into a useful tool.

Answer 3

You could use The example of newtons law with external forces $F_e$ and constraining forces $F_c$ (Lagrange equation of motion of first kind).

$ma=F_e+F_c.$

The constraining forces allow motion only in a plane. Hence, the constraining forces must be orthogonal to this plane. One can show that $F_c=\lambda n$, where $n$ is the normal of the plane. It turs out this $n$ is the gradient of the equation which describes the plane of motion.