Understanding L1 and L2 norms

Question

I am not a mathematics student but somehow have to know about L1 and L2 norms. I am looking for some appropriate sources to learn these things and know they work and what are their differences. I am asking this question since there are lots of stuff on this matter in the internet and I am looking for a simple applicable one to understand this.

To add more, I want to understand why/how/when these norms are used in optimization problems? And maybe, how these cost functions are solved.

Answer 1

Just the norm? The norm acts on a vector and any norm has to satisfy three properties. Given a vector space V over a field F, the norm must

1. $\|x\|\ge0$ for all x and $x=0\Leftrightarrow\|x\|=0$
2. $\|ax\|=|a|\|x\|$ for all $a\in\mathbb F, x\in V$
3. $\|x+y\|\le \|x\|+\|y\|$ (triangle inequality)

You ask about the L1 and L2 norms. The L1 norm is the sum of the absolute value of the entries in the vector. The L2 norm is the square root of the sum of the entries of the vector. In general, the Lp norm is the pth root of the sum of the entries of the vector raised to the pth power.

$$\begin{split}\|x\|_1&=\sum |x_i|\\ \|x\|_2&=\sqrt{\sum x_i^2}\\ \|x\|_p&=(\sum |x_i|^p)^{1/p}\end{split}$$

It is fairly common to square the l2 norm to get $\|x\|^2_2=\sum x_i^2$

For more use Wikipedia and go to the examples section.