I understand that for a set of vectors to be linearly independent, none of the vectors in the set should be a linear combination of some other vectors in that set. But why on earth should I care about it? How does it help me?
For example imagine a simple situation - I have a system of inequalities, which defines a set of points (vectors) which satisfy all these inequalities. Why should I care whether this set of solutions to the system is linearly dependent or independent?
On
Lots and lots of money can be made if a financial asset payoff is linearly dependent of other assets payoffs and its price is not (that's the idea behind arbitrage).
On
Let's say you have an $m\times n$ linear system with $Ax=0$ where $x$ is an $n\times 1$ matrix and $0$ is $m\times 1$.
The solution space is just the nullspace of the matrix $A$. If you know the solution space has dimension $k$ and you have $k$ linearly independent solutions, then you have all the solutions because the rest of the solutions are just the linear span of the solutions you currently have.
Having a basis for the solution space just allows you to compactly express what all solutions look like.
On
Do you like coordinates? A central fact of geometry is that a point in the Euclidean plane can be expressed uniquely as a pair of numbers, the first being the $x$-coordinate and the second being the $y$-coordinate.
But, there are many, many, many, many situations where we have to transform our coordinate system for one reason or another. We may need, for example to rotate the coordinate system. Or we may need to stretch the coordinate system, or to shear it, or to compose many such transformation into a single transformation. We would like to know that after we have transformed our coordinate system, the central fact of geometry remains true: each point can be expressed uniquely as a pair of numbers.
Linear independence is one of the main tools that let's us transform coordinate systems: any pair of linearly independent vectors $v = \langle a,b\rangle$, $w = \langle c,d \rangle$ in the plane determines a coordinate system whereby each point $p$ in the plane is represented by a unique pair of numbers $x,y$, namely $$p = x v + y w = \langle xa + yc, xb + yd \rangle $$
For example, consider the pair of vectors $$v = \langle \sqrt{2}/2, \sqrt{2}/2 \rangle, \,\,\, w = \langle -\sqrt{2}/2, \sqrt{2}/2 \rangle $$ You can verify for yourself that $v,w$ are linearly independent. The coordinate system for the plane that one gets by using $v,w$ is exactly the same as the coordinate system that one gets by rotating the "standard" coordinate system by $45^\circ$.
This answer is related to that of @math.noob, but I wanted to emphasize the underlying geometric meaning of linear independence. If you like geometry, you will find linear independence to be useful.
On
Linear dependence is, at its heart, about eliminating redundancy -- about having an efficiency way of describing something with no extra slack in the description.
It happens often that you have some set of things -- maybe a set of solutions to a differential equation, or a set of points on a hyperplane, or a set of equations satisfied by a physical system -- and you want to be able to pick out a small number (say 4, for the sake of definiteness) of those things and say "Everything in my set can be built up out of these 4 basic ones, in exactly one way." If you could do that, it would provide a simple and unambiguous "language" for describing everything in the set of interest.
If your set of 4 things is linearly dependent, that would mean it (a) has more things in it than you really need (so the set has some built-in redundancy), and (b) there is more than one way -- in fact infinitely many ways! -- to describe everything in the set of interest (so the set is not very useful as a language).
Why do mathematicians like to have a basis for a vector space? Because you can decompose any vector in the space and represent it as a finite linear combination of some of them.
To write any vector as a linear combination of some given vectors, they define the concept of a spanning set. But this isn't enough, because we also want the representation of a vector with respect to a given set of vectors to be unique, that is where linear independence comes in. It guarantees you the uniqueness of such a representation.
Let me explain it algebraically. Imagine that we're dealing with finite dimensional vector spaces, like $\mathbb{R}^n$. Imagine that a vector can be represented in two ways by using the same set of spanning vectors like $\{v_1,v_2,\cdots, v_n\}$. So, we can write:
$$\vec{v} = \sum_{k=1}^n \alpha_i \vec{v_i} = \sum_{k=1}^n \beta_i \vec{v_i}$$
Therefore:
$$\sum_{k=1}^n \alpha_i \vec{v_i} - \sum_{k=1}^n \beta_i \vec{v_i} = \sum_{k=1}^n (\alpha_i - \beta_i) \vec{v_i} = \vec{0}$$
Now, what does linear independence say if $\vec{v_i}$'s are linearly independent?
As a good exercise, imagine that you have a spanning set of vectors for a finite dimensional vector space which is not linearly independent. Find an example that shows you will have infinitely many different representations for the same vector!