$Ax = b$ describes a convex polyhedron, where $A$ is a real matrix and $b$ is a real vector. Now assume $A$ has less rows than columns.
If you take a look here:
http://en.wikipedia.org/wiki/System_of_linear_equations#Matrix_solution
The formula $$ x = Pb + (I-PA)w $$
where $P$ is the pseudoinverse produces all solutions. So every point inside the polyhedron.
But what I don't understand is:
The polyhedron could be bounded, but I have the feeling with a certain value of $w$ I can produce as big $x$ as I want since $Pb$ is a constant and $(I-PA)$ is also a constant matrix.
Could someone explain how to understand this formula?
$Ax = b$ describes an affine subspace, that is a point or an unbounded polyhedron. In the latter case, there is no problem, in the further case, we have that $PA = {\rm Id}$ (otherwise we had more than one solution), so $x = Pb$ is the only point.
If you want to consider all polyhedra, you have to look at $Ax \le b$.