Linearly independence after feature embedding

154 Views Asked by At

Problem

If feature vectors $\mathbf{x}_1, \mathbf{x}_2,\cdots,\mathbf{x}_m$ are linearly independent, argue whether or not their embedding $\psi(\mathbf{x}_1), \psi(\mathbf{x}_2),\cdots,\psi(\mathbf{x}_m)$ are linearly independent.

Some Thoughts

The complication here is that the choice of $\psi(\cdot)$ is not random and often requires that the resulting $\psi(\mathbf{x}_i)$'s being linearly separable (or we could always do $\psi:\mathbf{x}_i\mapsto \mathbf{0}$, but this is hardly useful). I am wondering whether this requirement could guarantee that the mapping keeps the linearly independence of original vectors $\mathbf{x}_1, \mathbf{x}_2,\cdots,\mathbf{x}_m$.

1

There are 1 best solutions below

0
On

It depends on what you mean by "requires that the resulting $\psi(\mathbf{x}_i)$'s being linearly separable". If you mean that any of the points $\psi(\mathbf{x}_i)$ can be linearly separated from the others, then the answer is false.

For instance, the points $(0,0)$, $(1,0)$ and $(0,1)$ are linearly separable in this sense, but are not linearly independent.