I really need some mathematical expertise, as am stumped.
If I have a small vector a $\in$ $\mathcal{R}^{r}$ where r is small (1,000) . And I have a mapping f(a) = $W_{2}(tanh(W_{1}(a^{T})+B_1)+B_{2})$=O, where $W_1$ $\in$ $\mathcal{R}^{dxr}$, $W_2$ $\in$ $\mathcal{R}^{lxd}$, l = 50,000 and d =800. The $B_1$ and $B2$ are rank 1 matrices, repeated bias vectors.
Can this f(a) = o $\in$ $\mathcal{R}^{l}$ represent all of $\mathcal{R}^{l}$, if f is fixed,and a function of a much smaller a $\in$ $\mathcal{R}^{r}$