The following is from Reconstructing a neural net from its output by Fefferman.
In here, I'm not sure about the notation. Are we fixing one $l$ such that $\gamma_l$ is identity, and the rest of the permutation $\gamma_0, \ldots \gamma_{l-1}, \gamma_{l+1} \ldots \gamma_{L}$ can be something other than identity function?
Thanks.
If I understood correctly, all the $(\gamma_\ell)_{1\le\ell\le L}$ are permutations on $(1,\ldots,D_\ell)$ that can be arbitrary except for $\gamma_0$ and $\gamma_L$ which are identity. Similarly, the authors set $\varepsilon_j^0 = \varepsilon_j^L = 1$.
Under these assumptions, the authors claim that the network with weights $(\tilde\omega^\ell_{jk})$ and biases $(\tilde\theta^\ell_{j}) $ as defined in the paper computes the same function as the original network with weights $(\omega^\ell_{jk})$ and biases $(\theta^\ell_{j})$.
To see this, work recursively with the depth $L$ and the expression of $x_j^{\ell+1}$ given in equation $(5)$ in the paper (Intuitively, the neural network doesn't "see" nodes swapping between adjacent layers).