Notation for applying a function to every entry in a vector/matrix?

611 Views Asked by At

Let's say you have a vector $\vec{v} \in \mathbb{R}^n$ with coordinates $(v_1,\ldots,v_n)$ in some basis, and a function $f:\mathbb{R}\rightarrow\mathbb{R}$.

In coordinates we can write $u_i = f(v_i)$ to build a vector $\vec{u}$ with coordinates $u_i$ that are the result of $f$ applied to each coordinate of $v$.

Is there a common notation to write $\vec{u}$ in terms of $\vec{v}$ and $f$ assuming the basis is understood? (I understand the operation I'm describing is basis-dependent but I wonder if there is a notation if the basis is understood. This seems similar to the Hadamard product.)

(This is coming up when I am writing down neural network formulas want to apply an activation function $\sigma$ to every entry of a vector $z^l = w^la^{l-1} + b^l$. I'm tempted to write $a^l = \sigma(z^l)$ but I'm not sure if that's accepted. I can settle for the clearly correct $a^l_j = \sigma(z^l_j)$ if need be.)

1

There are 1 best solutions below

0
On

I'll write $A\odot B$ for the Hadamard product of matrices $A$ and $B$, and define the Hadamard power $A^{\odot n}=A\odot A^{\odot (n-1)}$ for $n\geq 1$ and $A^{\odot 0}$ is defined to be the all-ones matrix. Then for an analytic function $f\colon\mathbb R\to\mathbb R$ given by $$ f(x)=\sum_{n\geq 0}a_n x^n, $$ your operation sends the matrix $A$ to $$ f_{\odot}(A):=\sum_{n\geq 0}a_n A^{\odot n}. $$

While I wrote "matrix" above, the Hadamard product (unlike the regular matrix product) is defined for any two matrices with the same dimensions, including $n\times 1$ matrices. In particular, this discussion applies equally well to both vectors and matrices.