Mathematical notation for dropout mask

193 Views Asked by At

I have this simple code:

D = np.random.rand(X.shape) # Create array of given shape with [0,1] bound random floats

D = ( D < k ) # Get boolean values from evaluation against [0,1] bound constant k

I would like to write this with the corresponding mathematical notations.

I believe a random number within [0,1] would be:

$$x\in\Bbb R:0\leq x\leq1$$

Although I have no clue how to feed a matrix of given shape from this.

For the conditional test, there may be:

$$x < k \rightarrow 0$$ $$x \geq k \rightarrow 1$$

Still, I have no clue how to deal with a matrix.

2

There are 2 best solutions below

4
On BEST ANSWER

For the first matrix, something like this: Let $D$ be a matrix of the same dimensions as $X$ (or more likely, if $X$ is already known to be $m\times n$, "Let $D$ be an $m\times n$ matrix"), where the entries are iid random variables uniformly distributed in $[0,1]$. You can also write this as $d_{ij}\sim\mathcal U(0,1)$.

For the second: Let $D'$ be the $m\times n$ matrix with entries $$ d_{ij}' = 1_{[d_{ij} < k]}, $$ where $d_{ij}$ are the entries of $D$ and $1_{[\cdot]}$ is the indicator function. This is equivalent to your version with the cases, but you would normally write the cases like this: $$ d_{ij}' = \begin{cases}1&d_{ij} < k\\0&d_{ij} \ge k. \end{cases} $$

If you don't care about $D$ but only $D'$, you can say that the entries of $D'$ are independent Bernoulli distributed variables with parameter $k$, or $d_{ij}'\sim\mathcal Bern(k)$. (And don't define $D$ at all).

1
On

$$\{ A \}\ {\rm s.t.}\ A \in {\bf M}^{n \times m}, m_{ij} = \sim {\cal B}[\theta]$$

The set of matrices $A$ in the set of $n \times m$ matrices in which every component is chosen from the Bernoulli distribution with parameter (threshold) $\theta$.