What is the meaning of taking the power of the set of reals $\mathbb{R}^2$

Question

What does this mean:

$\mathbb{R}^2$

or

$\mathbb{R}^{2\times500}$

I am trying to learn about neural networks and keep stumbling on set notation which I don't know how to look up on Google. (It took me 15 minutes just to figure out how to type $\mathbb{R}$ here, never mind on a search engine.)

The web page that has this notation is here: Implementing Neural Networks from scratch

The notation in question is in the section labeled "How our network makes predictions" in the 2nd paragraph.

I know from other Stack Exchange entries that a powerset is a relationship between one set and another. But this doesn't make sense to me when the second set is a number. In this case 2 or whatever we are to interpret $2\times 500$ to mean.

The article says: $W_1\in\mathbb{R}^{2\times 500}$

So $W_1$ is presumably in the set of reals. Is this suposed to mean that $W_1$ is an array that is $2$ wide by $500$ long?

Or something else?

Answer 1

$\mathbb{R}^2$ is commonly interpreted as $\mathbb{R}\times\mathbb{R} = \{(a,b)~:~a\in\Bbb R,~b\in\Bbb R\}$ is the set of all ordered pairs of real numbers. Here the $\times$ is in reference to the cartesian product.

Similarly $\mathbb{R}^n$ is commonly interpreted as the $n$-fold cartesian product $\times (\Bbb R,\Bbb R,\Bbb R,\dots,\Bbb R)$ or similarly as the successive cartesian products $\Bbb R\times \Bbb R\times \Bbb R\times \dots \times \Bbb R$. An astute observer will recognize that these are technically different, but for all intents and purposes they act exactly the same otherwise. As such, we are generally lazy and refer to this as the set of $n$-tuples of real numbers: $\Bbb R^n = \{(a_1,a_2,a_3,\dots,a_n)~:~\forall i~(a_i\in \Bbb R)\}$

On the other hand, if a product is explicitly written in the power such as $\Bbb R^{2\times 2}$ this is commonly taken to mean the set of $2\times 2$ matrices with entries in the reals. That is to say $\Bbb R^{2\times 2}=\left\{\begin{bmatrix}a&b\\c&d\end{bmatrix}~:~a,b,c,d\in\Bbb R\right\}$. This does act similarly to $\Bbb R^4$ in many regards and for many purposes can be thought of as essentially the same set, but it implies something more about how we represent the elements and also implies more about what operations are commonly used on the elements.

Similarly $\Bbb R^{m\times n}$ would be in reference to the set of $m\times n$ matrices with all elements as real numbers.

Answer 2

$\Bbb R^2$ is the set of ordered pairs of reals. It could represent the Euclidean plane. $\Bbb R^{1000}$ is the set of ordered sequences of length $1000$ of reals. They presumably wrote it as $2 \cdot 500$ to show where $1000$ came from. It could be an array that is $2 \times 500$.

The power set of the reals is something completely different. It is the set of subsets of the reals, but it is probably unimportant to you.