I followed the link and link to prove that $\mathbb{R}^J$ is open, and the following is my attempt:
Under uniform topology, $\mathbb{R}^J$ is metrizable with uniform metric, and is therefore normal. This gives that for any $x\in \mathbb{R}^J$ and a closed set $A$, we can find an open neighbourhood $U$ of $x$ in $\mathbb{R}^J$ such that $U\cap A=\emptyset$.
Since $\mathbb{R}^J$ with uniform metric is normal, given $x$ and $U$ as above, by Urysohn's Lemma there exists a continuous function $f:\mathbb{R}^J\rightarrow[0,1]$ such that $f(x)=0$ and $f(y)=1$ for all $y\in\mathbb{R}^J\setminus U$, since $\{x\}$ and $\mathbb{R}^J\setminus U$ are closed sets.
Since box topology is finer than uniform topology, $f$ defined above is also continuous in $\mathbb{R}^J$ with box topology. We also have $A\subseteq \mathbb{R}^J\setminus U$, so we have a continuous function $f$ such that $f(x)=0$ and $f(y)=1$ for all $y\in A$. This shows that $\mathbb{R}^J$ with box topology is completely regular.
My concern is that the closed set $A$ should be closed in the box topology. This only tells us that $\mathbb{R}^J\setminus A$ is open in box topology, but since uniform topology is coarser, we do not know whether it is open in uniform topology. I am thinking that as long as $A\cap U=\emptyset$, it does not matter. Will this be valid?
Let $X_i, i \in I$ be any family of completely regular Hausdorff spaces, and let $\Box_{i \in I} X_i$ be the set $\prod_{i \in I} X_i$ in the box topology. Then $\Box_{i \in I} X_i$ is completely regular and Hausdorff too.
Proof: the Hausdorffness is quite simple: if $x=(x_i)_i$ and $y=(y_i)_i)$ are distinct points in $\Box_{i \in I} X_i$, then there is at least one index $i_0$ such that $x_{i_0} \neq y_{i_0}$. As $X_{i_0}$ is Hausdorff we can find $U$ and $V$ disjoint in that space such that $x_{i_0} \in U$ and $y_{i_0} \in V$, and then $\pi_{i_0}^{-1}[U]$ and $\pi^{-1}[V_{i_0}]$ are disjoint open (projections are continuous on the box product) neighbourhoods of $x$ and $y$ resp.
Let $p=(p_i)_i$ be a point in $\Box_{i \in I} X_i$ and $U=\prod_i U_i$ be a basic neighbourhood of $p$ (each $U_i$ is thus an open set in $X_i$ containing $p_i$). It suffices to be find a continuous $f: \Box_{i \in I} X_i \to [0,1]$ such that $f(p)=0$ and $f(x)=1$ for all $x \notin U$. To this end, pick (as each $X_i$ is completely regular) a continuous $f_i: X_i \to [0,1]$ with $f(p_i)=0$ and $f_i[X\setminus U_i]=\{1\}$ for every $i \in I$.
Now define $f(x) = \sup \{f_i(x_i): i \in I\}$ which maps into $[0,1]$ (which is closed under taking sups). To see that $f$ is continuous it suffices to check that all sets of the form $(r,1]$, $r<1$ and $[0,s), s >0$, have open pre-image under $f$, i.e. all its points are interior points:
If $y \in f^{-1}[(r,1]]$ we know that $\sup \{f_i(y_i): i \in I\} > r$, which implies that $r$ is not an upper bound for $\{f_i(y_i): i \in I\}$, so for some $j \in I$, $f_j(y_j) > r$. But then $\pi_j^{-1}[f_j^{-1}[(r,1]]]$ contains $y$ and all points $x$ in it have $f_j(x_j) >r$ too and hence $f(x)>r$. So $y \in \pi_j^{-1}[f_j^{-1}[(r,1]]] \subseteq f^{-1}[(r,1]]$ and as $y$ is arbitrary, the latter set is open, as required.
If $y \in f^{-1}[[0,s)]$ we know that for all $i$: $f_i(y_i) \le f(y) < s$ so we can pick $t \in [0,1]$ such that $f(y) < t < s$ as well. Then for all $x \in W:=\prod_{i \in I} f_i^{-1}[[0,t)]$, which is box open by definition and continuity of the $f_i$, we have $f_i(x_i) < t$ so that $f(x)\le t < s$ and so $y \in W \subseteq f^{-1}[[0,s)]$, and so the latter set is again open as required.
There is an easier proof if you know about uniform structures: all $X_i$ are uniformisable by a uniformity given by entourages $\mathcal{D}_i$, and then it’s easy to check that $\mathcal{D}$ on $\prod_{i \in I} X_i$ given by $$(x,y) \in \mathcal{D} \iff \forall i \in I: (x_i,y_i) \in \mathcal{D}_i$$ is a uniformity on the product that is compatible with the box topology. (This too is not the product in the category of uniform spaces, just as in the topology category). The complete regularity of $\Box_{i \in I} X_i$ is then an immediate consequence. It turns out (see the chapter on box products by Scott Williams in the Handbook of Set-theoretic Topology) that a box product of topological groups is again a topological group and completeness (in the uniformities) is also preserved in the box product.
The strategy using the uniform metric topology on $\Bbb R^J$ cannot really work: we have to start with $U$ box-open and $x \in U$ and we need the Urysohn function for $U$. Suppose we could find a uniform-open $V$ such that $x \in V \subseteq U$. Then in the metric (hence completely regular) topology we can find a function $f: \Bbb R^J \to [0,1]$ with $f(x)=0, f[X\setminus V]=\{1\}$, continuous for the uniform topology, so also for the finer box topology and then indeed this $f$ would work. But e.g. in $\Bbb R^\omega$, the box open neighbourhood $\prod_{n \ge 1} (-\frac1n, \frac1n)$ of $0$ does not contain a uniform-metric ball around $0$, which shows we cannot work on our earlier optimistic assumption. Moreover, a meta-argument: we could have also applied this idea to closed sets and Urysohn-functions for them, while it is known (van Douwen) that there is a countable box product of completely metrisable spaces that is not not normal. The normality of $\Bbb R^\omega$ in the box topology is still open, AFAIK. (it is true under CH, e.g.) It might be undecidable. But complete regularity is no problem...