An infinite random sequence $(X_i)_{i \ge 1}$ is defined exchangeable if its distribution is invariant to finite permutations of the indexes, that is $$(X_i)_{i \ge 1} \overset{d}{=} (X_{\sigma(i)})_{i \ge 1},$$ for any finite permutation $\sigma$ of $\mathbb{N}$.
An infinite random matrix $(X_{ij})_{ij}$ is defined row-column exchangeable (RCE) if its distribution is invariant to two different permutations of the row indexes and column indexes, that is $$(X_{i j})_{ij} \overset{d}{=} (X_{\sigma(i) \sigma^{\prime}(j)})_{ij},$$ for any pairs of finite permutations $\sigma$ and $\sigma^{\prime}$ of $\mathbb{N}$.
This is equivalent to ask that the sequence of rows $(X_{i\cdot})_{i \ge 1}$ are exchangeable and also the sequence of the columns $(X_{\cdot j})_{j \ge 1}$, see Aldus1983.
Question: if fixed an arbitrary row $i \in \mathbb{N}$, $(X_{ij})_{j \ge 1}$ is an exchangeable sequence and fixed an arbitrary column $j \in \mathbb{N}$, $(X_{ij})_{i \ge 1}$ is an exchangeable sequence, the random matrix $(X_{ij})_{ij}$ is row-column exchangeable?
Guess: I believe that it is false. More precisely, RCE implies immediately exchangeability inside an arbitrary row and inside an arbitrary column, but I think that from the marginal assumption we can not recover the global one, but I can not find a counterexample.
I'm not sure whether I'm interpreting the definition in your Question correctly, but as I understand it the answer is no: your definition is different from row-column exchangeability, although row-column exchangeability is a special case of yours.
Consider for example a matrix with entries $$\begin{pmatrix} 1&1&1&\dotso\text{(all 1)} \\\ 0&0&0&\dotso \text{(all 0)}\\\ 0&0&0&\dotso\text{(all 0)} \\\ &\dotso&\text{(all 0)}&\end{pmatrix}$$ According to row-column exchangeability, it has the same probability as the matrix $$\begin{pmatrix} 0&0&0&\dotso \\\ 1&1&1&\dotso \\\ 0&0&0&\dotso \\\ &\dotso&&\end{pmatrix}$$ but in principle it can have a different probability from $$\begin{pmatrix} 1&0&1&\dotso \\\ 0&1&0&\dotso \\\ 0&0&0&\dotso \\\ &\dotso&&\end{pmatrix}$$ because this last matrix cannot be obtained from the first one through any sequence of exchanges of whole rows or whole columns.
According to your definition, instead, the third and first matrix should have the same probability because the third is obtained by exchanging the rows of the $j=2$ column, with $j$ arbitrary and fixed.