Let $R = \{(a,b)\mid a\cdot b=21\}$ be a relation on all integers.
so $R = \{(1,21), (3,7), (7,3), (21,1)\}$
I fail to understand if the domain here is all integers or only A, the domain, $= \{1,3,7,21\}$
Here it would mean its not reflexive since the domain is all integers (from minus infinity to infinity), it means $4\in$ domain and (4,4) its not part of the relationship? Im a bit confused. Thanks in advance.
A relation $R$ should be introduced over a specified universe $U$. This specification provides context for discussion and is important. It informs us that $R$ is to be considered a subset of $U \times U$, and allows us to apply to $R$ specific coordinate projections $U \times U \to U$.
For instance, you have called the image of $R$ under first coordinate projection $\alpha : (x,y) \mapsto x$ the domain of $R$, and would likely call its image under second coordinate projection $\beta : (x,y) \mapsto y$ the codomain of $R$, but not all texts would use such terminology - some might reserve both the terms domain and codomain for $U$ and call the sets $\alpha(R)$ and $\beta(R)$ something else.
While it is true that the only elements actually implicated by the relation $R$ are those in the set $V := \alpha(R) \cup \beta(R)$, reflexivity of $R$ is meant to be checked over its entire universe $U$. In your example $U$ is the set of integers, and indeed $R$ is not reflexive over $U$, as you have demonstrated.
You might still be wondering about reflexivity of this relation over a set like $V$, but you'd be wondering about reflexivity of a different relation, as the restriction $S = \left.R\right|_V$ is formally different from $R$.