The universal coefficient theorem gives $$H^k(X,X';N)\cong Ext^1_R(H_{k-1}(X,X';R),N))\oplus Hom_R(H_k(X,X';R),N).$$
But now for any free abelian group $A$, we have $Ext^1_{\mathbb Z}(A,\mathbb Z)\cong 0$. If we take the homology to also have coefficients in $\mathbb Z$ then it seems that $H_{k-1}(X,X';\mathbb Z)$ is also a free group. This gives that $H^k(X,X';\mathbb Z)\cong Hom_R(H_k(X,X';\mathbb Z),Z)$.
I am confused as this seems to be too strong of a result. Is $H_{k-1}(X,X';\mathbb Z)$ not always a free group?
Here is an example of $H_{k-1}(X, X';\mathbb{Z})$ not being free: let $X = \mathbb{R}P^{10}$ and $X'= \mathbb{R}P^5$. Then $H_i(X, X';\mathbb{Z}) \cong \mathbb{Z}/2$ for $i=7, 9$.
More generally, if $G$ is any abelian group you could take $X$ to be a CW complex with $H_{k-1}(X;\mathbb{Z})\cong G$ and take $X'$ to be a subcomplex of dimension $< k-2$, then the relative homology will still be $G$.