In the Lecture Notes in Algebraic Topology by Davis and Kirk, one has the following result
Proposition 1.14 (Adjoint Property of Tensor Products) There is an isomorphism of $R$-modules $$\operatorname{Hom}_R(A \otimes_R B, C) \simeq \operatorname{Hom}_R(A, \operatorname{Hom}_R(B,C))$$ natural in $A$, $B$, $C$ given by $\phi \leftrightarrow (a \mapsto (b \mapsto \phi(a \otimes b) ) ).$
My problem is it now to understand what exactly natural in $A$, $B$, $C$ means.
When one talks about naturality, then one probably means there is a natural isomorphism. At least this is what I assume.
Let us consider the naturality in $A$ and denote the category of $R$-modules by $R\operatorname{-MOD}$. Then we can define two functors $\mathcal{F}^A, \mathcal{G}^A: R\operatorname{-MOD} \to R\operatorname{-MOD}$ by $$\mathcal{F}^A(X) = \operatorname{Hom}_R(X \otimes_R B, C), \quad \mathcal{G}^A(X) = \operatorname{Hom}_R(X, \operatorname{Hom}_R(B,C)).$$
Now the natural isomorphism $\eta^A: \mathcal{F}^A \Rightarrow \mathcal{G}^A$ must satisfy that for every $R$-module $X$ we have an $R$-module isomorphism $\eta^A_X: \mathcal{F}^A(X) \longrightarrow \mathcal{G}^A(X)$. This is true by the universal property (one notes that $\mathcal{G}^A(X)$ can be interpreted as an bilinear map $X \times B \to C$).
The point where I am not sure about is if $\mathcal{F}^A$ and $\mathcal{G}^A$ are actually functors. For instance, given an $R$-module homomorphism $f: X \to Y$, how does the induced morphism $\mathcal{F}^A(f): \mathcal{F}^A(X) \to \mathcal{F}^A(Y)$ look like? My first try was it to take a $g \in \mathcal{F}^A(X) = \operatorname{Hom}_R(X \otimes_R B, C)$, $x \otimes b \mapsto g(x \otimes b)$ and set $\mathcal{F}^A(f)(g) (x \otimes b) = g(f(x) \otimes b)$. But this does not make sense since $f$ does not need to be surjective and the first component of $\mathcal{F}^A(f)$ has to be defined on the whole module $Y$.
Now I am totally stuck and have no idea how to continue. Could someone please unwind my messy thoughts? That would be great.
One thing you're missing is that the functor you call $\mathcal{F}^A$ is contravariant. We can compute it by unfolding notation.
Given $f : X \to Y$...
The morphism $f \otimes B : X \otimes B \to Y \otimes B$ is determined from the fact that $\otimes$ is a bifunctor. If mixing arrows and objects bothers you, note that $f \otimes B$ means the same thing as $f \otimes 1_B$.
This is, as you've presumably realized, the morphism $x \otimes b \mapsto f(x) \otimes b$.
Now, $\hom(-, C)$ is contravariant. Given any $g : U \to V$, $\hom(g, C) : \hom(V, C) \to \hom(U, C)$ is the mapping $h \mapsto h \circ g$. Note the reversal in the direction of the arrow.
So, $\hom(f \otimes B, C) : \hom(Y \otimes B, C) \to \hom(X \otimes B, C)$ is the function that sends $h : Y \otimes B \to C$ to the function $h \circ (f \otimes B) : X \otimes B \to C$. $h \circ (f \otimes B)$ is the mapping $x \otimes b \mapsto h(f(x) \otimes b)$.
And it will be true that this gives a natural isomorphism between functors
$$ \mathcal{F}^A, \mathcal{G}^A : (R\operatorname{-MOD})^{\mathrm{op}} \to R\operatorname{-MOD} $$
It is ultimately true that both sides of the equation define naturally isomorphic functors
$$ (R\operatorname{-MOD})^{\mathrm{op}} \times (R\operatorname{-MOD})^{\mathrm{op}} \times R\operatorname{-MOD} \to R\operatorname{-MOD} $$