The following theorem about composition of two derived functors is well known, see e.g. [Stacks Project, Tag 015M]:
Let $\mathcal{A}, \mathcal{B}, \mathcal{C}$ be abelian categories. Let $F : \mathcal{A} \to \mathcal{B}$ and $G : \mathcal{B} \to \mathcal{C}$ be left exact functors. If $\mathcal{A}$, $\mathcal{B}$ have enough injectives, then the canonical map $ R(G \circ F) \rightarrow RG \circ RF $ is an isomorphism of functors from $D^{+}(\mathcal{A})$ to $D^{+}(\mathcal{C})$ if (and only if) $F(I)$ is $G$-acyclic for each injective object $I$ of $\mathcal{A}$.
My question is about the situation with three such functors:
Let $\mathcal{A}, \mathcal{B}, \mathcal{C}, \mathcal{D}$ be abelian categories. Let $F : \mathcal{A} \to \mathcal{B}$, $G : \mathcal{B} \to \mathcal{C}$, and $H : \mathcal{C} \to \mathcal{D}$ be left exact functors. Assume $\mathcal{A}$, $\mathcal{B}$, $\mathcal{C}$ have enough injectives, $F(I)$ is $G$-acyclic for each injective object $I$ of $\mathcal{A}$, and $G(J)$ is $H$-acyclic for each injective object $J$ of $\mathcal{B}$.
Can we say something about the relation between $ R(H \circ G \circ F)$, $ RH \circ R(G \circ F)$, $ R(H \circ G) \circ RF$, and $RH \circ RG \circ RF$?
To get full associativity, your assumptions are not enough. You need to assume something stronger about $F$ - it should map injectives to $H \circ G$-acyclics or something similiar.
To see that such a condition is necessary, consider the case where $G$ is the identity functor - your assumptions then give you back the first case, but without any assumptions on $F$. As you already have remarked, this is not enough to guarantee $R(H \circ G \circ F) \cong R(H \circ G) \circ RF$.
On the other hand, if you assume that $F$ sends injectives to $H \circ G$-acyclics, then two times applying your first box gives $R(H \circ G \circ F) \cong RH \circ RG \circ RF$.