First of all, the equation you're using is incorrect. Please refer to the correct version.
Second, you have realized that the gravitational force of $m_1$ on $m_5$ and $m_4$ on $m_5$ cancel each other out, because the mass of $m_1$ and $m_4$ are the same, and so are their distance from $m_5$. This observation speeds things up.
Lastly, to lead you in the correct direction, the magnitude of the gravitational force of $m_2$ on $m_5$ is
$$
G\cdot\frac{m_2 m_5}{d^2},
$$
where $d$ is the distance from $m_2$ to $m_5$, and the direction is to the north east.
What about the gravitational force of $m_3$ on $m_5$? What is its magnitude and direction? What is the net effect of these two gravitational forces? What about the net effect of all four gravitational forces on $m_5$?
If you can figure out these issues step by step, you will have learnt something.
First of all, the equation you're using is incorrect. Please refer to the correct version.
Second, you have realized that the gravitational force of $m_1$ on $m_5$ and $m_4$ on $m_5$ cancel each other out, because the mass of $m_1$ and $m_4$ are the same, and so are their distance from $m_5$. This observation speeds things up.
Lastly, to lead you in the correct direction, the magnitude of the gravitational force of $m_2$ on $m_5$ is $$ G\cdot\frac{m_2 m_5}{d^2}, $$ where $d$ is the distance from $m_2$ to $m_5$, and the direction is to the north east.
What about the gravitational force of $m_3$ on $m_5$? What is its magnitude and direction? What is the net effect of these two gravitational forces? What about the net effect of all four gravitational forces on $m_5$?
If you can figure out these issues step by step, you will have learnt something.