Three horizontal ropes pull on a large stone stuck in the ground, producing the vector forces A⃗ , B⃗ , and C⃗ shown in the figure below
Find the magnitude of a fourth force on the stone that will make the vector sum of the four forces zero.
Part A I got 90.2 N
from:
Rx = 100cos30° + 80cos120° + 40cos233° + Dx = 0
Dx = -22.53N
Ry = 100sin30° + 80sin120° + 40sin233° + Dy = 0
Dy = -87.34N
then using magnitude function(sqrt(a^2+b^2) = 90.2N
Find the direction of a fourth force on the stone that will make the vector sum of the four forces zero. For Part B I used arctan[(-87.34N)/(-22.53N)] = 75.5° but it is wrong. Why?

Assuming your computation of the fourth force being $(-87.34,-22.54)$ is correct, then what you want is $75.5^\circ+180^\circ$. It may help to note that $\arctan(-87.34/(-22.53))=\arctan(87.34/22.53)$, which will give you one of the angles of a right triangle with legs $87.34$ and $22.53$. However, your fourth force lies in the 3rd quadrant, so you should add $180^\circ$.