I have ray's source coordinates (point D) and angle of that ray in radians. How can I find coordinates of intersection of that ray with square perimeter (point F) for arbitrary source and angle?
I've tried to derive equation of line and from angle determine which of two coordinates equal to $0$ or $5$. Another try was to find coordinates on square with inscribed unit circle centered at source of ray and from that coordinates return to source square, but no success.

Lets label the edges top ($AB$), bottom ($OC$), left ($OA$), and right ($BC$), and the angle of the ray (counterclockwise from the $x$ axis) $\varphi$.
The simple solution is to use the ray angle $\varphi$ (counterclockwise from positive $x$ axis; say, $-180° \le \varphi \le +180°$), and check each possible edge. The ray may hit two edges at most, so we simply calculate the distance from $D$ to the intersection. The correct edge and location corresponds to the smaller (nonnegative) distance.
(You can work this out for all axis-aligned rectangles, by using coordinates $y_{max}$, $y_{min}$, $x_{min}$, and $x_{max}$, so that $O = ( x_{min} , y_{min} )$, $A = ( x_{min} , y_{max} )$, $B = ( x_{max} , y_{max} )$, and $C = ( x_{max} , y_{min} )$. See the example pseudocode at the bottom.)
For the ray, let's use $D = ( x_0, y_0 )$, so that the coordinates for point $t$ on the ray (at distance $t$ from $D$), $0 \le t \in \mathbb{R}$, are $$\begin{cases} x(t) = x_0 + t \cos \varphi \\ y(t) = y_0 + t \sin \varphi \end{cases}$$
However, we can generalize this for any ray, using $$\begin{cases} x(t) = x_0 + t x_\Delta \\ y(t) = y_0 + t y_\Delta \end{cases} \tag{1}\label{NA1}$$ simply by treating OP's particular problem with $$\begin{aligned} x_\Delta &= \cos\varphi \\ y_\Delta &= \sin\varphi \end{aligned}$$
Note that whenever I use the term distance below, I mean Euclidean distance in units of $\sqrt{x_\Delta^2 + y_\Delta^2}$.
The distance $t_x$ from $( x_0 , y_0 )$ to a vertical line at $x = X$ is $$t_x = \frac{X - x_0}{x_\Delta} \tag{2a}\label{NA2a}$$ The distance $t_y$ from $( x_0 , y_0 )$ to a vertical line at $y = Y$ is $$t_y = \frac{Y - y_0}{y_\Delta} \tag{2b}\label{NA2b}$$
If $x_\Delta = 0$, the ray is vertical, and can only hit the top or bottom edge. If $x_\Delta \gt 0$, the ray can hit the right edge. If $x_\Delta \lt 0$, the ray can hit the left edge. Similarly for $y_\Delta$.
If $t_x$ and $t_y$ both exist, then the ray hits the one with the smaller positive value $t$, and the intersection occurs at $( x(t), y(t) )$.
In pseudocode, one might implement this as a function thus:
Note that it would be easy to generalize the above function to work for any ray and any axis-aligned rectangle, including starting points outside the rectangle; you'd simply expand the
OUTSIDEcase. You'd probably also want to return some kind ofOUTSIDEorINSIDEflag, so the caller can easily determine which edge or vertex/corner and from which side the ray hit the rectangle, if any.