Congrunece relation is a 4-ary relation between points: $\delta(x,y;z,t)$ iff $|xy|=|zt|$.
Betweennes relation is a 3-ary relation between points: $\beta(x,y,z)$ iff $y \in xz$.
Midpoint relation is a 2-ary operation on points: $x \circ y = z$ iff $\beta(x,z,y) \land \delta(x,z;z,y)$.
Is there any general way of defining congruence using only the notions of midpoints and betweenness in any affine space $\mathbb R^n$? Or, perhaps, is there a separate way for any $n$?
This cannot be done. If we work in $\mathbb R^2$, then the transformation $(x,y) \mapsto (x, 2y)$ preserves betweenness and midpoints. (In general, any affine transformation has this property.) So any property that you can define in terms of midpoints and betweenness will also be preserved by this transformation.
But the transformation $(x,y) \mapsto (x,2y)$ does not preserve congruence: the segment from $(0,0)$ to $(1,0)$ is congruent to the segment from $(0,0)$ to $(0,1)$, but after the transformation this is no longer the case.
In general, betweenness, midpoints, and other properties such as:
are properties of affine spaces: they are preserved by affine transformations. Lengths of line segments, and comparisons between lengths of non-parallel line segments, are not preserved in this way; they are outside the realm of affine geometry. (The same is true for circles, angles, and other more complicated objects.)