I see many different notations in context of sets, minimization.
For example
$$\min_{x \in X} f(x),$$
$$\min \{ f(x) \mid x \in X \}$$
$$\min \{ f(x) \colon x \in X\}$$
and so on..
Is there any difference between them? I feel very confused when I read books on optimization and they use these things very inconsistently.
Thank you for your help.
No, all these notations express exactly the same thing. The only difference that you might want to be aware of is between $$\arg\min_{x\in X} f(x)$$ and $$\min_{x\in X}f(x)$$ or it's equivalent forms that you refer to in your post. The $\arg\min$ refers to the $x_0 \in X$ at which the minimum of $f$ is attained and the $\min$ refers to the minimum value of the function $f$ which is in that case $f(x_0)\in \mathbb R$.
Otherwise, all the expressions that you have above are equivalent.