In my Abstract Algebra class, the professor defined a restriction as
Given $ X\xrightarrow{f} Y $ and a non-void subset $S$ of $X$ define $ f \mid S\xrightarrow{S} Y $ by $(f \mid S )(s) = f(s), \forall s \in S$
He literally just pulled this one out of a hat without explaining what he meant by restriction or even what was meant by the definition.
If anyone could enlighten me, I'd much appreciate it.
Restricted here means much the same as it does in ordinary English. For example, think of the function $f:\mathbb{R}\to\mathbb{R}$ defined by $f(x)=x^2$.
This function has some defects. For one thing, it is not one to one, since $f(-3)=f(3)$.
We can define a new function $g$, from the set $\mathbb{R}^+$ of non-negative integers to $\mathbb{R}$, by $g(x)=x^2$. This function is $f$ restricted to $\mathbb{R}^+$. Note that $g$ is one to one.
In the notation used in your course, $g$ is $f|\mathbb{R}^+$.
The only "inputs" that $g$ accepts are elements of $\mathbb{R}^+$, but it does the same thing to them as $f$ did.