In computer science there are what are known as higher-order functions. This means pretty much nothing, really. It is a property of a programming language.
I want to know if they are valid in mathematics, and if so, what the notation is. It would be like the following function definition:
$$g(x,y,f) = 20 + f(x + 3y)$$
The function itself is not really important. I just obscured it enough so that it cannot be made into a function composition.
I know that integrals, limits, and derivatives are sort of like functions that take functions as arguments. What about functions that return arguments?
Like:
$$h(f) = 3 \cdot f$$
The above would return the f function but where the terms are multiplied by three. Of course this would be f itself, not its image.
What is the difference between a function and its image anyway? I imagine it would be like the difference between source code and return values, but I have never heard of math having "source code" so it seems bizarre to me.
Yes, this is perfectly valid, and there's no particular notation for it (the notation you used is perfectly fine). In general, if $X$ and $Y$ are two sets then a function $g:X\to Y$ is a rule that assigns to each element of $X$ an element of $Y$ (more formally, "rule" here really means a set of ordered pairs $(x,y)$, where for each $x\in X$ there is a unique $y\in Y$ such that $(x,y)$ is in the set; we then say $y=g(x)$). The sets $X$ and $Y$ can be any sets at all, including sets of functions.
So, for instance, your example $g(x,y,f)=20+f(x+3y)$ could be considered as a function whose domain is the set $X$ of all ordered triples $(x,y,f)$, where $x,y\in\mathbb{R}$ and $f:\mathbb{R}\to\mathbb{R}$ is a function and whose range is $\mathbb{R}$. Given any triple $(x,y,f)$, $g$ gives you a real number, namely $20+f(x+3y)$.