My friend insists that it is valid to write $h\in f(x)$ where $f:A\to B$ is just a set-theoretic function.
He says one need not write $h=f(x)$ to be correct, since $f(x)$ is the image of the singleton $\{x\}$, and the image of a morphism $f\in\text{Mor}(\mathsf{Set})$ is of course a set, so we have $f(x)=f|_{\{x\}}(x)$ is a set. If $f$ is injective, where '$f(x)=h$', then $h\in f(x)$ is valid. Is he correct?
On
If $f\colon A\to B$ is a function (if you will, $f$ is a subset of $A\times B$ with certain properties: $\forall x\in A\exists y\in B\colon \langle x,y\rangle\in f$, $\forall x\in A\forall y,z\in B\colon \langle x,y\rangle \in f\land \langle x,z\rangle\in f\to y=z$), then $f(x)$ denotes the unique (by virtue f $f$ being a function) element $y\in B$ such that $\langle x,y\rangle \in f$. This (in general) does not make $y\in f$. In fact, in general $B\cap f=\emptyset$.
By abuse of notation, if $S\subseteq A$ is a set, we also write $f(S)$ for the set $\{\,f(y)\mid x\in S\,\}$ (but this is ambiguous as it may happen that bot $S\in A$ and $S\subset A$ and both notions of $f(S)$ differ). So if $f(x)=y$, we also have $f(\{x\})=\{y\}$ and so $y\in f(\{x\})$, but still not $y\in f(x)$.
You mention the category theoretical context. There, objects do not even have elements per se! If $A\in\text{Obj}(\mathsf{Set})$, the best way to intrduce elements of $A$ is in the form of morphisms $\bullet\stackrel x\to A$ from a terminal object (i.e., a "standard" singleton set). Then if $f\in\text{Mor}(A,B)$, we see that $f\circ x$ is an element $\bullet\stackrel f\to A$ of $B$ (which we then define and write as $f(x)$). Note that even in this context, $f(x)$ is not a set but a specific kind of morphism, and even that only because we need to introduce element-of relation into the abstacrtness of category theory.
Your friend is absolutely wrong, but one can see where his problems come from.
If $f\colon X\to Y$, we often want to consider its derived functions, direct and inverse image: $F\colon\mathcal P(X)\to\mathcal P(Y)$ given by $F(A)=\{f(a)\mid a\in A\}$, and $G\colon\mathcal P(Y)\to\mathcal P(X)$ given by $G(B)=\{a\in A\mid f(a)\in B\}$.
But often we write $f(A)$ instead of $F(A)$, or $f^{-1}(B)$ instead of $G(B)$. And we also have a tacit agreement that if $x\in X$, then since $F(\{x\})$ is a singleton $\{y\}$, and moreover $f(x)=y$, then we can confuse these without worry.
You can see this when $f$ is not injective, and then $f^{-1}(y)$ is "not well-defined", because we implicitly mean $f^{-1}(\{y\})$ or $G(\{y\})$. But then to make things simpler, we want to write $f^{-1}(f(x))$, so we really mean $\{f(x)\}$ there.
So while your friend is absolutely wrong, and writing $f(x)\in f(x)$ is confusing, misleading, and possibly even offensive to some people, it should be clear why he feels that this notation is even remotely "reasonable".