Related to image processing, I'm familiar with different types of geometric transformations. Translation, scaling, similarity, Affine, Perspectivity, and Projective Transformation.
Is there a transformation more general than an Affine transformation but less general than a perspectivity?
Let me start with some definitions. Given a (real) affine $n$-space $A$, I will denote by $\bar{A}$ its projective compactification, which is a projective $n$-space. Then a perspectivity between two affine planes $A_1, A_2$ in the affine 3-space $A$ is actually an (invertible) projective map $p: \bar{A}_1\to \bar{A}_2$ given by a projection from a point $o\in \bar{A}=P^3$ (the center) which does not belong to $\bar{A}_1\cup \bar{A}_2$ ($p(a)$ is defined by taking the unique intersection point with $\bar{A}_2$ of the projective line $oa$ through $o$ and $a\in \bar{A_1}$). An affine perspectivity is defined by choosing the center in the projective plane at infinity, $$ o\in P^2= P^3 \setminus A. $$ Affine perspectivities can be characterized as those perspectivities between $A_1$ and $A_2$ which send points at infinity to points at infinity; equivalently, they are restrictions to $A_1$ of those affine transformations of $A$ which send $A_1$ to $A_2$.
By abuse of terminology, one can call an affine perspectivity an "affine transformation", even though the word "transformation" is usually reserved for maps from a space to itself. Now, your question amounts to asking for a subset $S$ of $$ P^3 \setminus (\bar{A}_1\cup \bar{A}_2) $$ which contains $$ P^2_0:=P^2 \setminus (\bar{A}_1\cup \bar{A}_2). $$ (This will be the set of centers of perspectivities which are more general than affine.)
Claim. I claim that there is no natural choice of such subset apart from $P^2_0$ and and $P_0^3= P^3 \setminus (\bar{A}_1\cup \bar{A}_2)$. Here by natural I mean invariant under the group $G$ of affine transformations of $A$ which carry the pair of affine planes $A_1, A_2$ to themselves. Indeed, this follows from the fact that the group $G$ acts transitively on the set $$ A_0=A \setminus (A_1 \cup A_2). $$ Hence, if $S$ contains a point $o\in A_0$, then it contains the entire $A_0$.
Of course, if you do not insist on naturality, you can arbitrarily add to $P^2_0$ a subset of $P^3_0$. For instance, you can add all points with rational coordinates (assuming that you have a fixed coordinate system in the affine space). Or you can add all points which lie in a hyperplane of your choosing, etc.
In this answer I was working with the 3-dimensional affine space (you did not specify the dimension you are interested in, but I assumed it is 3); the same argument works in all dimensions.