If one has two vector bundles $E\to X$ and $F\to X$ over a scheme $X$, why is there a scheme $S$ over $X$ with points of $S$ over a point $x\colon\mathrm{spec}(R)\to X$ is precisely the set of isomorphisms of $x^\ast(F)$ and $x^\ast(E)$?
I think this is a standard result, but going back I can't find it in the literature.
On
A general result goes as follows.
Whenever you have a category fibered in groupoids $p:\mathscr V\to \textrm{Sch}$ (a particular covariant functor to the category of schemes), if you choose an object $X\in \textrm{Ob }\textrm{Sch}$ you get a groupoid $\mathscr V_X\subset \mathscr V$, the subcategory whose objects lie above $X$ and whose morphisms lie above $1_X:X\to X$. Now, for any two objects $E,F\in \textrm{Ob }\mathscr V_X$ there exists a (contravariant) functor $$\textrm{Isom}_X(E,F):\textrm{Sch}_X\to \textrm{Sets}$$ defined on objects by sending an $X$-scheme $f:T\to X$ to the set of isomorphisms between $f^\ast E$ and $f^\ast Y$, i.e. to the set $\hom_{\mathscr V_T}(f^\ast E,f^\ast F)$.
The upshot is that if $\mathscr V$ is the category of rank $r$ vector bundles and $p$ is the groupoid fibration sending a vector bundle to its base, then the above functor is represented by an $X$-scheme $\textbf{Isom}_X(E,F)$, which is the one you are after.
If you want to know more about the Isom scheme, I recommend:
Can't you take $S$ to be the subset of the vector bundle $\mathrm{Hom}(E,F)$ whose fiber is the set of isomorphisms? This is a $GL_n$-bundle over $X$, since if $E,F$ are trivialized over the open set $U$ then $S|_U \cong U \times GL_n$, where $n$ is the rank of the vector bundles $E$ and $F$.