This is Exercise 1.4.8(3) of Springer's book, "Linear Algebraic Groups (Second Edition)".
The Question:
A morphism of affine varieties $\phi: X\to Y$ is an isomorphism if and only if the algebra homomorphism $\phi^*$ is an isomorphism.
The Details:
Since definitions vary:
A topological space $(X,\tau)$ is a set $\tau$ of subsets of $X$, called closed subsets, such that
- $\varnothing, X\in\tau$,
- The intersection $$\bigcap_{i\in I}X_i$$ of any closed subsets $(X_i)_{i\in I}$ is closed, where $I$ is arbitrary, and
- The union of finitely many closed sets is closed.
Note that $\tau$ is omitted sometimes when the context is clear.
Let $k$ be an algebraically closed field.
We denote by $\mathcal{V}(I)$ the set of all zeros of $I$.
The topology on $V:=k^n$ whose closed subsets are $\mathcal{V}(I)$ for ideals $I$ of $S:=k[T_1,\dots, T_n] =k[T]$ is known as the Zariski topology.
For any $X\subseteq V$, let $\mathcal I(X)\subseteq S$ be the ideal of the $f\in S$ with $f(v)=0$ for all $v\in X$.
Let $X=\mathcal{V}(I)\subseteq V$ be an algebraic set.
The restriction to $X$ of the polynomials of $S$ form a $k$-algebra isomorphic to $S/\mathcal I(X)$; write this as $k[X]$. Then:
$k[X]$ is a $k$-algebra of finite type, i.e., there is a finite subset $\{f_1,\dots,f_r\}$ of $k[X]$ such that $k[X]=k[f_1,\dots, f_r]$;
$k[X]$ is reduced, i.e., $0$ is the only nilpotent element of $k[X]$.
A $k$-algebra with properties 1 and 2 is called an affine $k$-algebra.
For $f\in k[X]$ put
$$D_X(f)=D(f)=\{ x\in X\mid f(x)\neq 0\}.$$
Then the $D(f)$ are called principal open subsets of $X$.
On page 6,
A $k$-valued function $f$ defined in a neighbourhood $U$ of $x$ is called regular in $x$ if there are $g,h\in k[X]$ and an open neighbourhood $V\subseteq U\cap D(h)$ of $x$ such that $f(y)=g(y)h(y)^{-1}$ for $y\in V$.
A function $f$ defined in a nonempty open subset $U$ of $X$ is regular if it is regular in all points of $U$. [. . .] Denote by $\mathcal O_X(U)=\mathcal O(U)$ the $k$-algebra of regular functions in $U$.
The pair $(X,\mathcal O_X)$ is called a ringed space in a general setting; in particular, though, it is called an affine algebraic variety or an affine $k$-variety.
Let $(Y,\mathcal O_Y)$ be an affine algebraic variety. Let $\phi:X\to Y$ be a continuous map. If $f$ is a function on an open set $U\subseteq Y$, denote by $\phi_U^*f$ the function on the open subset $\phi^{-1}U$ of $X$ which is the composite of $f$ and the restriction of $\phi$ to that set. We call $\phi$ a morphism of affine algebraic varieties if, for each open $U\subseteq Y$, we have that $\phi^*_U$ maps $\mathcal O_Y(U)$ into $\mathcal O_X(\phi^{-1}U)$.
A morphism $\phi:X\to Y$ on affine varieties defines an algebra homomorpism
$$\mathcal O_Y(Y)\to\mathcal O_X(X),$$
giving a homomorphism $\phi^*:k[Y]\to k[X]$.
Humphreys calls $\phi^*$ the comorphism of $\phi$ (which comes as no surprise to me) in his "Linear Algebraic Groups".
Phew! I think that's all the definitions required!
Context:
This exercise is difficult for me due to my topology being rusty and the plethora of nomenclature I'm unfamiliar with. It was suggested by my academic supervisor that I give it a go, since he remembers struggling with it when he learnt the material and, once he finally did it, it illuminated things for him considerably.
He said I can invest as much time into it as I see fit. I think it's time I take the initiative and ask about it here.
To give you a flavour of where I am, I asked the following question nearly a month ago:
Understanding $\mathcal V(I)$, $\mathcal I(X)$, and their relationship to each other.
I offered a bounty there several days ago too.
I am likely to give a bounty here.
I'm looking for a detailed answer working from the ground up, please. My supervisor hinted that category theory might help, saying that $\phi^{\color{red}{*}}$ is chosen to signify a contravariant functor, but Springer's book doesn't cover that perspective explicitly yet.
I think we can start like so. Suppose $\phi: X\to Y$ is an isomorphism. Then I need to show there is a homomorphism $ \phi^{-1}:Y\to X$ such that
$$\phi\circ\phi^{-1}=1_Y\quad\text{and}\quad\phi^{-1}\circ\phi=1_X.$$
Then I need to show:
$$\begin{align}(\phi^{-1})^*\circ\phi^*&\stackrel{?}{=}(\phi\circ\phi^{-1})^*\\ &=(1_Y)^*\\ &\stackrel{?}{=}1_{k[Y]} \end{align}$$
and
$$\begin{align}\phi^*\circ(\phi^{-1})^*&\stackrel{?}{=}(\phi^{-1}\circ\phi)^*\\ &=(1_X)^*\\ &\stackrel{?}{=}1_{k[X]}. \end{align}$$
That would take care of the forward direction.
Update: 04/11/2022:
Let $f\in k[Y]$. Then
$$\begin{align} ((\phi\circ\phi^{-1})^*(f))(y)&=(f\circ(\phi\circ \phi^{-1}))(y)\\ &=((f\circ\phi)\circ\phi^{-1})(y)\\ &=((\phi^{-1})^*(f\circ\phi))(y)\\ &=((\phi^{-1})^*(\phi^*(f)))(y)\\ &=(((\phi^{-1})^*\circ\phi^*)(f))(y), \end{align}$$
so $(\phi\circ\phi^{-1})^*=(\phi^{-1})^*\circ \phi^*$ and similarly $(\phi\circ\phi^{-1})^*=\phi^*\circ (\phi^{-1})^*$, and
$$\begin{align} ((1_Y)^*(f))(y)&=(f\circ 1_Y)(y)\\ &=f(1_Y(y))\\ &=f(y)\\ &=(1_{k[Y]}(f))(y), \end{align}$$
so that $(1_Y)^*=1_{k[Y]}$. Similarly, $(1_X)^*=1_{k[X]}$.
(I would like to thank my secondary supervisor for his support with this.)
My problem with this update is that it doesn't use $\mathcal O_Y(Y)$, say, or $D(f)$. Plus, I keep changing my mind on what the subscripts should be . . .
The following question seems relevant to the converse:
Every $K$-algebra homomorphism of affine algebras is a comorphism
An example to consider is when $X=\{X_1^2+X_2^2-1\}\subseteq k^2$ and $Y=\{Y_1^2-Y_2\}\subseteq k^2$. Take
$$\begin{align} \phi: k^2&\to k^2,\\ (X_1,X_2)&\mapsto (X_1,1-X_2^2). \end{align}$$
My supervisor mentioned this example. It is clear that $\phi(X)\subseteq Y$. I'm not sure where to take this though. (I don't expect this to be an isomorphism.)
It was suggested here by @Thorgott that I should
try showing that a map $\phi\colon X\rightarrow Y$ is a morphism if and only if its coordinates are regular functions on $X$
Please help :)
I am totally aware that this is not the answer that you wanted, but I hope it can make the exercise easier.
What you are looking for is an equivalence of categories between the categories $\mathbf{Aff}_k$ and the opposite of $\mathbf{RedAlg}_k$, where $\mathbf{Aff}_k$ is the category whose objects are affine algebraic varieties over $k$ and the arrows are morphisms; and $\mathbf{RedAlg}_k$ is the category whose objects are finitely generated reduced $k$-algebras and the arrows are the usual homomorphisms.
In particular, if $\mathcal{F}:\mathbf{C}\to\mathbf{D}$ is an equivalence of categories, $\alpha$ is an isomorphism (resp. monomorphism, epimorphism) in $\mathbf{C}$ if and only if $\mathcal{F}\alpha$ is an isomorphism (resp. monomorphism, epimorphism) in $\mathbf{D}$.
You need to prove that the functor mapping an affine algebraic variety $X$ to its coordinate ring $\mathcal{A}(X)=\mathcal{O}_X(X)$ and mapping a morphism $\varphi:X\to Y$ to $\varphi^*:\mathcal{A}(Y)\to\mathcal{A}(X)$ is an equivalence of categories, i.e., you need to prove:
For the first point you don't need category theory. The second point is equivalent to prove that given two affine algebraic varieties $X\subseteq\mathbb{A}^n_k$ and $Y\subseteq\mathbb{A}^m_k$ and a homorphism $\phi:\mathcal{A}(Y)\to\mathcal{A}(X)$ then there's a unique morphism $\varphi:X\to Y$ such that $\varphi^*=\phi$. A morphism is given by $\varphi(x)=(\varphi_1(x),\dots,\varphi_m(x))$, where $\varphi_i(x)\in\mathcal{A}(X)$ for all $1\leq i\leq m$. You can check that defining $\varphi_i(x):=\phi(x_i)$ for all $1\leq i\leq m$, you get what you need.
I don't know how much you wanted to avoid using category theory, but you don't really need a lot of it to prove the result. Once you reduce your problem to the two points above, you are only working with algebraic geometry. Previous to that, all you need to know is the definition of category, functor, and equivalence of categories.