In elementary treatments of algebraic varieties, a regular morphism between affine varieties is one whose components are all polynomials, and the coordinate ring as a ring of polynomial functions.
When defining the sheaf of regular functions on a variety $X$, then you have for any basic open set $D(f)$, the nonvanishing locus of a polynomial $f$, that $\mathcal{O}(D(f)) = \mathcal{O}(X)_f,$ which is the ring obtained by adjoining $1/f$ to the coordinate ring of polynomial functions. This is for example explained by user D_S here or the sources linked by Nils Matthes in this answer. We started by looking at polynomials, and then decided to include $1/f$. I understand including $1/f$ when we look for the rational functions, but the multiplicative inverse of a polynomial is definitely not polynomial. What is the justification for adding it, is there something stronger than "because it's nonzero on $D(f)$ therefore $1/f$ is defined. We add it because we can".
The proofs in those linked sources start out as defining a regular function as "locally a quotient of the form $g/h$ where $h$ doesn't vanish. So they appear to be just assuming the thing I am looking for justification of.
I'm not asking what inverting elements has to do with localization,which has some good answers on this site already. I liked Viktor Vaughn's answer on this thread and Jesko Hüttenhain's in this one.
When you say
you're assuming that a variety $X$ is given as embedded in some affine space $\Bbb A^n$, and when you say "polynomial", you mean the restriction of a polynomial function on $\Bbb A^n$ to $X$. But what if we took a different embedding? It turns out that all embeddings will give the same ring, so the ring of regular functions is independent of the embedding.
Let's now consider $1/f$: if $f=0$ on some but not all points of $X\subset\Bbb A^n$, then $D(f)\subset X \subset \Bbb A^n$ is not an affine variety, because it is not closed in $X$ (and hence not closed in $\Bbb A^n$). But there is a variety in $\Bbb A^{n+1}$ which looks very much like $D(f)$: if $I$ is the ideal of polynomials cutting out $X\subset\Bbb A^n$, then the zero locus of the ideal $(I,x_{n+1}f-1)$ in $\Bbb A^{n+1}$ is (1) a variety and (2) projects down to exactly $D(f)$ under the projection $\Bbb A^{n+1}\to\Bbb A^n$, $(x_1,\cdots,x_n,x_{n+1})\mapsto (x_1,\cdots,x_n)$. If we broadened our class of functions to "polynomial on some closed embedding", then we would still get everything we've developed so far and we could declare that these two objects are isomorphic, then apply our tools to a few more scenarios. In this sense, it's a natural extension of the theory you've already talked about.
(This is slightly out of order with how the history developed - there are already parametrizations using rational functions running around pretty early on, perhaps as early as Greek antiquity, depending on how generous you are. If you're not as generous, Riemann's work leads to some amount of rational geometry and considering "locally regular" functions in the 1800s when talking about Riemann surfaces - so certainly the idea of looking at functions which behave well locally is already floating around in the fields which kicked off much of the development of algebraic geometry. One can reasonably start from considering functions which are locally given by a quotient of polynomials and then prove that on an affine variety, the only functions which are regular everywhere are the polynomial functions, and I'd say this might be more reflective of how the field actually developed.)