We know that a paraboloid, $x_1^2 + x_2^2 = x^T x$ is the generalization of a quadratic $x^2$.
Is there some function similar to $x^Tx$ that can be used to represent cubic, quartic, quintic...polynomials?
I was thinking of $x x^Tx$ for $x^3$ but obviously that doesn't work. Or perhaps $x_1^3 + x_2^3$ is the natural generalization that you simply cannot be put into a dot product between two vectors.
On
One possible generalization is $x$, $x^{\otimes 2}$, $x^{\otimes 3}$, ... where we take the $n$- fold tensor power. This gives rise to the general version of Taylor's theorem for functions $\mathbb R^{n}\to \mathbb R^{m}$:
$$\begin{aligned}f(x) &=f(x_{0})+(Df(x_{0}))\cdot (x-x_{0})+{}{\tfrac {1}{2}}(D^{2}f(x_{0}))\cdot (x-x_{0})^{\otimes 2}+\cdots \\ &\qquad \cdots +{\tfrac {1}{k!}}D^{k}f(x_{0})\cdot (x-x_{0})^{\otimes k}+{\tfrac {1}{(k+1)!}}R_{k+1}(x)\cdot (x-x_{0})^{\otimes (k+1)} \end{aligned}$$
Or in other words the Taylor series is $T_{f,x_0}(x+x_0)=\sum_{k\ge 0}\tfrac{1}{k!}D^kf(x_0)\cdot x^{\otimes k}$. Note that here, in the term $D^{k}f(x_{0})\cdot (x-x_{0})^{\otimes k}$, the "$\cdot$" denotes the appropriate tensor contraction. For example, a case you are probably familiar with is the Taylor polynomial of degree $2$ for a scalar, multivariate function $\mathbb R^{n}\to\mathbb R$, which can be expressed in terms of the Jacobian and Hessian:
$$ f(x+ x_0)\approx f(x_0) + Jf(x_0)\cdot x + \tfrac{1}{2}x^THf(x_0)x $$
here, note that the Hessian term can be equivalently be expressed in many ways, including the Frobenius inner product on $\mathbb R^{n\times n}$, which is equivalent to the induced inner product on the tensor product of Hilbert spaces $\mathbb R^{n}\otimes\mathbb R^{n}$ or, finally, simply the corresponding tensor contraction
$$\begin{aligned} \tfrac{1}{2}x^THf(x_0)x = \tfrac{1}{2}\operatorname{tr}(x^T Hf(x_0)x) &= \tfrac{1}{2}\operatorname{tr}(Hf(x_0) xx^T) \\&= \tfrac{1}{2}\langle Hf(x_0), xx^T\rangle_F \\&= \tfrac{1}{2}\langle Hf(x_0), xx^T\rangle_{\mathbb R^{n}\otimes\mathbb R^n} \\&= \tfrac{1}{2} Hf(x_0)\cdot x^{\otimes 2} \end{aligned}$$
Since this is a polynomial function of degree 3, it should be equal to its own Taylor expansion of degree 3. Taking $x_0=0$ we have:
And on the other hand:
And the tensor contraction in each case, since $f$ is a scalar function, is simply the total sum over the multi-index
$$D^k f(x_0) \cdot x^{\otimes k} = \sum_{\substack{\mathbf{i}=(i_1,\ldots, i_k) \\ i_j \in\{1, \ldots, n\}}} [D^k f(x_0)]_{\mathbf{i}} \cdot [x^{\otimes k}]_{\mathbf{i}}$$
So
$$ \tfrac{1}{3!}D^3 f(0) \cdot x^{\otimes 3} = \tfrac{1}{6}\sum_{ijk} \Big(\frac{\partial f}{\partial x_i x_j x_k}\Big)_{ijk} x_ix_jx_k = \tfrac{1}{6} (6x_1^3 + 6x_2^3) = x_1^3 + x_2^3$$
The symmetric tensors that Vercassivelaunos talked about you can think of in the rank $2$ case: every rank-2 tensor ($\leftrightsquigarrow$ matrix) of the form
$$ T = \sum_{i} \Big( (u_i \otimes v_i) + (v_i \otimes u_i)\Big) \leftrightsquigarrow \sum_{i} \Big( u_iv_i^T + v_i u_i^T\Big) $$
I.e. the symmetric matrices you are familiar with. And in the general rank-$n$ case everything that can be written as a linear combination of symmetric pure tensors, i.e. $\displaystyle T=\sum_{i}\sum_{\sigma\in S_n} \bigotimes_{j=1}^n u_{\sigma(j)}^{(i)}$
What you want is a special case of a homogenous function (that's functions satisfying $f(\lambda x)=\lambda^k f(x)$ for some $\lambda$, just like $x^k$ in 1d). In particular, you want those homogenous functions which are generated by a symmetric multilinear map: Let $M_k:V^k\longrightarrow W$ be a multilinear map, where $V,W$ are vector spaces. Then you want those functions $f:V\longrightarrow W$ of the form $f(x)=M_k(\underbrace{x,\dots,x}_{k~\textrm{times}})$. Such functions will always be homogenous, like powers are in 1d. Such functions come up in the generalized Taylor's theorem, for instance, similar to what Hyperplane mentions in their answer. These kinds of homogenous functions are also special because they allow us to express polynomials in multiple variables: For every polynomial $P\in\mathbb R[X_1,\dots,X_n]$ of degree $m$, there are unique symmetric multilinear maps $M_k:\underbrace{\mathbb R^n\times\dots\times\mathbb R^n}_{k~\textrm{times}}\longrightarrow\mathbb R$ for $k=0,\dots,m$ such that $$P(X)=M_0+M_1(X)+\dots+M_m(\underbrace{X,\dots,X}_{m~\textrm{times}}),$$ where $X:=(X_1,\dots,X_n)$. And that's what we're really doing with integer powers of $x$ most of the time, isn't it?
If you want to have it more abstractly, you can use the symmetric $k$-th power $S^k(V)$ of $V$, which is similar to the tensor product (a quotient space of the tensor product, to be precise), with a universal property uniquely suited to describe the above maps: The symmetric power $S^k(V)$ is the unique (up to unique isomorphism) vector space for which there exists a symmetric multilinear map $\vee :V^k\longrightarrow S^k(V),~(v_1,\dots,v_k)\mapsto v_1\vee\dots\vee v_k$, called the symmetric product, with the property that every other symmetric multilinear map $m:V^k\longrightarrow W$ can be written uniquely as $m(v_1,\dots,v_k)=\tilde m(v_1\vee\dots\vee v_k)$, where $\tilde m:S^k(V)\longrightarrow W$ is linear. So you can abstractly describe "integer powers of $x$" in multiple dimensions as linear maps whose domain is the symmetric power $S^k(V)$.
To get slightly more down-to-earth, a description of how the symmetric power looks: It's essentially made up of linear combinations of elements of the form $v_1\vee\dots\vee v_k$, where $$v_1\vee\dots\vee v_i\vee\dots\vee v_j\vee\dots\vee v_k=v_1\vee\dots\vee v_j\vee\dots\vee v_i\vee\dots\vee v_k$$ and $$v_1\vee\dots (v_i+\lambda w_i)\vee\dots\vee v_k=v_1\vee\dots v_i\vee\dots\vee v_k+\lambda(v_1\vee\dots w_i\vee\dots\vee v_k)$$ meaning that the $\vee$ is symmetric and linear. Essentially like the tensor product, which is made up of linear combinations of elements of the form $v_1\otimes\dots\otimes v_k$, but $\otimes$ is only linear, not symmetric.