Vector form of a polynomial in multiple variables.

Question

I've been coming across various polynomial functions in my machine learning course; and I'm trying to understand what the most general form of these would look like.

We started with a linear equation: $f(x) = ax + b$; which is obviously nice and simple.

We can generalise that to a linear function of $n$ input variables,

$f(x_1, x_2, \dots, x_n) = a_1x_1 + a_2x_2 + \dots + a_nx_n +b$,

and we noted that this has has a nice representation in matrix/vector form, since if we take $\mathbf x = [x_1, \dots, x_n]^T$ and $\mathbf a = [a_1, a_2, \dots, a_n]^T$ then $f(\mathbf x) = \mathbf a^T\mathbf x$ (i.e. the dot product of the two vectors).

If we wanted to have a quadratic polynomial function of $n$ input variables, analogous to $g(x) = ax^2 + bx + c$ then we can write:

$g(\mathbf x) = \sum_{i = 1}^n\sum_{j = 1}^n a_{ij}x_ix_j + \sum_{i = 1}^n b_i x_i + c$

And again this has a nice matrix/vector representation (I think):

$g(\mathbf x) = \mathbf x^T A \mathbf x + \mathbf b^T \mathbf x + c$, where $\mathbf A = [a_{ij}]$ and $\mathbf b = [b_i]$.

This has led to me wonder about the most general form of this: a $K$ polynomial function of $n$ input variables, which I'm struggling to write down in a neat way. I guess it is something like:

$h(x) = \sum_{k=1}^{K} (\sum_{i_1}^n \sum_{i_2}^n \dots \sum_{i_k}^n a_{i_1, \dots, i_k}x_1\dots x_k)$

but that doesn't seem like the most efficient notation.

So would anyone be able to:

1. Confirm this is the right idea for the general form.
2. Give a better way to notate it, if there is one.
3. Potentially give a way of writing it in terms of vectors and matrices as above (Google suggests maybe I can use "Kronecker products" but I'm not entirely sure what that means or how to do that)
4. Or provide some keywords I can use to help Google this kind of issue: searching things like "general form of a polynomial in $n$ variables"; "matrix form of polynomial"; "general polynomial regression model" aren't turning up what I'm looking for.

Many thanks!

Answer 1

The most compact and simple way for doing this is to consider the multi-index notation where $m$-variate polynomial $p(x)$ of degree $n$ can be written as

$$p(x)=\sum_{|\alpha|\le n}p_\alpha x^\alpha$$

where $\alpha\in\mathbb{N}^m$, $p_\alpha\in\mathbb{R}$, $|\alpha|=\alpha_1+\ldots+\alpha_m$, and $x^\alpha=x_1^{\alpha_1}\ldots x_m^{\alpha_m}$.

This is basically a tensor notation where $P=[p_\alpha]_{|\alpha|\le n}$ is a tensor of order $m$.

More information here: https://en.wikipedia.org/wiki/Multi-index_notation.