Given a group $G$ and field $k$ one can define the group algebra $k[G]$ in two ways:
The underlying vector space of $k[G]$ is the free $k$-vector space on $G$, and the multiplication on $k[G]$ is the unique bilinear extension $k[G] \times k[G] \to k[G]$ of the group multiplication $G \times G \to G$.
The elements of $k[G]$ are the functions $f \colon G \to k$ with finite support and the vector space operations are defined pointwise. The multiplication is given by the convolution product $$ (f_1 f_2)(g) := \sum_{h \in G} f_1(h) f_2(h^{-1} g) \qquad \text{for all $g \in G$}. $$
Since I first encountered the second definition I have always found it somewhat inferior to the first one. I assumed that it exists only to define $k[G]$ without requiring the notion of a free vector space, made possible by hiding the free vector space on a set $X$ behinds its technical construction as functions $X \to k$ with finite support.
But by now I have seen the second definition often enough to begin wondering if there is some benefit to taking it more seriously. While thinking about this, I began to wonder if there is some deeper connection between the two definition than I have given it credit for.
Question: What do we gain from thinking about the group algebra $k[G]$ as an algebra of functions on $G$, and how does this view point relate to the other definition?
(I realize that it could turn out be more appropriate to ask two separate questions instead. But I’m currently not sure how related the two are, and thus refrained from doing so.)
So far my thoughts on this have been the following:
According to the first definition, $G$ is a basis of $k[G]$. Functions $G \to k$ are then “the same” as elements of $k[G]^*$; the functions of finite support correspond to the image of the basis-dependent embedding $k[G] \hookrightarrow k[G]^*$, $g \mapsto \delta_g$. This seems to suggest some kind of duality between the two definitions, coming from the usual duality pairing of $k[G]$ and $k[G]^*$. (I know that for finite $G$ the Hopf algebra $k[G]$ is dual to the Hopf algebra of functions $G \to k$. But I don’t know if this is of use here.)
If $k[G]$ consists of functions on $G$, then I would strongly expect it to be contravariant in $G$. That this is not the case seem to be thanks to the convolution product. But I don’t really understand where the convolution product comes from when one doesn’t already have the first definition in mind.
The requirement of finite support is needed to assure that the sum $\sum_{h \in G} f_1(h) f_2(h^{-1} g)$ is well-defined even for infinite groups. It should be possible to lift this restriction if we have some notion of convergence for this sum. While this seems to be a thing, I don’t know if this sheds any light on a possible deeper relation between the two definitions in the algebraic setting.
Help is appreciated.
One way to understand why "functions (of some sort) on $G$" is a good notion is in terms of the representation theory of $G$. That is, if $G\times V\to V$ is a representation on a complex vector space $V$ (with some completeness properties if infinite-dimensional, and with continuity of $G\times V\to V$ also, in case $G$ is a non-finite (but locally compact, Hausdorff) topological group), the collection of (e.g.) continuous, compactly-supported complex-valued function $f$ on $G$ acts on $V$ in a natural way by "averaging": $$ f\cdot v \;=\; \int_G f(g)\;g\cdot v\;\;dg $$ with Haar measure (which could be counting measure in the discrete $G$ case) on $G$. Requiring associativity $f\cdot (\varphi\cdot v)=(f*\varphi)\cdot v$ for all $v$ determines the convolution $*$. This viewpoint is especially useful for non-compact $G$, where sometimes these integral operators can be compact, while a unitary action of $G$ on an infinite-dimensional Hilbert space is never by compact operators.