The monoid algebra $R[M]$, for a commutative ring $R$ and a monoid $M$, can be described as the free $R$-algebra on $M$. We think of $R[M]$ as the set of finite formal sums of elements of $M$ with coefficients in $R$, where we use $M$ for multiplication. In the most familiar case, $M$ is the free commutative monoid on a set $X$ (written multiplicatively), and $R[M]$ is the algebra of polynomials over $R$ in the indeterminates $X$. If $U$ is the forgetful functor from $R$-algebras to monoids which takes an algebra to its multiplicative monoid, $M \mapsto R[M]$ appears to be a left adjoint to $U$.
More generally, there is a concept of the category algebra or categorical algebra $R[C]$ for a category $C$. For example, choose $n \ge 1$ and consider the category $C$ whose objects are $\{0, 1, ..., n - 1\}$, with a single morphism from $i$ to $j$ for each pair $(i, j)$. An order-$n$ matrix over $R$ is a formal sum of these morphisms with coefficients in $R$. Regarding this matrix as a map from Mor$(C)$ to $R$, we can define the product of two matrices $\alpha$, $\beta$ by the convolution formula
$$(\alpha \beta)(m) = \sum_{m_0 m_1 = m} \alpha(m_0) \beta(m_1)$$
So $R[C]$ is the familiar matrix algebra. If instead we choose $C$ to be the totally ordered set with $n$ elements, $R[C]$ is the algebra of upper triangular matrices.
Question 1: What's the most general definition of a category algebra?
For example, suppose $U$ is the forgetful multiplicative-monoid functor from $R$-algebras to the category of all small categories. Is there a left adjoint to $U$ in this case?
We need some finiteness condition in order to define $R[C]$. If we require the formal sum to be finite, we will not have a multiplicative unit when $C$ is infinite. But if the formal sums can be infinite, the product of two elements may not be defined.
The matrix example above works because $C$ is finite. Fei Xu seems to be taking this approach in the paper A Mini Course on Category Algebras. This suggests that a suitable left adjoint can be defined in that case.
When $C$ is a partially ordered set, the category algebra is called an incidence algebra, and the usual requirement is that $C$ is locally finite: each morphism can be written as a product of two other morphisms in only a finite number of ways.
However, these approaches fail to include simple monoid algebras such as group algebras over an infinite group. My best guess is to define $R[C]$ as the set of formal sums $\alpha = \sum r_m m$ such that the morphisms $m$ involve only a finite number of objects. That is, for each $\alpha$, the set $\{m: r_m \neq 0\}$ of morphisms has a finite number of domain objects and a finite number of codomain objects. This does give us an algebra, but it's not clear to me that it has the required universal property.
Question 2: What other interesting algebras arise from this construction?
Presumably there are other matrix subalgebras. How can we describe those? And what about other small categories?
Maybe this article will be of interest for you:
Zbl 1224.16036 Khripchenko, N.S. Finitary incidence algebras of quasiorders. (English) Mat. Stud. 34, No. 1, 30-37 (2010).