Prof. Daphne Koller talks a lot about factors in her PGM course: youtube video
She defines product & marginalization operations for them. However this factor entity is either not widespread or called differently in other domains. At least the wikipedia page for "Factor" doesn't seem to have a page that corresponds to Koller's use of the term.
So I'm interested in:
- How else is this construction called? What are best analogies?
- What are other operations defined for them? Is there a theory that studies factors?
- Are there any smart algorithms for doing these operations?
The use of the term 'factor' in the context of PGM is computer science lingo and is practically never used in mathematics circles.
The factors you are talking about are just functions. In more detail, a factor $F$ of scope $X_1,\cdots, X_n$ (where the $X_i$ are sets) is a function $F:X_1\times \cdots X_n \to \mathbb R$. So, a different name for factors is: a real valued function on the cartesian product of sets.
Since this is such a general concept you can define a million different and crazy operations on factors. In the context of PGM there are some natural operations to consider since most factors encountered are either probability distributions, or unnormalized such, or somehow related to such.
In particular, the common operations performed directly on factors are, as you mention, product factors and marginalizations (and also nomrlizations etc.). The most straightforward algorithms for these operations are the most efficient ones and are very fast (often $O(n^2)$). If the factors are known to have certain properties (e.g., sparsity) then faster algorithms can be developed.