Can someone tell me if there is some natural algebraic structure given by an infinite (co)product that has no neutral element ( where a neutral element is one that witnesses $\exists u \forall a [u * a = a = a * u]$), but such that every "finite reduction" (somewhat intuitive, but I could try and come up with something formal if pressed) actually has a (not necessarily unique, not necessarily even 2-sided if you want) neutral element?
I am thinking something like this:
https://en.wikipedia.org/wiki/Rng_(algebra)#Example:_Quinary_sequences
but I cannot see why the infinite product does not have neutral element the (infinite) sequence of all $1$'s, and, furthermore, it seems a bit specific to $5$ arbitrarily, so I would like something more general. Also I don't know what it means when it talks about the supposed "identity/idempotent" element (it seems to switch between sentences) in the relevant "finite reductions" (again, my own terminology) is actually the neutral element (with respect to either multiplication or addition in their rng).
You're right that they chose that ring a bit arbitrarily. It has nothing to do with the property you're interested in.
If you take any infinite collection of nonzero rings (with identity of course) $\{R_i\mid i\in I\}$ then $R=\oplus_{i\in I} R_i$ is a rng without identity, but you can say that it has local identities. That is for every $x\in R$, there is an $e\in R$ such that $ex=xe$.
An identity for $R$ would have to be nonzero on every coordinate, and of course no element in that set has that property. So there's no identity.
To answer your first question after having said this, you might be thing of something like a "semigroup with local identities."