I am looking for an example (in ZFC) of a Boolean algebra $A$ such that no ultrafilter on $A$ is $\sigma$-complete. Since every principal ultrafilter is complete, there has to be no principal ultrafilter on $A$, so $A$ has to be atomless.
Alternatively, can one show (in ZFC) that every Boolean algebra has a $\sigma$-complete ultrafilter?
Well, the Boolean algebra with one element has no ultrafilters at all.
Less trivially, a simple counterexample is any countable atomless Boolean algebra $B$ (in fact, up to isomorphism, there is only one nontrivial such algebra, the clopen algebra of the Cantor set). If $U$ is any ultrafilter in $B$, note that $\bigwedge U=0$ (if $u\in B$ were any nonzero lower bound of $U$, then it would have to be an atom generating $U$, but $B$ is atomless). Since $U$ is countable, this implies it is not $\sigma$-complete.
[As I commented, there are multiple definitions of what "$\sigma$-complete ultrafilter" might mean in a Boolean algebra that is not itself $\sigma$-complete, but this example will fail all possible definitions, since the meet $\bigwedge U$ does exist in $B$.]
If you want a nontrivial example of a $\sigma$-algebra (or even a complete Boolean algebra) with no $\sigma$-complete ultrafilter, you can take the completion $C$ of a countable atomless Boolean algebra $B$ (explicitly, $C$ would be the regular open algebra of the Cantor set). If $U$ is an ultrafilter in $C$, then $U\cap B$ is an ultrafilter in $B$. Then as above, $\bigwedge U\cap B=0$ in $B$. But the inclusion $B\to C$ preserves all joins and meets that exist in $B$, so $\bigwedge U\cap B=0$ in $C$ as well. Since $U\cap B$ is a countable subset of $U$, this shows $U$ is not $\sigma$-complete.