I'm having a hard time proving the direct implication. This is my progress so far:
We prove the converse. Assume that for all $b\in B$, either $b\in F$ or $\neg b\in F$. Let $F'$ be a filter that properly contains $F$. By assumption, there exists $c\in B$ such that $c,\neg c\in F'$. Therefore, $0=c\land \neg c\in F'$. But $0\leq a$ for all $a\in B$. Hence $a\in F'$, implying that $F'=B$. This argument can be used to prove (a part of) the direct implication: if $F$ is an ultrafilter and $b,\neg b \in F$, then $F=B$, which is a contradiction.
But as you can see here, there is no guarantee here that either $b\in F$ or $\neg b\in F$. I tried to assume that both are not in $F$, and then construct a proper filter $U$ that is "generated" by $F$ and $b$, and then show that $\neg b\not\in F$, but I have no idea how to do it.
Suppose $F$ obeys
$$\forall b \in B: b \in F \text{ or } \lnot b \in F\tag{1}$$
Now suppose $F' \supsetneq F$ is a filter too. Let $c \in F'$ so that $c \notin F$ witness the proper inclusion. Then by $(1)$ applied to $c$ we have that $c \notin F$ hence $\lnot c \in F$ and by the inclusion $\lnot c \in F'$ too, which is a contradiction as then $c \land \lnot c = 0 \in F'$ too. So we cannot have a properly larger filter than $F$ and $(1)$ implies maximality of $F$.
OTOH if $F$ is maximal, assume there is some $c \in B$ such that $c \notin F$ and $\lnot c \notin F$ too. Then define $$F' = \{x \in B\mid \exists f_x \in F: f_x \land c \le x\}$$
Then this is a filter:
Use $f_x = x$ to see that $F \subseteq F'$ and the inclusion is proper as $c \in F'$ is witnessed by taking $f_x=1 \in F$ (using $F \neq \emptyset$).
Then $F'$ contradicts the maximality of $F$. So no such $c$ can exist and $F$ obeys $(1)$.