How to solve Bellman's optimal equation from the first principle

Question

How to solve the following set (finite) of equations

$$ v_*(s) = \max_{a\in A(s)} \sum_{s'} p(s'|s,a) [r(s,a,s') + \gamma v_*(s')]$$

$p$ and $r$ functions are given.

Answer 1

Since $$ \sum_{s'} p(s'|s,a) [r(s,a,s') + \gamma v_*(s')] $$ is a function of $a$ and $s$, we can denote it as $g(a,s)$ we can consider $$ \max_a g(a,s) $$ as a parametric optimization in $a$. If you assume that $s$ has discrete values only, it is possible to solve the maximization problem for each possible $s$ separately.