My instructor, in his notes uses repeatedly an argument about cofinality I cannot understand.
Say we have an ordinal $\alpha$ and let $f:cof(\alpha)\rightarrow\alpha$ be the cofinal map witnessing that the cofinality of $\alpha$ is $cof(\alpha)$. Then say we want to prove that a map $g: cof(\alpha)\rightarrow\alpha$ (usualy defined in terms of $f$) is well defined. Then, in various proofs he argues like this: let $\alpha'<cof(\alpha)$ be the least ordinal such that $g(\alpha')$ is not defined, then $\sup_{\gamma<\alpha'} g(\gamma)=\alpha$ then $g:\alpha'\rightarrow \alpha$ would be cofinal against $\alpha'<cof(\alpha)$.
I really cannot understand the highlighted passages.


Your two examples switch the role of $f$ and $g$; for consistency, I'll use $h$ for the "original" function and $k$ for the "new" function.
I think the language used here is a bit non-ideal. In all these cases we have a totally valid definition by recursion of a function $k$ with domain $cof(\alpha)$ based on a function $h$; the issue is justifying the claim $ran(k)\subseteq\alpha$.
The key point is the way $k$ is defined in each case: as a sup of previous values and things below $\alpha$ (the $h(\beta)$s). This means it can't shoot above $\alpha$ without first reaching $\alpha$, and if that were to happen we'd have "climbed up $\alpha$ in less than $cof(\alpha)$ many steps."
More precisely:
Basically, the points are:
Throwing the values of $h$ into the mix never pushes us past $\alpha$.
Using sups at each stage can't push us past $\alpha$ suddenly; we have to wind up at $\alpha$ at some point.
At such a point we have to have used ordinals arbitrarily high below $\alpha$ itself, but then that gives us a cofinal map from a too-small ordinal.