Let $\varphi:[0,\infty)\to [0,\infty]$ be increasing with $\varphi(0)=\lim_{t\to 0^{+}}\varphi(t)=0$ and $\lim_{t\to \infty}\varphi(t)=\infty$.
$\textbf{Proposition}$: As a convex function, $\varphi$ is continuous in $[0,\infty)$ if and only if $\varphi$ is finite on $[0,\infty)$.
The function $\varphi(t)= \begin{cases} 0, & 0\leq t\le 1/2 \\ \frac{2t-1}{1-t}, & 1/2< t< 1 \\ \infty, & t\geq 1 \end{cases} $ is convex and continuous but not finite on $[0,\infty)$.
Where is my mistake?
This is more of a comment than an answer, but its too long.
I think this is hard to evaluate without knowing the exact definitions that are being used for convexity and continuity: In my experience these are much more commonly discussed for functions taking finite values. It seems there are reasonable definitions of continuity and convexity for which you are correct. However, I think you lose some important aspects of convexity when you do this.
One can define a topology on $[0,\infty]$ by taking a basic of opens set of the form $(a,b)$ and $(b,\infty]$ where $0<a<b<\infty$. In this topology your function is continuous. You can then take the usual definition of convex function: $f$ is convex if $f(tx+(1-t)y) \leq tf(x)+(1-t)f(y)$. With this set up you are correct.
However, another typical definition of convexity for functions is that the epigraph of the function is convex. The epigraph of $f$ denote $epi(f)$ is the set $$ epi(f)=\{(x,y) \in \mathbb{R}^2 : y \geq f(x)\}. $$ Now the problem is, in our topology $epi(f)$ is a compact convex set. However, $epi(f)$ is NOT the convex hull of its extreme points.
You also lose existence of unique optimizers for many functions on compact sets in this topology. For example, the linear functional $y$ achieves a maximum on $epi(f)$ at every point with an infinite $y$ coordinate.
The point is, you should expect to lose things when going to the extend real numbers and you likely cannot get a totally satisfying notion of convexity here. Uniqueness optimizers and spanning properties of extreme points are some of the most interesting things about convex sets, so you have payed a big price here.