Since the given set, $C \subset \mathbb{N}$ is non empty, hence by well ordering principle there exists $\alpha \in C$ which is the lowest element in C. Also, since the set $C$ is finite, $\quad \exists \beta \in \mathbb{N}| \quad x\le\beta, \quad \forall x\in C $.
Algorithmic proof: You start from $\beta$ and check whether it belongs to $C$. If it does, then $\beta$ is the highest element in C, else, you go to $\beta -1$ and repeat the process. One only needs to continue this procedure till he or she reaches $\alpha$. If no element of $\mathbb{N}$ greater than $\alpha$ but less than or equal to $\beta$ belongs to $C$, then $\alpha$ will be the highest element of $C$ for we have already established that $\alpha \in C$.
Your informal algorithmic proof works, but establishing its correctness rigorously will require some form of induction on $\mathbb{N}$, and at that point you're probably better off using a more direct argument like the one suggested by @drhab in the comments.
You can also argue directly by induction on the size of the set. The result is clearly true for sets of size $n=1$. If the result is true for all sets of size $n$ and $C$ has size $n+1$ (where $n\ge 1$), then for any $\alpha\in C$, the largest element of $C$ is $\max(\alpha,\beta)$ where $\beta$ is the largest element of $C-\{\alpha\}$.