In the infinite dimensional setting, we know the unit ball is not compact so in general the $\sup_{\|x\|\leq 1}$ can not be replaced by $\max_{\|x\|\leq 1}$.
However, when the Banach space $X$ is reflexive, we could replace $\sup$ by $\max$. Due to Hahn-Banach theorem, for each $f\in X^*$, we know there exists an $u^{**} \in X^{**}$ with norm one such that $$\sup_{\|x\|\leq 1} f(x) = \|f\|_* =u^{**}(f)$$ and since $X$ is reflexive, we know there exists an $u \in X$ with norm one such that $$\sup_{\|x\|\leq 1}f(x) = f(u).$$
Now I want to give a counter example when the space is non-reflexive. Let us take $l^1$ and its dual $l^\infty$, take $$f=(0.9, 0.99, 0.999, \cdots ) \in l^\infty$$ then $\sup_{\|x\|\leq 1}f(x) = 1$ but there is no element in the unit ball would give $f(x) = \sum_i f_i x_i = 1$.
Is this correct? And I think if we could replace $\sup$ by $\max$ in the dual norm, can we conclude that the space has to be reflexive.
Looks good to me. The converse that you are suggesting is also true, according to Brezis' Functional Analysis (Chapter 1 Section 1):
In general, the “sup” in (5) is not achieved; see, e.g., Exercise 1.3. However, the “sup” in (5) is achieved if $E$ is a reflexive Banach space (see Chapter 3); a deep result due to R. C. James asserts the converse: if $E$ is a Banach space such that for every $f \in E^\ast$ the sup in (5) is achieved, then $E$ is reflexive; see, e.g., J. Diestel [1, Chapter 1] or R. Holmes [1].
The references from his appendix are:
Geometry of Banach spaces: Selected Topics, Springer, 1975.
Geometric Functional Analysis and Its Applications, Springer, 1975.