Let $\omega:G\rightarrow \Lambda^1 T^*G$ be a left invarinat $1$-form on $G$ i.e., $(L_g)^*\omega=\omega$ for each $g\in G$.
Is it true that $\omega(Y):G\rightarrow \mathbb{R}$ is constant for any vector field $Y:G\rightarrow TG$ of $G$?
I see that if $Y$ is of the form $A^*$ for some $A\in \mathfrak{g}$ then $\omega(Y)$ is the constant function $A$.
Given $A\in \mathfrak{g}$, we have $A^*:G\rightarrow TG$ is defined as $A^*(g)=(L_g)_{*,e}(A)$.
We have $\omega(Y):G\rightarrow\mathbb{R}$ given by $g\mapsto \omega(g)(Y(g))$.
As $\omega$ is left invariant, we have $\omega(g)(v)=\omega(e)((L_{g^{-1}})_{*,g}(v))$.
For $Y=A^*$, we have $$\omega(g)(Y(g))=\omega(g)(A^*(g)) =\omega(e)((L_{g^{-1}})_{*,g}((L_g)_{*,e}(A)))=\omega(e)(A).$$ Thus, $\omega(Y):G\rightarrow \mathbb{R}$ is the constant function $\omega(e)(A)$ when $Y=A^*$.
Is it true for an arbitrary vector field $Y$ on $G$ that $\omega(Y)$ is constant for a left invariant $1$-form $\omega$ on $G$?
We have $d\omega(X,Y)=\frac{1}{2}(X(\omega(Y))-Y(\omega(X))-\omega[X,Y])$.
As mentioned above, $\omega(A^*)$ is constant, so, $B^*(\omega(A^*))$ is zero map and for similar reason $A^*(\omega(B^*))$ is zero map for any $A,B\in \mathfrak{g}$. So, $d\omega(A^*,B^*)=-\frac{1}{2}\omega[A^*,B^*]$.
In case $\omega(Y)$ is constant for any vector field $Y$ on $G$ we will then have $d\omega(X,Y)=-\frac{1}{2}\omega[X,Y]$.
I do not think this is true but wikipedia article says it is true for any vector fields $X,Y$ on $G$ not necessarily left invariant vector fields i.e., of the form $A^*,B^*$ for some $A,B\in \mathfrak{g}$.
EDIT : Wikipedia section says the following :
In particular, if $X$ and $Y$ are left-invariant, then $X(\omega (Y))=Y(\omega (X))=0,$ so $$d\omega (X,Y)+\frac{1}{2}[\omega (X),\omega (Y)]=0$$ but the left-invariant fields span the tangent space at any point (the push-forward of a basis in $T_eG$ under a diffeomorphism is still a basis), so the equation is true for any pair of vector fields $X$ and $Y$. This is known as the Maurer–Cartan equation.
No. Let $G=\mathbb{R}$ with coordinate $t$. The 1-form $\omega=\mathrm{d}t$ is left invariant. Now choose any non-constant vector field, e.g. $Y=t\frac{\partial}{\partial t}$. Then $\omega(Y)=t$.
Of course, as you say, evaluating a left-invariant 1-form on a left-invariant vector field does result in a constant function.
EDIT: To conclude the Maurer-Cartan equation just take arbitrary vector fields $X,Y$ and express them as $X=f_iX^i,Y=g_iX^i$ for some left-invariant vector fields $X^i$. Then notice that the equation is (bi)linear over functions so the case of left-invariant vector fields implies the general case.