A pseudo-Riemannian manifold $(M,g)$ is a generalisation of the concept of a Riemannian manifold where we relax positive-definiteness to non-degeneracy.
$\alpha$) Non-degeneracy is still enough to give us the musical isomorphisms. I wonder if the implication goes the other way: Are pseudo-Riemannian manifolds precisely the ones for which the metric $g$ gives musical isomorphisms?
$\beta$) I also wonder if we take the stronger assumption that, in addition to be being non-degenerate pointwise, for any two vector field $X \in \chi(M)$, the function $f == g(X,X)$ is non-negative, that is $f \neq 0$ and $f(m) \geq 0$ for all $m \in M$.
$\gamma$) Of course the $\beta$-assumption will exclude Lorentzian manifolds, begging the question "Are there any interesting (non-Riemannian) examples of such manifolds"?
Your questions depend a lot on what your assumptions are. Let's start with rather minimal ones.
The classical definitions of pseudo-Riemannian manifold would require $g$ to be symmetric and non-degenerate, with the case $g$ is positive definite being the Riemannian case.
Not necessarily, since this mapping need not be symmetric (think a rotation matrix).
If you assume that $g$ is symmetric, then the answer is yes, since the contrapositive is just "if $g$ is symmetric but not pseudo-Riemannian, then the musical operation is not an isomorphism". But the definition of $g$ being degenerate is precisely equivalent to there existing some $X$ such that $g(X,Y) = 0$ for all $Y$, or that $g(X,\cdot)$ is the 0 covector, or that $X$ belongs to the kernel of the musical operation.
If $g$ is not required to be symmetric, not much can be said, as conditions on $g(X,X)$ is completely not seen by the antisymmetric part of $g$.
If $g$ is required to be symmetric, then notice that by the polarization identity you can write $$ g(X,Y) = \frac12 \left[ g(X+Y,X+Y) - g(X,X) - g(Y,Y) \right] $$ thereby recovering $g$ entirely from its diagonal part, so knowledge of the quadratic form $X\mapsto g(X,X)$ is enough to recover the whole bilinear form $g$.
Putting that aside: Sylvester's law of inertia can be applied. Every symmetric bilinear form has a basis in which it is diagonal. Your assumptions proved that the coefficients cannot be negative. Non-degeneracy tells you that the coefficients cannot be zero. Hence the form must by positive definite.
If $g$ is assumed to be symmetric, the argument above shows that no such examples exist.
If $g$ is not assumed to be symmetric, then any Riemannian metric $\mathring{g}$ augmented with an arbitrary antisymmetric part will do the job.