I'm having trouble understanding how to specify a transverse wave (including it's longitudinal axis and transverse direction) in 3d space.
I know this is called a "plane wave", and I know the formula for a plane wave along a unit vector $\hat{k}$ is:
$$ A( \vec{r}, t ) = A_0 cos( \vec{k} \cdot \vec{r} - \omega t + \phi ) $$
Which is from here
Where $\vec{k}$ is the longitudinal axis and $\vec{r}$ is the space position. The output of this function should be the magnitude of the wave at that point in space.
But I don't understand where you specify the transverse axis. From this picture it appears that the wave described must be a longitudinal wave with only one axis, modelling something like compressions and rarefactions in air.
What is the transverse axis here? How do you specify a transverse axis?
I want a single wave that looks like just either the red one or the blue one from here:
I can't see where the transverse axis comes in or how you specify it's direction.
Longitudinal waves are along the direction of propagation, e.g., along the $y$-direction in your second figure.
Transverse waves are not longitudinal---the displacement is perpendicular to the direction of propagation, e.g., in the $x$- or $z$-direction in your figure. These two types of transverse waves are in independent states of polarization. They can be combined to form linearly, circularly, or elliptically polarized waves.
A transverse wave in the $x$-$y$ plane (blue in the figure) is described by $A_x \cos(k y - \omega t) \hat x$. A general transverse wave propagating in the $y$-direction would look like $$A_x \cos(k y - \omega t)\hat x + A_z \cos(k y - \omega t + \phi)\hat z,$$ where $A_x,A_z \ge0$ and $-\pi < \phi \le \pi$.
If $\phi = 0$ or $\pi$ the wave is said to be linearly polarized.
If $A_x = A_z$ and $\phi = \pm \pi/2$ the wave is circularly polarized. (The circular polarization is left- or right-handed depending on the sign of $\phi$ and your convention for handedness.)
Otherwise the wave is elliptically polarized.
Addendum: To describe a transverse wave propagating with wave vector ${\bf k}$ we must first find a unit vector orthogonal to ${\bf k}$, call it $\hat n$. This is one of the transverse directions. The other can be chosen to be $\hat k\times \hat n$. (It's a good exercise to show this is a unit vector orthogonal to $\hat k$ and $\hat n$.) Then a general transverse wave will be represented by $$A \cos({\bf k}\cdot {\bf r} - \omega t)\hat n + B \cos({\bf k}\cdot {\bf r} - \omega t + \phi)\hat k\times \hat n,$$ where $A,B \ge0$ and $-\pi < \phi \le \pi$. Replace $A_x,A_y$ with $A,B$ above to determine the type of polarization.
For example, if we want a linearly polarized wave propagating along the $\frac{1}{\sqrt{2}}(1,0,1)$ direction with polarization vector $\hat y = (0,1,0)$ we will find the wave described by $$A \cos\left(\frac{1}{\sqrt{2}}k(x+z) - \omega t\right) \hat y.$$ (Note that ${\bf k}\cdot {\bf r} = \frac{1}{\sqrt{2}}k(x+z)$.)