In Euclidean geometry, there is exactly one line through a point that is perpendicular to a line that does not go through the point. In spherical geometry, how do perpendicular lines exist? I can see someone calling "latitude" and "longitude" lines perpendicular. This would mean the answer is infinite as all the lines meet at the equator. Is this true?
*Edit: Is this statement correct? 
Perpendicularity is the same in any geometry where you can measure angles.
Being perpendicular means meeting at a 90° angle.
In most geometries (spherical, Euclidean, hyperbolic, and others), lines have perpendiculars passing through them at every point along the line.
Yes, the statement in your question is correct.
Think of the sphere as centered at the origin of 3-dimensional Euclidean space.
Lines on the sphere correspond to planes through the origin. (The line is where the plane intersects the sphere.)
Two lines are perpendicular iff the two planes are perpendicular.
Given a spherical line $L$, we can consider the corresponding plane through the origin, and we can see that there is one Euclidean line through the origin perpendicular to that plane. This 3D Euclidean line intersects the sphere in two points. These two points are called the poles of the spherical line $L$.
In spherical geometry, every line through a pole of $L$ is perpendicular to $L$. (This is your "statement".)
And in fact, every line that is perpendicular to $L$ passes through both poles of $L$.
But for any point $P$ that is not a pole of $L$, there is exactly one line through $P$ that is perpendicular to $L$.
This is similar to observing that in 3D Euclidean geometry, if Euclidean line $P$ is perpendicular to Euclidean plane $L$, then every plane through $P$ is perpendicular to $L$. But if line $P$ is not perpendicular to plane $L$, then there is exactly one plane through $P$ that is perpendicular to plane $L$.