I am trying to figure out what the empirical distribution function is.
Wiki uses this figure to illustrate the empirical distribution function.
The grey hash marks represent the observations in a particular sample drawn from that distribution, and the horizontal steps of the blue step function (including the leftmost point in each step but not including the rightmost point) form the empirical distribution function of that sample.
What does the vertical line between each horizontal lines mean?
The vertical blue lines are just lines connecting the horizontal blue lines. They don't mean anything.
As the text explains, the blue graph is supposed to represent a step function, whose actual graph consists only of horizontal segments (which in this function are closed on the left and open on the right). So the horizontal segments are the only parts of the graph that actually represent the function.