A cone with a $90^\circ$ vertex angle can be parameterized by
$$(x, y, z) = (z \cos \theta, z \sin \theta, z).$$
The metric on the cone can then be found to be
$$dx^2 + dy^2 + dz^2 = 2 dz^2 + z^2 d \theta.$$
The only non-zero Christoffel symbols are
$$\Gamma^z_{\space \space \theta \theta} = -\frac{1}{z} \hspace{1 cm}\Gamma^\theta_{\space \space \theta z} = \frac{z}{2}$$
and the only non-zero independent component of the Riemann curvature tensor is
$$R_{z \theta z \theta} = 2 z^{-2} + 1.$$
It's possible I made a mistake in the above computations, but I believe they are correct.
This shows that a cone is not locally flat.
However, a cone in real life can be formed by curving a piece of paper. For example, to compute the surface area of a cone, you can consider the surface area of a pac-man shape laying flat on a table. When the top and bottom of the mouth of the pac-man are brought together, the paper is in the shape of a cone.
The Christoffel symbols and Riemann tensor contain no information on the embedding of a manifold in ambient space. Therefore, it would seem to me that the Riemann tensor would have to be $0$, as it would be for flat piece of paper. How can this be?
As Anthony Carapetis correctly points out, there was indeed a mistake in my calculation and the Riemann tensor of a cone is indeed $0$. I will say a few more words for those who stumble across this answer and want an extra lesson about cones and curvature.
I mistakenly thought that a cone cannot have $0$ Riemann curvature because if you take a vector tangent to the cone and parallel transport it $360^\circ$ around the cone, it will not come back in the same configuration as when you started. (Think about the cone in terms of the pac-man to convince yourself of this.) Because the cone has non-trivial parallel transport properties, I assumed it could not be flat.
Let's introduce a slope parameter $a$ to the cone. When $a = 1$, we have the cone from before. When $a = 0$ we have a flat plane.
$$(x, y, z) = (r \cos \theta, r \sin \theta, ar).$$
$$dx^2 + dy^2 + dz^2 = (1 + a^2) dr^2 + r^2 d \theta$$
$$\Gamma^r_{\space \space \theta \theta} = \frac{-r}{1 + a^2} \hspace{1 cm} \Gamma^\theta_{\space \space \theta r} = \frac{1}{r}$$
$$R_{r \theta r \theta} = 0$$
Let us now consider parallel transporting a vector $u^\mu = (u^r, u^\theta)$ around the cone. Our path will be
$$(r, \theta) = (A, t)$$
and the velocity vector around this path will therefore be
$$(v^r, v^\theta) = (0, 1).$$
From the parallel transport equation
$$\frac{d u^\nu}{dt} = - \Gamma^\nu_{\space \space \mu \sigma} u^\mu v^\sigma$$
we can easily work out the following two equations governing the evolution of $u^\mu$:
$$\frac{d^2 u^r}{dt^2} = -\frac{1}{1 + a^2} u^r$$ $$\frac{d^2 u^\theta}{dt^2} = -\frac{1}{1 + a^2} u^\theta$$
When $t = 2 \pi$ we will have completed a full loop around the cone.
We can see from the two equations above that when $a = 0$, the vector $u^\mu$ is returned to its original state. Otherwise, the vector will in general be different. (The solution for each equation is a linear combination of $\cos(t / \sqrt{1 + a^2} )$ and $\sin(t / \sqrt{1 + a^2} )$.)
I mistakenly thought that if the curvature was $0$ everywhere this cannot happen. Now I realize that this is only true for a simply connected smooth region. (The tip of the cone is the problem point.) Otherwise, the proof that zero curvature $\implies$ flat space does not work. My failure to realize this is why I was not more skeptical of my incorrect work. I hope this is interesting to others.