Let $X$ be a topological space and consider a sheaf $\mathcal{O}$ such that for each open set $U$ in $X$ we get a ring of functions $\mathcal{O}(U)$. Denote the set of germs at $p$ as $\mathcal{O}_p$. Show that the ideal $\mathfrak{m}_p$ of $\mathcal{O}_p$ consisting of functions that vanish at $p$ is maximal and that it's the only maximal ideal.
To show that $\mathfrak{m}_p$ is maximal we need to show that $\mathcal{O}_p/\mathfrak{m}_p$ is a field. Now I found that $$0 \longrightarrow \mathfrak{m}_p \longrightarrow \mathcal{O}_p \longrightarrow \mathbb{R} \longrightarrow 0$$ is exact, but I don't understand the map $\mathcal{O}_p \longrightarrow \mathbb{R}$. The first map is the inclusion, but the elements of $\mathcal{O}_p$ are sets of the form $$\{ (f,U) \mid p \in U, f \in \mathcal{O}(U) \}.$$
so how does a map from this kind of a set to real numbers look like and how will this result in the sequence being exact?
Lastly to show that $\mathfrak{m}_p$ is the only maximal ideal I was given a hint that I need to show that every element of $\mathcal{O}_p/\mathfrak{m}_p$ is invertible, but how does this imply that there can be no other maximal ideals?
The map $\mathcal{O}_p \to \mathbb{R}$ takes $(f, U)$ to $f(p)$. Unfortunately, your definition of $\mathcal{O}_p$ is slightly wrong -- you need to consider equivalence classes of pairs $(f, U)$ where two such pairs $(f_1, U_1)$ and $(f_2, U_2)$ are equivalent if there is some small neighborhood $U$ of $p$ contained in both $U_1$ and $U_2$ on which $f_1|_U = f_2|_U$. So you also have to check that the stated map is well-defined mod this equivalence relation.
For your last question, your hint should be to show that elements in $\mathcal{O}_p \setminus \mathfrak{m}_p$ are invertible (set difference, not quotient -- it's immediate that nonzero elements in the quotients are invertible since you've already shown the quotient is a field, namely $\mathbb{R}$!) Once you show this, you'll know that $\mathfrak{m}_p$ is the only maximal ideal (for any other element, any ideal containing it is the whole ring).