The title says it all. I often heard people say something like memory is unimportant in doing mathematics. However, when I tried to solve mathematical problems, I often used known theorems whose proofs I forgot.
EDIT Some of you may think that using theorems whose proofs one has forgotten does not seem to support importance of memory. My point is that it is not only useful, but often necessary to remember theorems(not their proofs) to solve mathematical problems. For example, you can't solve many problems of finite groups without using Sylow's theorem.
Your question reminded me of the following article (I happen to be very interested in research on math and cognition): Working Memory and Mathematics. It's a very long article, a review of the literature, citing a lot great references, and dated 2010.
You seem to be referring to "long term memory", though. So perhaps the article is not of any interest to you. Mathematical cognition (and thinking in general) involves many sorts of memory: working memory, long term memory, fluid memory, static memory, pattern recognition, etc., as well as the faculties of spatial ordering, temporal-sequential ordering, etc., each of which involves various parts of the human mind.
Here's a link to the abstract of a nice article that might be of interest Memory in Mathematical Understanding. The article discusses the correlation between learning, understanding, and memory. That is, memory can be a function of how well one learned and understood the material one is hoping to remember.
If it's of any consolation, I don't think that you are alone. (I can only speak for myself, here, in that I too wish I was able to recall many things, many times. That's where having access to good references and texts come to the rescue! It is better to know how to find what you need to know, than to simply rely on the hope that everything one has learned will surface when needed! Usually, one need only refresh one's memory, which takes only a fraction of the time spent having learned in in the first place.
On a light note:
I remember consoling myself a while back by visualizing "learning" as filling a suitcase(s), the contents of which is "what I know." The more you add to your luggage, (the more you learn and the more you know), the harder it is to locate any particular item amidst the cramped collection of what you've acquired!