A post for the rejected -- influential papers that had trouble getting published

Question

Having your paper rejected feels a lot like getting dumped. But while there are plenty of good ways to alleviate the pain of romantic rejection, there seem to be few outlets to alleviate intellectual rejection. Perhaps answers to this question can help:

What are examples of influential mathematicians with famous rejection stories? Or highly regarded mathematical papers that had particular trouble being published? Especially, can you provide excerpts from highly critical referee reports of such papers (names elided, of course)?

Answer 1

Makes me think of this comment from Hermann Grassmann's Wikipedia page:

Grassmann's mathematical ideas began to spread only towards the end of his life. 30 years after the publication of A1 the publisher wrote to Grassmann: “Your book Die Ausdehnungslehre has been out of print for some time. Since your work hardly sold at all, roughly 600 copies were used in 1864 as waste paper and the remaining few odd copies have now been sold out, with the exception of the one copy in our library”. Disappointed by the reception of his work in mathematical circles, Grassmann lost his contacts with mathematicians as well as his interest in geometry.

Also from the same page, this synopsis of the significance of his work:

The definition of a linear space (vector space)... became widely known around 1920, when Hermann Weyl and others published formal definitions. In fact, such a definition had been given thirty years previously by Peano, who was thoroughly acquainted with Grassmann's mathematical work. Grassmann did not put down a formal definition --- the language was not available --- but there is no doubt that he had the concept.

Beginning with a collection of 'units' e1, e2, e3, ..., he effectively defines the free linear space which they generate; that is to say, he considers formal linear combinations a1e1 + a2e2 + a3e3 + ... where the aj are real numbers, defines addition and multiplication by real numbers [in what is now the usual way] and formally proves the linear space properties for these operations. ... He then develops the theory of linear independence in a way which is astonishingly similar to the presentation one finds in modern linear algebra texts. He defines the notions of subspace, linear independence, span, dimension, join and meet of subspaces, and projections of elements onto subspaces.

...few have come closer than Hermann Grassmann to creating, single-handedly, a new subject.

Answer 2

Galois had a rather mixed history of publication. While the widespread narrative that his work was not understood and rejected by the great mathematicians of his time, then vindicated decades later is somewhat oversimplified, he did get rejected several times. Wikipedia says:

He submitted two papers on [the theory of polynomial equations] to the Academy of Sciences. Augustin Louis Cauchy refereed these papers, but refused to accept them for publication for reasons that still remain unclear. However, in spite of many claims to the contrary, it is widely held that Cauchy recognized the importance of Galois's work, and that he merely suggested combining the two papers into one in order to enter it in the competition for the Academy's Grand Prize in Mathematics.

[...]

He submitted his memoir on equation theory several times, but it was never published in his lifetime due to various events. As noted before, his first attempt was refused by Cauchy, but in February 1830 following Cauchy's suggestion he submitted it to the Academy's secretary Joseph Fourier, to be considered for the Grand Prix of the Academy. Unfortunately, Fourier died soon after, and the memoir was lost. The prize would be awarded that year to Niels Henrik Abel posthumously and also to Carl Gustav Jacob Jacobi. Despite the lost memoir, Galois published three papers that year, one of which laid the foundations for Galois theory. The second one was about the numerical resolution of equations (root finding in modern terminology). The third was an important one in number theory, in which the concept of a finite field was first articulated.

[...]

Siméon Poisson asked him to submit his work on the theory of equations, which he did on 17 January 1831. Around 4 July 1831, Poisson declared Galois's work "incomprehensible", declaring that "[Galois's] argument is neither sufficiently clear nor sufficiently developed to allow us to judge its rigor"; however, the rejection report ends on an encouraging note: "We would then suggest that the author should publish the whole of his work in order to form a definitive opinion." While Poisson's report was made before Galois's Bastille Day arrest, it took until October to reach Galois in prison. It is unsurprising, in the light of his character and situation at the time, that Galois reacted violently to the rejection letter, and decided to abandon publishing his papers through the Academy and instead publish them privately through his friend Auguste Chevalier. Apparently, however, Galois did not ignore Poisson's advice, as he began collecting all his mathematical manuscripts while still in prison, and continued polishing his ideas until his release on 29 April 1832.

[...]

Galois was so convinced of his impending death that he stayed up all night writing letters to his Republican friends and composing what would become his mathematical testament, the famous letter to Auguste Chevalier outlining his ideas, and three attached manuscripts. Mathematician Hermann Weyl said of this testament, "This letter, if judged by the novelty and profundity of ideas it contains, is perhaps the most substantial piece of writing in the whole literature of mankind." However, the legend of Galois pouring his mathematical thoughts onto paper the night before he died seems to have been exaggerated.