I have this question from reading QFT textbooks.
Consider harmonic oscillator as a simpler model for differential operators. The Lagrangian is given by
$$L=\frac{1}{2}\dot{x}^{2}-\frac{1}{2}\omega^{2}x^{2}.$$
After integration by parts, one can express the classical action as
$$S[x;j]=\int_{-\infty}^{\infty}dt\left[\frac{1}{2}x(t)\left(-\frac{d^{2}}{dt^{2}}-\omega^{2}\right)x(t)+x(t)j(t)\right],$$
where $j(t)$ is an external source.
One can define the following second order differential operator
$$L(t,s)\equiv\left(\frac{d^{2}}{dt^{2}}+\omega^{2}\right)\delta(t-s),$$
where $\delta(t-s)$ is the Dirac delta function. Using this notation, one can write the classical action in a compact form
$$S[x;j]=-\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}dtds\frac{1}{2}x(t)L(t,s)x(s)+\int_{-\infty}^{\infty}dtx(t)j(t)\equiv-\frac{1}{2}\langle x,Lx\rangle+\langle j,x\rangle.$$
In QFT, one defines the Greens's function of the differential operator $L$ as its "right inverse", i.e.
$$\left(\frac{d^{2}}{dt^{2}}+\omega^{2}\right)G(t-s)=\delta(t-s),$$
where $\delta(t-s)$ is treated as the "continuous generalization" of the Kronecker delta symbol. In our compact notation, this is just
$$LG=.$$
Then, it is well-known that the equation of motion
$$\ddot{x}(t)+\omega^{2}x(t)=j(t) \tag{1}$$
has a unique solution
$$x(t)=x_{0}(t)+\int_{-\infty}^{\infty}dsG(t-s)j(s), \tag{$\star$}$$
where $x_{0}$ is the unique solution of the homogeneous ODE
$$\ddot{x}(t)+\omega^{2}x(t)=0. \tag{2}$$
If we use the compact notation introduced above, the solution ($\star$) is very easy to understand:
$$Lx=Lx_{0}+LGj=LGj=j=j,$$
which is precisely equation (1). However, there is a problem. In linear algebra, if an operator has an inverse, then it cannot have zero-eigenmodes. But from equation (2), it is clear that $x_{0}$ is a zero mode of the linear operator $L$. Then, we cannot define its Green's function $G(t-s)$, which is absurd.
The same problem arises when we are dealing with QFT in higher dimensions, and $L$ becomes a second order partial differential operator.
What's going on? Where did I make mistakes?
The identity $LG={\bf 1}$ implies that $L$ is surjective and $G$ is injective, but not necessarily the other way around. It might be OK for $L$ to have zero-modes.
Note that for OP's action functional $S$ to govern a well-posed variational problem, one should impose appropriate boundary conditions. This choice in turn uniquely specifies a Greens function $G$ for the problem.