So, I'm considering a PDE and trying to find its Green's function first. To this end, I solve the following helmholtz equation:
$$\frac{d^2g}{dx^2}+\frac{d^2g}{dy^2}+\frac{d^2g}{dz^2}-\alpha^2g=\delta(x-\xi)\delta{(y-\eta)}\delta{(z-\rho)}$$
Well, I can solve this PDE for $g$ , but what happens to the solution when $x=\xi$,$y=\eta$, etc? In that case, would the Green's function fails to solve the PDE? So, it fails to solve the homogeneous case at the "separation point"? Just trying to understand what happens at this odd location.
Yes, the Green's function (a.k.a. "fundamental solution") does not solve the homogeneous equation "at" the separation point.
There are at least two ways to think about this. One is to approximate (weakly) the Dirac deltas by spikes that are nevertheless smooth pointwise-valued functions. For elliptic operators and such, solving such an equation provably approximates the genuine solution. This heuristic has certain limitations, but is the obvious starting point.
A philosophically slightly different attitude is to let go a little of the "pointwise" notion of "function", to allow thinking of distributions (such as Dirac delta) as "generalized functions", despite not necessarily having pointwise values everywhere. That is, instead of using the lack of pointwise sense at "the separation point" to "prove" that Dirac delta "is not a function", interpret that difficulty as an argument in favor of relaxing the notion of function to not-necessarily require pointwise values, etc.