I would like to solve this problem:
Let $\Omega = \lbrace{ (x,y) \in \mathbb{R}^2: 0<x<y\rbrace }, f \in C_{c}(\Omega) $. Find the solution of $ \Delta u =f \text{ in } \Omega\\ u=0 \text{ on the boundary of } \Omega $.
First, I can't use the fundamental solution of the Laplace equation, because $\Omega \neq \mathbb{R}^{2}$. I've found the solution for the homogeneous problem with $f=0$ - it is (in polar coordinates) $u(r,\phi)=\frac{rsin(4\phi)}{-15}$. I tried to compute the convolution of f and u (similarly idea as in the Dirichlet problem), which exists since the support of f compact is. But when I try to compute the Laplacian of my candidate for the solution,I differentiate under the integral and get 0, from the Laplacian of u.
The another idea, I've tried, was with the Green Function - I was thinking about constructing $\phi_{x}$ using the fundamental solution $\Phi = Clog(|x|)$ (recall the upper plane problem) and reflection of $\Omega$ w.r.t. the line $y=x$. But it does not help me at all, since f is not twice continuously differentiable.
I would be grateful if someone could help me. PS. I so sorry about my English.
Your second approach, using Green's function, is promising (the first one doesn't make sense to me). Extend $f$ to $\mathbb{R}^2$ by odd reflection across boundaries: $f(y,x) = - f(x,y)$, $f(-x,y) = -f(x,y)$, and $f(x,-y) = -f(x,y)$. Then the convolution of this extended $f$ with the fundamental solution $\frac{1}{4\pi}\log(x^2+y^2)$ satisfies the PDE and has the same symmetries as $f$, which implies it being zero on $\partial\Omega$.
You don't need $f$ to be twice differentiable. The function $f$ geing Hölder continuous is enough to imply that $u$ is twice differentiable and $\Delta u=f$ holds in the classical sense. Even for nastier functions $f$, the convolution still solves the PDE in some weaker sense, as long as we can make sense out of the integral that defines the convolution.