When solving
$ U_{xx}+U_{yy}=0$ with $u(0,y)=u(a,y)=u(x,b)=0,u(x,0)=f(x)$.
$0<=x<=a$ , $0<=y<=b$ by the method of separation of variables I have
$-X''(x)-\lambda X(x)=0 $
$Y''(y)-\lambda Y(y)=0$
In $\lambda>0$ case letting $\lambda=\omega^2$ I have come up to
$Y(y)=C e^{\omega y}+D e^{ -\omega y}$.
Is there a way I can write this in the form
$Y(y)=A_n cosh(\omega y)+B_n sinh(\omega y)$ .
After this using the boundary condition $u(x,b)=0$, I get $Y(b)=0$.
Then is there a way to write $Y(y)=A_n cosh(\omega (y-b))+B_n sinh(\omega (y-b))$
Yes, there is a way but only when $\lambda < 0$, then $Y$ can be written in form:
$Y(y)=C_1 e^{i\sqrt{-\lambda}x}+D_1 e^{-i\sqrt{-\lambda}x}$, but on the other hand:
$e^{i\sqrt{-\lambda}}=\cos{\sqrt{-\lambda}x}+i\sin{\sqrt{-\lambda}x}$ and
$e^{-i\sqrt{-\lambda}}=\cos{-\sqrt{-\lambda}x}+i\sin{-\sqrt{-\lambda}x}=\cos{\sqrt{-\lambda}x}-i\sin{\sqrt{-\lambda}x}$, so when we substitute to equation $Y(y)=C_1 e^{i\sqrt{-\lambda}x}+D_1 e^{-i\sqrt{-\lambda}x}$ we have (with other constans $C,D$)
$Y(x)=C\cos{-\sqrt{-\lambda}x}+D\sin{\sqrt{-\lambda}x}$
If you don't want to use complex numbers you can simply check that for $\lambda<0$ solution is like above ($\sin$ and $\cos$ are linear independent so it's general solution of this equation).
If you want to swap $e^{\omega x}$ and $e^{-\omega x}$ to $\sinh{\omega x}$ and $\cosh{\omega x}$ use this:
$\sinh(z)=\frac{e^z-e^{-z}}{2}$
$\cosh(z)=\frac{e^z+e^{-z}}{2}$
$Y(x)=Ce^{\omega x}+De^{\omega x}=\frac{C-D+C+D}{2}e^{\omega x}+\frac{C+D-(C-D)}{2}e^{-\omega x}$
if we substitute $A=C+D$ and $B=C-D$ we have
$\frac{A+B}{2}e^{\omega x}+\frac{B-A}{2}e^{-\omega x}=A\sinh \omega x+B\cosh \omega x$