I am creating an encryption application. when I use values of a & b as 2 & 3 respectively. My message get encrypted successfully, but while decrypting it does not work.
Is there any formula using that I can use any values for a & b, and my text get encrypt or decrypt easily.
encryption
E(x) = (ax+b) mod 26
decryption
D(x) = a^-1[(x-b) mob 26]
Your cipher is the affine cipher. So you encryption function is, as you have written $$ E(x) = (ax + b) \text { mod }26. $$ What this is, is a function $$ E: \mathbb{Z} / 26\mathbb{Z} \longrightarrow \mathbb{Z} / 26\mathbb{Z}. $$ And decryption corresponds to a way $D:\mathbb{Z} / 26\mathbb{Z} \longrightarrow \mathbb{Z} / 26\mathbb{Z}$ such that $D\circ E(x) = D(E(x)) = x$ (mod $m$) for all $x$. So $D$ is an inverse to $E$ and for such a $D$ to exist you need $E$ to be invertible. And $E$ is invertible exactly when $a$ has an inverse in $\mathbb{Z} / 26\mathbb{Z}$. And this is true exactly whenthe greatest common division of $a$ and $26$ is $\gcd(a,26) = 1$. So you can't just use any value of $a$ That is $a$ must be one of the numbers $1, 3, 5, 7, 9, 11, 15, 17, 19, 21, 23, 25$. And then $D(x) = a^{-1}(x - b)$ $(\text{mod } m)$.