PROBLEM.Let $ a,b,c > 0$ such that $a+b+c=3$. Prove that $$\sum_{cyc}{\frac{1}{2(a-b)^2+ab}} \geq \frac{16(ab+bc+ca)^3(abc-1)}{\sqrt{a}+\sqrt{b}+\sqrt{c}}+3.$$
By AM-GM : $$\sum_{cyc}\frac{1}{2(a^2+b^2)-3ab} +16(a+b+c) \geq 42abc +9$$ Proof $$42abc-16(a+b+c)+6 \geq \frac{16(ab+bc+ca)^3(abc-1)}{\sqrt{a}+\sqrt{b}+\sqrt{c}}\tag{$*$}$$ Thus, it's enough to prove that: $$\sqrt{a}+\sqrt{b}+\sqrt{c} \geq ab+bc+ca$$
Therefore $(*)$ is true by AM-GM. And we are done!
What other solutions are there?
Your first step leads to a wrong inequality.
It's enough to prove that: $$\frac{16(1-abc)(ab+ac+bc)^3}{\sqrt{a}+\sqrt{b}+\sqrt{c}}\geq42(1-abc)$$ and since by AM-GM $1-abc\geq0,$ we need to prove that: $$16(ab+ac+bc)^3\geq42(\sqrt{a}+\sqrt{b}+\sqrt{c}),$$ which is wrong for $b=c\rightarrow0^+$.
By the way, by C-S $$\sum_{cyc}\frac{1}{2a^2-3ab+2b^2}=\sum_{cyc}\frac{c^2}{c^2(2a^2-3ab+2b^2)}\geq\frac{(a+b+c)^2}{\sum\limits_{cyc}(4a^2b^2-3a^2bc)}$$ and since by AM-GM $$\sum_{cyc}\sqrt{a}\leq\sum_{cyc}\frac{a+1}{2}=3,$$ it's enough to prove that: $$\frac{9}{\sum\limits_{cyc}(4a^2b^2-3a^2bc)}+\frac{16(ab+ac+bc)^3(1-abc)}{3}\geq3,$$ which is true, but my proof of this statement is not nice.
I hope, it will help.