I have this confusion related to palindrome language. I have found it being mentioned if a language L is equal to its reverse, it is a palindrome language.
But lets say
L = {aaab, baaa}
Then $L^R= \{baaa,aaab\}$
Definitely,
L = $L^R$ in this case
But the language isn't palindrome isn't it. Any insights?
"Palindrome language" is not a technical term that has a generally accepted meaning. Just from the words one could imagine several different meanings for it:
Neither of theses definitions are, however, one that an unprepared reader can be expected to understand. But fear not! That is what definitions are for.
One can make "palindrome language" into a technical term for the duration of a book, an article, an exercise, simply by setting forth a definition that declares what it will mean. The definition can be one of the three suggested above, or something fourth and different ... so long as it defines something that you need to have a name for for your purposes.
So, if you're reading an exercise that defines a palindrome language to be an $L$ such that $L=L^R$ (which seems to be the most relevant use of the term in the first page of Google hits), then that's what "palindrome language" means for the purpose of doing that exercise. It's not as if any of the possible meanings is something that crops often enough to warrant attaching a name to it permanently.