if a real valued function defined for example as :$[a,b]\times[a,b] \to \mathbb{R}$ , then I ask if there is a function defined as :${[a,b]}^{[a,b]} \to \mathbb{R}$ or Does this ${[a,b]}^{[a,b]}$ make sense in mathematics?
Note: $a, b$ are real numbers with $a < b $
Yes. $[a,b]^{[a,b]}$ is simply the set of all functions $f:[a,b]\to [a,b]$. You can think of this as an uncountable version of a sequence space. For example $[a,b]^{\mathbb N}$ is simply the set of all functions $x:\mathbb N\to [a,b]$, i.e $x(n)\in[a,b]$ for all $n\in \mathbb N$. We can think of this as a sequence $(x_n)$, where $x_n=x(n)$.
$[a,b]\times [a,b]$ is simply $[a,b]^2=[a,b]^{\{0,1\}}$. It is the set of all functions $f:\{0,1\}\to [a,b]$, which you should convince yourself is the same as the more intuitive notion of $[a,b]\times[a,b]$ as a set of ordered pairs.
Now with $[a,b]^{[a,b]}$ defined, it is a simple matter to think of a function $\varphi:[a,b]^{[a,b]}\to \mathbb R$. It is just some function that takes a function as an argument and maps it to a real number. A classical example, that crops up in various forms all over the place, are the canonical maps $\varphi_x:[a,b]^{[a,b]}\to \mathbb R$ associated to each $x\in [a,b]$, defined by $\varphi_x(f)=f(x)\in [a,b]$.