Consider the single-valued functions $f: \mathbb{R}^n \to \mathbb{R}$, and the set valued function $F: \mathbb{R}^n \to 2^{\Bbb{R}}$.
Is $f$ a set-valued function as well?
My only trouble in understanding the difference is because the image of $F$ consists of sets, whereas the image of $f$ consists of an element in the reals, i.e., a number $6$ which is not a set $\{6\}$.
Can someone make clear to me the distinction between $f$ and $F$? Can we think of the set of $f$ as a sub-class of set-valued functions? What is the distinction between $f$ and a function whose output consists of singleton sets?
I would say yes, with the caveat that an identification has to be made.
Often in mathematics there is a bit of a discrepancy between how we technically define things and how we think of them. For example, what is $\mathbb{R}^3$? Is it $$(\mathbb{R} \times \mathbb{R}) \times \mathbb{R}$$ or $$\mathbb{R} \times (\mathbb{R} \times \mathbb{R})?$$ Technically these would be different, since the first has elements like $((a,b),c)$ and the second has elements $(a,(b,c))$. However, this distinction is not generally useful, so we identify them - that is, we forget about this difference between them. If you want to get technical, you can talk about isomorphisms, category theory, natural transformations and such. But it's generally considered fine to sweep this small technical difference under the rug.
Similarly, technically speaking a function $f: \mathbb{R} \rightarrow 2^{\mathbb{R}}$ cannot be a function $f: \mathbb{R} \rightarrow \mathbb{R}$ because no subset of $\mathbb{R}$ is also an element of $\mathbb{R}$. However, we can pretend that $\{2\} = 2$ (or in more usual jargon, identify $\{2\}$ with $2$). This is called an abuse of notation, and is generally considered fine as long as it is clear to the reader what the author intends. Math exposition can be very difficult, so most authors will take occasional shortcuts if it makes things easier to understand. Whether or not different kinds of abuse of notation are acceptable depends on the subject matter and the intended audience (and, if you're taking a class, what is expected by the grader.)