Now this might be a very dumb question but this has been bothering me from some days.
Imagine I want to create the real number line and for that I start with the rational numbers. So I start to put the numbers on the line (like toppings on a pizza). Now I know that between any two rational numbers I can add another rational number.
So I keep on adding infinitely many rationals between any two rational numbers by adding more and more digits in the decimal expansion.
I can keep on doing this forever but I don't think I will ever reach a point where I can't put a rational and only an irrational number can go there i.e. I don't think there will be any gap between two rationals which can be only fit in by an irrational (or is it ?)
So where do irrationals actually fit in on the line ?
I know this entire argument can be done the other way starting with irrationals too.
The gaps you're thinking of are very simple gaps - gaps of the form "greater than $a$ but smaller than $b$" for appropriate $a,b$ (that is, $a$ and $b$ are rational and $a<b$). However, this is not the only sort of gap we have to worry about! We could have a gap described by sets of rationals instead of individual rationals; that is, maybe we have (nonempty) sets $S,T$ of rationals such that $s<t$ holds whenever $s\in S$ and $t\in T$.
This sort of gap is exactly what irrational numbers are needed to fill. For example, consider the sets $$S=\{x\in\mathbb{Q}: x^2<2\}\quad\mbox{and}\quad T=\{y\in\mathbb{Q}: x^2>2\}.$$ Since every rational number appears in one or the other "sides" of this gap, this is a gap that can't be filled with a rational number.
This leads to the general idea of a Dedekind cut, and I strongly recommend Dedekind's original essays on the topic.
(Minor soapbox moment: in my experience students are more commonly shown the construction of the reals via Cauchy sequences; while this definitely has its advantages, in my opinion it lacks some of the philosophical beauty of Dedekind's approach. But reading Dedekind's essays was a formative mathematical experience for me, so I'm biased.)