I am having a hard time understanding why the division algorithm works during base conversion even after reading some of the answers given on this site.
Obviously I am missing a formal introduction to these topics hence the difficulty. So I am asking if introductory books on number theory like - "A Friendly Introduction to Number theory" or "A Gentle Introduction to Number Theory" would solve this problem?
I will try an answer to the question behind your question: why the division algorithm works for base conversion.
Consider the integer $238$, say, in ordinary base $10$ notation. The units digit is the remainder when you divide that number by $10$. That's something you can understand several ways. One is "it's obvious". Or you can inspect the expansion $$ 238 = 200 + 30 + 8 . $$ Or you can invoke the division algorithm explicitly: $$ 238 = 23 \times 10 +8 . $$ However you understand it, you can proceed by subtracting the remainder $8$ and dividing by $10$ to get to $23$. Use that to find the ten's digit of $238$, and so on (in general).
Now stop thinking about "base conversion" and remember that an integer $n$ is just what it is, independent of how we choose to represent it with marks on a page. Then "converting to base $b$" is just "finding the digits for the base $b$ representation". There is nothing special about $10$ and the argument in the previous paragraph works just fine.
In practice, it's easy to be confused when writing out an actual conversion because you have to write the integers in our ordinary base $10$ along the way.
For example, to write $238$ in binary, begin by imagining that you have the answer. Then $$ 238 = ? \times 2^7 + ? \times 2^6 + ? \times 2^5 + ? \times 2^4 + ? \times 2^3 + ? \times 2^2 + ? \times 2^1 + ? \times 2^0, $$ and the problem is to find the "digits" $?$, each of which is $0$ or $1$. (You know the first term will be $1 \times 2^7$ because $128$ the highest power of $2$ that's smaller than $238$.)
Since all but the last term in the expansion is even, the remainder when you divide by $2$ will be the last term. That tells you the units digit, which is $0$ in this case. Now subtract that remainder (which is $0$) and divide by $2$ to get $119$. That's odd, so division by $2$ leaves the remainder $1$, which is the $2$s digit in the binary expansion of $238$. And so on.