This is a statement made in Weibel Chpt 2, Sec 6, Adjoint Functors and Left/Right Exactness.
2.6.10 Let functor $L:\mathcal{A}\to\mathcal{B}$ be left adjoint to a functor $R:\mathcal{B}\to\mathcal{A}$ with $\mathcal{A,B}$ arbitrary categories. Then $L$ preserves all \operatorname{colim}its.
My thought goes as the following. Suppose $A=\operatorname{colim}(A_i)$. Then one has $$\operatorname{Hom}_{\mathcal{B}}(L(\operatorname{colim}(A_i),B)\cong \operatorname{Hom}_{\mathcal{A}}(\operatorname{colim}(A_i),RB)\cong \operatorname{Hom}_{\mathcal{A^I}}(A_i,\Delta RB)=RHS$$ where $\Delta$ is the diagonal functor $\Delta:i\in I\to RB$.
Now $\Delta R(B)=R\Delta(B)$ for sure. Hence $$RHS\cong \operatorname{Hom}_{\mathcal{A^I}}(A_i,R\Delta B)\cong \operatorname{Hom}_{\mathcal{B}}(LA_i,\Delta B)\cong \operatorname{Hom}_{\mathcal{B}}(\operatorname{colim}(LA_i),B).$$ Hence $\operatorname{Hom}_{\mathcal{B}}(L(\operatorname{colim}(A_i),-)\cong \operatorname{Hom}_{\mathcal{B}}(\operatorname{colim}(LA_i)),-)$ naturally equivalent. Then I conclude $L(\operatorname{colim}(A_i))\cong \operatorname{colim}(LA_i)$.
$\textbf{Q:}$ Is above reasoning correct? I just used naively adjointness of functors and obvious commutativity of $R\Delta=\Delta R$.
Your reasoning is essentially correct. In fact, it's completely correct if your target category has all colimits. (Ignoring some minor typos, and things that aren't really expressed clearly in places.)
The only major issue with it is the final natural isomorphism.
The statement you want to prove is that
You're currently assuming that $\colim_D L\circ J$ exists when you write $$\newcommand\Hom{\operatorname{Hom}}\Hom_{[I,D]}(L\circ J,\Delta_B)\simeq \Hom_D(\colim(L\circ J),B). $$ Then you're writing $$\Hom_D(L(\colim J),B)\simeq \Hom_D(\colim(L\circ J),B)$$ and concluding $L(\colim J)\simeq \colim(L\circ J)$ by the Yoneda lemma.
Instead, once you've proven $$\Hom_D(L(\colim J),B)\simeq \Hom_{[I,D]}(L\circ J,\Delta_B),$$ you can immediately conclude that $L(\colim J)$ is by definition a colimit of $L\circ J$, since it corepresents the functor $X\mapsto \Hom_{[I,D]}(L\circ J, \Delta_X)$. This way you're making it clear that we don't need to assume that the colimit of $L\circ J$ exists to prove that it must be $L(\colim J)$.
Side note
I also want to address Matematleta's comment on the question, concerning it being unclear where we get the morphisms forming the cocone from in this argument.
To me this part really makes me feel the beauty of the Yoneda lemma.
The morphisms determining the cocone are encoded in the natural transformation we obtain.
Let $A=\colim J$.
We have a natural isomorphism of functors in $B$, $$\Hom_D(L(A),B)\simeq \Hom_{[I,D]}(L\circ J,\Delta_B).$$ If we let $B=L(A)$, we have a natural isomorphism $$\Hom_D(L(A),L(A))\simeq \Hom_{[I,D]}(L\circ J, \Delta_{L(A)}).$$ The left hand set has a distinguished element, $1_{L(A)}$. This corresponds to a natural transformation $\lambda : L\circ J \to \Delta_{L(A)}$ on the right hand side.
In other words, for every object $X\in J$, we get a map $\lambda_X : L(J(X))\to L(A)$, and these maps commute with the maps in the diagram $L\circ J$, since $\lambda$ is a natural transformation. Thus our natural isomorphism encodes the cocone for us already.
The reason I say this makes me feel the beauty of the Yoneda lemma is that the construction I just gave is a special case of the statement in the Yoneda lemma that if $F:D\to \mathbf{Set}$ is a functor, then $$\operatorname{Nat}(h^X,F) \simeq F(X),$$ where $h^A$ is the contravariant Yoneda embedding of $X$.