$$x = c_1\cos (t) + c_2\sin (t) $$ is a two-parameter family of solutions of the second-order DE $$x'' + x = 0. $$
Find a solution of the second-order IVP consisting of this differential equation and the given initial conditions:
$$x\left(\frac{\pi}{6}\right) = \frac{1}{2} $$
and
$$x'\left(\frac{\pi}{6}\right) = 0 $$
I found that:
$$x'(t) = - c_1\sin (t) + c_2\cos (t) $$
and then I solved for $c_2$
$$c_2 = 1 - c_1\sqrt 3 $$
and plugged $c_2$ into $$x'\left(\frac{\pi}{6}\right) = 0 $$
where
$$x'\left(\frac{\pi}{6}\right) = - c_1\sin (t) + (1 - c_1\sqrt 3 )\cos (t) = 0 $$
and $$ = \frac{1}{2}c_1 + 1 - c_1\frac{{\sqrt 3 }}{2} = 0 $$
and finally got as far as
$$c_1 = \frac{{(\frac{1}{2} - \frac{{\sqrt 3 }}{2})}}{{\frac{1}{2} - \frac{{\sqrt 3 }}{2}}} = \frac{{ - 1}}{{\frac{1}{2} - \frac{{\sqrt 3 }}{2}}} = $$
However the book's solutions guide came up with
$$c_1 = \frac{{\sqrt 3 }}{4},c_2 = \frac{1}{4} $$
from
$$\frac{{\sqrt 3 }}{2}c_1 + \frac{1}{2}c_2 = \frac{1}{2} $$
and
$$ - \frac{1}{2}c_1 + \frac{{\sqrt 3 }}{2}c_2 = 0 $$
I just can't follow their logic, So any help would be appreciated.
From the equations $$ x'\left(\frac{\pi }{6}\right) = 0 \tag1 $$ and $$ x'(t) = - c_1\sin (t) + c_2\cos (t) \tag2 $$ you can set $t = \frac\pi6$ (which I assume is what you did) so that $x'(t)=0,$ $\sin (t)=\frac12,$ and $\cos (t) = \frac{\sqrt3}2,$ and plugging these values into Equation $(2)$ you would get $$ 0 = -c_1 \cdot \frac12 + c_2 \cdot \frac{\sqrt3}2. \tag3 $$ Solving Equation $(3)$ for $c_2$ in terms of $c_1,$ your first step might be to bring $c_1$ to the left hand side, so that $$ c_1 \cdot \frac12 = c_2 \cdot \frac{\sqrt3}2. $$ and then multiply by $\frac2{\sqrt3}$ on both sides to get $$ \frac1{\sqrt3} c_1 = c_2. \tag4 $$
Once you have $c_2$ in terms of $c_1$ (or vice versa) you still have to use the fact that $x\left(\frac{\pi }{6}\right) = \frac12,$ of course.
Actually it was unclear what you meant by "then I solved for $c_2$". I assumed you meant you solved the immediately preceding equation with $x'(t),$ but if you meant that you went all the way back to the first equation you wrote and solved that at $t = \frac\pi6,$ you would get $$ \frac12 = c_1 \frac{\sqrt3}2 \cdot + c_2 \cdot \frac12. \tag5 $$ And now if you solve for $c_2$ you do indeed get $c_2 = 1 - c_1 \sqrt3.$ But in the subsequent steps you should have found that when $t = \frac\pi6,$ \begin{align} 0 &= - c_1\sin(t) + (1 - c_1\sqrt3)\cos(t) \\ &= -c_1\cdot\frac12 + (1 - c_1\sqrt3)\cdot \frac{\sqrt3}2 \\ &= -\frac12 c_1 + \frac{\sqrt3}2 - \frac32 c_1 \\ &= \frac{\sqrt3}2 - 2 c_1, \end{align} and that is how we end up with $c_1 = \frac{\sqrt3}4.$
Now for what the book's author(s) apparently did: they derived Equation $(5)$ using the same method I described above, except that they wrote the equation in the form $$\frac{\sqrt3}2 c_1 + \frac12 c_2 = \frac12. \tag6$$ Then they derived Equation $(3)$ just as I did above, except they wrote it $$ - \frac12 c_1 + \frac{\sqrt3}2 c_2 = 0. \tag7$$ Now Equations $(6)$ and $(7)$ are two linear equations in two unknowns and there are standard methods (which you should study if you don't know them already) for solving such a set of equations. For example, multiply both sides of Equation $(7)$ by $\sqrt3$ so that when you add the left side of Equation $(7)$ to the left side of Equation $(6)$ the terms in $c_1$ cancel and you're left with $2c_2$; while adding the right-hand sides of these equations gives you $\frac12,$ so you conclude that $$2c_2 = \frac12$$ and therefore $c_2 = \frac14.$ Plug that into either equation and solve for $c_1.$ Of course if you notice that Equation $(7)$ has no constant term, you may see that it immediately gives you $c_1 = (\sqrt3)c_2,$ which you can plug into Equation $(6)$ to solve for $c_2.$
A word of advice: lay out your steps much more carefully and methodically and be very clear about what you're doing and why at each step. Whoever grades your papers will appreciate it.