I am trying to do this exercise but I am not having much luck. First, my understanding of a $\sigma$-finite set is that it is the countable union of Borel sets which have finite measure.
Let such a set be denoted as $A=\cup_{i=1}^{\infty}A_{i}$ with $\mu(A_{i})<\infty$ for each $i$.
We may assume also that the $A_{i}$ are disjoint, by utilizing the standard trick below
$B_{1}=A_{1}$
$B_{2}=A_{2}\cap A_{1}^{c}$
$B_{3}=A_{3}\cap (A_{1}\cup A_{2})^{c}$ etc...
At this stage I don't really see how to progress. I tried to use outer regularity of each of the $A_{i}$ to assert the existence of $A_{i}$ such that $A_{i}\subset V_{i}$ and $\mu(V_{i})-\mu(A_{i})<\frac{\epsilon}{2^{i}}$. This would mean I have and open set, namely $\cup_{i=1}^{\infty}V_{i}$ such that
$$\mu(\cup_{i=1}^{\infty}V_{i})-\mu(\cup_{i=1}^{\infty}A_{i})<\epsilon$$
Now, one could use inner regularity on $\cup_{i=1}^{\infty}V_{i}$ to assert the existence of a compact $K$ such that $\mu(\cup_{i=1}^{\infty}V_{i})-\mu(K)<\epsilon$ and $K\subset \cup_{i=1}^{\infty}V_{i}$. If this subset $K$ was a subset of $A$ then we would be done but that is not necessarily the case....
The idea is first to show that for a every borel set $A \subset X$ of finite measure $\mu(A) < \infty$ and for every $\varepsilon > 0$ there exists a compact set $K \subset A$ such that $\mu(K) > \mu(A)- \varepsilon$, i.e. inner regularity on sets of finite measure. To do this, pick $U \supset A$ open with $$\mu(U) < \mu(A) + \varepsilon/10.$$ (Here $\varepsilon/10$ is probably more than enough). Pick $K \subset U$ compact such that $$\mu(K) > \mu(U) - \varepsilon /10.$$ Now the trick is to apply outer regularity on the set $U\setminus A$: There exits an open $V \supset (U \setminus A)$ such that $$ \mu(V) < \mu(U \setminus A) + \varepsilon/10\,. $$ Define then $C:= K \setminus V$. This is the intersection of a closed set with a compact set, hence closed. Then $C \subset A$ and $\mu(C) > \mu(A) - \varepsilon$.
For a general $\sigma$-finite set $A = \bigcup_{n \geq 1}{A_n}$ with $\mu(A_n) < \infty$ there are two cases: Either $\mu(A) < \infty$ and we are done by the above proof. The other case is $\mu(A) = \infty$. Then inner regularity on $A$ means that $$ \mu(A) = \sup{\{\mu(K)\, : \, K \subset A \, \text{compact}\}} = \infty $$ So given any $M > 0$ we have to find a compact set $K \subset A$ such that $\mu(K) > M$. To do this, assume that the $A_n$ are pairwise disjoint to which you have already reduced. Then choose compact sets $K_n \subset A_n$ such that $\mu(K_n) > \mu(A_n) - 2^{-n}$. For every integer $N \geq 1$ we then have $$ \mu(\cup_{n= 1}^N{K_n}) \geq \mu(\cup_{n= 1}^N{A_n}) - 1 $$ As the right hand side tends to $\infty$ as $N \rightarrow \infty$, so does the left hand side and you can take a finite union the $K_n$ to get the desired $K$.
This answer is partly taken from D. Salamon's notes on measure theory that you can find on his webpage.