There are quite a few sources online that say something along the lines of :
"As a rule of thumb, if the condition number $\kappa(A)=10^k$ then you may lose up to $k$ digits of accuracy on top of what would be lost to the numerical method due to loss of precision from arithmetic methods."
What's the reason for this?
It is a rule of thumb, but there are some estimates backing that rule.
Given a matrix $A∈ℝ^{n×n}$ and a vector $b∈ℝ^n$ the linear equation system ought to be solved is $$Ax=b.$$ Now we have perturbation in both $A$ and $b$, e.g. measurement errors in the rhs or integration errors in $A$ (thinking about FEM for PDEs). So actually we solve the perturbed system $$\tilde{A}x=\tilde{b},$$ with $\tilde{A}=A+δA$, and $\tilde{b}=b+δb$, with the perturbation $δA,\ δb$.
The measure of how $x$ and $\tilde{x}=x+δx$ are related is the relative error $$\frac{\|x-\tilde{x}\|}{\|x\|}=\frac{\|δx\|}{\|x\|}.$$ In the following there are two intermediate results, which lead to the desired estimate.
Perturbation of $b$:
Let $\tilde{x}=x+δx$ be the solution of the perturbed equation system $A\tilde{x}=\tilde{b}$, then it holds $$\frac{\|δx\|}{\|x\|}\leqslant \text{cond}(A)\frac{\|δb\|}{\|b\|}.$$ The condition number of $A$ is defined as $\text{cond}(A)=\|A\|\|A^{-1}\|$.
So the relative error in the solution can be estimated by the relative error in the rhs amplified with the condition number. Since that proof is quite simple I will state it for completeness.
Proof: It is $$δx=\tilde{x}-x = A^{-1}(A\tilde{x}-Ax)=A^{-1}(\tilde{b}-b)=A^{-1}δb$$ Hence, we get $$\frac{\|δx\|}{\|x\|}\leqslant\|A^{-1}\|\frac{\|δb\|}{\|x\|}\frac{\|b\|}{\|b\|} =\|A^{-1}\|\frac{\|δb\|}{\|b\|}\frac{\|Ax\|}{\|x\|} \leqslant\|A^{-1}\|\|A\|\frac{\|δb\|}{\|b\|}.$$
Perturbation of $A$:
If $\|δA\|<\|A^{-1}\|^{-1}$ then it holds $$\frac{\|δx\|}{\|x\|}\leqslant \frac{\text{cond}(A)}{1-\text{cond}(A)\|δA\|/\|A\|}\frac{\|δA\|}{\|A\|}.$$ with $\tilde{x}=x+δx$ the solution of the perturbed system $\tilde{A}\tilde{x}=b$.
That proof is a bit more involved and need a lemma, so I will skip it here. The condition $\|δA\|<…$ ensures regularity of $\tilde{A}$. (Note: Not sure if $<$ or $\leqslant$.)
Combining both results
Given the requirements of the theorem about perturbation in $A$ it holds $$\frac{\|δx\|}{\|x\|}\leqslant \frac{\text{cond}(A)}{1-\text{cond}(A)\|δA\|/\|A\|}\left(\frac{\|δb\|}{\|b\|}+\frac{\|δA\|}{\|A\|}\right),$$ with $\tilde{x}$ being the solution of the system $\tilde{A}\tilde{x}=\tilde{b}$.
Again I skip the proof.
Remark
Note that you can ignore the denominator: $$\frac{\text{cond}(A)}{1-\text{cond}(A)\|δA\|/\|A\|} \approx \text{cond}(A),$$
That is because we assumed $\|δA\|<\|A^{-1}\|^{-1}.$ So the denominator is just some small factor $c$.
So you get an estimate, like in your question. $$\frac{\|δx\|}{\|x\|}\leqslant [c\cdot]\text{cond}(A)\left(\frac{\|δb\|}{\|b\|}+\frac{\|δA\|}{\|A\|}\right)$$
So you can use that to estimate the error propagation from data to solution. But you should keep in mind, that the theorems state estimates from above. So replacing $"\leqslant"$ with $"\approx"$ will give you a very negative result. The math in physics is usually not that bad.
What you can see, when dealing with ill-conditioned problems, is that iterative solvers have a hard time converging. They will usually take a lot more steps to achieve an improvement of the solution guess. Here you can precondition your system to improve convergence. I personally only had one (real world) problem that I could not solve due to conditioning.
(If you want to see the missing proofs, let me know.)