I know that $$\lim_\limits{x \to 0} \frac{f(x)}{g(x)} = \frac{\lim_\limits{x \to 0} f(x)}{\lim\limits_{x \to 0} g(x)}$$ but only if $\lim_\limits{x \to 0} g(x) \neq 0$, $\lim_\limits{x \to 0} f(x) = A \in \mathbb{R},$ $\lim_\limits{x \to 0} g(x) = B \in \mathbb{R}$.
And my question is, can we omit the condition $\lim_\limits{x \to 0} g(x) \neq 0$? I mean, is it true that
If $\lim_\limits{x \to 0} \frac{f(x)}{x} = L \in \mathbb{R}$ then $\lim_\limits{x \to 0} f(x) = 0$.
because otherwise, if $\lim_\limits{x \to 0} f(x) = A \leq \infty, A \neq 0 $ then $\lim_\limits{x \to 0} \dfrac{f(x)}{x} = A \times \lim_\limits{x \to 0} \dfrac{1}{x} = A \times \infty$ cannot be equal to real number.
Formal proof for the marked claim:
You're given that for every $\varepsilon>0$ there exists $\delta>0$ such that for every $x\in (-\delta,\delta), x\not = 0$ we have $$|\frac{f(x)}{x}-L|<\varepsilon$$
Fix $\varepsilon=1$, so for a fixed $\delta$ we have that $L-1<\frac{f(x)}{x}<L+1$ for all $x\in (-\delta,\delta),x\not= 0$.
In particular it follows that $|f(x)|<\max(|x|(L+1),|x|(L-1))$ for all $x\not=0$ in an interval around zero. Thus, $|f(x)|$ convergence to zero as $x$ goes to zero.
Now it is not quit trivial (not that hard as well) to show that $\lim_{x\rightarrow 0}|f(x)|=0$ implies that $\lim_{x\rightarrow 0} f(x)=0$ which is what you wanted to prove.
Edit: It turns out to be even simpler than that (See the comment of Theo Bendit) using the algebra of limits theorems since $\lim_{x\rightarrow 0}\frac{f(x)}{x} = L$ and $\lim_{x\rightarrow 0} x = 0$ we have that $\lim_{x\rightarrow 0} f(x) = \lim_{x\rightarrow 0}\frac{f(x)}{x} \cdot \lim_{x\rightarrow 0} x = L\cdot 0 = 0$