Let X be the real-valued number of sides on a six-sided dice. So $\Omega(X) = ${$6$}.
Therefore, the probability density function is:
\begin{equation} f_{X}(x) = \begin{cases} \infty, x = 6 \\ 0, else \end{cases} \end{equation} which integrates to 1.
This is acceptable in engineering (delta signal). But is it "correct" mathematically? If yes, then does it also mean any probability density function can be expressed as some of delta signal?
On
Here we have a probability space $(\Omega,\mathcal F,\mathbb P)$ where $\Omega=\{1,2,3,4,5,6\}$, $\mathcal F = \mathcal P(\Omega)$, and $\mathbb P(\{\omega\})=\frac1{|\Omega|}=\frac16$ for each $\omega\in\Omega$. We have defined $X(\omega)=6$. Therefore $$\mathbb P(X = 6) = \mathbb P(\omega\in\Omega : X(\omega) = 6) = \mathbb P(\Omega) = 1.$$ So the distribution function of $X$ is $F_X(t) = \mathsf 1_{[6,\infty)}(t)$, which is manifestly discontinuous at $t=6$. Therefore the distribution of $X$ is not absolutely continuous with respect to Lebesgue measure and does not admit a density $f$ such that $\mathbb P(X\leqslant t) = \int_{-\infty}^t f(s)\ \mathsf ds$.
However, $X$ DOES admit a density with respect to counting measure, i.e. the measure $\mu:\mathbb R\to[0,\infty]$ defined by $\mu(S) = |S|$ for finite $S$ and $\mu(S)=+\infty$ for infinite $S$. In this setting, the density of $X$ is $f_X(t) = \mathsf 1_{\{6\}}(t)$, and $\mathbb P(X\leqslant t) = \int_{-\infty}^t \mathsf 1_{\{6\}}(s)\ \mathsf d\mu(s).$
There is a sense in which the delta signal is well-defined mathematically. The delta signal is not a function, but a distribution, which can be regarded as a generalized form of a function (and ironically, it can also be used to define distributions in a probabilistic sense).
Define some measure space $(S,\mathcal{S},\mu)$. Define,
$$\mathcal{E} \subseteq \{f:S\rightarrow \mathbb{R} \text{ measurable.}\}. $$
We call the elements of $\mathcal{E}$ test functions. In applications related to partial differential equations, $\mathcal{E}$ is usually $C^\infty_0$ or $C^\infty_c$ (the set of infinitely continuously differentiable functions that vanish at infinity resp. with compact support). These spaces have very nice interactions with $L^p$ and Sobolev spaces, which are heavily used in theoretical PDEs.
In probability, we typically let $\mathcal{E}$ be the set of bounded, continuous functions. There is some work that has to be done to ensure that $\mathcal{E}$ is a large enough set for whatever application you are interested in, but we'll ignore that for now. Then a distribution is a (bounded?) linear operator from $\mathcal{E}$ to the real numbers. For each measurable function $g:\mathcal{E} \rightarrow\mathbb{R}$, we can define a distribution:
$$\psi(f) = \int_S f(x)g(x)\,d\mu(x).$$
This is why we can consider distributions to be generalized functions. In this way, the distribution $\delta_6$ is well-defined when $S = \mathbb{R}$ by (forgive my slight abuse of notation here),
$$\delta_6(f) = \int_{-\infty}^\infty f(x)\delta_6(x)\,dx = f(6).$$
This is useful in PDEs because it allows us to define a notion of weak-derivatives for functions that are not traditionally differentiable. This also allows for weak solutions to PDEs that lack classical solutions. For example, the Heaviside step function,
$$H(x) = \begin{cases} 1 &\text{ if } x \geq 0\\ 0 &\text{ otherwise} \end{cases},$$
is generally not differentiable, as it isn't even continuous. However, intuitively it should be clear that it's derivative is the delta function $\delta_0$ because for $a < b$,
$$H(b) - H(a) = \int_a^b \delta_0(x)\,dx.$$
That is, the delta signal satisfies the fundamental theorem of calculus with respect to the Heaviside step function. In general, for a function $g$ satisfying some conditions, we define the weak derivative $\psi_{g'}$ to be the distribution given by,
$$\psi_{g'}(f) = -\int_S f'(x)g(x)\,d\mu(x)\text{ for all } f \in \mathcal{E} = C^\infty_0.$$
Applying this to the Heaviside step function,
$$\psi_{H'}(f) = -\int_{-\infty}^\infty f'(x)H(x)\,dx = -\int_0^\infty f'(x)\,dx = -\left(\lim_{x\rightarrow\infty} f(x)\right) + f(0) = f(0).$$
Distributions (in the generalized function sense) are also extremely useful in probability because it gives us a general way to describe distributions (in the probabilistic sense) of random variables even if they are not continuous random variables. Let $\mathcal{E} = C_b$ be the set of bounded, continuous functions and assume that $S$ is a Polish space. Then by using the Monotone Class Theorem, we can prove that if,
$$\mathbb{E}[f(X)] = \mathbb{E}[f(Y)]\text{ for all }f \in \mathcal{E},$$
then $X$ and $Y$ have the same distribution (in the probabilistic sense). Then for any random variable $X$, we can define the distribution (in the generalized function sense) of $X$ as the operator $\psi_X: \mathcal{E} \rightarrow \mathbb{R}$ defined by,
$$\psi_X(f) = \mathbb{E}[f(X)].$$
Thus, the probabilistic definition of distribution and the generalized function definition of distribution are equivalent.