Understanding an inequality related to Diffusion Processes in Machine Learning

47 Views Asked by At

I'm trying to understand an equality from a blog post about diffusion models here in the Reverse diffusion process section.

Here $\beta_t \in (0, 1)$, $\alpha_t = 1-\beta_t$, and $\bar{\alpha_t} = \Pi_{i=1}^{t} \alpha_i$.

The author states the following equality $$\tilde{u_t} = \frac{\sqrt{\alpha_t}(1-\overline{\alpha}_{t-1})}{1 - \overline{\alpha_t}}x_t + \frac{\sqrt{\overline{\alpha_{t-1}}}\beta_t}{1-\overline{\alpha_t}}\frac{1}{\sqrt{\alpha_t}}(x_t - \sqrt{1-\overline{\alpha_t}}\epsilon_t) = \frac{1}{\sqrt{\alpha_t}} (x_t - \frac{1-\alpha_t}{\sqrt{1- \overline{\alpha_t}}}\epsilon_t)$$

Matching coefficients for LHS/RHS of $x_t$, I get

$$ \frac{\sqrt{\alpha_t}(1-\overline{\alpha}_{t-1})}{1 - \overline{\alpha_t}} + \frac{\sqrt{\overline{\alpha_{t-1}}}\beta_t}{1-\overline{\alpha_t}}\frac{1}{\sqrt{\alpha_t}} = \frac{1}{\sqrt{\alpha_t}}$$

$$ \implies \frac{\alpha_t(1-\overline{\alpha}_{t-1})+\sqrt{\overline{\alpha_t}}(1-\alpha_t)}{(1 - \overline{\alpha_t})(\sqrt{\alpha_t})} = \frac{1}{\sqrt{\alpha_t}}$$

Examining the left side this doesn't seem right...

$$ \frac{\alpha_t - \overline{\alpha_t} + \sqrt{\alpha_t}- \sqrt{\alpha_t}\alpha_t }{(1 - \overline{\alpha_t})(\sqrt{\alpha_t})} = \frac{1}{\sqrt{\alpha_t}}$$

I'm not sure what I'm doing wrong, any insights appreciated.

1

There are 1 best solutions below

0
On BEST ANSWER

In the left hand side, in the beginning, you are missing a bar:

$$ \frac{\sqrt{\alpha_t}(1 - \bar{\alpha}_{t-1})}{1 - \bar{\alpha}_t} \mathbf{x}_t + \frac{\sqrt{\bar{\alpha}_{t-1}}\beta_t}{1 - \bar{\alpha}_t} \frac{1}{\sqrt{\color{red}{\bar{\alpha}_t}}}(\mathbf{x}_t - \sqrt{1 - \bar{\alpha}_t}\boldsymbol{\epsilon}_t) $$

Thus after doing the obvious cancellation with the missed term we see the $\mathbf{x}_t$ coefficient is:

$$\frac{\alpha_{t}(1-\bar{\alpha}_{t-1})+(1-\alpha_{t})}{\sqrt{\alpha}_{t}(1-\bar{\alpha}_{t})} =\frac{1-\bar{\alpha}_{t}}{\sqrt{\alpha_{t}}(1-\bar{\alpha}_{t})} = \frac{1}{\sqrt{\alpha_{t}}}$$