Upperbound VCdimension of a hypothesisclass that is the product of two other hypothesisclasses

12 Views Asked by At

The product class Fix non-empty hypothesis classes $H_1$ and $H_2$ for binary classification, mapping from a common instance space $X$ to label space $\{-1, +1\}$. Consider the product class $H_X$, $H_X = \{x \mapsto h_1(x)h_2(x) \mid h_1 \in H_1, h_2 \in > H_2\}$ Let $d_1 = \text{VCdim}(H_1)$, $d_2 = \text{VCdim}(H_2)$, and $d = d_1 + d_2$.

Show that there is a constant $C > 0$ such that $\text{VCdim}(H_X) \leq C \cdot d \log_2(d)$. Hint: you may use the tangent upper bound $\log_2(m) \leq \log_2(m_0) + > \frac{m-m_0}{m_0}\log_2(e)$.

In the previous part of the question I have shown that $\text{VCdim}(H_X)\geq \text{max}\{\text{VCdim}(H_1), \text{VCdim}(H_2)\}$.

I was thinking about something like: $|H_X|=|H_1||H_2|\geq 2^{\text{VCdim}(H_X)}$.(1)

To find a bound on $|H_1|$ and $|H_2$ use that $|H_1|\geq d_1$ and $|H_2|\geq d_2$. Therefore by the hint:

$$log_2(|H_1|) \leq log_2(d_1)+\frac{|H_1|-d_1}{d_1}log_2(e)$$

and similar for $H_2$.

Then from (1),

$$log_2(|H_1||H_2|) =log_2(|H_1|+log_2(|H_2|) \geq \text{VCdim}(H_x)$$ $$log_2(d_1)+\frac{|H_1|-d_1}{d_1}log_2(e)+log_2(d_2)+\frac{|H_2|-d_2}{d_2}log_2(e)\geq \text{VCdim}(H_X)$$

Which does not get me to the desired upperbound...

Can someone give me an idea on how to approach or how to start this problem? Thanks in advance