This function is related with a previous question.
I need to estimate a function $y=f(x): \mathbb{B}^d \rightarrow \mathbb{R}$, where $\mathbb{B}^d$ stands for $\{0,1\}^d$. The unknown function $f(x)$ is like a black box, what I can do is to input $x^{(i)}$ to it, and obtain $y^{(i)}$ ($i=1,2,\cdots, N$). Then I get a dataset $(x^{(i)}, y^{(i)})$ and am able to fit a function (or use some fancy machine learning technique) on it.
When $d$ is small, I can enumerate all $2^d$ possibilities and get perfect knowledge of $f(x)$. However, this is not possible when $d$ is large...
In the case of very high dimension, is it possible to use a reasonable amount of data to have a good estimate of $f(x)$? How could we do it?
This question has two follow-up questions:
- What if the values of $x$ are integers in a finite set (e.g. $x \in \{0,1,2,3\}^d $)?
- What if the function $f(x,z)$ (where $ x\in \mathbb{R}^m, z \in \{0,1,2,3\}^n $) is defined on both continuous and discrete variable?
For simplicity, we could assume $f(x)$ is infinitely differentiable ($f \in C^\infty$).