Suppose I have a Function $F(t,x,y) = t*l(x,y)$ where $x,y,t$ are independent variables and $l$ is another variable that depends on further on $x$ and $y$. Now I wanted to ask that if: $\frac{\partial F}{\partial(\frac{\partial l}{\partial x})} = 0$
I dont know if it is correct or incorrect, for further context here, I am manually implementing a Neural Network with a custom cost function.And I am trying to calculate the gradient with respect to nework parameters using Reverse mode automatic differentiation And I came Across the following problem :
$\frac{\partial L}{\partial x} = f'(A)*\frac{\partial A}{\partial x}$ where $A = \Sigma_{i}W^i*M^i(x,y) + B$ where $W^i$ and $B$ are independent variables and $M^i$ further depends on $x$ and $y$ and So $\frac{\partial A}{\partial x} = \Sigma_{i}W^i*\frac{\partial M^i}{\partial x}$
Now I want to calculate $\frac{\partial \frac{\partial L}{\partial x}}{\partial \frac{\partial M^i}{\partial x}} = \frac {\partial (f'(A)*\frac{\partial A}{\partial x})}{\partial \frac{\partial M^i}{\partial x}}$ and I want to know that if $f'(A)$ depends on $\frac{\partial M^i}{\partial x}$ and If it does not depend on it then I can treat it as a constant in above equation and will become :
$\frac{\partial \frac{\partial L}{\partial x}}{\partial \frac{\partial M^i}{\partial x}} = f'(A)*\frac {\partial (\frac{\partial A}{\partial x})}{\partial \frac{\partial M^i}{\partial x}}$ Is it correct or am I making some mistake here. Sorry for such a long post This is my first time asking a question here, so if I make any mistakes, please correct me so that I can learn from them in the future.
Consider the counter-example: $l(x,y)=xy$.
Then $\frac{\partial l}{\partial x}=y$, and so $\frac{\partial F}{\partial\left( \frac{\partial l}{\partial x}\right)}=\frac{\partial}{\partial y}(txy)=tx\ne0$.
So this is not necessarily true.