After studying backpropagation for neural networks, I have a question: why can't we use full derivatives for backpropagation? I understand why partial derivatives work in backpropagation. However I wonder why we cannot (or should not) use full derivatives.
2026-03-28 12:55:52.1774702552
Backpropagation: why partial derivative, not full derivative?
402 Views Asked by Bumbble Comm https://math.techqa.club/user/bumbble-comm/detail At
1
There are 1 best solutions below
Related Questions in DERIVATIVES
- Derivative of $ \sqrt x + sinx $
- Second directional derivative of a scaler in polar coordinate
- A problem on mathematical analysis.
- Why the derivative of $T(\gamma(s))$ is $T$ if this composition is not a linear transformation?
- Does there exist any relationship between non-constant $N$-Exhaustible function and differentiability?
- Holding intermediate variables constant in partial derivative chain rule
- How would I simplify this fraction easily?
- Why is the derivative of a vector in polar form the cross product?
- Proving smoothness for a sequence of functions.
- Gradient and Hessian of quadratic form
Related Questions in PARTIAL-DERIVATIVE
- Equality of Mixed Partial Derivatives - Simple proof is Confusing
- Proving the differentiability of the following function of two variables
- Partial Derivative vs Total Derivative: Function depending Implicitly and Explicitly on Variable
- Holding intermediate variables constant in partial derivative chain rule
- Derive an equation with Faraday's law
- How might we express a second order PDE as a system of first order PDE's?
- Partial derivative of a summation
- How might I find, in parametric form, the solution to this (first order, quasilinear) PDE?
- Solving a PDE given initial/boundary conditions.
- Proof for f must be a constant polynomial
Related Questions in GRADIENT-DESCENT
- Gradient of Cost Function To Find Matrix Factorization
- Can someone explain the calculus within this gradient descent function?
- Established results on the convergence rate of iterates for Accelerated Gradient Descent?
- Sensitivity (gradient) of function solved using RK4
- Concerning the sequence of gradients in Nesterov's Accelerated Descent
- Gradient descent proof: justify $\left(\dfrac{\kappa - 1}{\kappa + 1}\right)^2 \leq \exp(-\dfrac{4t}{\kappa+1})$
- If the gradient of the logistic loss is never zero, does that mean the minimum can never be achieved?
- How does one show that the likelihood solution for logistic regression has a magnitude of infinity for separable data (Bishop exercise 4.14)?
- How to determinate that a constrained inequality system is not empty?
- How to show that the gradient descent for unconstrained optimization can be represented as the argmin of a quadratic?
Related Questions in NEURAL-NETWORKS
- Retrain of a neural network
- Angular values for input to a neural network
- Smooth, differentiable loss function 'bounding' $[0,1]$
- How to show that a gradient is a sum of gradients?
- Approximation rates of Neural Networks
- How does using chain rule in backprogation algorithm works?
- Computing the derivative of a matrix-vector dot product
- Need to do an opposite operation to a dot product with non square matrices, cannot figure out how.
- Paradox of square error function and derivates in neural networks
- Momentum in gradient descent
Trending Questions
- Induction on the number of equations
- How to convince a math teacher of this simple and obvious fact?
- Find $E[XY|Y+Z=1 ]$
- Refuting the Anti-Cantor Cranks
- What are imaginary numbers?
- Determine the adjoint of $\tilde Q(x)$ for $\tilde Q(x)u:=(Qu)(x)$ where $Q:U→L^2(Ω,ℝ^d$ is a Hilbert-Schmidt operator and $U$ is a Hilbert space
- Why does this innovative method of subtraction from a third grader always work?
- How do we know that the number $1$ is not equal to the number $-1$?
- What are the Implications of having VΩ as a model for a theory?
- Defining a Galois Field based on primitive element versus polynomial?
- Can't find the relationship between two columns of numbers. Please Help
- Is computer science a branch of mathematics?
- Is there a bijection of $\mathbb{R}^n$ with itself such that the forward map is connected but the inverse is not?
- Identification of a quadrilateral as a trapezoid, rectangle, or square
- Generator of inertia group in function field extension
Popular # Hahtags
second-order-logic
numerical-methods
puzzle
logic
probability
number-theory
winding-number
real-analysis
integration
calculus
complex-analysis
sequences-and-series
proof-writing
set-theory
functions
homotopy-theory
elementary-number-theory
ordinary-differential-equations
circles
derivatives
game-theory
definite-integrals
elementary-set-theory
limits
multivariable-calculus
geometry
algebraic-number-theory
proof-verification
partial-derivative
algebra-precalculus
Popular Questions
- What is the integral of 1/x?
- How many squares actually ARE in this picture? Is this a trick question with no right answer?
- Is a matrix multiplied with its transpose something special?
- What is the difference between independent and mutually exclusive events?
- Visually stunning math concepts which are easy to explain
- taylor series of $\ln(1+x)$?
- How to tell if a set of vectors spans a space?
- Calculus question taking derivative to find horizontal tangent line
- How to determine if a function is one-to-one?
- Determine if vectors are linearly independent
- What does it mean to have a determinant equal to zero?
- Is this Batman equation for real?
- How to find perpendicular vector to another vector?
- How to find mean and median from histogram
- How many sides does a circle have?
It is because you ultimately want to find "in which direction" you should change the network's parameters (biases and weights), one by one, in order to minimize the loss. So you look at every parameter and tweak it slightly from its value and see if the loss increases or decreases: this is equivalent to computing
$$\frac{\partial \mathcal{L}}{\partial b^{(l)}_j} \hspace{0.5cm} or \hspace{0.5cm} \frac{\partial \mathcal{L}}{\partial w^{(l)}_{ij}}$$
where $(l)$ is the layer the bias or weight refers to.
To compute these quantities you use backpropagation (which is just the chain rule applied to neural networks) and this involves partial derivatives.