I am aware of the phenomenon of duality in linear programming.
Given a linear optimization problem $P$, let's say in canonical form:
$\begin{cases} \max_x cx\\ Ax \leq b\\ x \geq 0 \end{cases}$
Its dual is the problem
$\begin{cases} \min_y b^T y\\ A^Ty \geq c^T\\ y \geq 0 \end{cases}$
And vice versa.
On the other hand, if $P$ is in standard form:
$\begin{cases} \min_x cx\\ Ax = b\\ x \geq 0 \end{cases}$
Its dual is
$\begin{cases} \max_y b^Ty\\ A^Ty \leq c\\ \end{cases}$
And that's how I learned the subject, on a case-by-case basis.
This really irks me, because, for example. Given the same problem in canonical and standard form, the dual is different. This shows that the dual is a function of, not the problem, but the way you state the problem.
Now, my question is: what's really going on here? Why do we say the dual of so-and-so is that-and-that, and why not some other choice? Furthermore, how would I, given an arbitrary linear optimization problem, find its dual? Since the duals depend on the way chosen to state the problem, it is not a good idea to turn the problem to canonical form and then take the dual of that...
Furthermore, how does one interpret the dual problem? Is there some way to see intuitively why it should even give good results?
Note: I took a module on linear optimization last semester, but it did not cover nonlinear optimization, so any examples or ideas you might want to use would best come from linear optimization, if possible. Thanks in advance.