Here are two quotes from Leinster's book:
According to the first quote a dual is obtained by reversing all the arrows.
Accrording to the second quote, the second pair of functors is the dual of the first.
But if so, why aren't the arrows (in the category of categories and functors) in the second pair reversed (they still go from left to right)? And why aren't the arrows in $\mathbf{Set}$ reversed (i.e., why isn't $\mathbf{Set}$ replaced with $\mathbf{Set}^{op}$)? As far as I can see, only the arrows in $\mathscr A$ are reversed. Why so?
If it's easier to answer what exactly "dual" means (in the framework of Leinster up to p.90), that would be great too.


The way the duality principle is phrased is meant to give some initial intuition, but is not suitable (and, considering the qualification as 'informal', not intended to be used) for formal application. One reason you've already mentioned: There are categorical constructions where the 'dual' is not obtained by reversing 'all' arrows. Secondly, while the introductory paragraph can be read to suggest this, dual constructions or proofs need not be carried out 'again', but they are in fact formal consequences of the original construction.
An attempt for a formal statement of the duality principle would be:
Note that the context of the statement stays the same - dualization applies to the single category the statement is parametric over. In your example of the Yoneda Lemma, the parameter is ${\mathscr A}$, and the statement is that a concrete canonical assignment describes a functor ${\mathscr A}\to [{\mathscr A}^{\text{op}},\textsf{Set}]$. The dual statement hence says that for any ${\mathscr A}$ there is a concrete canonical assignment describing a functor ${\mathscr A}^{\text{op}}\to [{\mathscr A}^\text{op op}(\equiv {\mathscr A}),\textsf{Set}]$.