This seems like a pretty obvious thing, so it's never really explained, but I can't understand it.
Many book chapters use the expression: $\boldsymbol w^{T}\boldsymbol x$ as a form of denoting something like $w_{1}x_{1} + w_{2}x_{2} + ... + w_{n}x_{n}$.
However if I treat $\boldsymbol w$ as a vector $\boldsymbol w = \begin{bmatrix} w_{1} & w_{2} & ... & w_{n} \end{bmatrix}$ and $\boldsymbol x$ as a vector $\boldsymbol x = \begin{bmatrix} x_{1} & x_{2} & ... & x_{n} \end{bmatrix}$ this doesn't hold because:
$\boldsymbol w^{T}\boldsymbol x = \begin{bmatrix} w_{1} \\ w_{2} \\ ... \\ w_{n} \end{bmatrix}\times \begin{bmatrix} x_{1} & x_{2} & ... & x_{n} \end{bmatrix} = \begin{bmatrix} w_{1}x_{1} & w_{1}x_{2} & ... & w_{1}x_{n} \\ w_{2}x_{1} & w_{2}x_{2} & ... & w_{2}x_{n} \\ ... & ... & ... & ... \\ w_{n}x_{1} & w_{n}x_{2} & ... &w_{n}x_{n} \end{bmatrix}$
Funny thing is that the EXACT reverse produces the desired output, that is a matrix with just one element being the above mentioned sum:
$\boldsymbol w\boldsymbol x^{T} = \begin{bmatrix} w_{1} & w_{2} & ... & w_{n} \end{bmatrix} \times \begin{bmatrix} x_{1} \\ x_{2} \\ ... \\ x_{n} \end{bmatrix} = \begin{bmatrix} w_{1}x_{1} + w_{2}x_{2} + ... + w_{n}x_{n} \end{bmatrix}$
So I guess the only explanation left is that both $\boldsymbol w$ and $\boldsymbol x$ are already in the "vertical" form:
$\boldsymbol w = \begin{bmatrix} w_{1} \\ w_{2} \\ ... \\ w_{n} \end{bmatrix} \ \ \ \boldsymbol x = \begin{bmatrix} x_{1} \\ x_{2} \\ ... \\ x_{n} \end{bmatrix}\ \ \ $ which indeed makes $\ \ \boldsymbol w^{T} = \begin{bmatrix} w_{1} & w_{2} & ... & w_{n} \end{bmatrix}$
My question is: how do I know what vector / one-dimensional matrix direction the authors had in mind? Is is a convention that is ALWAYS followed that by default it's a "vertical" direction?
Or am I just supposed to guess based on knowing what the probable direction will be given the answer I expect? Here it's doable, but when I encounter more difficult equations it takes lots of time...
While column vectors are more common than row vectors, usually, books will mention whether they are using column vectors or row vectors. For example, they might write the following: $$\mathbf w=[1, 2, 3]^T$$ where the $T$ exponent means that this is the transpose of a row vector, making it a column vector. Although column vectors are usually more often referred to, linear algebra also makes use of row vectors sometimes, so make sure you are paying attention to the author's intentions and their mention of row/column vectors to make sure you know what kind of vector they are using.