Comprehension of linear algebra notations and concepts

Question

There is a line of math, whose mechanics I'm unsure about: multiplying {1} times a vector... In "Learning From Data" by Abu-Mostafa, Magdon-Ismail and Lin (1.1.2 A Simple Learning Model) (pg 7):

---

"To simplify the notation of the perceptron forumula, we will treat the bias b as a weight w₀ = b and merge it with the other weights into one vector w = [w₀,w1₁,...,w_d]^T, where ^T denotes the transpose of a vector, so w is a column vector. We also treat x as a column vector and modify it to become x=[x₀,x₁,...,x_d]^T, where the added coordinate x₀ is fixed at x₀=1. Formally speaking, the input space is now:

χ = {1} x ℝ^d = {[x₀,x₁,...,x_d]^T | x₀=1, x₁ ∈ ℝ,..., x_d ∈ ℝ }"

---

To explain what I think is going on here is an example with an arbitrary 3-dimensional vector A: let A = [a₀,a₁,a₂] where a₀ = 2.3, a₁= 8, a₂ = 7.6, so...

χ = {1} x ℝ3 = [new vector with one more element that increases dimension to ℝ4 ?]

χ = {1} x A = [ 1,2.3,8,7.6] ?

I also encountered set theory which explained the Cartesian product/cross product of two sets... {1,2} x {3,4} = {(1,3),(1,4),(2,3),(2,4)} so I'm inferring that [a₀,a₁,a₂] is a set of one element whose element is a vector and so, {1} x {[a₀,a₁,a₂]} = {1, [a₀,a₁,a₂]} or just {1,a₀,a₁,a₂}?

Answer 1

Let $D$ be the set of your original training data points. $D$ is a subset of $\mathbb{R}^n$.

The procedure that you are describing is just constructing the Cartesian product of $\{1\} \times D$.

That is if $\begin{bmatrix} x_1, \ldots, x_n \end{bmatrix}^T \in D$, then we are constructing the vector $\begin{bmatrix} 1, x_1, \ldots, x_n\end{bmatrix}^T.$

It is a common trick to first work with a version of the algorithms with no non-zero intercept and then visualize the problem with intercept as a problem without an intercept in a higher dimensional space since

$$b + \sum_{i=1}^n c_ix_i=\begin{bmatrix} b, c_1, \ldots,c_n\end{bmatrix}^T\begin{bmatrix} 1\\ x_1, \\ \vdots\\x_n\end{bmatrix}$$