design matrices

Question

Given a linear model $Y = X\beta + \epsilon$ with three treatments and six subjects where $X$ is the design matrix, suppose $X = \begin{matrix}1 & 1 & 0\\ 1 & 1 & 0\\ 1 & 0 & 1\\ 1 & 0 & 1\\ 1 & -1 & -1\\ 1 & -1 & -1 \end{matrix}$ and

X'= \begin{matrix}1 & 0 & 0\\ 1 & 0 & 0\\ 1 & 1 & 0\\ 1 & 1 & 0\\ 1 & 0 & 1\\ 1 & 0 & 1 \end{matrix}

with response vector $Y=[Y_{11} Y_{12} Y_{21} Y_{22} Y_{31} Y_{32}]^{T}$

Do these two matrices give you the same model?

I've had some trouble trying to figure out how, given that $\beta = (\beta_1 \beta_2 \beta_3)^T$, these two designs can give the equivalent model. Don't they have entirely different values of $\beta$ for each $Y_{ij}$?

EDIT: The constraints are $\sum_{i=1}^{3}\beta_i = 0$

Answer 1

$\beta=X^\dagger Y$ where $X^\dagger=(X^t X)^{-1} X^t$ is the pseudo inverse of X.

If you compute the pseudo inverse of both $X$ and $X'$ you will see that they give different matrix (octave code below). Hence your models will, most likely, be different. Now there might be some $Y$ such that the $\beta$ will be the same (i.e. such that $(X')^\dagger Y = (X)^\dagger Y$) but I'm thinking this is somehow irrelevant here (however, as J.M. points out, this case can happen).

X = [1 1 0; 1 1 0; 1 0 1; 1 0 1; 1 -1 -1; 1 -1 -1];

Xb = [1 0 0; 1 0 0; 1 1 0; 1 1 0; 1 0 1; 1 0 1];

inv(X'*X)*X'

inv(Xb'*Xb)*Xb'

Edit: a counter example shows the same: let Y=[1;2;3;4;5;6], then X\Y yields [3.5;-2;0] and Xb\Y yields [1.5;2;4] where A\b operation amounts to applying the pseudo inverse of A to the vector b.

Answer 2

I suspect that what is intended is for you to check if the models give the same fit to the data after fitting a least-squares model, i.e. that the predicted value of the outcome vector, $\hat Y$ is the same between models. $\hat Y$ turns out to be the projection of $Y$ onto the column space of $X$, so this is equivalent to $X$ and $X'$ having the same columns space.

$X$ and $X'$ do in fact have the same column space and so give the same fit. To get the second column of $X'$ from $X$, so $\frac 1 3 x_1 - \frac 1 3 x_2 + \frac 2 3 x_3$ where $X = [x_1 x_2 x_3]$, and to get the third column do $\frac 1 3 x_1 - \frac 1 3 x_2 - \frac 1 3 x_3$.

Alternatively, we can check that the projection matricies $P(X) = X (X^TX)^{-1} X^T$ of $X$ and $X'$, and they do: both are $6 \times 6$ block diagonal with diagonal matricies all equal to $$D = \begin{pmatrix} \frac 1 2 & \frac 1 2 \\ \frac 1 2 & \frac 1 2 \end{pmatrix}.$$

An alternative way of phrasing this equivalency between models is that if the "true" model $M_0$ can be expressed as $Y = X\beta + \epsilon$ then it can also be expressed as $Y = X'\beta' + \epsilon$ for a different parameter $\beta'$ and so the models are equivalent.

EDIT: With the added constraint, they no longer result in the same model, see comment below. I think the point is that even though without the constraints these models are equivalent in some sense, adding constraints is enough to turn them into different models.