How to work with an expression involving matrix products and Hadamard products in least squares?

157 Views Asked by At

With the system $\boldsymbol{A} \boldsymbol{x} = \boldsymbol{b}$, where $\boldsymbol{A}$ is a known matrix, $\boldsymbol{b}$ is a measured vector and $\boldsymbol{x}$ the vector we want to estimate, the ordinary least squares solves our problem: $$\hat{\boldsymbol{x}} = (\boldsymbol{A}^\mathrm{H} \boldsymbol{A})^{-1} \boldsymbol{A}^\mathrm{H} \boldsymbol{b} = (\boldsymbol{A}^\mathrm{H} \boldsymbol{A})^{-1} \boldsymbol{A}^\mathrm{H} \boldsymbol{A} \boldsymbol{x} = \boldsymbol{x}.$$

Now, I have a system slightly more complicated: $(\boldsymbol{A} \odot \boldsymbol{C}) \boldsymbol{x} = \boldsymbol{b}$, where $\odot$ denotes the Hadamard product, and $\boldsymbol{C}$ contains random values that I don't know. In this case, if I try to solve it like the ordinary least squares, I have:

$$(\boldsymbol{A}^\mathrm{H} \boldsymbol{A})^{-1} \boldsymbol{A}^\mathrm{H} \boldsymbol{b} = (\boldsymbol{A}^\mathrm{H} \boldsymbol{A})^{-1} \boldsymbol{A}^\mathrm{H} (\boldsymbol{A} \odot \boldsymbol{C}) \boldsymbol{x} = \:?$$

Then, I am stuck here, because from what I know and what I read, we cannot simplify an expression where matrix products and Hadamard products are mixed (there are some things possible when vector or diagonal matrices are involved, but this is not the case here).

Can I go any further with this equation, and try to have an expression for the error introduced by the presence of $\boldsymbol{C}$ ? Or can I do something else than the ordinary least squares ?

I have another similar problem where I am also stuck. I have $((\boldsymbol{A} \boldsymbol{C}) \odot \boldsymbol{D}) \boldsymbol{x} = \boldsymbol{b}$, where I know $\boldsymbol{D}$ but not $\boldsymbol{C}$, and if I try to solve it like the ordinary least squares, I have: $$((\boldsymbol{A} \odot \boldsymbol{D})^\mathrm{H} (\boldsymbol{A} \odot \boldsymbol{D}))^{-1} (\boldsymbol{A} \odot \boldsymbol{D})^\mathrm{H} \boldsymbol{b} = ((\boldsymbol{A} \odot \boldsymbol{D})^\mathrm{H} (\boldsymbol{A} \odot \boldsymbol{D}))^{-1} (\boldsymbol{A} \odot \boldsymbol{D})^\mathrm{H} ((\boldsymbol{A} \boldsymbol{C}) \odot \boldsymbol{D}) \boldsymbol{x} = \:?$$

1

There are 1 best solutions below

1
On

The best way to write the least squares solution of $Ax=b$ is $$\hat x = A^+b$$ where $A^+$ denotes the Moore-Penrose pseudoinverse, because when you introduce a Hadamard factor and replace $A\to(A\odot C)$ it can destroy the full-rank property such that $(A^HA)$ becomes singular.

With this in mind, the solution to your first problem is $$\hat x = (A\odot C)^+b$$ and the solution to the second one is $$\hat x = (AC\odot D)^+b$$