Is the variance of a linear combination of independent variables always less than or equal to the variance of the same linear combination of dependent variables?
I.e. in general
$Var(\sum_i a_i X_i)=\sum_ia_i^2 var(X_i)+2\sum_{j>i}a_i a_j cov(X_i, X_j) $
If $X_i$ are independent the second term is 0. If dependent, is the second term positive definite? I believe so, but am unable to prove it. Thanks!
Edit: See drhab's response below. Its a great counterexample to the question as asked. However, suppose I add a constraint such that $\sum_i a_i X_i>0$. I.e. we rule out the case given in his response below. Is the second term now positive definite? If not is there some constraint that will make this so?
I'll update if I find the answer myself.
No.
The second term can be negative as well.
E.g. let it be that $X$ is a non-degenerated rv with $\mathbb EX^2<\infty$.
Further let $X_1=X$ and $X_2=-X$ and let $X_1,X_3$ be independent where $X_3$ has the same distribution as $X$.
We find $\mathsf{Cov}(X_1,X_2)=-\mathsf{Cov}(X,X)=-\mathsf{Var}(X)<0$.
In that case we find: $$\mathsf{Var}(X_1+X_2)=0<\mathsf{Var}X_1+\mathsf{Var}X_3$$