Need help calculating entropy using huge kronecker products

18 Views Asked by At

I am no expert in tensor algebra. I am stuck with computing the following function of kronecker products (you will recognize an entropy-like equation):

$H(\mathbf{T}) = \mathbf{T}^t \cdot \log_2(\mathbf{T})$

where $\mathbf{T}'$ is the transpose of $\mathbf{T}$ and $\mathbf{T}$ is the sum of two tensors

$\mathbf{T} = \bigotimes_{i=1}^{N} \mathbf{p}_i^1 +\bigotimes_{i=1}^{N} \mathbf{p}_i^2$

and $\mathbf{p}_i^2$ are $5 \times 1$ arrays of integers. Note that the logarithm above is not the matrix-logarithm but the logarithm computed for each element of the tensor.

As long as $N$ is between 10 to 15, a normal computer can handle the computation. However, the problem is that in my case $N \ge 100$. Even for $N = 100$, $\mathbf{T}$ will have approx $10^{69}$ elements. Is there any trick for computing this in less than eternity?