Math and science::INF ML AI
Covariance matrix
Let \( X \) and \( Y \) be two random variables. The covariance between \( X \) and \( Y \) is defined as:
\[\begin{aligned}
Cov[X,Y] &:= E[(X-E[X])(Y-E[Y])] \\
&= [...]
\end{aligned} \]
Let the vector \( Z \) be defined like so: \( Z := \begin{bmatrix} X \\ Y\end{bmatrix} \). Thus, \( Z \) is a vector of random variables.
The covariance matrix for \( Z \) is defined as:
\[ \begin{aligned}
Cov[Z] &:= E[(Z - E[Z])(Z - E[Z])^T] \\
&= [...] \\
\end{aligned} \]
Where the expectation is an elementwise operation. The covariance matrix is a result of a matrix multiplication of two vector-like matrices, which produces a 2x2 matrix. (Yes, it is valid!).