Covariance matrix

Gaussian-2d

A covariance matrix is a square matrix that contains the covariances between elements of a random vector. It is a key concept in probability theory and statistics, particularly in the fields of multivariate statistics and linear algebra.

Definition

Given a random vector \(\mathbf{X} = \begin{bmatrix} X_1 & X_2 & \cdots & X_n \end{bmatrix}^T\) of \(n\) random variables, the covariance matrix \(\Sigma\) is defined as: \[ \Sigma = \text{Cov}(\mathbf{X}) = \mathbb{E} \left[ (\mathbf{X} - \mathbb{E}[\mathbf{X}])(\mathbf{X} - \mathbb{E}[\mathbf{X}])^T \right] \] where \(\mathbb{E}[\mathbf{X}]\) is the expected value (mean) vector of \(\mathbf{X}\).

Properties

• **Symmetry**: The covariance matrix is symmetric, i.e., \(\Sigma = \Sigma^T\).
• **Positive Semi-Definiteness**: The covariance matrix is positive semi-definite, meaning that for any non-zero vector \(\mathbf{a}\), \(\mathbf{a}^T \Sigma \mathbf{a} \geq 0\).
• **Diagonal Elements**: The diagonal elements of the covariance matrix are the variances of the individual random variables, i.e., \(\Sigma_{ii} = \text{Var}(X_i)\).

Applications

Covariance matrices are used in various applications, including:

• **Principal component analysis (PCA)**: PCA uses the covariance matrix to identify the principal components of the data.
• **Portfolio theory**: In finance, the covariance matrix is used to model the returns of different assets and to optimize the portfolio.
• **Kalman filter**: The covariance matrix is used in the Kalman filter algorithm to estimate the state of a dynamic system.

Example

Consider a random vector \(\mathbf{X} = \begin{bmatrix} X_1 & X_2 \end{bmatrix}^T\) with the following properties: \[ \mathbb{E}[\mathbf{X}] = \begin{bmatrix} \mu_1 \\ \mu_2 \end{bmatrix}, \quad \text{Cov}(X_1, X_2) = \sigma_{12} \] The covariance matrix \(\Sigma\) is: \[ \Sigma = \begin{bmatrix} \sigma_1^2 & \sigma_{12} \\ \sigma_{12} & \sigma_2^2 \end{bmatrix} \] where \(\sigma_1^2\) and \(\sigma_2^2\) are the variances of \(X_1\) and \(X_2\), respectively.

See also

• Variance
• Correlation matrix
• Multivariate normal distribution
• Linear regression
• Eigenvalues and eigenvectors

Related pages

• Probability theory
• Statistics
• Linear algebra
• Principal component analysis
• Portfolio theory
• Kalman filter

This linear algebra related article is a stub. You can help WikiMD by expanding it.