A Gaussian Mixture Model (GMM), which is a generative model for data , is defined by the following set of parameters:

- K Number of mixture components
- A d-dimensional Gaussian for every
- : Mixture weights
- The parameters of a K-component GMM can be represented as:
- The likelihood of a point in a GMM is given as

## Observed case

Let and let be three observed points in cluster 1 and be three observed points in cluster 2 .

## Unobserved Case: EM Algorithm

Estimates of Parameters of GMM: The Expectation Maximization (EM) Algorithm

We observe data points in , we would like to maximize the GMM likelihood with respect to the parameter set:

Maximizing is not tractable using the settings of GMMs.

The EM algorithm is an iterative algorithm that finds a locally optimal solution to the GMM likelihood maximization problem.

**E-Step**

Assume that the initial means and variances of two clusters in a GMM are as follows:

Let

Let

Using the formula of E-Step:

**M-Step**

Using the formulae corresponding to the M-step,

A Gaussian mixture model can provide information about how likely it is that a given point belongs to each cluster.