Marginal Likelihood

In statistics, a marginal likelihood function, or integrated likelihood, is a likelihood function in which some parameter variables have been marginalized. In the context of Bayesian statistics, it may also be referred to as the evidence or model evidence.

Given a set of independent identically distributed data points where according to some probability distribution parameterized by θ, where θ itself is a random variable described by a distribution, i.e. the marginal likelihood in general asks what the probability is, where θ has been marginalized out (integrated out):

The above definition is phrased in the context of Bayesian statistics. In classical (frequentist) statistics, the concept of marginal likelihood occurs instead in the context of a joint parameter θ=(ψ,λ), where ψ is the actual parameter of interest, and λ is a non-interesting nuisance parameter. If there exists a probability distribution for λ, it is often desirable to consider the likelihood function only in terms of ψ, by marginalizing out λ:

Unfortunately, marginal likelihoods are generally difficult to compute. Exact solutions are known for a small class of distributions, particularly when the marginalized-out parameter is the conjugate prior of the distribution of the data. In other cases, some kind of numerical integration method is needed, either a general method such as Gaussian integration or a Monte Carlo method, or a method specialized to statistical problems such as the Laplace approximation, Gibbs sampling or the EM algorithm.

It is also possible to apply the above considerations to a single random variable (data point) x, rather than a set of observations. In a Bayesian context, this is equivalent to the prior predictive distribution of a data point.