G-test - Relation To Mutual Information

Relation To Mutual Information

For analysis of contingency tables the value of G can also be expressed in terms of mutual information.

Let

, and

Then G can be expressed in several alternative forms:

where the entropy of a discrete random variable is defined as

and where

is the mutual information between the row vector and the column vector of the contingency table.

It can also be shown that the inverse document frequency weighting commonly used for text retrieval is an approximation of G applicable when the row sum for the query is much smaller than the row sum for the remainder of the corpus. Similarly, the result of Bayesian inference applied to a choice of single multinomial distribution for all rows of the contingency table taken together versus the more general alternative of a separate multinomial per row produces results very similar to the G statistic.

Read more about this topic:  G-test

Other articles related to "relation to mutual information, mutual information":

Jensen–Shannon Divergence - Relation To Mutual Information
... Jensen–Shannon divergence is the mutual information between a random variable from a mixture distribution and a binary indicator variable where if is from and if is ... in analogy to Kullback–Leibler divergence and mutual information) and to measure how reliably one can decide if a given response comes from the joint distribution or the ...

Famous quotes containing the words relation to, information, relation and/or mutual:

    In relation to God, we are like a thief who has burgled the house of a kindly householder and been allowed to keep some of the gold. From the point of view of the lawful owner this gold is a gift; From the point of view of the burglar it is a theft. He must go and give it back. It is the same with our existence. We have stolen a little of God’s being to make it ours. God has made us a gift of it. But we have stolen it. We must return it.
    Simone Weil (1909–1943)

    I am the very pattern of a modern Major-Gineral,
    I’ve information vegetable, animal, and mineral;
    I know the kings of England, and I quote the fights historical,
    From Marathon to Waterloo, in order categorical;
    Sir William Schwenck Gilbert (1836–1911)

    Parents ought, through their own behavior and the values by which they live, to provide direction for their children. But they need to rid themselves of the idea that there are surefire methods which, when well applied, will produce certain predictable results. Whatever we do with and for our children ought to flow from our understanding of and our feelings for the particular situation and the relation we wish to exist between us and our child.
    Bruno Bettelheim (20th century)

    Ties of blood are not always ties of friendship; but friendship founded on merit, on esteem, and on mutual trust, becomes more vital and more tender when strengthened by the ties of blood.
    Philip Dormer Stanhope, 4th Earl Chesterfield (1694–1773)