The conditional entropy H(Y|X) is the amount of information needed to describe the outcome of a random variable Y given that the value of another random variable X is known.
In order to calculate the conditional entropy we need to know joint distribution of X and Y. Below you should enter the matrix where the cell value for any i row and j column represents the probability of the outcome, . Rows represent the values of X random variable and columns - the values of Y random variable .
Note that you can click "Show details" to view details of the calculation. Formula used in calculation is explained below the calculator.
Conditional entropy formula
The conditional entropy of Y given X is defined as
It is assumed that the expressions and should be treated as being equal to zero.
for each row is calculated by summing the row values (that is, summing cells for each value of X random variable), and are already given by the input matrix.
What is the meaning of this formula?
In fact, it is the weighted average of specific conditional entropies over all possible values of X.
Specific Conditional Entropy of Y for the X taking the value v is the entropy of Y among only those outcomes in which X has value v. That is,
So, the conditional entropy as the weighted sum of specific conditional entropies for each possible value of X, using p(x) as the weights, is