MATHEMATICAL ENTROPY AND A CRITICISM OF A USAGE OF THE PRINCIPLE OF MAXIMUM ENTROPY
The ‘principle of maximum entropy’ has been advocated for choosing a probability distribution to represent individual unknowns such as systematic deviations. Supporting claims like ‘the maximum-entropy distribution is minimally committal’ are indefensible in this context. The idea that entropy measures information only has meaning with sequences of categorical random variables.