codekansas

A blog of language, neuroscience, and deep learning

Entropy

P(a) P(b) E(a, b)
50% 50% 1
75% 25% 0.81
25% 75% 0.81
10% 90% 0.47

Experiment

Neuron Count Entropy Expected
S 0 1 0.00
A 0 1 0.81
B 0 1 0.81
C 0 1 1.00
D 0 1 1.00
O 0 1 0.54

What does this mean?