Badr F. Albanna, Christopher Hillar, Jascha Sohl-Dickstein, Michael R. DeWeese
Maximum entropy models are increasingly being used to describe the collective activity of neural populations with measured mean neural activities and pairwise correlations, but the full space of probability distributions consistent with these constraints has not been explored. We provide lower and upper bounds on the entropy for both the minimum and maximum entropy distributions over binary units with fixed mean and pairwise correlation, and we construct distributions for several relevant cases. Surprisingly, the minimum entropy solution has entropy scaling logarithmically with system size, unlike the linear behavior of the maximum entropy solution, resolving an open question in neuroscience. Our results show how only small amounts of randomness are needed to mimic low-order statistical properties of highly entropic distributions, and we discuss some applications for engineered and biological information transmission systems.
View original:
http://arxiv.org/abs/1209.3744
No comments:
Post a Comment