Darwinian Neurodynamics (2): Perceptual Unsupervised Learning Quality, as a Guide to Action.

An idea for an intrinsic motivation function for child development based on quality of unsupervised learning. 

Compressibility (Schmidhuber) 

Short term goal. Produce a range of python functions for Alex that take in a senosirmotor time series one timestep at a time, and returns at the end a score for how well an unsupervised learning system (of various types) did, e.g. the quality of unsupervised learning. One such measure is naturally compressibility, which is how well the data can be reproduced given an informational bottleneck, so this could be done with an autoassociator by trying to reduce the size of the hidden layer to a minimum. The greater the compressibility the deeper the learning. Deep Belief Networks could also be used for this. In fact, at a deep level, the goal of unsupervised learning is to achieve maximal compression of the data. 

We've returned to Schmidhuber's compression progress as a measure of interestingness of an action. A game which is trying to compress an input vector s and is achieving greater compression rates over time, as a superior game to a game which is trying to compress input vector s' and is not improving. 

How do we measure the compressibility of the sensorimotor subset we're interested in then?