Tuesday, 5 November 2013

Arduino for psychophysics 

Chrisantha Fernando 

I've been thinking about spending this week on a quite different project involving doing some psychophysics experiments with an arduino involving an array of LEDs and a button. Nothing too complicated, but I want the system to log the data and send it through serial to the computer for collection. 

The idea is to replicate and extend the following experiment 2, which is intended to look at the reaction time of subjects when required to identify whether an LED on one OR the other side of a fixation point is active. This can be contrasted with RT in AND and XOR tasks, with two stimuli L and R of a fixation point. As the discrimination becomes more complex, how does the RT scale, and can this be explained by parallel or serial processing? 


Arduino has already been used for this purpose here... 



Basically I need a variant of the Posner test.


Modifying Sensory Dynamics and Morphologies - pt.1


This project has arisen from my interests around sensory experience. The few years I had working with people with disability before I came to 'ac(k)ademia has given me a certain perspective on the multitude of ways in which humans interface with the world on a sensorimotor level.

My recently completed MSc on 'Evolving Behaviour through Sensorimotor Contingencies' gave me a chance to explore the fascination I have around this topic from a robotics perspective, as framed by Dr Fernando's and Prof Szathmäry's theory of Darwinian Neurodynamics.

Furthermore, a invitation to the eSMCS summer school on embodiment and morphological computation allowed me the opportunity to learn about the role of the physical body in interactions with the world.

So, taking a line of enquiry that seeks to explore how sensorimotor contingencies and morphology affect behaviour - particularly what one finds 'interesting' during autonomous explorations in the world - I shall be modifying people's sensory dynamics and morphologies before releasing them into the wilds of Snowdonia.

Sensorimotor contingencies and active sensing/perception.

A sensorimotor contingency is a lawful relation between the sensor and motor functions of a system, specifically the manner in which a sense is dependant upon the motor actions or routines affecting it.

An early definition comes from J. Kevin O'Regan and Alva Noë:
"[...] the structure of the rules governing the sensory changes produced by various motor actions, that is, what we call the SMCs..." (2001, p. 941).
Sensorimotor contingencies are specific to each sense according to how we use that sense. Vision is used according to the rapid eye movements one utilises to scan the visual field as well as head, neck, body positioning that all determine a motor routine specific to the act of seeing. Smell is used according to the subsumption of normal automatic breathing processes in favour of a regulated breathing pattern that allows one to explore the field of odours present.

The structure of the relations between sensor and motor functions then describes the nature of particular sensorimotor contingencies. Furthermore, the role of motor actions in a continuous exploration of the senses suggests an intricate link between action and perception ...an active sensing of the world.

It can also be observed that we are still able to sense when we are neither acting nor performing motor routines, eg. sensing the wind against one's skin or hearing. This is a discussion around afferent nervous system and attentional processes that I do not wish to jump into at this point but I simply bring the idea into play to highlight the perspective from which my research here is studied from. That is, the process of active sensing is perhaps most salient to the fact of motor exploration in the world.

Of course, audition is the sense which, above all others, may be said to be performed better by indeed not acting, but rather paying attention. Though the movements of the head and pinnae and motor action of the outer hair cells - allowing active control of auditory gain (Roberts & Rutherford, 2008; Thomas & Vater, 2004) -  are certainly constitutive of the sensorimotor contingencies around audition, the sense of audition is experienced as "remarkably divorced from the subject's ongoing motor activity" (Schroeder et al, 2010).

It is this dichotomy between audition and the other senses in respect of active perception that I wish to remedy! 

Here's how...

Modifying the dynamics of audition.

If there is a tendency to be more still, to move less, to take a stationary position in order to listen then how would the reverse be? That is, what if to listen is to move, to scan the soundfield with successive motor movements, perhaps even something of a routine like that of the saccadic eye movements that constitute active vision and which allow one to form a more complete image of the world by rapidly shifting focus around the visual field?

I am going to modify the dynamics of audition according to motion in a manner which makes this reverse scenario a possibility.

System and signal path.

Kit needed:
  • binaural microphones/earphones: Roland CS-10EM
  • binaural microphone preamp & recording device: Zoom H4N
  • duplex stereo audio interface: Behringer UCA202
  • audio processing: Raspberry Pi (Rpi) running Pure Data
  • motion detection: accelerometer/gyroscope/magnetometer somewhere on the subject's body


The microphones in the earpiece of the Roland's capture the soundfield from the natural perspective of the human head. This signal will be preamplified and recorded for later analysis by the Zoom before being sent to the Behringer interface and into the Rpi to be processed in Pure Data. The signal is then sent from Pure Data out of the Rpi through the Behringer and back to the earphone of the Roland's.

System dynamics.

If no processing is performed on the signal passing through Pure Data then audition may be experienced as 'normal', while accounting for any filtering, colouration or latency observed in the audio signal path. This mode may be used as a ground truth for subjects' experience with the system. 

In order that the subjects can experience audition as a predominantly active sense there are a number of ways that the audio may be processed according to motion (as measured from the subject's head or torso).

A few modified audition-motion contingencies to start:
  • a general non-directional measure of motion can be taken, with a lower value of motion resulting in a higher degree of masking of the stereo audio signal by white/pink/brown noise.
  • a 2-dimensional measure of motion (left-right, forward-backward), with motion toward left or right resulting in less noise masking of that side of the stereo audio image starting from the centre and working further toward the periphery as a greater angle is covered in that motion, while the degree of masking is inversely proportional to the velocity of the motion so that continued forward motion results in an auditory fovea of sorts.
  • a multi dimensional measure of motion (up to 9D possible in theory), with the similar motion-masking relations as in the above scenario but the audio being processed with a spatial audio algorithm.
The practical feasibility of the higher dimensional scenarios rests largely on the processing power of the Rpi. This will be tested in due course.

Progress and things to work on...

After spending the best part of a week getting a stable Raspberry Pi setup running full duplex stereo audio and Pure Data processing the audio I am now at the very exciting stage of working on the audio algorithms which will govern the dynamics of the modified audition-motion contingencies.There's plenty of good spatial algorithms out there for Pure Data to look at and base my work on.

As well as exploring some of the examples above I might also look at processing the signal with some sort of temporal shifting to give a sense of sonic memory. I imagine this could be similar to the idea of vision as an external memory being built up with saccadic eye movements around the field, as espoused by O'Regan and Noë (2001). 

At this stage I imagine the experimental procedure will be prepared for by encouraging subjects to take a silent walk through the woods while voluntarily engaging in a process of open listening, simply allowing their attention to be drawn to whatever sounds are most salient, whatever appears most interesting before any thoughts about what those sounds might mean arise. This could be done as a group or in pairs perhaps. The field of ecological audio acoustic ecology (see: Westerkamp, Truax, Schafer) offers rich methodologies for this kind of activity.

Analysis of the actual modified listening session could be performed against a ground truth of the subjects using the modified audition system while no processing is performed on the signal. Much more needs to be thought of as to how to then analyse the data. Live commentary performed by the subjects or commentary post-session?

Next post... more of this + modified morphologies!


O'Regan, J. K., & Noë, A. (2001). A sensorimotor account of vision and visual consciousness. Behavioral and brain sciences, 24(5), 939-972.

Thomas, J. A., Moss, C. F., & Vater, M. (Eds.). (2004). Echolocation in bats and dolphins. University of Chicago Press.

Roberts, W. M., & Rutherford, M. A. (2008). Linear and nonlinear processing in hair cells. Journal of Experimental Biology, 211(11), 1775-1780.

Schroeder, Charles E., et al. "Dynamics of active sensing and perceptual selection." Current opinion in neurobiology 20.2 (2010): 172-176.

Monday, 4 November 2013

Voxel Project (Part 2)

Control System

In the original VoxCAD simulator there are four materials; soft, hard and two which inflate to 20% their original value but out of phase with each other. For the single cube project we want to be able to control each cube edge independently so that we can explore what dynamics a cube can accomplish. This does however somewhat complicate the control system, we now need to be able to control 12 edges via a PC and to be able to set the extension value (or rate of expansion) for each edge.

The cube itself is designed with hydraulic syringes so we need some mechanism to apply and reduce pressure to the system. The obvious choice is again to use syringes connected to some form of motor control. There are many comercial systems available, mostly used in the medical profession, but these are large and expensive machines used for delivering exact millilitre dosses and not practical for our requirement.

On the hobby scale there are a few examples of syringe pumps, these mostly use a threaded bar and guide-rail system to pull the syringe plunger back and forth. YouTube syringe pump example

The quick prototype I have below is designed on an oil rig system of rotating wheel and pulley bar. It does take a lot of torque to move the syringe forward and the original Lego Mindstorm motor does not have the power required (although this is not what it was designed for). We have some high torque servos which should hopefully have the power to move the mechanism. The design does have the advantage that there are limited components and these are joined by simple dowel joints. The required parts could be easily manufactured in bulk using the 3D-printer.

More designes to come :)

Tuesday, 29 October 2013

Voxel Project

Introduction to Voxels

Traditional robots are typically composed of hard fixed components linked together by hinges, joints and powered by motors and gears. These have success in may areas but lack the adaptability and precision that we see in the natural world, the simplest example is to list the robots capable of lifting an egg and placing in a saucepan. In recent years work has progressed on a new range of 'soft body' robots that use new composite materials and manufacturing methods to create more biologically inspired robots. The most notable of these is the robot tentacle [1], which has shown workable examples of potential use for; bomb disposal, surgery tools and cleanup tasks in hazardous environments. As impressive as this design is it is still crafted by the human hand and can only ever be as good as it's human inventor.

A parallel line of research is into evolving soft body robots within virtual worlds. These are able to utilise a number of simulated materials and produce designs optimally evolved for a given fitness function within their environment. Karl Sims is probably the father of this field producing evolved creatures back in 1994 [2], hardware and software have moved on since then and new techniques commonly use the VoxCAD voxel simulator as a testbed. A voxel is a 3D simulated cube that can be given particular material properties (stiffness, stretch, periodic inflation) these voxels are stacked together and due the the periodic inflating nature of some of these voxels the whole structure can move.

Hackademia Project

Voxels within a simulated environment can be evolved to perform a range of motion tasks and can be optimised via a fitness function to generate designs optimised for speed, energy use or specific environmental factors. However can an evolved solution within the simulated environment perform equally well when that design is copied over to the real world?

To test this hypothesis we are first going to need some voxels. Within the simulator there are typically four materials used; hard, soft and passive, periodic volume increase of 20% in 500 time steps, and out of phase 20% periodic volume increase of 20%.

Hard and soft are relatively easy to manufacture, however a periodic volume increase is much more of an issue. The biggest problem is to create a cube that will increase in volume and yet still retain the cuboid shape and not simply turning into a balloon. The second issue is how to get all faces of the cube to expand equally and also allow for a certain amount of deformation in the cube required to allow motion to happen. The final issue is the same, but how to get a consistant inflation / deflation in all of the phase and out of phase cubes simultaneously.

Below are some of the prototypes that I have been working on:

1.0 : Voxels to cuboids

After much experimenting with different ideas the only way to retain the cube shape was to inflate the design along the cubes edges. The first of may simplifications was to assume that the overall structure would only be one voxel deep, so in effect a 2D collection of voxels. This allowed me to only have to worry about inflating the front face, but to retain stability I would also inflate the rear face leaving the internal edges fixed so rather than a cube we more technically have a cuboid design.

The first step is to be able to push out the required edges, after a little trip to B&Q and raiding the lego pneumatics tray.

Syringes were cut down to reduce the size of the cuboid, this in turn reduces the amount the cuboid would need to be inflated. Next step is to link the edges together, this time a trip to JD Sport.

With squash balls (blue spot for maximum hang time), elastic bands and dowels to complete the design we can build a full cuboid.

[ After discussions with Chris Jack on how to control a collection of 25 (min required for a locomotion design) of these cuboids and a good discussion with Chrisantha on the project direction the decision was made to concentrate on controlling a single cube. This allows me to fully understand the dynamics of the problem and how to effectively scale up and link the cubes in future. This also allows us to explore the rather interesting property of controlling each edge of the cube independently and seeing what dynamics we can get and control with 12 degrees of freedom. ]

2.0  Codename: Squeaky
The first task was to move from cuboid to full cube.  For this I created a quick mockup allowing expansion in all three dimensions that would allow me to check for issues and work on the control system for independently controlling each of the 12 faces. [ Mental note; dowel, syringes and zip ties makes an awful squeaky racket!]

Awaiting a few connectors so that I can complete this project but a few issues have arose, with the extra  flexibility introduced with the extra dimension the design has a habit of rotating on the horizontal plane and collapsing upon itself. This may be simply due to unequal pressure from the elastic bands or too much flexibility in the squash balls. The design will work for testing a control systems as long as the cube does not go over it's collapse threshold point, but a better connection method is top of the todo list.

To Do:
1) New connection method (codename: ******** - well that would give it away!)
2) Control system prototype (control 12 faces independently)

[1] http://www.seas.harvard.edu/suo/papers/279.pdf
[2] http://creativemachines.cornell.edu/sites/default/files/ALIFE10_Hiller.pdf
[3] http://creativemachines.cornell.edu/sites/default/files/GECCO09_Hiller.pdf

Monday, 26 August 2013

Just published: From Blickets to Synapses. How brians do causal inference.


From Blickets to Synapses: Inferring Temporal Causal Networks by Observation


  • Causal inference;
  • Rational process model;
  • Neuronal replicator hypothesis;
  • Polychronous groups;
  • Backwards blocking;
  • Screening-off


How do human infants learn the causal dependencies between events? Evidence suggests that this remarkable feat can be achieved by observation of only a handful of examples. Many computational models have been produced to explain how infants perform causal inference without explicit teaching about statistics or the scientific method. Here, we propose a spiking neuronal network implementation that can be entrained to form a dynamical model of the temporal and causal relationships between events that it observes. The network uses spike-time dependent plasticity, long-term depression, and heterosynaptic competition rules to implement Rescorla–Wagner-like learning. Transmission delays between neurons allow the network to learn a forward model of the temporal relationships between events. Within this framework, biologically realistic synaptic plasticity rules account for well-known behavioral data regarding cognitive causal assumptions such as backwards blocking and screening-off. These models can then be run as emulators for state inference. Furthermore, this mechanism is capable of copying synaptic connectivity patterns between neuronal networks by observing the spontaneous spike activity from the neuronal circuit that is to be copied, and it thereby provides a powerful method for transmission of circuit functionality between brain regions.
Read it, or don't. 

Thursday, 22 August 2013

Projects to continue in September

Dear Hackademics, 

The new current list of people is above. We're going to be very much focused on Darwinian neurodynamics and building unconventional robots using 3D printing and soft sensors to test those algorithms. Boris is working on efficient representations of policies. Chris Jack plans to modify our sensorimotor contingencies somehow, but he'll know better after he gets back from Zurich I hope. Mark is making an unconventional worm bot. 

NOTE OF CAUTION: You should begin your projects by September. The week itself is a final get-together to write up papers together in a peaceful and contemplative environment, and demonstrate hardware, along with sci fi poetry writing and sci-fi cooking of-course. 

Travel arrangements will be as follows; We leave from London on 11th in 2-3 large people carriers which we hire. Mark Roper/Chris Jack and I will drive these. We take equipment 3D printer etc… electronics equipment and so forth.. as needed, we get food delivered from Waitrose or take some as well, we arrive in the evening and make dinner. 


Friday, 2 August 2013

Android experiments in child development

Android SDK: http://developer.android.com/sdk/index.html

Instructions for Processing in Android: http://wiki.processing.org/w/Android

Aim: Make a set of simple apps that can be used by Frida to test things like causal inference, associative learning, etc... in a flexible manner. Then extend this to Bamara's attempts to make a doll based on firm principles from child development.

Possible Experiments: 

1. Implement a pictorial version of a 3-node temporal causal network. Reward child with a noise if they can predict the next states of the network, given the current states. This is a game in which one must predict the next states of the system.

Sunday, 14 July 2013

Darwinian Neurodynamics (3): Accumulation of Adaptations & NEAT.

Accumulation of Adaptations

I think this represents the results we should work towards for the Living Machines 2013 exhibit. 

The closest literature that tries to accumulate adaptations in the way we do is the options framework I think (and MAXQ), but they have not demonstrated transfer learning. A review of transfer learning appears here by Taylor and Stone (2009). I summarise some aspects of that article here.

Q1. What knowledge to transfer?

a. Low-level knowledge: i. <s, a,r, s'> instances, action-value function (Q), policy (pi), full task model (model), prior distributions (pri).
b. High-level knowledge: i. What subset of actions A to use in which situations. ii. partial policies or options (basically what we're doing with the archive I think), iii. rules or advice, iv. important features for learning (e.g. perceptual atoms), v. proto-value functions?, vi. shaping rewards (subgoal functions suitable for full goals, subtask definitions).

Examples of Hierarchical Transfer Learning (Including only tasks that can be done in a Jolly Jumper)

Primitive Tasks

1. Agent learns to bounce on a cushion using its knees and ankles.
2. Agent learns to swing forwards and backwards on a cushion using its knees and ankles.
3. Agent learns to rotate around the vertical axis using its knees and ankles.

Permitting variability operators a. and b. will allow new molecule types to come into existence that may be more suitable for recombination. If these operators are evolved along with a recombination operator, then molecule types that suit those operators will evolve.

Purely Sequentially Composite Tasks

1. Bounce (1) for 5 seconds then Swing (2) for 5 seconds.
2. Bounce (1) for 5 seconds then Rotate (3) for 5 seconds.
3. Swing (2) for 5 seconds then Bounce (1) for 5 seconds.
4. Swing (2) for 5 seconds then Rotate (3) for 5 seconds.
5. Rotate (3) for 5 seconds then Bounce (1) for 5 seconds.
6. Rotate (3) for 5 seconds then Swing (2) for 5 seconds.

Purely Parallel Composite Tasks

1. Bounce + Swing
2. Bounce + Rotate
3. Swing + Rotate

at the same time. Fitness is the sum of both primitive fitnesses.

Methods of Transfer

1. FIXED CHAIN the timing of sub-actions. A -(delay = 3)--> B. with fixed delays.
2. CONDITIONAL CHAIN the timing of sub-actions based on termination conditions A --(terminate)-->B.
3. MERGE two compartments with equal priority, such that all motor atoms controlling the same motors now average their motor commands in the motor buffer.

Transfer method 1 would be expected to be able to satisfy the purely sequential composite tasks, if it was possible to execute compartment (x) followed by compartment (y). Currently, our archive retrieval system does not permit combining distinct whole compartments in this way. Neither does it permit the termination condition based method of sequencing whole compartments relative to each other. This is a failing of the current EROS algorithm.

Parallel task recombination would be expected to be helped by (3) the equal priority Merge operator.

Primitive Compartment Mechanisms and Variability Operators.  

1. A large DMP molecule moves knees and ankles taking sensory input from joints and accelerometers.
2. Two small reflex molecules operate ankle and knee movements as a function of FSRs on that leg.

TO DO: Currently, by limiting the molecules to the two above types, we severely restrict the kinds of knowledge transfer that can occur by recombination.

a. Structural changes should be permitted, e.g. removal and formation of edges. Removal is first order, and addition is zeroth order. This will result in the formation of new molecule structures.

b. Mutation of sensor identities should be permitted. Motor identities should also be mutable, within the subset of permitted motors i.e. knees and ankles only.

c. Priorities should also be allowed to mutate.

d. Recombination between compartments in the population (and between compartments in the archive and population should allow molecules to be added (with/without replacement).

e. These operators should be tested with the primitive tasks above.


NEAT Stanley and Miikkulainen (2002) Evolutionary Computation 10(2):99-127 

Neuroevolution of Augmented Topologies attempts to answer these questions. These have been the same questions we've been asking so we should look to see if they have solved all our problems for us. Lets hope they have, otherwise we might need to do some more work. Their questions were...

(1) Is there a genetic representation that allows disparate topologies to cross over in a meaningful way? 

(2) How can topological innovation that needs a few generations to be optimized be protected so that it does not disappear from the population prematurely? 

(3) How can topologies be minimized throughout evolution without the need for a specially contrived fitness function that measures complexity?

Topology andWeight Evolving Artificial Neural Networks (TWEANNs) are of two types. Direct and indirect encoding.

1. Binary connectivity matrix [crossover requires equal sized matrices, crossover has competing conventions problem], (Dasgupta and McGregor, 1992)

2. Nodes contain incoming and outgoing edge list (Pujol and Poli, 1997). [Subgraph swapping is possible] " Subgraph swapping is representative of a prevailing philosophy
in TWEANNs that subgraphs are functional units and therefore swapping them
makes sense because it preserves the structure of functional components.  However, we
cannot be sure whether the particular subgraphs being combined in PDGP are the right
ones to create a functional offspring."

1. Gruau's cellular encoding (I programmed this fully in C during my MSc thesis I remember now!). [Biased search space, utility of bias not known.]

Q(1) Is there a genetic representation that allows disparate topologies to cross over in a meaningful way? 

The biological solution is homology (lining up of equivalent genes) prior to crossover, done by RecA. Neat uses the HISTORICAL lineage knowledge to provide direct evidence of common ancestry from which true homology can be determined, to constrain crossover [Can this really work well, because there could have been functional divergence. Is this the best that can be done, is direct structural analysis really not possible? How does RecA work?]
" Tracking the historical origins requires very little computation. Whenever a new gene appears (through structural mutation), a global innovation number  is incremented and assigned to that gene. The innovation numbers thus represent a chronology of the appearance of every gene in the system."
" When crossing over, the genes in both genomes with the same innovation numbers are lined up. These genes are called matching  genes. Genes that do not match are either disjoint  or excess , depending on whether they occur within or outside the range of the other parent’s innovation numbers. They represent structure that is not present in the other genome. In composing the offspring, genes are randomly chosen from either parent at matching genes, whereas all excess or disjoint genes are always included from the more fit parent."

Q(2) How can topological innovation that needs a few generations to be optimized be protected so that it does not disappear from the population prematurely? 

A2. Protecting Innovation (NEW STRUCTURAL MUTANTS) with Speciation 
[EROS SUFFERS FROM THIS CURRENTLY]  " Frequently, adding new structure initially causes the fitness of a network to decrease" " By adding new genes to the population and sensibly mating genomes representing  different structures, the system can form a population of diverse topologies. However, it turns out that such a population on its own cannot maintain topological innovations. Because smaller structures optimize faster than larger structures, and adding nodes and connections usually initially decreases the fitness of the network, recently augmented structures have little hope of surviving more than one generation even though the innovations they represent might be crucial towards solving the task in the long run. The solution is to protect innovation by speciating the population" 

Adding new nodes and connections can reduce fitness suddenly before it's parameters have had a chance to optimise, so new topological additions must be protected/ This is achieved by fitness sharing based on genetic similarity [?]

Solution: Genomes are clustered on the basis of evolutionary similarity. Individuals in the same species must share fitness as in Goldberg and Richardson, 1987 (explicit fitness sharing).


1. Initialise population with a set of non-random MINIMAL initial topologies to reduce the number of parameters that have to be searched at the same time. [Instead of penalising larger networks like Zhang and Muhlenbein, 1993), they start with a minimal topology and grow it carefully.] 

Some parmeters from an example run below... 

2. Genetic Encoding [Designed to allow homologous crossover.] Linear representations of network connectivity.  

"Genomes are linear representations of network connectivity (Figure 2). Each genome includes a list of connection genes , each of which refers to two node genes  being connected. Node genes provide a list of inputs, hidden nodes, and outputs that can be connected. Each connection gene specifies the in-node, the out-node, the weight of the connection, whether or not the connection gene is expressed (an enable bit), and an innovation number , which allows finding corresponding genes (as will be explained below)."

[I do not like the fact that connectivity is not a self-organizing consequence of atom structures, which makes them fully PSS like.] 

"In the add connection mutation, a single new connection gene with a random weight is added connecting two previously unconnected nodes. In the add node  mutation, an existing connection is split and the new node placed where the old connection used to be. The old connection is disabled and two new connections are added to the genome.  The new connection leading into the new node receives a weight of 1, and the new connection leading out receives the same weight as the old connection. This method of adding nodes was chosen in order to minimize the initial effect of the mutation." [Vera's idea as well, in our system. But I'm not sure this way of adding nodes is the best. What is the justification for it in our system? How to add new nodes in our system in order to make smooth operators?] 


1. Introduce historical markings to atoms. [We have an issue with edges which are currently encoded IN atoms, rather than as separate genes to which a separate historical marking could be assigned. Is this a problem? Consider the entities to which historical markings, and hence species definition, could be assigned, and perhaps how the definition of a species (and inter-individual distance) might itself be evolvable, because surely it is rather arbitrary how the 3 parameters that define distance is NEAT are chosen. COULD BETTER SPECIES EVOLVE OVER TIME IF THE DEFINITION OF SPECIES DISTANCE WAS ITSELF EVOLVABLE< OR NOT?

2. Initialize with a more minimal topology, e.g. only a sensor and motor atom with a linear transform. 

3. [Protect innovation] Introduce speciation based on historical similarity and niching (based on markings).  

4. Make the initial parameters of a new structural variant likely to be not too detrimental, e.g. [neutral addition of variation (e.g. insertion of an identity transform between sensor and motor atom). Atoms and their variation operators should be designed such that they have this property.] 

5. Increase parameter mutation rate (allowed by speciation!, not otherwise, see paragraph above.. 

Darwinian Neurodynamics (2): Perceptual Unsupervised Learning Quality, as a Guide to Action.

An idea for an intrinsic motivation function for child development based on quality of unsupervised learning. 

Compressibility (Schmidhuber) 

Short term goal. Produce a range of python functions for Alex that take in a senosirmotor time series one timestep at a time, and returns at the end a score for how well an unsupervised learning system (of various types) did, e.g. the quality of unsupervised learning. One such measure is naturally compressibility, which is how well the data can be reproduced given an informational bottleneck, so this could be done with an autoassociator by trying to reduce the size of the hidden layer to a minimum. The greater the compressibility the deeper the learning. Deep Belief Networks could also be used for this. In fact, at a deep level, the goal of unsupervised learning is to achieve maximal compression of the data. 

We've returned to Schmidhuber's compression progress as a measure of interestingness of an action. A game which is trying to compress an input vector s and is achieving greater compression rates over time, as a superior game to a game which is trying to compress input vector s' and is not improving. 

How do we measure the compressibility of the sensorimotor subset we're interested in then? 

Thursday, 11 July 2013

Work-in-progress: exploring sensorimotor contingencies in robots

This blog shows the work-in-progress by me (Chris Jack) on building simple robots to demonstrate sensorimotor contingencies as they explore themselves and the world around them.

The 1st video below shows a robot called 'Spongebot' (aka noisiest-robot-in-the-world) that is designed to demonstrate the sensorimotor loops involved in squeezing diferent objects or tapping its fingers together. Here you see it at the stage of demonstrating a straightforward reflexive behaviour whereby the fingers retreat backward when a certain pressure of grip is sensed on the object - in this case a ball of steel or, at the end of the video, between the fingers themselves.

This second video shows a now defunct design to build a robot which could autonomously play the traditional English pub game known as Devil Amongst the Tailors or, more plainly, Bar Skittles.

Wednesday, 10 July 2013

Sub-Project Darwinian Neurodynamics: Video of Nao in the Jolly Jumper

Darwinian Neurodynamics

Chrisantha Fernando and Alex Churchill are the principle investigators in the Darwinian neurodynamics subproject which develops the EROS (Evolutionary Robot Operating System) on a range of 3D printed arduino-based robot platforms and the Nao humanoid robot, including the one that Berit and Chris are making. DN algorithms may serve as methods for autonomous epistemic exploration for curiosity and creativity in a range of sub-projects during the week. 

This page contains notes of cutting-edge developments in our thought about Darwinian cognitive architectures. 

Video of Nao on the Jolly Jumper: Experiment 1.  

The video below shows our first experiment with the Nao in the Jolly Jumper, evolving an actor molecule parameters, with the fitness being the sum of the first derivative of accelerometer values in rostro-caudal axis (up-down), i.e. the amount of bounciness. Dynamic movement primitive atoms are used to encode a range of movements. The coupling consists and the internal parameters of the DMPs are evolved using a microbial GA of pop size = 10. The whole experiment runs for about 800 evaluations, each evaluation being about 8 seconds. The fitness graph is shown. 

Half way through, the cushion is removed and the Nao has to readapt to the hanging condition where its feet cannot touch the ground any longer. It successfully adapts to this new condition with a different motor strategy being used. We did not test whether it had forgotten the original solution with the cushion by returning the cushion, because I am sure it will have forgotten because the archive is not being used.

An early description of the architecture can be found here and here, but a paper is expected in Frontiers in Cognitive Science shortly with a full description. Also we're talking and giving a demo at Living Machines 2013 here, where Terrence Deacon is also talking.

The actor molecule we evolve the parameters of is shown below. We hold it to this fixed topology. The sensor and motor identities are also not allowed to mutate. The DMG parameters (coupling constants, a, b, time constants, weights and properties of the radial basis functions) are allowed to mutate, as are the weights in the linear transforms of the two foot reflexes that move the knee and ankle as a linear function of the sum of the 4 force sensors on the feet.

The next experiment is to try another fitness function which punishes electric current used by the robot so that it tries to maximise "bounce per ounce" as Esther Thelen puts it in this wonderful paper here, which is what inspired us to apply Darwinian neurodynamics to child development explicitly. Thelen used the latest modelling of exploration and selection dynamics available in the brain at the time, i.e. the work of Edelman on Neural Darwinism, and tended to be anti-cognitivist, but our approach tries to unify the dynamical systems and cognitivist positions, and extend the work to include the accumulation of adaptation. The above results do not show the accumulation of adaptation yet.

Compare to a child who has optimised performance in the jolly jumper here. 

Experiment 2: Including accelerometers in the DMG molecule. 

In the above experiment, the z accelerometer reading is not accessible in real time to the controller, but only to the game molecule for calculating the fitness of a controller. Each of the sensor atoms in the large DMG molecule now contains as its second sensor the z accelerometer [143]. A linear transform (green atom) linearly combines the two sensor values and transmits them to the motor DMG atoms (red). In this way the robot can now use the real-time z accelerometer values if they are useful for performance.

Observations during the run (10th July 2013, afternoon):

1. 40 Evaluations: Two strategies have been discovered, a. Maximally Flex knees and ankles and just passively swing. b. Oscillate right knee pushing rhythmically on cushion (not clear if this uses the FSRs). There seems to be no tendency for 'circular reactions' learned on one side to be transferred to the other side. Whilst passively swinging in the air there are some knee movements that seem to be accelerometer dependent,i.e. oscillating in rhythm to the overall robot swinging.

2. 70 Evaluations: Both legs are used by in a somewhat unsynchronized fashion. Alternating pushing off. The total independence of strategies in both legs seems biologically implausible, but I have no idea how much bilateral skill transfer there is in infants. What kinds of molecule variation operator would permit bilateral skill transfer? [Copying of parameters between atoms would produce transfer, or copying of the weights in a linear transform molecule.]

3. 100 Evaluations: Perhaps (wishfully) slightly more coordinated kick-off action between the two legs, with both knees extending at the same time sometimes. Yes, I'm not imagining it, coordination between knees is definitely there.

4. 200 Evaluations: There does not seem to be any great improvement in coordination over the last 100 generations. I'm concerned that with the cushion the Nao is too close to the floor, so that there isn't much room for it to actually bounce.

5. 230 Evaluations: Discovered a left leg bouncing method which looks altogether more elegant than the half-cocked two leg strategies. Lets hope that one survives! Well it seems to have spread in the population now, and by 260 Evaluations the fitness has gone up to around 10 from less than 8 where it was around 200 evaluations, so this left foot based hopping method works much better than trying to coordinate the two legs. [in a sense this is a kind of freezing but bilateral freezing. The right leg is now stuck in a flexed position so it doesnt get in the way].

6. 290 Evaluations: It sounds like that left ankle is getting quite tired. I think this means it automatically reduces its stiffness, and this means that actor molecules that use that ankle may now no longer do so well. So in combination with a routine that reduces force when a joint is tired, this drives perhaps the evolution of other actor molecules that can use other joints. There is no need to explicitly code energy saving in the fitness function for actors if joints can become tired and work less well when overheated.

7. 300 Evaluations: Tuning of the single left leg hopping strategy is taking place. There are micro changes in timing and fixed attractor position on the left leg knee and ankle joints to match the details of the jolly jumper spring and the cushion.

8. 380 Evaluations: Not much change in strategy or fitness. The population seems to have become converged, with only minor tuning of the parameters. Fitness remains around 10.0. Perhaps this is a local optimum that a behavioural insight would be required to break out of? Perhaps for more radical changes to be possible we need to add recombinatorial complexity, i.e. add stuff from the archieve, or use variation operators that modify the topology of the system. This should definately be a next step to prevent getting stuck on these local optima. Perhaps the more complex operators could ONLY be tried during an impasse of this sort where no fitness improvement is being made. 

Experiment 3: Including accelerometers in the DMG molecule + Punishment for Electric Current Used. 

The fitness function is modified to divide the amount of bounciness by the sum of the electriccurrent used by the 4 active joints (knees and ankles). This is basically Thelen's notion of "bounce per ounce". It is a simple multi-objective function but we are not using any sophisticated methods to maintain diversity along a pareto front. I wonder whether explicitly punishing electric current used will produce more elegant and efficient looking behaviours?

Further ideas: 

The same kind of one legged jumping behaviour arose. OK, I'm bored of that behaviour now, it was fun at the beginning of he week, now its boring. I'm quite brain dead, need to sleep and think about how to get some accumulation of adaptation, or as Alex prefers, transfer learning... Right, slept! A few information theory measures have been brought to my attention, e.g sensorimotor mutual information (SMMI) and this may be an interesting selection criterion for good games, see.


and the preceeding paper


for the methods used. a Hidden Markov Network is used as the method of encoding activation logic between atoms which is interesting. They use a table of probabilities to encode the logic which is costly in terms of genome length however, but this is the most general encoding. We might wish to try it as a means of encoding the activation between atoms. It is a probabilistic encoding which may aid evolvability, i.e. you get rare paths that come into existence while the standard path works. It also bring the system closer to the path evolution algorithm here.

Probabilistic activation of atoms. 

A HMN is a general encoding for determining I/O functions for multiple inputs and outputs, its like a Boolean network but with probabilities. Its a very general formulation. It might be something we want to consider implementing later to achieve probabilistic actor molecules which may be more evolvable.

Accumulation of adaptation 

SMMI (Sensorimotor mutual information) as a fitness measure over games

To work this out in a continuous system, the simplest way is to artificially discrete the system by binning in time and magnitude. Thats what people do... e.g. Ay et al 2008 in a Sante Fe working paper describe how to calculate a simpler version of sensory mutual information for single channels for autonomous robots here. The mutual information between successive timesteps is equal to predictive information given a Markovian system. The authors claim that maximizing this measure results in robots that are both explorative and have predictable future events. White Gaussian noise is assumed (unrealistically). They split the sensor stream into 30 bins and calculated probabilities p(x), p(y) and p(x(t+1), x(t)) by frequencies obtained over long sampling over the run. They evaluated the MI for each of the sensor channels separately! To do this for vectors of x and y it is necessary to have multidimensional bins probably of larger sizes, and then we have the curse of dimensionality. 

Perceptual interestingness (unsupervised learning rate) as a fitness measure over games [POWERPLAY]

This relates to Schmidhuber's compressibility. A good game is one that results in interesting perceptual input. If one wants to get into this line of investigation one should really read Schmidhuber's stuff.


Powerplay may be of importance for task description methods. The introduction to powerplay implies a rich task description. SLIM RNN neurons internal states define the goal! These SLIM RNNs have a halting neuron. They are universal dynamical function approximators like CTRNNs, so one can think of them as CTRNNs with a threshold halt neuron I think. A new task is learned for which the network can reach the SAME internal goal state in the new environment, AND reproduce the same internal goal state in all previous environments! Its like saying, the CTRNNs 10th neuron must be above 0.5 when the CTRNN halts. Wow. How internal. If the SLIM RNN can't halt in the new environment, then random weight changes (mutations) are made.


The algorithm is simple. Get a new network s, and mutate it to s+1. Make a new task k. See if the mutated s can solve the new k task AND all prev. tasks (by having the same internal state at halting). If it can, then store the task k and internal state of the network s+1, into the archive. 

A compression version is also used where the task is to compress the data, in which case the fitness criteria is that after mutation of the network, the internal states can be produced with smaller weights or fewer connections being used. This apparently is equivalent to a better unsupervised learning ability of the classification. But I'm not sure the classification is interesting to begin with (what is a trivial set of internal states/classifications is learned at the beginning, e.g. what if s classifies all pictures with a black pixel on the bottom right as 0 and all those without as 1. Then an infinite number of pictures could be classified thus, and the new network would evolve just to consider the pixel on the bottom right and ignore everything else. Eventually no simpler more compressed network would be possible. Without the compression criterion then new tasks could always be solved. 

There is absolutely no demonstration in Schmidhuber's paper that in the fovea task the behaviours learned are subjectively interesting!!!! 


What is interesting is that he is using a hill-climbing strategy, with fitness = new task performance + old task performances (to promote accumulation of adaptation and no forgetting) + compression together as a criterion. Compression being a simplification of the network capable of solving the tasks. In our work we don't have multiple tasks yet, and we don't require a solution to be good at all previous tasks. I don't think thats a sensible solution for open-endedness, because there are too many dependencies. The solution to french shouldnt need to also do unicycling. Unicycling may be a quite different controller. I do like the idea of tasks as being defined as certain internal states of the RNN because this is a very powerful and open-ended method of specifying situated and embodied goals, BUT I think that a measure of INTERESTING internal states must be added, possibly constrained by one of the information theory measures of Tononi et al to make those internal states informationally interesting about the sensrimotor states?? 

Implication for EROS. Set up a game molecule containing a SLIM RNN atom (or equivalent CTRNN with fixed halting time). The game molecule gives a high fitness to the actor molecule if a subset of the neurons in the CTRNN are in a given GOAL state at the end of a fixed time trial. Choose CTRNN + GOAL state pairs (i.e. dynamical game definitions) in which PROGRESS is being made in achieving this goal state over actor generations. Do not require that a new actor molecule is able to satisfy all previous CTRNN+ GOAL pairs. Mutate CTRNNs and GOALs. Problem I see with this is that additional constraints on the fitness of CTRNNs and GOALs will be needed that prevent trivial CTRNNs and GOALs from evolving perhaps, which achieves progress in some trivial way, e.g. by classifying the pixel second from the right, and third from the right. etc... 

To solve this problem another kind of game atom may be a more traditional method for unsupervised learning, with the quality of unsupervised learning being a fitness measure. e.g. the game may be a k-means classifier. Actor molecules that result in classifier improvement are selected for, i.e. a good k-means classifier is one which has the greatest rate of reduction of the cost function over the last 100 data points perhaps. For k = 2 this means, behaviour which is able to distinguish maximally between two classes of event is favoured, behaviours that are able to divide observed data into two distinct clusters is favoured. Wait, the cost-function of k-means does not reward being able to MAXIMALLY SEPARATE the two classes, so instead, we should divide the actual energy of the classification, with the energy if the class labels were swapped. If the two centroids are close to each other then this will be near one. But if the centroids are far apart then this value will approach zero (I think). Another method may be to use a PCA game, in which the task is to do things that tend to maximize the variance of sensorimotor inputs to the minimum number of dimensions. 

References (Reading List) 

Object Action Complexes: http://en.wikipedia.org/wiki/Object_Action_Complex [This seems like a very promising evolvable representation of complex action that we should consider. How is it related to our notion of action molecules?] OACs are state-action-prediction triplets roughly, with a device that tries to make predictions more accurate, and tests how well it satisfies a goal, and records how well its prediction is doing. It is closely related to XCS/TGNG it seems. It is useful to know what existing concepts have inspired them, for similar concepts inspire our own molecular representations, although we did not know about STRIPS or Situation/Event calculi, but will certainly now look at these concepts because it might inform how our molecules should be organised (or not). See...


OAC Relation to Existing Concepts and Representations
OACs combine three elements:

The object (and situation)‐oriented concept of affordance
(Gibson 1950; Duchy, Warren and Kaelbling 1998; Stoytchev 2005; Fitzpatrick
et al. 2003; Sahin et al. 2007; Grupen et al. 2007, Gorniak and Roy 2007);

The representational and computational efficiency for planning and
execution monitoring (the original Frame Problem) of STRIPS rules
(Fikes and Nilsson 1971; Shahaf and Amir 2006; Shahaf, Chang and Amir 2006;
Kuipers et al. 2006; Modayil and Kuipers 2007);

The logical clarity of the situation/event calculus
(McCarthy and Hayes 1969; Kowalski and Sergot 1986; Reiter 2001; Poole 1993;
Richardson and Domingos 2006; Steedman 2002)

Lets consider each of the above influences in reverse order.

The situational and event calculi are temporal logics. I'd only once heard of these in a theory group meeting at QMUL I accidentally attended, but it did seem very interesting at the time. Didn't click until now that there could be a relation between that and our actor and game molecules. So what are these temporal logics all about? Can they help with modularity/task decomposition/action sequencing/action recombination, and search and accumulation of action adaptations? These are the critical questions to be asked and answered.

Kowalski, Robert, and Fariba Sadri. "Reconciling the event calculus with the situation calculus." The Journal of Logic Programming 31.1 (1997): 39-58.

Tuesday, 9 July 2013

Work-in-progress: Soft sensorial skin for the tentacle

HERE is a link to the work-in-progress documentation of the soft parts of the arm.
Links to textile research for smart skins to follow bit by bit.

Thursday, 4 July 2013

Essential references for Pollie's group.

Methods for establishing group cooperation 

There is an influential and well respected literature on the evolution of cooperation by economists, game theorists, evolutionary and theoretical biologists, that you should understand when working on automatic methods to promote group coherence, cooperation and happiness. The following references provide inroads into this literature. I would like to discuss these ideas with you Pollie, Rob and Simon.


1. Read Axelrod's stuff + simulations: http://en.wikipedia.org/wiki/The_Evolution_of_Cooperation
2. From the Adami Lab: Punishment catalyses the evolution of cooperation:  http://arxiv.org/abs/1210.5233

Monday, 1 July 2013

First meeting June 28th 2013.


1. Pollie, Rob and Simon: Emotional monitoring for cooperation.

2. Mark Roper: Soft Bodied /3D printed hybrid robots

3. Chrisantha Fernando/Vera Vasas/Alex Churchill : Darwinian neurodynamics in 3D printed robots and cuddly toys.

4. Chris Jack: Modification of sensorimotor contingencies

5. Boris Mitavskiy: Applying a new mathematical theorum about Monte-Carlo Tree Search/Recombination and hidden Markov decision processes to real world issues in Wales.

6. Berit Greinke: Sensorial fabrics for robotics

7. Thomas Bangert: A new theory of colour vision

8. Frida Fernando: A study on trying to woalk?wourk? [Youth ambassador]

9. Rollo Hollins: Film director. A robotic editor, the perfect cut, automatic emotion controlled go-pro videoing.

I suggest that each group initiates their own post so that we can keep comments about each topic together. I've got some references for Pollie/Rob and Simon especially about the mathematics of cooperation which will be essential.

Sunday, 30 June 2013

Greetings from TRUCE!

On behalf of the TRUCE project, welcome to the Hackademia Retreat project! TRUCE (Training and Research in Unconventional Computation in Europe) is a three-year project, generously supported by the European Commission via their Future and Emerging Technologies (FET) programme.

Our over-arching aim is to support, nurture and develop research and wider community-based outreach activities in the somewhat nebulous area of "unconventional computation" (UC). What actually qualifies as "UC" is an open question, but our project includes topics as diverse as quantum computing, biological engineering, and embodied computation. What researchers in all of these areas have in common is a desire to explore the boundaries of computing, not just in terms of the physical hardware (or, indeed, wetware), but to redefine the very notion of "computation". We do this by seeking alternatives to silicon (for example, using living cells as biological "micro-machines"), investigating new formalisms or computational models, using natural processes to guide the development of algorithms and structures, or asking how the idea of computation changes when it takes place as part of a physical "host" structure.

What we actively seek to avoid is navel-gazing; the temptation would be for us to play it safe, and to deliver a tidy set of standard activities and workshops. This conventional approach is clearly at odds with the very raison d'être of a project focussing on "unconventional computation", so we have, from the outset, sought to (in the slightly ungrammatical words of Apple) "think different".

The Hackademia Retreat is just one example of how we're trying to challenge the way in which research development and outreach can be successfully achieved. Another ongoing project involves seven acclaimed writers of short fiction, who will collaborate with scientists to develop a collection of stories and afterwords based on their shared vision of "Unconventional Computation in 2070".

Whether you've come to this blog as a casual visitor, or as a participant in the retreat, I hope you'll take a moment to comment on what you find, and to share your thoughts. Research is an inherently cooperative and collaborative enterprise, and you are welcome to join us. I hope that you find your involvement with TRUCE to be rewarding, challenging, and - most importantly - fun.

Best wishes for a successful retreat.

Prof. Martyn Amos
TRUCE Coordinator
Manchester Metropolitan University

Truce Hackademia Retreat 11th to 17th November. Introduction


The TRUCE EU project in unconventional computing has generously funded us to have a creative meeting in a secluded grand house in Wales to work on hardware based bioinspired computing projects that explore human experience in its various domains. A kind of theme is sort of emerging, and its unconventional in many respects.

This blog will describe our thoughts in the run up to the event.