See interview here
Friday 26 July 2013
Sunday 14 July 2013
Darwinian Neurodynamics (3): Accumulation of Adaptations & NEAT.
Accumulation of Adaptations
I think this represents the results we should work towards for the Living Machines 2013 exhibit.
The closest literature that tries to accumulate adaptations in the way we do is the options framework I think (and MAXQ), but they have not demonstrated transfer learning. A review of transfer learning appears here by Taylor and Stone (2009). I summarise some aspects of that article here.
Q1. What knowledge to transfer?
a. Low-level knowledge: i. <s, a,r, s'> instances, action-value function (Q), policy (pi), full task model (model), prior distributions (pri).
b. High-level knowledge: i. What subset of actions A to use in which situations. ii. partial policies or options (basically what we're doing with the archive I think), iii. rules or advice, iv. important features for learning (e.g. perceptual atoms), v. proto-value functions?, vi. shaping rewards (subgoal functions suitable for full goals, subtask definitions).
Examples of Hierarchical Transfer Learning (Including only tasks that can be done in a Jolly Jumper)
Primitive Tasks
1. Agent learns to bounce on a cushion using its knees and ankles.
2. Agent learns to swing forwards and backwards on a cushion using its knees and ankles.
3. Agent learns to rotate around the vertical axis using its knees and ankles.
Permitting variability operators a. and b. will allow new molecule types to come into existence that may be more suitable for recombination. If these operators are evolved along with a recombination operator, then molecule types that suit those operators will evolve.
Purely Sequentially Composite Tasks
1. Bounce (1) for 5 seconds then Swing (2) for 5 seconds.
2. Bounce (1) for 5 seconds then Rotate (3) for 5 seconds.
3. Swing (2) for 5 seconds then Bounce (1) for 5 seconds.
4. Swing (2) for 5 seconds then Rotate (3) for 5 seconds.
5. Rotate (3) for 5 seconds then Bounce (1) for 5 seconds.
6. Rotate (3) for 5 seconds then Swing (2) for 5 seconds.
Purely Parallel Composite Tasks
1. Bounce + Swing
2. Bounce + Rotate
3. Swing + Rotate
at the same time. Fitness is the sum of both primitive fitnesses.
Methods of Transfer
1. FIXED CHAIN the timing of sub-actions. A -(delay = 3)--> B. with fixed delays.
2. CONDITIONAL CHAIN the timing of sub-actions based on termination conditions A --(terminate)-->B.
3. MERGE two compartments with equal priority, such that all motor atoms controlling the same motors now average their motor commands in the motor buffer.
Transfer method 1 would be expected to be able to satisfy the purely sequential composite tasks, if it was possible to execute compartment (x) followed by compartment (y). Currently, our archive retrieval system does not permit combining distinct whole compartments in this way. Neither does it permit the termination condition based method of sequencing whole compartments relative to each other. This is a failing of the current EROS algorithm.
Parallel task recombination would be expected to be helped by (3) the equal priority Merge operator.
Primitive Compartment Mechanisms and Variability Operators.
1. A large DMP molecule moves knees and ankles taking sensory input from joints and accelerometers.
2. Two small reflex molecules operate ankle and knee movements as a function of FSRs on that leg.
TO DO: Currently, by limiting the molecules to the two above types, we severely restrict the kinds of knowledge transfer that can occur by recombination.
a. Structural changes should be permitted, e.g. removal and formation of edges. Removal is first order, and addition is zeroth order. This will result in the formation of new molecule structures.
b. Mutation of sensor identities should be permitted. Motor identities should also be mutable, within the subset of permitted motors i.e. knees and ankles only.
c. Priorities should also be allowed to mutate.
d. Recombination between compartments in the population (and between compartments in the archive and population should allow molecules to be added (with/without replacement).
e. These operators should be tested with the primitive tasks above.
Notes:
NEAT Stanley and Miikkulainen (2002) Evolutionary Computation 10(2):99-127
Neuroevolution of Augmented Topologies attempts to answer these questions. These have been the same questions we've been asking so we should look to see if they have solved all our problems for us. Lets hope they have, otherwise we might need to do some more work. Their questions were...
Topology andWeight Evolving Artificial Neural Networks (TWEANNs) are of two types. Direct and indirect encoding.
Direct
1. Binary connectivity matrix [crossover requires equal sized matrices, crossover has competing conventions problem], (Dasgupta and McGregor, 1992)
2. Nodes contain incoming and outgoing edge list (Pujol and Poli, 1997). [Subgraph swapping is possible] " Subgraph swapping is representative of a prevailing philosophy
Indirect
1. Gruau's cellular encoding (I programmed this fully in C during my MSc thesis I remember now!). [Biased search space, utility of bias not known.]
A1. Competing Conventions Problem (CROSSOVER OF DIFFERENT TOPOLOGY CONTROLLERS)
[EROS SUFFERS FROM THIS CURRENTLY]
The biological solution is homology (lining up of equivalent genes) prior to crossover, done by RecA. Neat uses the HISTORICAL lineage knowledge to provide direct evidence of common ancestry from which true homology can be determined, to constrain crossover [Can this really work well, because there could have been functional divergence. Is this the best that can be done, is direct structural analysis really not possible? How does RecA work?]
" Tracking the historical origins requires very little computation. Whenever a new gene appears (through structural mutation), a global innovation number is incremented and assigned to that gene. The innovation numbers thus represent a chronology of the appearance of every gene in the system."
Q(2) How can topological innovation that needs a few generations to be optimized be protected so that it does not disappear from the population prematurely?
A2. Protecting Innovation (NEW STRUCTURAL MUTANTS) with Speciation
[EROS SUFFERS FROM THIS CURRENTLY] " Frequently, adding new structure initially causes the fitness of a network to decrease" " By adding new genes to the population and sensibly mating genomes representing different structures, the system can form a population of diverse topologies. However, it turns out that such a population on its own cannot maintain topological innovations. Because smaller structures optimize faster than larger structures, and adding nodes and connections usually initially decreases the fitness of the network, recently augmented structures have little hope of surviving more than one generation even though the innovations they represent might be crucial towards solving the task in the long run. The solution is to protect innovation by speciating the population"
Adding new nodes and connections can reduce fitness suddenly before it's parameters have had a chance to optimise, so new topological additions must be protected/ This is achieved by fitness sharing based on genetic similarity [?]
Solution: Genomes are clustered on the basis of evolutionary similarity. Individuals in the same species must share fitness as in Goldberg and Richardson, 1987 (explicit fitness sharing).
NEAT ALGORITHM OVERVIEW
1. Initialise population with a set of non-random MINIMAL initial topologies to reduce the number of parameters that have to be searched at the same time. [Instead of penalising larger networks like Zhang and Muhlenbein, 1993), they start with a minimal topology and grow it carefully.]
Some parmeters from an example run below...
2. Genetic Encoding [Designed to allow homologous crossover.] Linear representations of network connectivity.
"Genomes are linear representations of network connectivity (Figure 2). Each genome includes a list of connection genes , each of which refers to two node genes being connected. Node genes provide a list of inputs, hidden nodes, and outputs that can be connected. Each connection gene specifies the in-node, the out-node, the weight of the connection, whether or not the connection gene is expressed (an enable bit), and an innovation number , which allows finding corresponding genes (as will be explained below)."
MUTATIONS:
"In the add connection mutation, a single new connection gene with a random weight is added connecting two previously unconnected nodes. In the add node mutation, an existing connection is split and the new node placed where the old connection used to be. The old connection is disabled and two new connections are added to the genome. The new connection leading into the new node receives a weight of 1, and the new connection leading out receives the same weight as the old connection. This method of adding nodes was chosen in order to minimize the initial effect of the mutation." [Vera's idea as well, in our system. But I'm not sure this way of adding nodes is the best. What is the justification for it in our system? How to add new nodes in our system in order to make smooth operators?]
IMPLICATIONS FOR EROS FROM NEAT.
1. Introduce historical markings to atoms. [We have an issue with edges which are currently encoded IN atoms, rather than as separate genes to which a separate historical marking could be assigned. Is this a problem? Consider the entities to which historical markings, and hence species definition, could be assigned, and perhaps how the definition of a species (and inter-individual distance) might itself be evolvable, because surely it is rather arbitrary how the 3 parameters that define distance is NEAT are chosen. COULD BETTER SPECIES EVOLVE OVER TIME IF THE DEFINITION OF SPECIES DISTANCE WAS ITSELF EVOLVABLE< OR NOT?
2. Initialize with a more minimal topology, e.g. only a sensor and motor atom with a linear transform.
3. [Protect innovation] Introduce speciation based on historical similarity and niching (based on markings).
4. Make the initial parameters of a new structural variant likely to be not too detrimental, e.g. [neutral addition of variation (e.g. insertion of an identity transform between sensor and motor atom). Atoms and their variation operators should be designed such that they have this property.]
5. Increase parameter mutation rate (allowed by speciation!, not otherwise, see paragraph above..
I think this represents the results we should work towards for the Living Machines 2013 exhibit.
The closest literature that tries to accumulate adaptations in the way we do is the options framework I think (and MAXQ), but they have not demonstrated transfer learning. A review of transfer learning appears here by Taylor and Stone (2009). I summarise some aspects of that article here.
Q1. What knowledge to transfer?
a. Low-level knowledge: i. <s, a,r, s'> instances, action-value function (Q), policy (pi), full task model (model), prior distributions (pri).
b. High-level knowledge: i. What subset of actions A to use in which situations. ii. partial policies or options (basically what we're doing with the archive I think), iii. rules or advice, iv. important features for learning (e.g. perceptual atoms), v. proto-value functions?, vi. shaping rewards (subgoal functions suitable for full goals, subtask definitions).
Examples of Hierarchical Transfer Learning (Including only tasks that can be done in a Jolly Jumper)
Primitive Tasks
1. Agent learns to bounce on a cushion using its knees and ankles.
2. Agent learns to swing forwards and backwards on a cushion using its knees and ankles.
3. Agent learns to rotate around the vertical axis using its knees and ankles.
Permitting variability operators a. and b. will allow new molecule types to come into existence that may be more suitable for recombination. If these operators are evolved along with a recombination operator, then molecule types that suit those operators will evolve.
Purely Sequentially Composite Tasks
1. Bounce (1) for 5 seconds then Swing (2) for 5 seconds.
2. Bounce (1) for 5 seconds then Rotate (3) for 5 seconds.
3. Swing (2) for 5 seconds then Bounce (1) for 5 seconds.
4. Swing (2) for 5 seconds then Rotate (3) for 5 seconds.
5. Rotate (3) for 5 seconds then Bounce (1) for 5 seconds.
6. Rotate (3) for 5 seconds then Swing (2) for 5 seconds.
Purely Parallel Composite Tasks
1. Bounce + Swing
2. Bounce + Rotate
3. Swing + Rotate
at the same time. Fitness is the sum of both primitive fitnesses.
Methods of Transfer
1. FIXED CHAIN the timing of sub-actions. A -(delay = 3)--> B. with fixed delays.
2. CONDITIONAL CHAIN the timing of sub-actions based on termination conditions A --(terminate)-->B.
3. MERGE two compartments with equal priority, such that all motor atoms controlling the same motors now average their motor commands in the motor buffer.
Transfer method 1 would be expected to be able to satisfy the purely sequential composite tasks, if it was possible to execute compartment (x) followed by compartment (y). Currently, our archive retrieval system does not permit combining distinct whole compartments in this way. Neither does it permit the termination condition based method of sequencing whole compartments relative to each other. This is a failing of the current EROS algorithm.
Parallel task recombination would be expected to be helped by (3) the equal priority Merge operator.
Primitive Compartment Mechanisms and Variability Operators.
1. A large DMP molecule moves knees and ankles taking sensory input from joints and accelerometers.
2. Two small reflex molecules operate ankle and knee movements as a function of FSRs on that leg.
TO DO: Currently, by limiting the molecules to the two above types, we severely restrict the kinds of knowledge transfer that can occur by recombination.
a. Structural changes should be permitted, e.g. removal and formation of edges. Removal is first order, and addition is zeroth order. This will result in the formation of new molecule structures.
b. Mutation of sensor identities should be permitted. Motor identities should also be mutable, within the subset of permitted motors i.e. knees and ankles only.
c. Priorities should also be allowed to mutate.
d. Recombination between compartments in the population (and between compartments in the archive and population should allow molecules to be added (with/without replacement).
e. These operators should be tested with the primitive tasks above.
Notes:
NEAT Stanley and Miikkulainen (2002) Evolutionary Computation 10(2):99-127
Neuroevolution of Augmented Topologies attempts to answer these questions. These have been the same questions we've been asking so we should look to see if they have solved all our problems for us. Lets hope they have, otherwise we might need to do some more work. Their questions were...
(1) Is there a genetic representation that allows disparate topologies to cross over in a meaningful way?
(2) How can topological innovation that needs a few generations to be optimized be protected so that it does not disappear from the population prematurely?
(3) How can topologies be minimized throughout evolution without the need for a specially contrived fitness function that measures complexity?
Topology andWeight Evolving Artificial Neural Networks (TWEANNs) are of two types. Direct and indirect encoding.
Direct
1. Binary connectivity matrix [crossover requires equal sized matrices, crossover has competing conventions problem], (Dasgupta and McGregor, 1992)
2. Nodes contain incoming and outgoing edge list (Pujol and Poli, 1997). [Subgraph swapping is possible] " Subgraph swapping is representative of a prevailing philosophy
in TWEANNs that subgraphs are functional units and therefore swapping them
makes sense because it preserves the structure of functional components. However, we
cannot be sure whether the particular subgraphs being combined in PDGP are the right
ones to create a functional offspring."
1. Gruau's cellular encoding (I programmed this fully in C during my MSc thesis I remember now!). [Biased search space, utility of bias not known.]
Q(1) Is there a genetic representation that allows disparate topologies to cross over in a meaningful way?
A1. Competing Conventions Problem (CROSSOVER OF DIFFERENT TOPOLOGY CONTROLLERS)
[EROS SUFFERS FROM THIS CURRENTLY]
The biological solution is homology (lining up of equivalent genes) prior to crossover, done by RecA. Neat uses the HISTORICAL lineage knowledge to provide direct evidence of common ancestry from which true homology can be determined, to constrain crossover [Can this really work well, because there could have been functional divergence. Is this the best that can be done, is direct structural analysis really not possible? How does RecA work?]
" Tracking the historical origins requires very little computation. Whenever a new gene appears (through structural mutation), a global innovation number is incremented and assigned to that gene. The innovation numbers thus represent a chronology of the appearance of every gene in the system."
" When crossing over, the genes in both genomes with the same innovation numbers are lined up. These genes are called matching genes. Genes that do not match are either disjoint or excess , depending on whether they occur within or outside the range of the other parent’s innovation numbers. They represent structure that is not present in the other genome. In composing the offspring, genes are randomly chosen from either parent at matching genes, whereas all excess or disjoint genes are always included from the more fit parent."
Q(2) How can topological innovation that needs a few generations to be optimized be protected so that it does not disappear from the population prematurely?
A2. Protecting Innovation (NEW STRUCTURAL MUTANTS) with Speciation
[EROS SUFFERS FROM THIS CURRENTLY] " Frequently, adding new structure initially causes the fitness of a network to decrease" " By adding new genes to the population and sensibly mating genomes representing different structures, the system can form a population of diverse topologies. However, it turns out that such a population on its own cannot maintain topological innovations. Because smaller structures optimize faster than larger structures, and adding nodes and connections usually initially decreases the fitness of the network, recently augmented structures have little hope of surviving more than one generation even though the innovations they represent might be crucial towards solving the task in the long run. The solution is to protect innovation by speciating the population"
Solution: Genomes are clustered on the basis of evolutionary similarity. Individuals in the same species must share fitness as in Goldberg and Richardson, 1987 (explicit fitness sharing).
NEAT ALGORITHM OVERVIEW
1. Initialise population with a set of non-random MINIMAL initial topologies to reduce the number of parameters that have to be searched at the same time. [Instead of penalising larger networks like Zhang and Muhlenbein, 1993), they start with a minimal topology and grow it carefully.]
Some parmeters from an example run below...
2. Genetic Encoding [Designed to allow homologous crossover.] Linear representations of network connectivity.
"Genomes are linear representations of network connectivity (Figure 2). Each genome includes a list of connection genes , each of which refers to two node genes being connected. Node genes provide a list of inputs, hidden nodes, and outputs that can be connected. Each connection gene specifies the in-node, the out-node, the weight of the connection, whether or not the connection gene is expressed (an enable bit), and an innovation number , which allows finding corresponding genes (as will be explained below)."
[I do not like the fact that connectivity is not a self-organizing consequence of atom structures, which makes them fully PSS like.]
"In the add connection mutation, a single new connection gene with a random weight is added connecting two previously unconnected nodes. In the add node mutation, an existing connection is split and the new node placed where the old connection used to be. The old connection is disabled and two new connections are added to the genome. The new connection leading into the new node receives a weight of 1, and the new connection leading out receives the same weight as the old connection. This method of adding nodes was chosen in order to minimize the initial effect of the mutation." [Vera's idea as well, in our system. But I'm not sure this way of adding nodes is the best. What is the justification for it in our system? How to add new nodes in our system in order to make smooth operators?]
IMPLICATIONS FOR EROS FROM NEAT.
1. Introduce historical markings to atoms. [We have an issue with edges which are currently encoded IN atoms, rather than as separate genes to which a separate historical marking could be assigned. Is this a problem? Consider the entities to which historical markings, and hence species definition, could be assigned, and perhaps how the definition of a species (and inter-individual distance) might itself be evolvable, because surely it is rather arbitrary how the 3 parameters that define distance is NEAT are chosen. COULD BETTER SPECIES EVOLVE OVER TIME IF THE DEFINITION OF SPECIES DISTANCE WAS ITSELF EVOLVABLE< OR NOT?
2. Initialize with a more minimal topology, e.g. only a sensor and motor atom with a linear transform.
3. [Protect innovation] Introduce speciation based on historical similarity and niching (based on markings).
4. Make the initial parameters of a new structural variant likely to be not too detrimental, e.g. [neutral addition of variation (e.g. insertion of an identity transform between sensor and motor atom). Atoms and their variation operators should be designed such that they have this property.]
5. Increase parameter mutation rate (allowed by speciation!, not otherwise, see paragraph above..
Darwinian Neurodynamics (2): Perceptual Unsupervised Learning Quality, as a Guide to Action.
An idea for an intrinsic motivation function for child development based on quality of unsupervised learning.
Compressibility (Schmidhuber)
Short term goal. Produce a range of python functions for Alex that take in a senosirmotor time series one timestep at a time, and returns at the end a score for how well an unsupervised learning system (of various types) did, e.g. the quality of unsupervised learning. One such measure is naturally compressibility, which is how well the data can be reproduced given an informational bottleneck, so this could be done with an autoassociator by trying to reduce the size of the hidden layer to a minimum. The greater the compressibility the deeper the learning. Deep Belief Networks could also be used for this. In fact, at a deep level, the goal of unsupervised learning is to achieve maximal compression of the data.
We've returned to Schmidhuber's compression progress as a measure of interestingness of an action. A game which is trying to compress an input vector s and is achieving greater compression rates over time, as a superior game to a game which is trying to compress input vector s' and is not improving.
How do we measure the compressibility of the sensorimotor subset we're interested in then?
Thursday 11 July 2013
Work-in-progress: exploring sensorimotor contingencies in robots
This blog shows the work-in-progress by me (Chris Jack) on building simple robots to demonstrate sensorimotor contingencies as they explore themselves and the world around them.
The 1st video below shows a robot called 'Spongebot' (aka noisiest-robot-in-the-world) that is designed to demonstrate the sensorimotor loops involved in squeezing diferent objects or tapping its fingers together. Here you see it at the stage of demonstrating a straightforward reflexive behaviour whereby the fingers retreat backward when a certain pressure of grip is sensed on the object - in this case a ball of steel or, at the end of the video, between the fingers themselves.
This second video shows a now defunct design to build a robot which could autonomously play the traditional English pub game known as Devil Amongst the Tailors or, more plainly, Bar Skittles.
The 1st video below shows a robot called 'Spongebot' (aka noisiest-robot-in-the-world) that is designed to demonstrate the sensorimotor loops involved in squeezing diferent objects or tapping its fingers together. Here you see it at the stage of demonstrating a straightforward reflexive behaviour whereby the fingers retreat backward when a certain pressure of grip is sensed on the object - in this case a ball of steel or, at the end of the video, between the fingers themselves.
This second video shows a now defunct design to build a robot which could autonomously play the traditional English pub game known as Devil Amongst the Tailors or, more plainly, Bar Skittles.
Wednesday 10 July 2013
Sub-Project Darwinian Neurodynamics: Video of Nao in the Jolly Jumper
Darwinian Neurodynamics
Video of Nao on the Jolly Jumper: Experiment 1.
Observations during the run (10th July 2013, afternoon):
1. 40 Evaluations: Two strategies have been discovered, a. Maximally Flex knees and ankles and just passively swing. b. Oscillate right knee pushing rhythmically on cushion (not clear if this uses the FSRs). There seems to be no tendency for 'circular reactions' learned on one side to be transferred to the other side. Whilst passively swinging in the air there are some knee movements that seem to be accelerometer dependent,i.e. oscillating in rhythm to the overall robot swinging.
2. 70 Evaluations: Both legs are used by in a somewhat unsynchronized fashion. Alternating pushing off. The total independence of strategies in both legs seems biologically implausible, but I have no idea how much bilateral skill transfer there is in infants. What kinds of molecule variation operator would permit bilateral skill transfer? [Copying of parameters between atoms would produce transfer, or copying of the weights in a linear transform molecule.]
3. 100 Evaluations: Perhaps (wishfully) slightly more coordinated kick-off action between the two legs, with both knees extending at the same time sometimes. Yes, I'm not imagining it, coordination between knees is definitely there.
4. 200 Evaluations: There does not seem to be any great improvement in coordination over the last 100 generations. I'm concerned that with the cushion the Nao is too close to the floor, so that there isn't much room for it to actually bounce.
5. 230 Evaluations: Discovered a left leg bouncing method which looks altogether more elegant than the half-cocked two leg strategies. Lets hope that one survives! Well it seems to have spread in the population now, and by 260 Evaluations the fitness has gone up to around 10 from less than 8 where it was around 200 evaluations, so this left foot based hopping method works much better than trying to coordinate the two legs. [in a sense this is a kind of freezing but bilateral freezing. The right leg is now stuck in a flexed position so it doesnt get in the way].
6. 290 Evaluations: It sounds like that left ankle is getting quite tired. I think this means it automatically reduces its stiffness, and this means that actor molecules that use that ankle may now no longer do so well. So in combination with a routine that reduces force when a joint is tired, this drives perhaps the evolution of other actor molecules that can use other joints. There is no need to explicitly code energy saving in the fitness function for actors if joints can become tired and work less well when overheated.
7. 300 Evaluations: Tuning of the single left leg hopping strategy is taking place. There are micro changes in timing and fixed attractor position on the left leg knee and ankle joints to match the details of the jolly jumper spring and the cushion.
8. 380 Evaluations: Not much change in strategy or fitness. The population seems to have become converged, with only minor tuning of the parameters. Fitness remains around 10.0. Perhaps this is a local optimum that a behavioural insight would be required to break out of? Perhaps for more radical changes to be possible we need to add recombinatorial complexity, i.e. add stuff from the archieve, or use variation operators that modify the topology of the system. This should definately be a next step to prevent getting stuck on these local optima. Perhaps the more complex operators could ONLY be tried during an impasse of this sort where no fitness improvement is being made.
Experiment 3: Including accelerometers in the DMG molecule + Punishment for Electric Current Used.
The fitness function is modified to divide the amount of bounciness by the sum of the electriccurrent used by the 4 active joints (knees and ankles). This is basically Thelen's notion of "bounce per ounce". It is a simple multi-objective function but we are not using any sophisticated methods to maintain diversity along a pareto front. I wonder whether explicitly punishing electric current used will produce more elegant and efficient looking behaviours?
Further ideas:
The same kind of one legged jumping behaviour arose. OK, I'm bored of that behaviour now, it was fun at the beginning of he week, now its boring. I'm quite brain dead, need to sleep and think about how to get some accumulation of adaptation, or as Alex prefers, transfer learning... Right, slept! A few information theory measures have been brought to my attention, e.g sensorimotor mutual information (SMMI) and this may be an interesting selection criterion for good games, see.
http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003111
and the preceeding paper
http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1002236#pcbi.1002236.s003
for the methods used. a Hidden Markov Network is used as the method of encoding activation logic between atoms which is interesting. They use a table of probabilities to encode the logic which is costly in terms of genome length however, but this is the most general encoding. We might wish to try it as a means of encoding the activation between atoms. It is a probabilistic encoding which may aid evolvability, i.e. you get rare paths that come into existence while the standard path works. It also bring the system closer to the path evolution algorithm here.
Probabilistic activation of atoms.
A HMN is a general encoding for determining I/O functions for multiple inputs and outputs, its like a Boolean network but with probabilities. Its a very general formulation. It might be something we want to consider implementing later to achieve probabilistic actor molecules which may be more evolvable.
Accumulation of adaptation
SMMI (Sensorimotor mutual information) as a fitness measure over games
To work this out in a continuous system, the simplest way is to artificially discrete the system by binning in time and magnitude. Thats what people do... e.g. Ay et al 2008 in a Sante Fe working paper describe how to calculate a simpler version of sensory mutual information for single channels for autonomous robots here. The mutual information between successive timesteps is equal to predictive information given a Markovian system. The authors claim that maximizing this measure results in robots that are both explorative and have predictable future events. White Gaussian noise is assumed (unrealistically). They split the sensor stream into 30 bins and calculated probabilities p(x), p(y) and p(x(t+1), x(t)) by frequencies obtained over long sampling over the run. They evaluated the MI for each of the sensor channels separately! To do this for vectors of x and y it is necessary to have multidimensional bins probably of larger sizes, and then we have the curse of dimensionality.
Perceptual interestingness (unsupervised learning rate) as a fitness measure over games [POWERPLAY]
This relates to Schmidhuber's compressibility. A good game is one that results in interesting perceptual input. If one wants to get into this line of investigation one should really read Schmidhuber's stuff.
http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0023534
http://arxiv.org/pdf/1210.8385v1.pdf
Powerplay may be of importance for task description methods. The introduction to powerplay implies a rich task description. SLIM RNN neurons internal states define the goal! These SLIM RNNs have a halting neuron. They are universal dynamical function approximators like CTRNNs, so one can think of them as CTRNNs with a threshold halt neuron I think. A new task is learned for which the network can reach the SAME internal goal state in the new environment, AND reproduce the same internal goal state in all previous environments! Its like saying, the CTRNNs 10th neuron must be above 0.5 when the CTRNN halts. Wow. How internal. If the SLIM RNN can't halt in the new environment, then random weight changes (mutations) are made.
Algorithm
The algorithm is simple. Get a new network s, and mutate it to s+1. Make a new task k. See if the mutated s can solve the new k task AND all prev. tasks (by having the same internal state at halting). If it can, then store the task k and internal state of the network s+1, into the archive.
A compression version is also used where the task is to compress the data, in which case the fitness criteria is that after mutation of the network, the internal states can be produced with smaller weights or fewer connections being used. This apparently is equivalent to a better unsupervised learning ability of the classification. But I'm not sure the classification is interesting to begin with (what is a trivial set of internal states/classifications is learned at the beginning, e.g. what if s classifies all pictures with a black pixel on the bottom right as 0 and all those without as 1. Then an infinite number of pictures could be classified thus, and the new network would evolve just to consider the pixel on the bottom right and ignore everything else. Eventually no simpler more compressed network would be possible. Without the compression criterion then new tasks could always be solved.
There is absolutely no demonstration in Schmidhuber's paper that in the fovea task the behaviours learned are subjectively interesting!!!!
http://arxiv.org/pdf/1210.8385v1.pdf
What is interesting is that he is using a hill-climbing strategy, with fitness = new task performance + old task performances (to promote accumulation of adaptation and no forgetting) + compression together as a criterion. Compression being a simplification of the network capable of solving the tasks. In our work we don't have multiple tasks yet, and we don't require a solution to be good at all previous tasks. I don't think thats a sensible solution for open-endedness, because there are too many dependencies. The solution to french shouldnt need to also do unicycling. Unicycling may be a quite different controller. I do like the idea of tasks as being defined as certain internal states of the RNN because this is a very powerful and open-ended method of specifying situated and embodied goals, BUT I think that a measure of INTERESTING internal states must be added, possibly constrained by one of the information theory measures of Tononi et al to make those internal states informationally interesting about the sensrimotor states??
Implication for EROS. Set up a game molecule containing a SLIM RNN atom (or equivalent CTRNN with fixed halting time). The game molecule gives a high fitness to the actor molecule if a subset of the neurons in the CTRNN are in a given GOAL state at the end of a fixed time trial. Choose CTRNN + GOAL state pairs (i.e. dynamical game definitions) in which PROGRESS is being made in achieving this goal state over actor generations. Do not require that a new actor molecule is able to satisfy all previous CTRNN+ GOAL pairs. Mutate CTRNNs and GOALs. Problem I see with this is that additional constraints on the fitness of CTRNNs and GOALs will be needed that prevent trivial CTRNNs and GOALs from evolving perhaps, which achieves progress in some trivial way, e.g. by classifying the pixel second from the right, and third from the right. etc...
To solve this problem another kind of game atom may be a more traditional method for unsupervised learning, with the quality of unsupervised learning being a fitness measure. e.g. the game may be a k-means classifier. Actor molecules that result in classifier improvement are selected for, i.e. a good k-means classifier is one which has the greatest rate of reduction of the cost function over the last 100 data points perhaps. For k = 2 this means, behaviour which is able to distinguish maximally between two classes of event is favoured, behaviours that are able to divide observed data into two distinct clusters is favoured. Wait, the cost-function of k-means does not reward being able to MAXIMALLY SEPARATE the two classes, so instead, we should divide the actual energy of the classification, with the energy if the class labels were swapped. If the two centroids are close to each other then this will be near one. But if the centroids are far apart then this value will approach zero (I think). Another method may be to use a PCA game, in which the task is to do things that tend to maximize the variance of sensorimotor inputs to the minimum number of dimensions.
Chrisantha Fernando and Alex Churchill are the principle investigators in the Darwinian neurodynamics subproject which develops the EROS (Evolutionary Robot Operating System) on a range of 3D printed arduino-based robot platforms and the Nao humanoid robot, including the one that Berit and Chris are making. DN algorithms may serve as methods for autonomous epistemic exploration for curiosity and creativity in a range of sub-projects during the week.
This page contains notes of cutting-edge developments in our thought about Darwinian cognitive architectures.
Video of Nao on the Jolly Jumper: Experiment 1.
The video below shows our first experiment with the Nao in the Jolly Jumper, evolving an actor molecule parameters, with the fitness being the sum of the first derivative of accelerometer values in rostro-caudal axis (up-down), i.e. the amount of bounciness. Dynamic movement primitive atoms are used to encode a range of movements. The coupling consists and the internal parameters of the DMPs are evolved using a microbial GA of pop size = 10. The whole experiment runs for about 800 evaluations, each evaluation being about 8 seconds. The fitness graph is shown.
Half way through, the cushion is removed and the Nao has to readapt to the hanging condition where its feet cannot touch the ground any longer. It successfully adapts to this new condition with a different motor strategy being used. We did not test whether it had forgotten the original solution with the cushion by returning the cushion, because I am sure it will have forgotten because the archive is not being used.
An early description of the architecture can be found here and here, but a paper is expected in Frontiers in Cognitive Science shortly with a full description. Also we're talking and giving a demo at Living Machines 2013 here, where Terrence Deacon is also talking.
The actor molecule we evolve the parameters of is shown below. We hold it to this fixed topology. The sensor and motor identities are also not allowed to mutate. The DMG parameters (coupling constants, a, b, time constants, weights and properties of the radial basis functions) are allowed to mutate, as are the weights in the linear transforms of the two foot reflexes that move the knee and ankle as a linear function of the sum of the 4 force sensors on the feet.
An early description of the architecture can be found here and here, but a paper is expected in Frontiers in Cognitive Science shortly with a full description. Also we're talking and giving a demo at Living Machines 2013 here, where Terrence Deacon is also talking.
The actor molecule we evolve the parameters of is shown below. We hold it to this fixed topology. The sensor and motor identities are also not allowed to mutate. The DMG parameters (coupling constants, a, b, time constants, weights and properties of the radial basis functions) are allowed to mutate, as are the weights in the linear transforms of the two foot reflexes that move the knee and ankle as a linear function of the sum of the 4 force sensors on the feet.
The next experiment is to try another fitness function which punishes electric current used by the robot so that it tries to maximise "bounce per ounce" as Esther Thelen puts it in this wonderful paper here, which is what inspired us to apply Darwinian neurodynamics to child development explicitly. Thelen used the latest modelling of exploration and selection dynamics available in the brain at the time, i.e. the work of Edelman on Neural Darwinism, and tended to be anti-cognitivist, but our approach tries to unify the dynamical systems and cognitivist positions, and extend the work to include the accumulation of adaptation. The above results do not show the accumulation of adaptation yet.
Compare to a child who has optimised performance in the jolly jumper here.
Experiment 2: Including accelerometers in the DMG molecule.
In the above experiment, the z accelerometer reading is not accessible in real time to the controller, but only to the game molecule for calculating the fitness of a controller. Each of the sensor atoms in the large DMG molecule now contains as its second sensor the z accelerometer [143]. A linear transform (green atom) linearly combines the two sensor values and transmits them to the motor DMG atoms (red). In this way the robot can now use the real-time z accelerometer values if they are useful for performance.
Compare to a child who has optimised performance in the jolly jumper here.
Experiment 2: Including accelerometers in the DMG molecule.
In the above experiment, the z accelerometer reading is not accessible in real time to the controller, but only to the game molecule for calculating the fitness of a controller. Each of the sensor atoms in the large DMG molecule now contains as its second sensor the z accelerometer [143]. A linear transform (green atom) linearly combines the two sensor values and transmits them to the motor DMG atoms (red). In this way the robot can now use the real-time z accelerometer values if they are useful for performance.
Observations during the run (10th July 2013, afternoon):
1. 40 Evaluations: Two strategies have been discovered, a. Maximally Flex knees and ankles and just passively swing. b. Oscillate right knee pushing rhythmically on cushion (not clear if this uses the FSRs). There seems to be no tendency for 'circular reactions' learned on one side to be transferred to the other side. Whilst passively swinging in the air there are some knee movements that seem to be accelerometer dependent,i.e. oscillating in rhythm to the overall robot swinging.
2. 70 Evaluations: Both legs are used by in a somewhat unsynchronized fashion. Alternating pushing off. The total independence of strategies in both legs seems biologically implausible, but I have no idea how much bilateral skill transfer there is in infants. What kinds of molecule variation operator would permit bilateral skill transfer? [Copying of parameters between atoms would produce transfer, or copying of the weights in a linear transform molecule.]
3. 100 Evaluations: Perhaps (wishfully) slightly more coordinated kick-off action between the two legs, with both knees extending at the same time sometimes. Yes, I'm not imagining it, coordination between knees is definitely there.
4. 200 Evaluations: There does not seem to be any great improvement in coordination over the last 100 generations. I'm concerned that with the cushion the Nao is too close to the floor, so that there isn't much room for it to actually bounce.
5. 230 Evaluations: Discovered a left leg bouncing method which looks altogether more elegant than the half-cocked two leg strategies. Lets hope that one survives! Well it seems to have spread in the population now, and by 260 Evaluations the fitness has gone up to around 10 from less than 8 where it was around 200 evaluations, so this left foot based hopping method works much better than trying to coordinate the two legs. [in a sense this is a kind of freezing but bilateral freezing. The right leg is now stuck in a flexed position so it doesnt get in the way].
6. 290 Evaluations: It sounds like that left ankle is getting quite tired. I think this means it automatically reduces its stiffness, and this means that actor molecules that use that ankle may now no longer do so well. So in combination with a routine that reduces force when a joint is tired, this drives perhaps the evolution of other actor molecules that can use other joints. There is no need to explicitly code energy saving in the fitness function for actors if joints can become tired and work less well when overheated.
7. 300 Evaluations: Tuning of the single left leg hopping strategy is taking place. There are micro changes in timing and fixed attractor position on the left leg knee and ankle joints to match the details of the jolly jumper spring and the cushion.
8. 380 Evaluations: Not much change in strategy or fitness. The population seems to have become converged, with only minor tuning of the parameters. Fitness remains around 10.0. Perhaps this is a local optimum that a behavioural insight would be required to break out of? Perhaps for more radical changes to be possible we need to add recombinatorial complexity, i.e. add stuff from the archieve, or use variation operators that modify the topology of the system. This should definately be a next step to prevent getting stuck on these local optima. Perhaps the more complex operators could ONLY be tried during an impasse of this sort where no fitness improvement is being made.
Experiment 3: Including accelerometers in the DMG molecule + Punishment for Electric Current Used.
The fitness function is modified to divide the amount of bounciness by the sum of the electriccurrent used by the 4 active joints (knees and ankles). This is basically Thelen's notion of "bounce per ounce". It is a simple multi-objective function but we are not using any sophisticated methods to maintain diversity along a pareto front. I wonder whether explicitly punishing electric current used will produce more elegant and efficient looking behaviours?
Further ideas:
The same kind of one legged jumping behaviour arose. OK, I'm bored of that behaviour now, it was fun at the beginning of he week, now its boring. I'm quite brain dead, need to sleep and think about how to get some accumulation of adaptation, or as Alex prefers, transfer learning... Right, slept! A few information theory measures have been brought to my attention, e.g sensorimotor mutual information (SMMI) and this may be an interesting selection criterion for good games, see.
http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003111
and the preceeding paper
http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1002236#pcbi.1002236.s003
for the methods used. a Hidden Markov Network is used as the method of encoding activation logic between atoms which is interesting. They use a table of probabilities to encode the logic which is costly in terms of genome length however, but this is the most general encoding. We might wish to try it as a means of encoding the activation between atoms. It is a probabilistic encoding which may aid evolvability, i.e. you get rare paths that come into existence while the standard path works. It also bring the system closer to the path evolution algorithm here.
Probabilistic activation of atoms.
A HMN is a general encoding for determining I/O functions for multiple inputs and outputs, its like a Boolean network but with probabilities. Its a very general formulation. It might be something we want to consider implementing later to achieve probabilistic actor molecules which may be more evolvable.
Accumulation of adaptation
SMMI (Sensorimotor mutual information) as a fitness measure over games
To work this out in a continuous system, the simplest way is to artificially discrete the system by binning in time and magnitude. Thats what people do... e.g. Ay et al 2008 in a Sante Fe working paper describe how to calculate a simpler version of sensory mutual information for single channels for autonomous robots here. The mutual information between successive timesteps is equal to predictive information given a Markovian system. The authors claim that maximizing this measure results in robots that are both explorative and have predictable future events. White Gaussian noise is assumed (unrealistically). They split the sensor stream into 30 bins and calculated probabilities p(x), p(y) and p(x(t+1), x(t)) by frequencies obtained over long sampling over the run. They evaluated the MI for each of the sensor channels separately! To do this for vectors of x and y it is necessary to have multidimensional bins probably of larger sizes, and then we have the curse of dimensionality.
Perceptual interestingness (unsupervised learning rate) as a fitness measure over games [POWERPLAY]
This relates to Schmidhuber's compressibility. A good game is one that results in interesting perceptual input. If one wants to get into this line of investigation one should really read Schmidhuber's stuff.
http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0023534
http://arxiv.org/pdf/1210.8385v1.pdf
Powerplay may be of importance for task description methods. The introduction to powerplay implies a rich task description. SLIM RNN neurons internal states define the goal! These SLIM RNNs have a halting neuron. They are universal dynamical function approximators like CTRNNs, so one can think of them as CTRNNs with a threshold halt neuron I think. A new task is learned for which the network can reach the SAME internal goal state in the new environment, AND reproduce the same internal goal state in all previous environments! Its like saying, the CTRNNs 10th neuron must be above 0.5 when the CTRNN halts. Wow. How internal. If the SLIM RNN can't halt in the new environment, then random weight changes (mutations) are made.
Algorithm
The algorithm is simple. Get a new network s, and mutate it to s+1. Make a new task k. See if the mutated s can solve the new k task AND all prev. tasks (by having the same internal state at halting). If it can, then store the task k and internal state of the network s+1, into the archive.
A compression version is also used where the task is to compress the data, in which case the fitness criteria is that after mutation of the network, the internal states can be produced with smaller weights or fewer connections being used. This apparently is equivalent to a better unsupervised learning ability of the classification. But I'm not sure the classification is interesting to begin with (what is a trivial set of internal states/classifications is learned at the beginning, e.g. what if s classifies all pictures with a black pixel on the bottom right as 0 and all those without as 1. Then an infinite number of pictures could be classified thus, and the new network would evolve just to consider the pixel on the bottom right and ignore everything else. Eventually no simpler more compressed network would be possible. Without the compression criterion then new tasks could always be solved.
There is absolutely no demonstration in Schmidhuber's paper that in the fovea task the behaviours learned are subjectively interesting!!!!
http://arxiv.org/pdf/1210.8385v1.pdf
What is interesting is that he is using a hill-climbing strategy, with fitness = new task performance + old task performances (to promote accumulation of adaptation and no forgetting) + compression together as a criterion. Compression being a simplification of the network capable of solving the tasks. In our work we don't have multiple tasks yet, and we don't require a solution to be good at all previous tasks. I don't think thats a sensible solution for open-endedness, because there are too many dependencies. The solution to french shouldnt need to also do unicycling. Unicycling may be a quite different controller. I do like the idea of tasks as being defined as certain internal states of the RNN because this is a very powerful and open-ended method of specifying situated and embodied goals, BUT I think that a measure of INTERESTING internal states must be added, possibly constrained by one of the information theory measures of Tononi et al to make those internal states informationally interesting about the sensrimotor states??
Implication for EROS. Set up a game molecule containing a SLIM RNN atom (or equivalent CTRNN with fixed halting time). The game molecule gives a high fitness to the actor molecule if a subset of the neurons in the CTRNN are in a given GOAL state at the end of a fixed time trial. Choose CTRNN + GOAL state pairs (i.e. dynamical game definitions) in which PROGRESS is being made in achieving this goal state over actor generations. Do not require that a new actor molecule is able to satisfy all previous CTRNN+ GOAL pairs. Mutate CTRNNs and GOALs. Problem I see with this is that additional constraints on the fitness of CTRNNs and GOALs will be needed that prevent trivial CTRNNs and GOALs from evolving perhaps, which achieves progress in some trivial way, e.g. by classifying the pixel second from the right, and third from the right. etc...
To solve this problem another kind of game atom may be a more traditional method for unsupervised learning, with the quality of unsupervised learning being a fitness measure. e.g. the game may be a k-means classifier. Actor molecules that result in classifier improvement are selected for, i.e. a good k-means classifier is one which has the greatest rate of reduction of the cost function over the last 100 data points perhaps. For k = 2 this means, behaviour which is able to distinguish maximally between two classes of event is favoured, behaviours that are able to divide observed data into two distinct clusters is favoured. Wait, the cost-function of k-means does not reward being able to MAXIMALLY SEPARATE the two classes, so instead, we should divide the actual energy of the classification, with the energy if the class labels were swapped. If the two centroids are close to each other then this will be near one. But if the centroids are far apart then this value will approach zero (I think). Another method may be to use a PCA game, in which the task is to do things that tend to maximize the variance of sensorimotor inputs to the minimum number of dimensions.
References (Reading List)
Object Action Complexes: http://en.wikipedia.org/wiki/Object_Action_Complex [This seems like a very promising evolvable representation of complex action that we should consider. How is it related to our notion of action molecules?] OACs are state-action-prediction triplets roughly, with a device that tries to make predictions more accurate, and tests how well it satisfies a goal, and records how well its prediction is doing. It is closely related to XCS/TGNG it seems. It is useful to know what existing concepts have inspired them, for similar concepts inspire our own molecular representations, although we did not know about STRIPS or Situation/Event calculi, but will certainly now look at these concepts because it might inform how our molecules should be organised (or not). See...
http://mirror.umd.edu/roswiki/attachments/Events(2f)CoTeSys(2d)ROS(2d)School/tamim.pdf
Lets consider each of the above influences in reverse order.
The situational and event calculi are temporal logics. I'd only once heard of these in a theory group meeting at QMUL I accidentally attended, but it did seem very interesting at the time. Didn't click until now that there could be a relation between that and our actor and game molecules. So what are these temporal logics all about? Can they help with modularity/task decomposition/action sequencing/action recombination, and search and accumulation of action adaptations? These are the critical questions to be asked and answered.
Kowalski, Robert, and Fariba Sadri. "Reconciling the event calculus with the situation calculus." The Journal of Logic Programming 31.1 (1997): 39-58.
http://mirror.umd.edu/roswiki/attachments/Events(2f)CoTeSys(2d)ROS(2d)School/tamim.pdf
OAC Relation to Existing Concepts and Representations
OACs combine three elements:
The object (and situation)‐oriented concept of affordance
(Gibson 1950; Duchy, Warren and Kaelbling 1998; Stoytchev 2005; Fitzpatrick
et al. 2003; Sahin et al. 2007; Grupen et al. 2007, Gorniak and Roy 2007);
The representational and computational efficiency for planning and
execution monitoring (the original Frame Problem) of STRIPS rules
(Fikes and Nilsson 1971; Shahaf and Amir 2006; Shahaf, Chang and Amir 2006;
Kuipers et al. 2006; Modayil and Kuipers 2007);
The logical clarity of the situation/event calculus
(McCarthy and Hayes 1969; Kowalski and Sergot 1986; Reiter 2001; Poole 1993;
Richardson and Domingos 2006; Steedman 2002)
The situational and event calculi are temporal logics. I'd only once heard of these in a theory group meeting at QMUL I accidentally attended, but it did seem very interesting at the time. Didn't click until now that there could be a relation between that and our actor and game molecules. So what are these temporal logics all about? Can they help with modularity/task decomposition/action sequencing/action recombination, and search and accumulation of action adaptations? These are the critical questions to be asked and answered.
Kowalski, Robert, and Fariba Sadri. "Reconciling the event calculus with the situation calculus." The Journal of Logic Programming 31.1 (1997): 39-58.
Tuesday 9 July 2013
Work-in-progress: Soft sensorial skin for the tentacle
HERE is a link to the work-in-progress documentation of the soft parts of the arm.
Links to textile research for smart skins to follow bit by bit.
Links to textile research for smart skins to follow bit by bit.
Thursday 4 July 2013
Essential references for Pollie's group.
Methods for establishing group cooperation
There is an influential and well respected literature on the evolution of cooperation by economists, game theorists, evolutionary and theoretical biologists, that you should understand when working on automatic methods to promote group coherence, cooperation and happiness. The following references provide inroads into this literature. I would like to discuss these ideas with you Pollie, Rob and Simon.
References
1. Read Axelrod's stuff + simulations: http://en.wikipedia.org/wiki/The_Evolution_of_Cooperation
2. From the Adami Lab: Punishment catalyses the evolution of cooperation: http://arxiv.org/abs/1210.5233
There is an influential and well respected literature on the evolution of cooperation by economists, game theorists, evolutionary and theoretical biologists, that you should understand when working on automatic methods to promote group coherence, cooperation and happiness. The following references provide inroads into this literature. I would like to discuss these ideas with you Pollie, Rob and Simon.
References
1. Read Axelrod's stuff + simulations: http://en.wikipedia.org/wiki/The_Evolution_of_Cooperation
2. From the Adami Lab: Punishment catalyses the evolution of cooperation: http://arxiv.org/abs/1210.5233
Monday 1 July 2013
First meeting June 28th 2013.
Projects
1. Pollie, Rob and Simon: Emotional monitoring for cooperation.
2. Mark Roper: Soft Bodied /3D printed hybrid robots
3. Chrisantha Fernando/Vera Vasas/Alex Churchill : Darwinian neurodynamics in 3D printed robots and cuddly toys.
4. Chris Jack: Modification of sensorimotor contingencies
5. Boris Mitavskiy: Applying a new mathematical theorum about Monte-Carlo Tree Search/Recombination and hidden Markov decision processes to real world issues in Wales.
6. Berit Greinke: Sensorial fabrics for robotics
7. Thomas Bangert: A new theory of colour vision
8. Frida Fernando: A study on trying to woalk?wourk? [Youth ambassador]
9. Rollo Hollins: Film director. A robotic editor, the perfect cut, automatic emotion controlled go-pro videoing.
I suggest that each group initiates their own post so that we can keep comments about each topic together. I've got some references for Pollie/Rob and Simon especially about the mathematics of cooperation which will be essential.
1. Pollie, Rob and Simon: Emotional monitoring for cooperation.
2. Mark Roper: Soft Bodied /3D printed hybrid robots
3. Chrisantha Fernando/Vera Vasas/Alex Churchill : Darwinian neurodynamics in 3D printed robots and cuddly toys.
4. Chris Jack: Modification of sensorimotor contingencies
5. Boris Mitavskiy: Applying a new mathematical theorum about Monte-Carlo Tree Search/Recombination and hidden Markov decision processes to real world issues in Wales.
6. Berit Greinke: Sensorial fabrics for robotics
7. Thomas Bangert: A new theory of colour vision
8. Frida Fernando: A study on trying to woalk?wourk? [Youth ambassador]
9. Rollo Hollins: Film director. A robotic editor, the perfect cut, automatic emotion controlled go-pro videoing.
I suggest that each group initiates their own post so that we can keep comments about each topic together. I've got some references for Pollie/Rob and Simon especially about the mathematics of cooperation which will be essential.
Subscribe to:
Posts (Atom)