Genetic Evolution of a Neural Network Driven Robot

@thrandell @inq Is it possible to put 4 of those on a chassis? I don't know if that gives 360 degrees of coverage, my quick read of the data didn't disclose that. In any case, is it possible to combine the outputs of 4 or more units in a way that makes sense?

Posted by: @zander

↑

I don't know if that gives 360 degrees of coverage, my quick read of the data didn't disclose that. In any case, is it possible to combine the outputs of 4 or more units in a way that makes sense?

These particular sensors have a 15 deg. field of view, so no you won’t get a 360 deg view.  I don’t understand what you mean by combine the output of four units.   For what purpose??  Maybe you should re-read the the product description less quickly.

Tom

Posted by: @zander

↑

@thrandell @inq Is it possible to put 4 of those on a chassis? I don't know if that gives 360 degrees of coverage, my quick read of the data didn't disclose that. In any case, is it possible to combine the outputs of 4 or more units in a way that makes sense?

For my robot, I'm going with the concept that I don't have eyes in the back of my head.  Although my Mom did, I didn't acquire that genome trait. 😉 I'm assuming you have a more 360° application...

I don't use Tom's particular brand, but the ones I use that are single beam (VL53L0X) that seem similar in concept.  They have FoV of 25°.  The 8x8, I use (VL53LCX) has a 45° FoV.  I believe there is even a wide angle version of it with higher.  But to answer your questions, what is your familiarity with I2C?  It takes 2 pins to access data on a bus.  You can have many sensors on the same bus as long as they have different addresses.  Strictly speaking there is an upper limit.  Like a LAN network.  The units I use have programmable addresses so... yes you could put more than one on the same bus and in software query their reading in a round-robin technique.  

@inq Good, understood re I2C.

Posted by: @thrandell

↑

What I’ve done so far was more of a “proof of concept” for me.  My immediate plans are to put together a purpose built robot and not cannibalize one of my swarm bots.  It will be about the same size but without a battery, battery charger, bumper, LEDs, buzzer and voltage meter.  It will have about 8 IR Lidar distance sensors, 2 photo transistors on top, a reflectance sensor on the bottom, a SD card breakout to log data and one free GPIO pin.  Same motors, driver, wheel encoders and MCU from my pile of swarm bot parts…

I've studied the first 3 chapters of Goldberg until I've got "a picture".  The one aspect that I'm not sure if I getting is the genome size.  As I think I understand it, the genome must have enough non-linear capability to handle the real world situations.  Unfortunately, the Goldberg only uses 5 bits so they can illustrate without having to show reams of numbers.  I get that.  The ACAA paper with it's 8 sensors, 2 motor has blatantly/glaringly avoided mention of their genome size. 

In the Back Propagation (BP) method, there are the weights and biases that make up the Genome.  With this plethora of interactions, along with the sigmoid function, there are plenty of opportunities for the solution to mimic the non-linear world.  The number of these is defined by the number of inputs, outputs and some nebulous number of hidden layers.  In example and depending on how many and how large their hidden layers are, the genome for the ACAA bot's Genome might be:

• Weights from input sensors to hidden layer (8 x 5) = 40
• Hidden layer biases = 5
• Hidden layer to output (5 x 2) = 10
• Hidden layer biases = 2
• Total = 57 * 4 bytes = 228 bytes.

Although GA is generalized and should work at any size genome, I'm wondering is there some recommended strategy for designing a GA genome or is simply evolving the same 228 byte BP Genome using GA techniques instead of BP techniques?

Or cutting to the chase, how large do you expect your upcoming bot's genome to be... ballpark?  

Posted by: @inq

↑

Or cutting to the chase, how large do you expect your upcoming bot's genome to be... ballpark?

Sorry about my slow response.  I noticed that the first page in this topic is trashed so I’m going to re-post some stuff from that page.

This is the neural network that I believe Floreano and Mondada used.  The two nodes that I labeled context should be considered hidden nodes.  

The first part of the answer to your question is how many neural network parameters did I use:

8 sensor weights  * 2 output nodes       = 16

+ 2 context weights * 2 output nodes   =  4

+ 2 output bias weights                              = 2

  Total                                                                 22 parameters for the neural network

The second part of the answer to your question is how are those parameters encoded?  From what I’ve read, the authors started their experiment by encoding each parameter as a floating point number.  When a parameter was mutated by the GA they added something like 0.02, BUT they also tried a binary encoding and got the same result in 25% fewer generations!  So that’s what I’m using, 4-bits per parameter for a total length of 88 bits.  I decode the 4-bit parameters into a real number in the range [-0.5,0.5] before I pass to through the neural network.  If you’re still curious I tried to explain this to @robotbuilder on page 2 of this topic.

Tom

Posted by: @thrandell

↑

Sorry about my slow response.  I noticed that the first page in this topic is trashed so I’m going to re-post some stuff from that page.

That bug is really bizarre.  I wonder if it is something that can be fixed, even if Bill was a WordPress guru.  He can't be expected to be an expert with everything.  😉 

Posted by: @thrandell

↑

This is the neural network that I believe Floreano and Mondada used.  The two nodes that I labeled context should be considered hidden nodes.  

When you first posted this, I thought it was more a conceptual picture than an actual model of the NN.  I understand that the two Context nodes are the velocities of the motors from the previous time slice.  Being hidden nodes is more literal in this context, but they're really just the same as two more inputs that are run through the calculation.  In the Back Propagation method, hidden layers are just intermediate outputs from a matrix to then be used as inputs to another matrix.  In that light, this picture indicates there isn't any hidden (BP) layers in the calculation.

Posted by: @thrandell

↑

If you’re still curious I tried to explain this to @robotbuilder on page 2 of this topic.

Posted by: @thrandell

↑

I’ve posted the answers to some of your questions elsewhere in this topic which makes me think that you might try some memory aids.

Oh!  I'm way curious.  I now remember this.  Apologies for using you as a memory tool.  I think I was still at a stage where my real neural net glazed over when I saw this data dump.  I have more questions, but let me first re-read your thread and see if you've already answered them.   This time around this data dump and posting makes complete sense.

My Status... and why I might have some questions.

I believe I have a C++ Genetic Algorithm working.  I finally just did it from the Goldberg's explanation instead of trying to monkey-see, monkey-do translating his Pascal code.  It's really quite simple once I got into it.  The leap came when I realized that the calculation from input to the output fitness score is really outside of the GA.  Obviously, his example was simply y = x^10.

The problem is... the only real-world test set I have that I can see if my algorithm is working is this digit recognition database and using the forward propagation for the calculation.  With this problem, all the weights matrices are the Genome.  Unfortunately, this makes the Genome string... 12,704 floats or 50,816 bytes.  I'm not getting any convergence even after a 1000 generations, but I can't tell if its a bug.  After working with it a while, I think its because the Genome is so large.  Using the population of 80, the differences in the fitness values are minute and so when using the Roulette wheel algorithm, the strongest individual doesn't really exert any advantage over the next generation.  So, in essence, it's just generates another random set of values in the children.  

@thrandell 

My question is: Are you able to provide the detail and results of your experiments?

I think @inq is a little bit too optimistic about what any of these networks can do. I don't see them reflecting what I know about biological brains.

The context neurons appear to be a requirement, making it a recurrent neural network. It is coding time events not just spatial events. This is a requirement for the chatGPT networks that have to predict the next word based on all that went before it.

https://www.kaggle.com/datasets/uciml/wall-following-robot

"The wall-following task and data gathering were designed to test the hypothesis that this apparently simple navigation task is indeed a non-linearly separable classification task. Thus, linear classifiers, such as the Perceptron network, are not able to learn the task and command the robot around the room without collisions. Nonlinear neural classifiers, such as the MLP network, are able to learn the task and command the robot successfully without collisions.

If some kind of short-term memory mechanism is provided to the neural classifiers, their performances are improved in general. For example, if past inputs are provided together with current sensor readings, even the Perceptron becomes able to learn the task and command the robot successfully. If a recurrent neural network, such as the Elman network, is used to learn the task, the resulting dynamical classifier is able to learn the task using less hidden neurons than the MLP network."

Posted by: @robotbuilder

↑

 

My question is: Are you able to provide the detail and results of your experiments?

Nope, I had no way of recording the data.  My new robot will have a Micro SD card reader.

Posted by: @robotbuilder

↑

The context neurons appear to be a requirement, making it a recurrent neural network. It is coding time events not just spatial events. This is a requirement for the chatGPT networks that have to predict the next word based on all that went before it.

I know I mentioned this before, but Elman wrote about word order in his paper on the properties of context neurons in recurrent networks.

Elman J.L. (1990)   Finding structure in time. Cognitive Science, 14:179-211.

Posted by: @robotbuilder

↑

"The wall-following task and data gathering were designed to test the hypothesis that this apparently simple navigation task is indeed a non-linearly separable classification task. Thus, linear classifiers, such as the Perceptron network, are not able to learn the task and command the robot around the room without collisions. Nonlinear neural classifiers, such as the MLP network, are able to learn the task and command the robot successfully without collisions.

If some kind of short-term memory mechanism is provided to the neural classifiers, their performances are improved in general. For example, if past inputs are provided together with current sensor readings, even the Perceptron becomes able to learn the task and command the robot successfully. If a recurrent neural network, such as the Elman network, is used to learn the task, the resulting dynamical classifier is able to learn the task using less hidden neurons than the MLP network."

Let’s see: linearly separable classification problems, recurrent networks, context neurons, dynamical systems.  These are all topics I’ve been reading about in Floreano’s tome:

Floreano, D. and Mattiussi, C. (2008) Bio-Inspired Artificial Intelligence: Theories, Methods, and Technologies.  MIT Press, Cambridge, MA.

Good post @robotbuilder, you've got this stuff.

Tom

@thrandell 

One thought is you might have a LED on the robot's head and use a video camera to record its path. It is possible interesting path patterns are there that aren't obvious from just watching it move about.

I have had the gist of all this stuff a long time ago in my twenties but could only spend some spare time reading about it when not living in the real world. You and @inq have an interest in theoretical research on these things whereas I am limited to occasionally reading about what other people are doing.

I found this link interesting.

https://archive.org/details/2008BioInspiredArtificialIntelligenceTheoriesMethodsAndTechnologiesDarioFloreano/page/n277/mode/1up?view=theater

Around page 461 it references Floreano and Mondada work in 1994

I noticed they make reference to the work of Ashby on page 264 the author of the book "An Introduction to Cybernetics" I mentioned as the source that gave me the means to think about these things. Another little book that gave me insight into evolutionary mechanisms was "Chance and Necessity" by Jacques Monod.

The link above gets into robotics in chapter 6.

On page 442 it references the use of vision based homing that interested me and which is used by some insects.

On page 447 the topic is central pattern generators which have evolved in biological systems for things like locomotion control and which I first read about in another insightful book "Life's Other Secret: New Mathematics of the Living World" by Ian Stewart.

Posted by: @robotbuilder

↑

I think @inq is a little bit too optimistic about what any of these networks can do. I don't see them reflecting what I know about biological brains.

You may be right that I'm expecting (assuming I'll get there) that ANN can do the things I would like to accomplish.  Although my knowledge on AI/ANN directly is limited, my background focused on numerical analysis using the Finite Element Method (FEM).  That field is well established and is based on discretizing any structure into small pieces "finite elements".  The differential equations of these tiny cubes in any material (steel, aluminum, carbon composites, water, air, the sun, earth tectonics, world atmosphere weather prediction, etc) are well characterized.  These are all accumulated into a huge matrix that is then solved and answers spit out.  Computational Fluid Dynamics (CFD) solves modeling of air or water flow using similar discretizing and matrices.  The point being... highly non-linear behavior can be modeled effectively, especially when doing it incrementally in finite time slices.  I did these types of analyses for many years.  The fact that AI/ANN and especially using the Back Propagation method look nearly identical to FEM under the covers, leads me to this optimistic view... especially since we are also talking about small increments in time.  So although the calculations are linear, they can approximate non-linear behavior successfully with small enough time slices. 

So my optimism stems from the knowledge if the AI weights matrix is large enough (or multi-tiered using multiple hidden layers) it should mimic extensive non-linear behavior with a decent enough probability of being able to accomplish a task.  The fact that the ACAA experiments and @thrandell's current bot is able to successfully do obstacle avoidance with a tiny, one-dimensional "matrix" must surely highlight the potential capabilities of more complex configurations.  

What is new to me is the method of deriving these weight matrices (solving them).  The Genetic Algorithm stuff is completely new to me and there is no corollary in FEM/CFD.  But I do find it interesting and worth playing with even if I don't come out with a practical thing. 

ROBOT HISTORY

I'd like you to elaborate on why you believe a wall following bot requires knowing its past time slices.  I'm not sure I understand this supposition.  I see driving the bot till it comes near a wall.  It has a decision to make whether to turn left or right.  Once that decision is made, I see the current incoming sensor data as being complete enough to follow the wall.  With the multiple distance sensors, they should see it (say to the right), its distance and the next distance sensor in the bot's array (around the bot or the spinning flavor) sees the wall further away.  In a completely hard-coded bot, the current data is enough, so I'm not seeing the aspect, that the bot's AI should need to know anything about previous time slices inputs. 

And even if some task does require hind-sight, I don't see why that can't be incorporated into an AI, matrix.  As illustrated, the ACAA bot does this already be re-using velocity of the previous slice in the current calculations.   

Posted by: @robotbuilder

↑

I found this link interesting.

https://archive.org/details/2008BioInspiredArtificialIntelligenceTheoriesMethodsAndTechnologiesDarioFloreano/page/n277/mode/1up?view=theater

Thank... another something to read.  🙄 

After way too many interruptions I finally finished a small external power robot to use in this experiment.  I wanted to post a picture but i wasn’t sure if WordPress had had its breakfast today so I decided not to bother with it.  It’s circular with differential drive and is about 4” in diameter and 3.5” tall.

I’m using new (for me) distance sensors, a micro SD breakout and a metal gear motor with an odd number of gears so I’ve had to tweak my first Genetic Algorithm program to make sure everything works as expected.

I wasn’t sure what to log, so for starters I decided on run level and generation level statistics.  At the run level I’m grabbing the population size, number of generations, chromosome length, cross-over probability, and mutation probability.  At the generation level I’m logging max fitness, min fitness, average fitness, the three components of the max fitness value (speed, straightness, avoidance), the neural network parameters of the fittest individual and the Hamming distance, which is a measure of population diversity. https://en.wikipedia.org/wiki/Hamming_distance

Should be interesting…

Tom

So @robotbuilder, I need some help brainstorming.

I’m at a point where I want to do some analysis on the activations of the neural network when the robot moves around in the environment to understand what’s going on with the evolved controller.  I can get at things like position and pose from the robot’s wheel encoders but there’s always the problem of drift with odometry…  Besides the wheel encoders I’ve got a micro SD card, 8 IR proximity sensors and a few analog GPIO pins available.  I remember you proposed using a video camera to track behavior, but how can I correlate that to the neural activations?  Any other ideas?

Here's a picture of my workhorse.  I've already worn the tread off the little tires.

Thanks Tom

@thrandell 

I think @inq is the man to address on evolving neural networks as I don't know how I would evolve and use one in any practical robot project. He also seems to have a good grasp on the mathematical techniques required. I am just a casual observer.

Essentially I see your robot as a reflex, single task, machine. It acts on the current input (which may include previous outputs) but it doesn't have any overall goals or planning levels. If I remember correctly you want to evolve a set of reflexes (i/o associations) that enable the robot to avoid hitting objects (walls?) while moving around as much as possible?

I am not sure what you mean by "track behavior". Maybe you mean an overhead camera that records the pose of the robot as it moves about?

I guess you could associate the current pose of the robot as seen by an overhead camera with the other sensor inputs like the data from time of flight (TOF) sensors?

@inq has his robot following a wall but it is not doing it with purpose it is just a complex reflex. He intends I think to give it a purpose and that is to map everything as it runs about.