Genetic Evolution of a Neural Network Driven Robot

Posted by: @thrandell

↑

As far as connecting it all together I connect the external power from the wall wart to a slip ring I picked up at Adafruit and some soft 30AWG wire down to the robot…Works great in my little 3’ by 2’ arena.

Did I go off the deep end on this one?

"Deep end"... No, not at all.  I did remember you had both capabilities and banged my head on the 30+ hour reminder.  I'll be using considerably different stuff, but about the same size bot.  I don't have a good feel for tiny wire gauges power carrying ability over it's length.  I'm guessing your about 3' or 4' on the 30AWG for that size arena?  I'll be using steppers/A4988 for driving force and if I have a buck converter, I'll run like 30VDC down.  @davee reminded me of that trick in another thread.

Thanks.

Posted by: @inq

↑

I'm guessing your about 3' or 4' on the 30AWG for that size arena?

Yep

@inq OK, I didn't realize the motors were involved. I was thinking that would be a hella lot of processing.

🤣 

@thrandell,

Based on your hardware motor/voltage/wheel diameter, do you know what your bot's top speed is?  Not that I'm in a race... Since the learn time is more dependent on the robot motoring speed than CPU speed, and your above 30 hours, I wondering beforehand if my bot is going to be fast enough or will it take weeks to learn.  

Posted by: @inq

↑

Since the learn time is more dependent on the robot motoring speed than CPU speed, and your above 30 hours, I wondering beforehand if my bot is going to be fast enough or will it take weeks to learn. 

I haven’t measured this robot’s top speed, but it’s probably in the 0.19 to 0.22 meters/second range, but that does not influence the time it takes to run the test.

The time it takes to run their experiment depends on these parameters:

1) size of the population
2) number of generations
3) number of actions
4) duration of each action
5) duration of the repositioning strategy

For example, my 33 hour run has these parameter

1) pop size = 80
2) generations = 50
3) actions = 80
4) action time = 300ms
5) reset position time = 5000ms

Here’s the math:

• 80 X 300ms = 24 seconds; for one individual to run its parameters (chromosome) 80 times
• 5 seconds; to reset the position of the robot in preparation for the next individual
• 24 + 5 = 29 seconds; total time for one individual to run with its parameters and then reset its position
• 80 X 29 = 2,320 seconds or 38.6 minutes; time for one generation of 80 individuals to run their parameters
• 50 * 38.6 = 1,930 minutes or 32.1 hours; execution time for all individuals to run for 50 generations

So this doesn’t include any time for doing selection, crossover, mutation, fitness testing , …

Tom

@thrandell,

Reset Position Time - Did you use the same methodology?

It sounds like ACAA, p.7,

Between one individual and the next, a pair of random velocities was applied to the wheels for 5 seconds. This procedure was aimed at limiting the artifactual inheritance of particular locations between adjacent individuals in the populations.

I'm have troubles seeing how this is fair... it would seem like some chromosomes that would otherwise be great, get punted because they might have their head stuck in the corner.

VBR,

Inq

@thrandell,

Fitness Forumula

I'm also having difficulties with the Fitness formula concept.  I understand the need.  You have to have something equivalent to the evolutionary struggle (Darwin Awards) to kill off the stupid and weak.  But in ACAA, I'm the equation seems like cheating.   

 For instance, this function could just as well be used.  I don't know if it is better or worse, but it basically says the same thing about one being better than another.

Φ = V + (1- sqrt(Δv)) + (1-i)

... how did they decide that the square-root was called for and be simply? 

Φ = V(1- Δv)(1-i)

I guess what I'm suggesting, it's rather arbitrary and when the problem gets far harder like a AI controlled hand.  It has to pick up an egg and a bowling ball.  There has to be something to define the fitness besides... dropped one or the other or crushed the egg.  

Those related to have extensive history, not just current speed, wall distance.

How does a fitness formula quantify something like:

... when you can't get to the fitness point of walking forward unless you do 50 million steps to even get turned over and stand up.

Obviously it can, but I wouldn't want to tackle this multi-year programming project with hard-coded, first principle equations of motion.

Posted by: @inq

↑

I'm have troubles seeing how this is fair...

Is evolution fair? LOL

At first blush I thought that an individual stuck in a corner would have an activation level of 1 (the highest), so the fitness function would be 0.

V * (1 - sqrt(v) * (1 - i) = 0, where i = 1.

and that this would result in the individual being a low achiever.   But the i value is an average of 80 tries, so there’s probably a very small probability that it would even happen.

Anyway, because the robot does an auger into a corner doesn’t necessarily mean that it won’t be selected to reproduce.  The wheels can still spin so the V and v terms will have some value.  I think that the point is to give the next individual a ‘fighting’ chance by not starting it facing a corner.  Right?

So I think it’s a valid question and one I’ve thought about, but from a slightly different viewpoint.  Like at what generation do you stop doing the reset position?  When does the reset start to interfere with the next individuals starting position?  How do you reset the robot into a neural position and not a worse one?

Tom

Posted by: @thrandell

↑

Is evolution fair? LOL

🤣 You're right!  Unfortunately its too fair now.  😉 

Posted by: @thrandell

↑

But the i value is an average of 80 tries, so there’s probably a very small probability that it would even happen.

So now... that your bot is intelligent.  If you place it head first into a corner, will it get out on its own?

Posted by: @thrandell

↑

Like at what generation do you stop doing the reset position?  When does the reset start to interfere with the next individuals starting position?  How do you reset the robot into a neural position and not a worse one?

Oh!... I see.  I was only thinking about the initial baby phases.  You're right... while one with a clear path can reach a far higher speed and one facing a wall has to turn around and hunt and peck till it finds the clear path wouldn't have a fighting chance in the next generation but could have been a real sprinter.

p.6 - The evolutionary training was a standard genetic algorithm as described by Goldberg [7] with fitness scaling
and roulette wheel selection, biased mutations [15], and one-point crossover.

p.10 - Genetic algorithm parameters:
Population size 80
Generation number 100
Crossover probability 0.1
Mutation probability 0.2
Mutation range 0:5
Initial weight range 0:5
Final weight range Not bounded
Life length 80 actions
Action duration 300 ms

ACAA - They didn't give any meat and potatoes in the paper.  Did you find code directly related to their or did you just start out using Goldberg's code (which it sounds like they followed explicitly)?  I'll be doing my first attempt using the forward/backpropagation methods because they more to what I'm used to, but I'd like to add Goldberg GA and I've got a few ideas that I'd like to experiment with the genetics.  The one-point crossover concept seems rather different than biology suggests.

Are you doing everything on the MPU or do you have some back/forth comms?

Where do your plans go from here that you have a working AI model?  How do you see this playing into Swarm Robot methodology?

VBR,

Inq

@therandell
@inq

In the biological world everything evolves including the fitness factor. The genes code not just for connections but for the neural architecture and the body as well. When things are hard coded, like the neural architecture, the fitness factor and the neurons themselves, it is always possible that pathways to more advanced solutions in more complex situations will never be possible by just changing the weights

The fitness factor in nature is reproductive success.

With Darkin's biomorphs he was the fitness factor just as we are the fitness factor when we practice selective breeding of animals and plants. Probably not practical in Tom's experiments.

With obstacle avoidance you would also need to trial not just the weights but many different wall layouts.

Posted by: @inq

↑

... how did they decide that the square-root was called for

I was wondering about the square root too, so let’s look at some examples.

Phi = V (1 - sqrt(v)) (1 - i)

given that the wheel rotational speed falls between [-0.5, 0.5]

1 - sqrt(v), where v = | (left-right) |  (the absolute value of the algebraic difference of the wheel speeds)

  left         right      v            sqrt(v)     1-sqrt(v)       Notes

-0.40     +0.50     0.90       0.948      0.052          left turn

+0.01     +0.02      0.01       0.100      0.900          inching forward

-0.48     -0.50     0.02       0.141      0.859          fast backwards

+0.30    -0.30     0.60       0.774      0.226          right spin

This component of the fitness function is largest when the wheels move in the same direction, regardless of speed and direction.  When you take the first two components together they encourage speed in the same direction.  The square root gives stronger weight to small differences?!

Tom

Posted by: @inq

↑

I'll be doing my first attempt using the forward/backpropagation methods because they more to what I'm used to

Interesting.  

Back propagation needs a training set, right?  Maybe you could cook up something like a Braitenberg vehicle with a neural network that looks like his circuit drawings and just hand code a training set that you could apply multiple times...  Can your sensors detect distance to an obstacle?  Maybe the training set could have the motor values adjust as the obstacle gets closer.  You know, like obstacle on the left is 5" away so increase the pivot turn sharpness to the right...

Tom

Posted by: @thrandell

↑

Posted by: @inq

↑

... how did they decide that the square-root was called for

I was wondering about the square root too, so let’s look at some examples.

 

Phi = V (1 - sqrt(v)) (1 - i)

given that the wheel rotational speed falls between [-0.5, 0.5]

 

1 - sqrt(v), where v = | (left-right) |  (the absolute value of the algebraic difference of the wheel speeds)

 

  left         right      v            sqrt(v)     1-sqrt(v)       Notes

-0.40     +0.50     0.90       0.948      0.052          left turn

+0.01     +0.02      0.01       0.100      0.900          inching forward

-0.48     -0.50     0.02       0.141      0.859          fast backwards

+0.30    -0.30     0.60       0.774      0.226          right spin

 

This component of the fitness function is largest when the wheels move in the same direction, regardless of speed and direction.  When you take the first two components together they encourage speed in the same direction.  The square root gives stronger weight to small differences?!

 

Tom

Ok... I see your point.  I'm just wondering if all the non-linear behavior built into the learning process would still result in a workable solution...

or... maybe, it would take longer to converge...

or... maybe, it'd work, but you might see some non-optimum behavior... like it wallows while following a obstruction less bearing.

Fundamentally, I'm not so obsessed on the sqrt or not of this specific problem... but ACAA obviously used their physics knowledge to define the fitness function.  What happens when I want to map an entire building with chair and table legs or you want to form some cool interaction between your swarming bots.  Bee hive or Ant hill mentality.  What will we use... sqrt, cubed, inverse, +/- or are do we just use different fitness functions and we learn the hard way from trial and error??? 😉 

Posted by: @thrandell

↑

Interesting.  

Back propagation needs a training set, right? 

Working through the hand-written number digit database scenario...

1. it uses images of 28x28 pixels.  Each pixel is 0-256 so it even has anti-aliasing type information in the images.
2. In this example, there is 784 sensor values of 1 byte coming in.  It uses multiple matrix multiplications to arrive at the output.  
3. The starting matrix is totally filled with random data.
4. Based on error between what it spit out and what it should have spit out (fitness function) it back-propagates to those matrices until it has learned.  In several examples, it got the answer right between 94 and 98% of the time.
5. The reason it makes sense to me - The code to do it is only about a dozen lines long if you don't count the matrix math library which is totally generalized.  

In our situation... the input goes from 784 sensors values to what... 8 so the matrices are tiny and easily done on the MPU.  Instead of all the Goldberg steps and abstraction to bit and byte swapping, it is well easier to conceptualize... at least for me.  Again, I have no idea if it better or worse, but I aim to find out... by doing both!

Posted by: @thrandell

↑

Can your sensors detect distance to an obstacle?  Maybe the training set could have the motor values adjust as the obstacle gets closer.  You know, like obstacle on the left is 5" away so increase the pivot turn sharpness to the right...

Well we all (meaning me doing and the forum making suggestions) messed with the sensor... ad nauseum.  In a nutshell, it'll does an 8 ray over a 45 degree spread to some obstacle.  (Actually an 8x8, but I'll limit it to one horizontal line.  The distance is real accurate to about 2 meters and better than an ultrasonic sensor out to 4 meters.  Its I2C so you don't have mess with all the timing crap like an ultrasonic.  It just gives you the answers to all the points at about 15Hz.

ACAA used ONLY the nearest sensor.  Maybe if I use all 8 points and it "sees" a wall, it might inherently make the corrections... as you suggest.

I'm trying not to have any pre-conceived opinions between ANN or GA solutions.  I want to try both.  Inqster is too big and can't be wired like yours (and several other lame excuses) so I'm CAD'ing a small bot to do similar to yours.  Only difference it'll have steppers and will use the ToF sensor instead of infrared sensors.  I'll be making my excuses and introducing the new bot thread after I have something to show.  I'll have to come back to Inqster later once I've got this AI better handled.