Inqster - The Next Generation

@thrandell, @robotbuilder,

It has been a very long time (also graduating in the mid-80's) where I really studied a textbook for full understanding (not just reading).  I'm guessing that this book is kind of the bible for this field of studies and that most all others stand on the shoulders of Goldberg.  However, with the video "Old Schooling" the Sigmoid function, I'm wondering how much of what I learn from Goldberg will have to be un-learned with newer teachings.  

I'm still in Chapter 1.  Between two careers of eyeballing computer screens having left my eyes worse for wear, Goldberg is the best sleeping aide one could hope for. 😴  It actually is interesting reading.  I just catch myself day-dreaming about

• how something can be applied
• why he did something the way he did
• did he or some later researcher consider something else
• will I need to un-learn this part, with later reading in this book or newer teachings

...until I finally nod off. 

Case in point... although using inputs like 01101 and treating them like the genetic letters A,C,G,T in the biological evolution of genetic manipulation may bring student understanding, I wonder if he fully intends to use string manipulation as I move forward in the book.  I'm not one to skip to the end of a book to see how it ends.  🙄 

It sounds like you have a misconception about how the sigmoid function is used.  You won’t find it mentioned in Goldberg’s text.  It is used as an activation function in an artificial neural network.

In a biological nervous system there are electrical signals transmitted between neurons, there are emitting neurons and receiving neurons.  There is also an electrical threshold in a neuron’s membrane that needs to be exceeded in order for the synapse on the receiving neuron to fire.  Obviously, I’m leaving all the wonderful biological details out.

In an artificial neural network those neural connections are represented as vectors.  Each connection between neural nodes has two vectors, one is the input signal and the second is its (synaptic) weight.  So the receiving node computes its output by summing the product of the two vectors for all incoming connections and subtracting the node’s threshold. In some examples of ANNs you’ll find that the node threshold is replaced with a  (bias) connection that uses the node threshold as its input signal vector and a weight vector with a fixed value of -1.  Clear as mud?

Anyway, there are a number of ways to mathematically describe a node’s output, one of which is the sigmoid function.

Tom

@inq wrote:

"I'm wondering how much of what I learn from Goldberg will have to be un-learned with newer teachings."

On the internet many people have contributed tutorials that anyone can use to increase their understanding on any subject. As a visual learner this series by Josh Stammer looked helpful in understanding what neural nets are actually doing.

https://www.youtube.com/c/joshstarmer

Posted by: @thrandell

↑

It sounds like you have a misconception about how the sigmoid function is used.  You won’t find it mentioned in Goldberg’s text.  It is used as an activation function in an artificial neural network.

I've taken your note that I won't find Sigmoid (or ReLU) in Goldberg.  My post was not trying to specifically connect Sigmoid and Goldberg.  My point was a request to both of you mentors for a little guidance on how much I should hinge my learning on Goldberg that might have been subsequently replaced with newer and stronger theories.  The illustration of "Old Schooling" by a PhD in the field stating that Sigmoid has been largely replaced with the ReLU.  I merely brought it up that things might have moved-on from Goldberg to a lesser or greater extent. 

Posted by: @thrandell

↑

In some examples of ANNs you’ll find that the node threshold is replaced with a  (bias) connection that uses the node threshold as its input signal vector and a weight vector with a fixed value of -1.  Clear as mud?

Actually, your statement is quite clear.  Still in chapter 1, I don't know what is coming in Goldberg.  I'm stating the following based on the video series referenced above.  I fully understand what the Sigmoid function does from my mathematical background better than my limited high-school level biology background.  The first forward step as described is to simply multiply the input vector (in my robot case, the sensor data) by the first Matrix of weighting factors to arrive at the first Hidden layer.  Since the Matrix values are merely randomly generated in the first generation, the outputs can be any magnitude, (say -1E10 to +1E10).  The Sigmoid and ReLU are simply to squish (video's jargon) that range into numbers between 0 and 1 before subsequent steps.  In Linear Algebra this would simply be called Normalizing.

I thought it was very interesting how Goldberg (in Chapter 1 still) abstracted all meaning away from the incoming data by just concatenating them into strings (instead of vectors).  I don't even know (yet), if Goldberg will switch gears to using vectors.  Doing string manipulations seems terribly inefficient and I assume it is merely to help conceptualize the problem for the student (me in this case).

I started thinking about just using Hex input from the sensors placed into a string.  It doesn't matter even what the sensor is (say ToF or Gyro)  or whether its a 8, 16 or 32 bit value... simply jam all of the values into a string.  In example, two parents selected randomly from the first generation might be...

• 1743DC06BCF
• 74534C21B0B

Their mating could simply be, from a biologicals view point, randomly selecting one of the parent's letters at each character location.  i.e.  The third letter of the child can only have a 4 or 5 and it's randomly picked.  So in the next generation, a child might be

• 1753DC26B0F

I find this kind of controlled randomness to be alien, but very interesting.  I've never seen anything like it in my math courses.  I might have to build an engine on this principle just to try it out.

Posted by: @robotbuilder

↑

On the internet many people have contributed tutorials that anyone can use to increase their understanding on any subject.

That is just the problem.  There is way too much information available and it can't all be seen and digested at one time and knowing which to start with and knowing which parts are valid and which are old school aren't known till its all studied.  I merely planted a flag in the ground based on @thrandell advice that Goldberg is one starting point and might be the grand-father of all subsequent work.  I'm just curious, if I should take 100% or 70% of Goldberg as current gospel.

@inq
"I'm just curious, if I should take 100% or 70% of Goldberg as current gospel."

@thrandell  FYI

I wouldn't take anything as gospel, current or otherwise, they are just ideas being experimented with.

Everything is up for grabs.

There are many different types of architectures with many different types of "neurons". For example there is research into using spiking neurons to better replicate the biological neuron. The more spikes per second the more the muscle contracts or that motor speeds up.

With genetic algorithms you are essentially inventing methods to select innate connections in a non evolved architecture and you can test them by running the program. The programs that work the best will survive and the others will be deleted.

In other words the genetic algorithms you are playing with is about evolving fixed solutions in a population,  it is not about individuals capable of learning in real time.

You end up with an innate fixed solution not an individual brain that can learn and adapt its behaviors from experience.

Posted by: @robotbuilder

↑

Everything is up for grabs.

There are many different types of architectures with many different types of "neurons". For example there is research into using spiking neurons to better replicate the biological neuron. The more spikes per second the more the muscle contracts or that motor speeds up.

😎 Cool!  The point of the spear!  "If you're not on the edge, you're just taking up space!" - Jim Whittaker

Posted by: @robotbuilder

↑

The programs that work the best will survive and the others will be deleted.

Evolution inside the program and of the programs!

Good enough and what I was looking for? 

@thrandell FYI

@inq wrote:

Good enough and what I was looking for?

Remind me 🙂

I have known about using genetic algorithms for a long time and have never read Goldberg's book. I have played with various types of neural nets for a long time as well.

My feeling is that using real hardware to test genetic algorithms would be very time consuming whereas a simulated robot can try thousands of configurations at high speed if need be as proof of concept.

A neural net maps inputs to outputs and in a simple example that can be duplicated as a table of outputs for every possible input. In the example below I used four inputs with two values (0 or 1) and two motor outputs with 3 values (-1 or 0 or +1). A hit sensor (bumper bar all the way around the robot) tells it that the net is not working as desired and thus it needs to modify the connections. Another issue is a robot may go into some repetitive movements without going anywhere or hitting anything but that is still a fail.

I don't know if there is a set of input/output values that actually exist to be found that would cause it to avoid obstacles for such a simple robot. In the example below if the robot gets a hit signal it changes the connections otherwise it is up to me to hit a key to change the connections when I see it go into a repetitive sequence. The display shows the current input and the current set of outputs. In the first example below it is just spinning on the spot because the inputs are at zero which with this set of outputs sets the motors to go in opposite directions. After restarting with another set of connections it came up a set that resulted in it moving around obstacles while doing little omega loop 🙂

Navigation to a particular goal position would require another input to tell the robot if it was going in the right direction.

Once you have evolved your net you can duplicate it whenever and wherever you want its behaviors are now innate.

To enlarge an image, right click image and choose Open link in new window.

Posted by: @robotbuilder

↑

My feeling is that using real hardware to test genetic algorithms would be very time consuming whereas a simulated robot can try thousands of configurations at high speed if need be as proof of concept.

I totally agree, but the point is the simulated robot is not real-world and doesn't have all the idiosyncrasies of real motors, non-perfect wheels, slippage, noisy, spikey sensors, etc.  So a simulated my show that the GA does converge on A solution, but until it works on a real robot, it's, just a proof of concept.  I think that was the whole point of https://www.research-collection.ethz.ch/bitstream/handle/20.500.11850/82611/1/eth-8405-01.pdf ...that only a real robot can be the end-solution.

Posted by: @robotbuilder

↑

I don't know if there is a set of input/output values that actually exist to be found that would cause it to avoid obstacles for such a simple robot.

I'm not sure what you are considering a simple robot.  The one in the above link is very simple - 2 motors, 2 encoders, 10 cheesy infrared sensors that aren't good beyond several inches.  Yet it learned how to navigate around the track.  And from what I understand from the paper, it would then be able to drive around a different track just as well.  It's not memorizing the track, just that it likes to go as strait and fast as possible without getting near items.

Posted by: @robotbuilder

↑

Navigation to a particular goal position would require another input to tell the robot if it was going in the right direction.

Now, if you're saying by not simple being something that would have a goal such as go to the refrigerator or go to the bedroom, that robot study certainly is too simple.  The fact is it did learn to travel by itself without anyone teaching it or by explicit instructions saying what each sensor means.

@inq

Perhaps you could make the experiment closer to real behaviour by introducing random errors into the distance and sensor data based on your experience with your existing robots' configurations and behaviours. If you could provide reasonable error rates and times, you could more closely simulate observed behaviour and perhaps feel more comfortable transferring the results from a virtual to a physical robot.

You'd probably need to make judgements like there's a 1% chance that a sensor is wrong by 1-10%, a .1% chance that it's wrong by 10-20% and a .01% chase of an error between 20 and 100% or something of that ilk. A similar strategy could apply for wheel errors and slippage.

It seems to me that, even in the real world, a solution may only apply to a particular robot since each individual may have different deviations in sensors and propulsion systems. Does 100,000 learning steps produce a 'clean' set of weights that will apply to all robots or only to ones sufficiently "close" in construction and operation ?

@will,

This project Inqster is very interesting to me and I'm thinking I might have bit off more than I can chew in such a large leap from Inqling Jr.  The last couple of pages of this thread are delving into the mind of Inqster, yet I haven't even really finalized the chassis.    

As you know in our other correspondence, I'm looking into a new 3D Printer as mine is certainly showing its hard miles.  It no longer will self calibrate, the heater element went, the SD card no longer works... blah, blah, blah.  I have parts coming for it (today hopefully) but it's old and tired.  (The printer and the print-ee 😉) 

I really like the CAD and Mechanical Engineering type design work, leading to the 3D Printing.  Even after years of 3D Printing, it still thrills me that I can draw my own design of parts of something on the computer and press a button and out pops a real, usable thing.

Even after the getting a chassis built, the next step is getting my InqPortal library working on the ESP32 so I can handle the higher workload of sensors I expect with this one.  That is not going well.  Just getting a working IDE to work with these ESP32's is a pain.

After that, I need to delve into the workings of the TMC2209 stepper drivers so I can fully utilize the power of these large stepper motors AND get the feedback from the driver (2-way serial communications) as an input to the ANN.

Then... I can really jump into the Artificial Neural Network (ANN) and/or the Genetic Algorithms (GA).  At the moment, they appear (to me) to be two distinct, different things.  The ANN seems to be brute-force matrix/vector, Linear Algebra, with a feedback loop.  This sounds quite strait forward to me.  On the other hand, the GA where it talks about random mating and random blending of parents  DNA leading to new generations, feels very alien, but sounds like it will offer up some very interesting possibilities.  @robotbuilder and @thrandell have been instrumental in keeping me between the fence posts.  

Posted by: @will

↑

It seems to me that, even in the real world, a solution may only apply to a particular robot since each individual may have different deviations in sensors and propulsion systems. Does 100,000 learning steps produce a 'clean' set of weights that will apply to all robots or only to ones sufficiently "close" in construction and operation ?

I am sure you are correct.  Even The Paper that @thrandel introduced and I continually reference is pre-geared to ONLY optimize for 3 things - (1) faster is better (2) turning is worse and (3) something near sensors is worse.  That's it!  It can't get my beer, it can't vacuum the floor.  BUT!...  It learns to optimize those three things to drive around a track by itself.  I find that alone is WAY COOL! 😎 But I think you're right... to do multiple things, I think a different, learned ANN Matrix will have to be loaded to do different tasks.  The goals are completely different.  For my eventual goals of mapping a room, then house, then building... would not have speed as a better.  It would be striving to find the walls and obstructions and then mapping them... not avoiding them.  The ANN matrix for this would be completely different!

Really, this robot motion ability is the same as many of the robots on this forum have done.  It is a known algorithm to hard-code if something gets near the left sensor, turn right... etc... etc.  We've all seen these type of robots duck and weave their way around a course avoiding obstacles.  But, The Paper version, once it learned, was far smoother around the course.  

But where as the hard-coded version could easily be put on another, different design of robot and work (just as badly), I believe the ANN version could not simply be transplanted.  Even the difference in the sensors and motors would require its own learning session.  It might start with that transplant, and converge far quicker, but I'd bet the matrix of weighting factors and biases would be significantly different... especially if using plain non-linear DC motors.  I still contend that stepper motors are far superiors in this regard (consistency)... I just have to prove to myself that they can be just as strong and fast hauling butt around my yard.  On paper they should both pull stumps and out-run the fastest human. 😊 I know I haven't convinced anyone else yet.

Posted by: @will

↑

Perhaps you could make the experiment closer to real behaviour by introducing random errors into the distance and sensor data based on your experience with your existing robots' configurations and behaviours. If you could provide reasonable error rates and times, you could more closely simulate observed behaviour and perhaps feel more comfortable transferring the results from a virtual to a physical robot.

You'd probably need to make judgements like there's a 1% chance that a sensor is wrong by 1-10%, a .1% chance that it's wrong by 10-20% and a .01% chase of an error between 20 and 100% or something of that ilk. A similar strategy could apply for wheel errors and slippage.

I hadn't thought about doing a simulation.  Maybe I should... just to learn more about ANN.  Though... seeing the data from various sensors over the years... ultrasonic, infrared proximity, and lastly ToF... they're all very noisy.   The hard coded version can receive a false spike and veer wildly.  I'm thinking an ANN version will be far more tolerant of noise and spikes.  In the first cut I plan on using VL53L5CX ToF sensor as the sole input to solve the same problem as The Paper did.  But recall how noisy it was from our study of it... and this wasn't even moving.

Could be fun!

@inq wrote:

"... but the point is the simulated robot is not real-world and doesn't have all the idiosyncrasies of real motors, non-perfect wheels, slippage, noisy, spikey sensors, etc."

As I discovered when I tried to run my simulator code using encoder outputs to navigate a real robot. Unlike the simulated robot the real robot's hardware failed to give consistent encoder counts which made it useless and had inertia so it kept going a bit after the motors were turned off. Maybe dumping the encoders and using stepper motors would have given better results.

===========================
@will wrote to @inq
If you could provide reasonable error rates and times, you could more closely simulate observed behaviour and perhaps feel more comfortable transferring the results from a virtual to a physical robot.

It is simpler to just use a real robot than to try and simulate the physics of the real world.

All I was trying to do is test if I could code a network that would end up with working connections as a result of the weights being selected via random chance and selection of the most fit and sexual reproduction. It is just easier at this stage to do it in software alone.

============================

At the moment I am having fun playing with the code and seeing if I can use a genetic algorithm to move the network to an i/o mapping that achieves some goal which in this case is moving a simulated robot around without hitting an obstacle. On top of this a higher module can bias the movement in the direction of a goal location.

Of course they already have hard coded methods of avoiding obstacles and planning a path to a goal location.

Posted by: @robotbuilder

↑

The meaning of any sensor is only ever in its relationship to other sensors. We might give them meaningful labels but they have no more meaning than a variable has in a program.

The distinction I was trying to make was that with what I'm calling the hard-coded version, we (the developers) knows that the output of say an infrared sensor indicates the distance to some object.  That if it is rising, we need to veer, slow down or back up.  We have to make those abilities in the code and we conceptualize them in our brains.  The ANN has no such code.  It has all the Linear Algebra, matrix manipulations, but there is no discernable, correlation to a sensor and no readable code (with comments) saying sensor did X and we perform Y.  

There is just the matrix and the algorithm that learns by doing the matrix math and learns by feeding back by adjusting the matrix for the next trial matrix.  Once a matrix is finalized, the robot simply avoids the obstacles.  The sensors could totally be randomly connected.  And the same matrix learning code, would happily learn to do the same steps, but with different inputs.  Yes, it would have to start over on the learning process, but us as coders would not have to do anything different in the code.  The hard-coded version would have to be re-worked.

Posted by: @robotbuilder

↑

It is simpler to just use a real robot then try and simulate the physics of the real world.

All I was trying to do is test if I could code a network that would end up with working connections as a result of the weights being selected via random chance and selection of the most fit and sexual reproduction.

I think at your experience level, you are right.  I think @will was making that suggestion in relation to my experience level.  To me... ANN and/or GA are still kind of nebulous.  I think it might be instructive for me and gain some insight by doing even the pure, non-noise case.

Posted by: @robotbuilder

↑

At the moment I am having fun playing with the code and seeing if I can use a genetic algorithm to move the network to an i/o mapping that achieves some goal which in this case is moving a simulated robot around without hitting an obstacle. On top of this a higher module can bias the movement in the direction of a goal location.

Of course they already have hard coded methods of avoiding obstacles and planning a path to a goal location.

That is great you already have the infrastructure to challenge the ANN/GA code you're experimenting with.  Please, let us know how it goes!

@inq That is not going well.  Just getting a working IDE to work with these ESP32's is a pain.

Can you elaborate? I program esp32's all the time using the Arduino IDE with no problems. What issues are you encountering?

Posted by: @will

↑

Perhaps you could make the experiment closer to real behaviour by introducing random errors into the distance and sensor data

Interesting point.  

I’ve been reading about Robotic experiments in evolution and learning and your comment reminded me of one where they wanted to test how a robot that was evolved in simulation could re-adapt to the physical world.

The short story is that they ran this experiment using three different conditions.  The first did not add any noise to the simulated sensors, the second added uniform (white) noise in the range of [0.0,0.4] to the sensors and in the third condition the sensor input was read as if the robot had been displaced by a small random amount in the range [-30,30] mm along the x and y coordinates (conservative noise). 

So after all that they found the robot re-adapted to the physical environment without a drop in performance when the conservative-noise was used in the simulation…

Tom