... and had some success re-creating a robot learning to navigate an environment.
I based the initial work on the same paper @thrandellsuggested - Automatic Creation of an Autonomous Agent (ACAA) that performed the study with a physical robot. I felt I didn't have the patience to watch a robot run around in circles for months and thus started out by simulating it in a virtual world. I had every intention of moving it to the real robot and continuing the study. Life kind of got in the way.
Three years later, l find I'm resurrecting my interest in ANN by starting a new journey. My long term goals of a self-balancing robot include adding environment awareness among other "intelligent" features. I acknowledge that artificial intelligence is a figment and that it is merely statistics. But... I wanted to get started by using ANN early in the design. The typical route for self-balancing robots is to use PID to balance and dampen out oscillations. I've already used PID for other projects, so I decided to use ANN as the method for this robot. https://forum.dronebotworkshop.com/user-robot-projects/inqster-the-re-awakening (I am not a masochist. I am not a masochist. I'm...)
At one point I had every intention of offering up the source code in case someone might be interested. Because of the organic feature creep it became very ugly coding and something I wouldn't want to be associated with. This time around, having a little experience under the belt, I have started restructuring and compartmentalizing it into separate, readable and hopefully concise functional classes. I will be publishing the code.
Although I hope this topic will be of interest and help others in their learning process, I also admit that I often write on the forum as a place to keep verbose notes that I can (and will) call on to refresh my memory on how to do something. I also use it as a peer review by others that will avidly catch my mistakes or make suggestions. 😉
My background is mainly doing static and dynamic analysis of structures and software development. This will almost exclusively be about software development, but the end-use case will be very much about dynamics. For this post, I want to give an overview of what an Artificial Neural Network is. There are a lot of very intelligent people on this forum with vastly different skill sets. For some, this will be boring or trivial. But as we long term members know, a lot of people come here to learn. I am going to drastically simplify the subject, because frankly, I believe most people consider it a black box only rocket scientists, and brain surgeons can fathom. I also think the Computer Science world doesn't really try to dispel this view. Job Security!
This does involve Mathematics, but it's typically taught at the High School level. Some people hate Math... if my wife sees Math on my screen or on a piece of paper, her eyes roll up and she slips back out of the room quietly (most of the time). Most of ANN is some basic Matrix Math. This includes everything from my simple project to ChatGPT, Claud and other AI related technologies.
This is the most fundamental and most common step of an ANN. This is a matrix multiplication. In this example, the matrix "A" has six parameters (numbers) in it (a11 to a32). A matrix is often enclosed in square brackets. It is multiplied by a matrix "X" with two numbers in it. The answer will form a matrix "Y" with three numbers in it. Often matrices with only one row or one column will be calledVectors. X and Y are vectors.
Matrix Multiplication
The actual Math to solve this is typically taught in First Grade, Primary School.
y1 = w11 * x1 + w12 * x2
y2 = w21 * x1 + w22 * x2
y3 = w31 * x1 + w32 * x2
Matrix Addition
The actual Math to solve this is even easier.
h1 = y1 + b1
h2 = y2 + b2
h3 = y3 + b3
These two procedures are typically written simplified as:
W * X = Y
Y + B = H
They just use the capital letters to indicate it is a matrix. If you understand these two matrix steps, you understand 99% of work done by things like ChatGPT. The difference between what the simple models I'll be using and ChatGPT - mine will likely have less than a hundred parameters in various matrices while ChatGPT reportedly has 1.8 trillion parameters. This is where all the electricity, and water come into the discussion... simply to power and cool the computers and Nvidia GPU's.
Nomenclature Used in ANN
In ANN, the matrixWis called aWeights matrix. The vectorBis called aBias. This vectorHis called aLayer. In this case, it is aHidden Layer.
Is that just an American phrase or does it translate in your country?
In the previous post, I discussed doing Matrix Mathematics. If this was all that we did, we would have only what is called a linear set of equations. Newtonian Theory says certain things act linearly. Push a block of ice across the table with a given force, it accelerates at a given rate. This is F=ma. Push twice as hard and it will accelerate twice the rate. This is linearity. But... What if you start pushing it uphill? What if it runs over rocks? What happens as it melts? All these reality items get in the way and make things difficult to analyze. They induce Non-linearity. Some of these can be solved with other equations like friction or Thermodynamics and Heat Transfer. But first you have to know those theories exist and you need to know how to apply them.
To give ANN a chance to think about these non-linear, real-world issues an Activation function is added to the mix. Besides adding non-linear behavior to all this Mathematics, it also keeps things in check. Say one of these "parameters" is very large. This means it has a lot of impact on the final answer. However, if it is too large, it can start drowning out more subtle aspects of the other parameters. The Activation function also brings these very large or very small numbers back into a useful range. This is called normalizing the data.
But really... All this Math has nothing to do with reality. Matrices don't know its ice, a story that ChatGPT creates or a robot trying to balance. The Activation function doesn't know if it's linear or doing a Faceplant. You might as well call it a Fudge Factor!
The Activation function works on theLayer Vector. We'll be putting all this together in later posts, but for now, you can focus on this Activation Function as if it is a stand alone equation. It only works on only one parameter. It has no other dependencies and has only one output. In and of itself, it has no meaning. Simply use the equation and be done with it. All versions of an Activation function have their pros and cons and will affect the results. But the beauty of an ANN, if one doesn't work for your specific case, just plug in another version. There isn't necessarily one conquering all. Here are some common Activation functions:
The Sigmoidfunction is the classical (meaning the original) function used. These days, it is not often used because it is very costly (compared to others) for a computer to calculate. No matter what the input value, the output value ranges from 0 to 1.0 and looks like:
y = 1.0 / (1.0 + Exp(-x))
The ReLU function is very easy on the computer and has some very significant benefits. It ranges from 0 to ∞ and looks like:
y = Max(0, x)
The FastSigmoid function is computationally cheap and doesn't have the one-sided effects of the two above. It ranges from -1.0 to +1.0 and looks like:
y = x / (1.0 + Abs(x))
There are others, but for now, these are the ones we'll be using.
In use in an ANN calculation, for every parameter in a layer that comes out of the previous step, we simply calculate the Activation value and place it back in the same place that it came from.
There are many other Activation functions and I'm not going into all the pros and cons of these, but just know Activation functions are a main part of ANN and really have no meaning by themselves. Some do a better job on some problems, but not others. Sometimes you can see the behavior of your model when it is learning and rationalize one of the other Activation functions might work better. Then again, you might just try them and see if you get better results. It may sound rather arbitrary, but that is what ANN is about. At the very least you have a set of keywords (ANN Activation function) and you dive as deep as you want with... AI. 😆
My schooling mathematics only covered a bit a algebra and geometry. For some reason I took the Accounting stream, and normal maths was supplanted with Commercial Arithmetic (whatever that was) and Statistics (and Debits and Credits) so these days I find it to too much effort to follow the likes of Signoids as they were not in my noddy algebra classes as far a I remember.
But I note this to see if I feel up to it at a later date. It all looks very well presented. 👍
At this point you've been exposed to everything that is needed to run what is called an Inference. This is the series of steps completed to give an answer. This is what happens when you ask Gemini or ChatGPT a question. It runs an Inference. Actually, it probably runs an Inference for every word in its response. When I first started out, I'd usually get lost trying to interpret the steps of an Inference when they just described the mechanics of the flow. I'll try to clear that up by also including a concrete, yet basic example. You'll notice there is a lot of hand-waving and fuzzy logic that seem to have no bearing on writing a story, or doing Physics. But here we go:
(1) Defining the Problem
For the initial prototype of myInqsterrobot https://forum.dronebotworkshop.com/user-robot-projects/inqster-the-re-awakening I want it to be a two-wheeled robot able to self-balance. For this how-to example, I'll simplify the problem even further by stating it doesn't have to drive around yet and we'll represent the Physics as a simple, 2D problem. The physical robot will have a 6DoF sensor located about half way up the body. This contains the 3 axis accelerometer and the 3 axis gyroscope. It will be driven with stepper motors. At this point, it really shouldn't matter if you have stepper motors or electric DC motors combined with encoders.
(2) Defining Inputs and Outputs
In the case of a self balancing robot, you might choose to use a PID loop. This will take tuning of the same difficulty and time that teaching an ANN to balance. However, the PID can not work with the raw data. You will need to take the raw data from the 6DoF sensor, run it through a Kalman filter to clean up the noise. You can't simply use the accelerometer results to determine down because any motion laterally or vertically or rotating will confuse the results. When the robot is still and balanced... you have to read the gyroscope data and integrate it over time to estimate the robot's angle. The problem with this is the Gyro data drifts. Set up a program to output the gyro data. Simply put the sensor flat on the workbench and watch the data... it will move even if you don't touch the sensor! Once you get that all worked out then you have to convert that information into commands for your motors and monitor the encoders to see if they are doing what you expect. Building a self-balancing robot isn't easy even if you just copy it out of some YT tutorial. Any minor differences in weight, weight distribution, the variation of the motors will result in you having to custom tune your PID.
Now this is where creating a self balancing robot using an ANN is incredibly easy. Every time you sample the 6DoF sensor, you simply put the data in one end of the ANN and the answers spit out the other end. Send them to the motors. Rinse and repeat! Easy-peasy.
Now what does putmean you might say. As you hopefully saw in the previous posts, all these matrices and vectors and Activation functions combine to abstract... well... anything. We'll start out with defining the input and output. Each of these are vectors.
We'll start with the output since it's easier. In the case of a normal self-balancing robot, you'll have two motors. For steppers, it'll tell them how fast to turn. For plain DC motors, I'd guess you set a voltage. Either way you need two outputs. For this simulation being 2D, we only need one output + or - tells it which direction and the magnitude tells how fast. The output vector O has one column and one row. IOW, it has one parameter and you send it to your motor driver.
The input vector for our 2D case will need three of the values from the 6DoF sensor. Using the sketch above, we see we'll need the Acceleration in the X direction, Acceleration in the Z direction and the Gyro in the Y direction. If doing a PID version, you'd use the same ones. So, we need an input vectorI with one column and three rows.
[ i1 ]
[ i2 ] (Stuff to be discussed next goes between these.) [ o1 ]
[ i3 ]
To run an Inference on the ANN, you would stuff the threerawvalues intoi1, i2andi3, call the ANN and it'll spit out theo1that you send to the motors. You do this for every timestep you get a new set of readings from the 6 DoF sensor.
That is the bare bones version. What you'll note because of all the abstraction built into the ANN, you don't have to convert the raw 16 bit integers coming out of you 6 DoF sensor to something meaningful like degrees/second and meter/s^2. You don't have to filter them, you don't have to integrate them over time... you just plug them in and go. The ANN will just figure it out as necessary.
My schooling mathematics only covered a bit a algebra and geometry. For some reason I took the Accounting stream, and normal maths was supplanted with Commercial Arithmetic (whatever that was) and Statistics (and Debits and Credits) so these days I find it to too much effort to follow the likes of Signoids as they were not in my noddy algebra classes as far a I remember.
Actually, this makes you far more qualified than me! The Activation functions are merely ways to massage the data, they adjust it, but in a blind/dumb way. Although they are equations, they are not like an equation in Geometry like...
sin(theta) = opposite / hypotenuse
... that gives a definitive answer, the Activation functions simply nudge data in a organized, but unknowing way if it'll make things better or worse... Just non-linear.
Later on, when we get to seeing some of this in action, I can describe further where one "might" be better than another, but most of the time, its easer to just swap one out and see if things get worse or get better.
But when we get to teaching side of the problem, it is ALL about statistics and I've never had one single course that even used the word. Maybe you'll slide in knowing why something is converging or not.
But when we get to teaching side of the problem, it is ALL about statistics and I've never had one single course that even used the word. Maybe you'll slide in knowing why something is converging or not.
Ha, when doing statistics back in the days we were allowed to take a slide rule into the exam. I remember scratching some formula's into the rule to cheat in the exams. But cheating did not pay off as now I don't remember anything much about statistics at all, and never had much use for them. Lies, Dam Lies and Statistics and all that. Debits and Credits yes, I became an Accountant, (I know, I know, bloody bean counter, though not for too long as computers crept in and diverted my attention 😎 ) but I've never had need to make any use of statistics in all these years. Should @inq educational enterprises ltd. take off, I'll give you a bargain price to audit your books. 😀
This is off topic so I hope @inq is ok with it but I will comment no more after.
In everyday life most people don't need math beyond arithmetic. In some jobs they might need to learn some math related to their job. But without the language of math the modern world would never have emerged and is essential for those working in science. One of my regrets is I never appreciated the power of math until the third year in secondary school. It does not come easy to me but was essential to have some understanding of how the world works and that includes the amazing advance they have made with neural network architectures that we can now run on super computer systems using specialized chips.
There are libraries that can do the heavy lifting for neural networks in your code and we all know you can train and use a trained neural network without having any idea how it works.
Does the ANN description so far sound too simple and totally removed from the Physics of a balancing robot? No gravity calculations. No mass, inertia calculations. Certainly none of those ugly trigonometry equations. No PID. No Kalman filtering. How is this possible?
I was formally trained with all that heavy vector, trigonomic Math describing 3D space with accelerations, masses, inertias, gyroscopic effects and more stuff I've long since flushed. And for the life of me, I can't see how a bunch of dumb matrices and a few brain-dead Activation functions can figure all that out on its own! I am here to tell you... it's not going to get any more logical sounding after this post!
Now, don't let all these images and verbiage discourage you. This is only to be completely transparent about what is going on in an ANN. If you use an ANN library or even a Matrix library, I wanted to show what is actually going on inside those black boxes. In the actual code using those libraries, you'll be surprised just how small the program is when done. Right now, I'm expecting the full Arduino, physical robot code to do this self balancing, including reading the 6DoF sensor, running the Inference and output to the stepper drivers to be about 20 lines. So strap in for the finale!
From the last post, we had our input vector of the three components from the 6DoF IMU (ax, az, gy) and the one output to the motor doing the balancing. I used the highly technical term: Stuff
There's no easy way to say this... we put some, semi (meaning almost) arbitrary matrices between these two vectors. In the ANN world, they are called Hidden Layers. They are simply intermediate vectors (1 column, n rows). The number of these intermediate, hidden layers seem to be State Secrets. For the past several years, I've asked Google (before AI) for references about how to estimate the number and sizes of the layers. I've asked professors with PhD's in this subject and now I've asked Gemini and ChatGPT for this specific project.
Gemini
Gemini says I should use 1 hidden layer with 4 Neurons and explicitly mentioned that I shouldn't overdo The AI field just loves to use brain analogies.
ChatGPT
ChatGPT has a more shotgun recommendation. But these two opinions totaling eight recommendations illustrate a couple of important points. NO ONE! knows what's going on inside these matrices and certainly not how to estimate them.
I've already run a learning process having 1 Hidden Layer using 8 neurons parameters, so I'll illustrate the rest of this post using my first attempt. I'm not completely satisfied with the results and will likely try all of these suggestions, but will start with this:
Again, the general term for these is layers, but in Mathematics they are one dimensional matrices (vectors) - an ANN problem always has one Input Layer, one or more Hidden Layers and one Output Layer. As was mentioned, a Hidden Layer is simply another vector, but you can't get from a vector of 3 (input) to this hidden layer of 8 directly. Working with ANN, a Weights Matrix is multiplied and a Bias Vector is added between each and every layer. If you recall from the second post of this topic, the size of a matrix used in multiplication will have the same number of columns as the input vector and the same number of rows as the output. So the matrices will look like:
Step 1 Input to Hidden
Step 2 Hidden to Output
This shows all the parameters needed for the entire ANN. Using Matrix notation all this turns into:
W * I + B = H
W' * H + B' = O
If you are using a matrix library this makes your Arduino sketch look trivial. I'll be supplying the library for C# on the computer's virtual robot and C++ on the physical robot just because I said, "...from the ground up." Now if you wanted to use something like Python, it is my understanding (don't know) there is a library called Numpy that I know has all the matrix mathematics and I've also seen it has the functions like Sigmoid, and FastSigmoid. It wouldn't surprise me if it had a full ANN routine. It wouldn't be that difficult.
I'm following this with interest. It's been many moons (ahem...65 or so years) since I've had any formal training in mathematics (at one time, I even thought about it as my major in college, but alas Criminal Psychology won out and later I was able to finish a MSEE/CS degree).
As you may recall, I'm building a self-balancing robot (it's built), but I'm having some difficulty in getting the robot to reliably stand up and remain standing. Finding the values for Kp, Ki, Kd, seem to be some sort of 'black magic' (read: trial and error) exercise. I guess I'm not very good at picking numbers.
Yes, Python has a NumPy library that does all this math 'stuff'!
I'm following this with interest. It's been many moons (ahem...65 or so years) since I've had any formal training in mathematics (at one time, I even thought about it as my major in college, but alas Criminal Psychology won out and later I was able to finish a MSEE/CS degree).
As you may recall, I'm building a self-balancing robot (it's built), but I'm having some difficulty in getting the robot to reliably stand up and remain standing. Finding the values for Kp, Ki, Kd, seem to be some sort of 'black magic' (read: trial and error) exercise. I guess I'm not very good at picking numbers.
Yes, Python has a NumPy library that does all this math 'stuff'!
Regards,
First, it is nice to hear if/when something is appreciated.
Too bad distances prevent, but I'd love to hear some stories from YOUR profession... over a beer or two. 🍻
PID
Manual - Have you followed any strategies so far? If I recall correctly, it's been years (I did it for a COVID aid project) you start out doing it with each parameter, in turn starting with "P". You get it to where it looks like you've bested as much as you can. Then you refine it with "I" getting supposedly better. Then the final touches with "D".
Either way, how are you protecting it from face plants? 😳 I'm behind on my project, but keep thinking how one good drop on mine will break something. I've considered tethering it to the ceiling so it can't go past a certain angle or just putting padding all over it. Neither sounds like a good plan.
Also, I would like to highly encourage you (cattle prod time) to update your robot topic. Your trials are just as important as your successes. I'm sure others have and will have the same types of problems you're running into. And seeing you work through them might give them hope whereas posting only a final success just discourages them. Plus, us engineering types like pictures and videos... being reading and writing impaired. I'm certainly subscribed to your topic!
Either way, how are you protecting it from face plants? I'm behind on my project, but keep thinking how one good drop on mine will break something. I've considered tethering it to the ceiling so it can't go past a certain angle or just putting padding all over it. Neither sounds like a good plan.
Could you have training guards that prevent the robot falling more than say 20 degrees or at least not actually falling completely over? These guards could provide a learning signal as well.
I seem to remember reading somewhere that a sudden burst of acceleration will throw the base backward into a upright position?
Could you have training guards that prevent the robot falling more than say 20 degrees or at least not actually falling completely over? These guards could provide a learning signal as well.
I seem to remember reading somewhere that a sudden burst of acceleration will throw the base backward into a upright position?
I think I can. They'll be pretty big, and add a lot of inertia, but I think that is the best idea.
Could be built out of light weight foam. It just seems a good idea so you can leave it alone while it uses the guards as feedback to the neural network when it tips too far. Maybe with sonar obstacle avoidance so it learns to balance while changing direction all by itself.
We use cookies on the DroneBot Workshop Forums to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept”, you consent to the use of ALL the cookies.
This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
I'm behind on my project, but keep thinking how one good drop on mine will break something. I've considered tethering it to the ceiling so it can't go past a certain angle or just putting padding all over it. Neither sounds like a good plan.
