Evolution of Adaptive Synapses: Robots with Fast Adaptive Behavior in New Environments

@thrandell

Clear as mud?

Yeah about that clear 🙂

I asked ChatGPT to explain it further.

Suppose you're designing a neural network where each neuron learns by updating its incoming weights using the same learning rule (e.g., gradient descent). The rule might specify that if the neuron’s output is too low, all incoming weights should be increased proportionally to improve the output. This shared rule simplifies learning because it ensures all synapses adapt coherently based on the same principle.

I asked it to provide a simple code example of this using the Hebbian rule.

@thrandell

Been reading your link and trying to get it clear in my head.

I have no idea what that self limiting mechanism was…normalization? sigmoid()? they don’t say.

While trying to learn what a Hebbian network is I notice the output in this example generated by ChatGPT uses the dot product.

import numpy as np

# Define input patterns (e.g., 3 patterns with 12 features each)
inputs = np.array([
    [1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0],
    [0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1],
    [1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0]
])

# Define target outputs for each pattern (e.g., 2 output neurons)
targets = np.array([
    [1, 0],
    [0, 1],
    [1, 1]
])

# Initialize weights randomly for 12 input neurons and 2 output neurons
np.random.seed(42)
weights = np.random.randn(inputs.shape[1], targets.shape[1])

# Hebbian learning parameters
learning_rate = 0.1
epochs = 10

print("Initial weights:\n", weights)

# Training with Hebbian learning
for epoch in range(epochs):
    for x, y in zip(inputs, targets):
        # Hebbian update: weights += learning_rate * input * output
        weights += learning_rate * np.outer(x, y)

    # Optional: print weights after each epoch
    print(f"\nWeights after epoch {epoch + 1}:\n{weights}")

# Testing the network
print("\nTesting the network:")
for x in inputs:
    output = np.dot(x, weights)
    print(f"Input: {x}\nOutput: {output}\n")

@thrandell

To keep the synaptic values from growing indefinitely the output is bound to the range [0,1] with a self-limiting mechanism.  I have no idea what that self limiting mechanism was…normalization? sigmoid()? they don’t say. 

Is this what they might be do?

# Function to normalize weights so their sum equals 1
def normalize_weights(w):
    total = np.sum(w)
    if total != 0:
        return w / total
    return w

@thrandell

Sorry for all the posts. Not sure how many others have an interest in these things.  When I get interested in a subject I tend to become obsessed for a while until I am exhausted and move on ...

I don't know if you ever use AI to extract useful information from the internet. AI has sucked up all the information and you can ask it questions which you can't do when reading an article on some subject.

I asked it:

"Can you give more detailed cases of robots using Hebbian Learning for Rapid Adaptation?"

Certainly! Here are detailed case studies of robots using Hebbian learning for rapid adaptation:

### 1. **Robotic Navigation in Unstructured Environments**

**Scenario:**
A mobile robot is deployed in an unknown, unstructured environment such as a disaster site with debris and irregular terrain.

**Implementation:**
- The robot is equipped with sensors (proximity, touch, camera) and a neural network controller with Hebbian learning rules.
- As the robot moves and encounters obstacles or different terrain types, the connections between sensor inputs and motor outputs are strengthened or weakened based on co-activation patterns.
- For example, if the robot repeatedly bumps into an obstacle when sensing a particular pattern, the synapses linking that sensory pattern to avoidance behaviors are reinforced.

**Outcome:**
- The robot rapidly learns to associate specific sensor inputs with appropriate responses, improving its navigation efficiency without pre-programmed maps.
- This on-the-fly adaptation allows it to handle novel obstacles and terrain more effectively than traditional rule-based systems.

---

### 2. **Adaptive Robotic Gripper for Variable Object Shapes**

**Scenario:**
A robotic arm tasked with grasping objects of various shapes and sizes in a warehouse.

**Implementation:**
- The robot uses Hebbian learning to adapt its grip control based on successful or failed grasps.
- Sensors in the gripper detect contact and force. When a grasp succeeds, the neural connections between sensory inputs (contact points) and motor commands (grip strength, finger positioning) are strengthened via Hebbian rules.
- Conversely, unsuccessful grasps lead to the weakening of those connections.

**Outcome:**
- Over a series of trials, the robot quickly fine-tunes its grip parameters for new objects, learning optimal strategies for grasping different shapes without explicit programming for each object.
- This rapid, experience-driven adaptation increases efficiency and flexibility in handling diverse items.

---

### 3. **Visual-Motor Learning in Robotic Manipulation**

**Scenario:**
A robot learns to reach for objects placed randomly within its workspace using visual cues.

**Implementation:**
- The robot's neural network links visual input patterns to motor commands controlling arm movement.
- Hebbian learning occurs as the robot practices reaching—successful attempts strengthen the connections between observed object positions and the corresponding motor actions.
- When the robot encounters a new position or object, it adapts its movement strategy based on the learned associations.

**Outcome:**
- The robot quickly adapts to new visual environments, improving its reaching accuracy over a few trials.
- This form of learning mimics biological sensorimotor adaptation, allowing for rapid behavior modification in changing environments.

---

### 4. **Swarm Robots for Collective Adaptation**

**Scenario:**
A swarm of small robots working together to explore and map an unknown environment.

**Implementation:**
- Each robot uses Hebbian learning to adapt its local interactions with neighbors based on successful communication and task completion.
- When a robot's signal leads to successful collaboration (e.g., sharing information about detected obstacles), the synapses between communication signals and behavioral responses are strengthened.
- This process enables the swarm to evolve cooperative strategies automatically.

**Outcome:**
- The swarm collectively adapts to environmental changes and task requirements without centralized control, leading to efficient exploration and mapping of new environments.

This is a really cool topic...I’ve always thought adaptive behavior could be a big step forward for real robots! The idea of adaptive synapses that let a robot modify its responses in new environments feels much more practical than fixed control strategies alone. The approach you shared here (Evolution of Adaptive Synapses) makes me think of how online learning could help with things like terrain changes, sensor noise, or unexpected obstacles in the real world: https://forum.dronebotworkshop.com/neural-networks/evolution-of-adaptive-synapses-robots-with-fast-adaptive-behavior-in-new-environments/#post-52098

I’m wondering if anyone here has tried even simple versions of this on hobby platforms (Raspberry Pi, ROS, or microcontrollers) — is it still mostly in research demos, or starting to show up in real robot projects? Would love to hear thoughts!

@oliver2003

I’ve always thought adaptive behavior could be a big step forward for real robots! The idea of adaptive synapses that let a robot modify its responses in new environments feels much more practical than fixed control strategies alone.

Adaptive behavior like habituation and sensitisation can be built into a neural network. (see Aplysia, Kendal)

I’m wondering if anyone here has tried even simple versions of this on hobby platforms (Raspberry Pi, ROS, or microcontrollers) — is it still mostly in research demos, or starting to show up in real robot projects? Would love to hear thoughts!

I guess that is what @thrandell is wanting to do with his little robots and sees the idea of synaptic change is something worth trying.

First you have to build your robot and then train its neural network using some desired goal states (travel in a straight line, don't hit a wall) to produce the desired outputs for any given inputs. Then change the environment in some way to test its ability to adapt to the changes and thus still achieve its goals.

One simple network @thrandell used in another project was composed of two motor neurons with inputs from distance sensors around its physical body. It used a genetically evolved set of weights.

https://forum.dronebotworkshop.com/neural-networks/genetic-evolution-of-a-neural-network-driven-robot/

https://www.youtube.com/channel/UCHpGX-lIqesZpwDjo6fUvgA

Can you use synaptic adaption as opposed to continual relearning to achieve the goals when the layout of the walls are changed?

Posted by: @robotbuilder

↑

Is this what they might be do?

I’m not sure.  I think what they meant was that after adding the old synapse weight to the product of the learning rate and the hebb rule results, the final weight should be in the range of [0,1].  In my code I’m just using the sigmoid function…  I’ll try it if you can explain what the code is doing?  What is the variable w?  What is np.sum(w) doing?

 ——

Does your AI tool cite references?  I’ve read about projects like your #4 with the swarm robots, but not using Hebb adaptation rules.  The robots are the neural network nodes and the synapses are the communication links between them.  It will have trouble scaling up but it’s a cool idea.

Tom

The Light-Switching Problem

The experimenters gave the robot a sequential task to solve.  Here is my paraphrase from the paper.  A robot is positioned in a rectangular environment.  A light bulb is attached on one side of the environment.  This light is normally off, but it can be switched on when the robot passes over a black painted area on the opposite side of the environment.  A black strip is painted on the wall over the light-switch area.  Each individual must find the light-switch area, to switch the light on, then move to the area under the light.  Each individual will be evaluated on how long they stay under the light.

Their results.

https://www.youtube.com/watch?v=GdKvQtmcq_A

The neural network

The neural controller is a fully recurrent, discrete-time neural network, where each node is connected to all other nodes.  It has  12 neurons for a total of 12 * 12 = 144 synapses.  10 sensory neurons receive input from a group of sensors on the body of the robot and the activation of 2 motor neurons set the wheel speeds.  There are 4 IR distance sensors, 3 ambient light sensors & 3 vision photoreceptors.  

With discrete-time recurrence the activation value from each neuron at time t - 1 is an input to itself at time t.  These connections also have a synaptic weight. 

Another interesting thing about the design of this neural network is how the sensor inputs are used.  Instead of giving them a synaptic weight and using them as input to a node, the sensory input is just added to the activation value at the very end.  How on earth do these guys come up with this stuff? 

At first I tried to implement the ANN like I did before with layers of neural nodes representing the input, hidden, recurrent & and output nodes, but it just didn’t work very well.  So I settled on a two dimensional array of synaptic weight values.  The rows are the post-synapse neurons and the columns the pre-synapse neurons.  The neuron and recurrent activation values are held in two one dimensional arrays.  It seems to be working ok, I think because it’s easier to match it up with the Hebb rule calculations given in the paper.

This is the routine I came up with to apply the Hebb rules to the neural network.

          
// Neural Network
#define NUMNODES  4         // 12 nodes: 10 sensors + 2 motors
#define NUMINPUTS  3         // 10 sensors, receptors 
#define NUMOUTPUTS  1        // 2 motor neurons 
#define MAXSTRING 21         // Starting with subset of ANN 4 nodes X 5 bits = 20 bits
#define MAXPARMS 5           // number of neuron parms defined on a chromosome, 1 base indexing
          
// Genetic Algorithm, just playing around with a chromosome not populations 
typedef bool allele;
typedef allele chromosome[MAXSTRING]; 
chromosome chrom;                    
          
struct encoding {       
  chromosome chrom;
  int8_t sign;   // could this be bool?
  uint8_t rule;         
  float rate;
};    
typedef struct encoding adaptivespecs[NUMNODES];  // adaptiveparms??  adaptiverules???
adaptivespecs nodespecs;   // nodeparms???  noderules??
          
// Neural Network, moving away from the notion of layers because all node are connected...
float  inputLayer[NUMNODES];               // holds the normalized IR sensor readings and zeros for motor neurons
float  nodeActivation[NUMNODES];           // holds the neural activation values
float  contextActivation[NUMNODES];        // holds the t-1 node output
float  synapseWeights[NUMNODES][NUMNODES]; // the last two in both dimensions are the motor neurons, left - right
          
float learning[4] = {0.0, 0.3, 0.6, 0.9};
            
float sigmoid(float x) { return 1 / (1 + exp(-x)); }   // e raised to the -x power
            
float diffPrePost(float preActivity, float postActivity) { // tanh = hyperpolic tangent
  return tanhf(4.0f * (1.0f  - fabs(preActivity - postActivity)) - 2.0f);
}  

void weight_delta() {
// Using the hebb rules calculate the delta weight for all synapses, runs once per sensory motor cycle
// The sigmoid function will be applied to all final synapseWeights to bound the synapse values to the range [0,1].

  float deltaWeight, f, tempWeight;

  for (uint8_t i=0; i<NUMNODES; i++) {    // i indexes the post synaptic neuron

    for (uint8_t j=0; j<NUMNODES; j++) {  // j indexes the pre synaptic neuron
      if (nodespecs[i+1].rule == 0) {  // plain hebb eventually use a case statement
        if (i == j)   // means the neuron's recurrent connection
          deltaWeight =  (1.0f - synapseWeights[i][j]) * nodeActivation[j] * contextActivation[i];
        else
          deltaWeight =  (1.0f - synapseWeights[i][j]) * nodeActivation[j] * nodeActivation[i];
      }
      else if (nodespecs[i+1].rule == 1) {  // post-synaptic rule
        if (i == j)   // means the neuron's recurrent connection
          deltaWeight = synapseWeights[i][j] * (-1.0f + nodeActivation[j]) * contextActivation[i] +
                        (1.0f - synapseWeights[i][j]) * nodeActivation[j] * contextActivation[i];
        else
          deltaWeight = synapseWeights[i][j] * (-1.0f + nodeActivation[j]) * nodeActivation[i] +
                        (1.0f - synapseWeights[i][j]) * nodeActivation[j] * nodeActivation[i];
      }
      else if (nodespecs[i+1].rule == 2) {  // pre-synaptic rule
        if (i == j)   // means the neuron's recurrent connection
          deltaWeight = synapseWeights[i][j] * nodeActivation[j] * (-1.0f + contextActivation[i]) +
                        (1.0f - synapseWeights[i][j]) * nodeActivation[j] * contextActivation[i];
        else
          deltaWeight = synapseWeights[i][j] * nodeActivation[j] * (-1.0f + nodeActivation[i]) +
                        (1.0f - synapseWeights[i][j]) * nodeActivation[j] * nodeActivation[i];
      }
      else if (nodespecs[i+1].rule == 3) {  // covariance rule
        if (i == j) {   // means the neuron's recurrent connection
          f = diffPrePost(nodeActivation[j],contextActivation[i]);
          if (f > 0)
            deltaWeight = (1.0f - synapseWeights[i][j]) * f;
          else
            deltaWeight = synapseWeights[i][j] * f;
        }
        else {
          f = diffPrePost(nodeActivation[j],nodeActivation[i]);
          if (f > 0)
            deltaWeight = (1.0f - synapseWeights[i][j]) * f;
          else
            deltaWeight = synapseWeights[i][j] * f;
        }
        //printf("delta weight = %f\n", deltaWeight);
      }
    tempWeight = synapseWeights[i][j] + nodespecs[i+1].rate * deltaWeight;
    synapseWeights[i][j] = sigmoid(tempWeight);
    } // end of the loop of the pre synaptic neuron j

  }  // end loop of the post synaptic neuron i
  return;
} // end weight_delta

Tom

@thrandell 

I’ll try it if you can explain what the code is doing? What is the variable w? What is np.sum(w) doing?

The Python code `total = np.sum(w)` calculates the sum of all elements in the array or list `w` using the NumPy library.

Here's a breakdown:
- `np` is the common alias for the NumPy library, which provides efficient numerical operations.
- `np.sum(w)` adds up all the values in `w`. If `w` is a list or array of numbers, it returns their total sum.
- The result is stored in the variable `total`.

For example, if `w = [1, 2, 3, 4]`, then `np.sum(w)` will give `10`.

@thrandell 

This is the routine I came up with to apply the Hebb rules to the neural network.

Somewhat hard for me to follow in a short time. I will have to find time to study it more...

I asked ChatGPT to write a Python and FreeBASIC version along with an explanation as to what the program did and how it did it. 

### **Overview of the Program**

This program models a simple neural network with a set of interconnected neurons, where the connections (synapses) between neurons can adapt over time. The network's behavior is inspired by biological neural processes and uses rules similar to Hebbian learning, with some variations.

---

### **Key Components**

1. **Neurons (Nodes):**
The network has multiple neurons (called nodes). These neurons receive inputs (like sensors), process information, and communicate with each other through weighted connections (synapses).

2. **Connections (Synapses):**
Each pair of neurons is connected by a synapse with a weight value that indicates how strongly one neuron influences another. These weights are dynamically adjusted during the program's execution.

3. **Neuron Rules:**
Each neuron has an associated rule that determines how its synapses are updated. The rules include:
- **Hebbian rule (rule 0):** Strengthens connections based on simultaneous activity.
- **Post-synaptic rule (rule 1):** Considers activity after the synapse.
- **Pre-synaptic rule (rule 2):** Considers activity before the synapse.
- **Covariance rule (rule 3):** Adjusts weights based on activity correlation, using a hyperbolic tangent function to measure similarity.

4. **Input and Activation:**
The network receives simulated sensor inputs (random values). These inputs influence the neurons' activation levels, which are real-valued signals between 0 and 1.

5. **Learning and Adaptation:**
During each cycle:
- The network receives new inputs.
- The neurons' activations are updated based on the inputs.
- The synapse weights are adjusted according to the selected rule, based on the current activations and previous activity (context).

6. **Recurrent Connections:**
The neurons can have recurrent (self) connections, meaning a neuron can influence itself, adding a feedback loop.

---

### **What the Program Actually Does Step-by-Step**

1. **Initialization:**
- Sets up the neurons with specific rules and learning rates.
- Randomly initializes the synaptic weights.
- Prepares the input and activation arrays.

2. **Repeated Cycles:**
For a number of iterations (cycles):
- Generates new random sensor inputs.
- Updates the neurons' activation levels based on these inputs.
- Stores the current activation as previous activity (context).
- Calls the weight update function, which:
- Loops over each neuron pair.
- Calculates a change in the connection weight based on their activity and the selected rule.
- Applies a sigmoid function to keep weights in a bounded range (0 to 1).
- Prints the updated weights for analysis.

3. **Learning Over Time:**
- Over multiple cycles, the weights adapt based on the activity patterns, simulating learning.
- Different rules influence how connections are strengthened or weakened.
- The network can potentially develop patterns or associations based on input sequences.

---

### **In essence:**

This program simulates a small, interconnected neural network that learns by adjusting connection strengths based on activity correlations, mimicking some basic aspects of biological learning. It can be used to explore how different learning rules affect the development of connections in a neural system over repeated experiences.

---

A Python code example based on your C code.

import math
import numpy as np
import random

# Constants
NUMNODES = 4
NUMINPUTS = 3
NUMOUTPUTS = 1
MAXSTRING = 21
MAXPARMS = 5

# Structures
class Encoding:
    def __init__(self, sign=0, rule=0, rate=0.0):
        self.sign = sign
        self.rule = rule
        self.rate = rate

# Initialize node specifications
nodespecs = [Encoding() for _ in range(NUMNODES + 1)]  # index from 0

# Initialize network variables
inputLayer = [0.0] * NUMNODES
nodeActivation = [0.0] * NUMNODES
contextActivation = [0.0] * NUMNODES
synapseWeights = np.zeros((NUMNODES, NUMNODES))
learning = [0.0, 0.3, 0.6, 0.9]

# Functions
def sigmoid(x):
    return 1 / (1 + math.exp(-x))

def diffPrePost(preActivity, postActivity):
    return math.tanh(4.0 * (1.0 - abs(preActivity - postActivity)) - 2.0)

def weight_delta():
    for i in range(NUMNODES):  # post-synaptic
        for j in range(NUMNODES):  # pre-synaptic
            rule = nodespecs[i + 1].rule
            if rule == 0:
                if i == j:
                    delta_weight = (1.0 - synapseWeights[i][j]) * nodeActivation[j] * contextActivation[i]
                else:
                    delta_weight = (1.0 - synapseWeights[i][j]) * nodeActivation[j] * nodeActivation[i]
            elif rule == 1:
                if i == j:
                    delta_weight = (synapseWeights[i][j] * (-1.0 + nodeActivation[j]) * contextActivation[i] +
                                    (1.0 - synapseWeights[i][j]) * nodeActivation[j] * contextActivation[i])
                else:
                    delta_weight = (synapseWeights[i][j] * (-1.0 + nodeActivation[j]) * nodeActivation[i] +
                                    (1.0 - synapseWeights[i][j]) * nodeActivation[j] * nodeActivation[i])
            elif rule == 2:
                if i == j:
                    delta_weight = (synapseWeights[i][j] * nodeActivation[j] * (-1.0 + contextActivation[i]) +
                                    (1.0 - synapseWeights[i][j]) * nodeActivation[j] * contextActivation[i])
                else:
                    delta_weight = (synapseWeights[i][j] * nodeActivation[j] * (-1.0 + nodeActivation[i]) +
                                    (1.0 - synapseWeights[i][j]) * nodeActivation[j] * nodeActivation[i])
            elif rule == 3:
                if i == j:
                    f = diffPrePost(nodeActivation[j], contextActivation[i])
                    delta_weight = (1.0 - synapseWeights[i][j]) * f if f > 0 else synapseWeights[i][j] * f
                else:
                    f = diffPrePost(nodeActivation[j], nodeActivation[i])
                    delta_weight = (1.0 - synapseWeights[i][j]) * f if f > 0 else synapseWeights[i][j] * f
            else:
                delta_weight = 0.0

            temp_weight = synapseWeights[i][j] + nodespecs[i + 1].rate * delta_weight
            synapseWeights[i][j] = sigmoid(temp_weight)

# Initialize node specs with example rules and rates
for i in range(1, NUMNODES + 1):
    nodespecs[i].rule = i % 4  # cycle through rules 0-3
    nodespecs[i].rate = learning[i % len(learning)]

# Run multiple cycles
NUM_CYCLES = 10

for cycle in range(NUM_CYCLES):
    print(f"\nCycle {cycle + 1}")

    # Simulate sensor input with random values
    for i in range(NUMINPUTS):
        inputLayer[i] = random.uniform(0, 1)
    # Zero for motor neurons
    for i in range(NUMINPUTS, NUMNODES):
        inputLayer[i] = 0.0

    # Set activations from input
    for i in range(NUMNODES):
        nodeActivation[i] = inputLayer[i]

    # Save previous activations for context
    for i in range(NUMNODES):
        contextActivation[i] = nodeActivation[i]

    # Update weights based on current activations
    weight_delta()

    # Print current weights
    print("Synapse weights:")
    print(np.round(synapseWeights, 3))

And a FreeBASIC version as I use it as well.

' FreeBASIC version of the neural network with genetic rules and weight updates
' Define tanh function since FreeBASIC doesn't have it built-in
Function TanH(x As Single) As Single
    Dim As Single ePos = Exp(x)
    Dim As Single eNeg = Exp(-x)
    Return (ePos - eNeg) / (ePos + eNeg)
End Function

Const NUMNODES = 4
Const NUMINPUTS = 3
Const NUMOUTPUTS = 1
Const MAXSTRING = 21
Const MAXPARMS = 5

Type encoding
    sign As Integer ' could be boolean, but using Integer for simplicity
    rule As Integer
    rate As Single
End Type

Dim shared As encoding nodespecs(NUMNODES + 1) ' index from 0 to NUMNODES

Dim shared As Single inputLayer(NUMNODES)
Dim shared As Single nodeActivation(NUMNODES)
Dim shared As Single contextActivation(NUMNODES)
Dim shared As Single synapseWeights(NUMNODES, NUMNODES)

Dim shared As Single learning(4) = {0.0, 0.3, 0.6, 0.9}

' Sigmoid function
Function sigmoid(x As Single) As Single
    sigmoid = 1 / (1 + Exp(-x))
End Function

' diffPrePost function
Function diffPrePost(preActivity As Single, postActivity As Single) As Single
    Return TanH(4.0 * (1.0 - Abs(preActivity - postActivity)) - 2.0)
End Function

Sub weight_delta()
    Dim As Single deltaWeight, f, tempWeight
    Dim As Integer i, j
    For i = 0 To NUMNODES - 1
        For j = 0 To NUMNODES - 1
            Dim As Integer rule = nodespecs(i + 1).rule
            If rule = 0 Then
                If i = j Then
                    deltaWeight = (1.0 - synapseWeights(i, j)) * nodeActivation(j) * contextActivation(i)
                Else
                    deltaWeight = (1.0 - synapseWeights(i, j)) * nodeActivation(j) * nodeActivation(i)
                EndIf
            ElseIf rule = 1 Then
                If i = j Then
                    deltaWeight = synapseWeights(i, j) * (-1.0 + nodeActivation(j)) * contextActivation(i) + (1.0 - synapseWeights(i, j)) * nodeActivation(j) * contextActivation(i)
                Else
                    deltaWeight = synapseWeights(i, j) * (-1.0 + nodeActivation(j)) * nodeActivation(i) + (1.0 - synapseWeights(i, j)) * nodeActivation(j) * nodeActivation(i)
                EndIf
            ElseIf rule = 2 Then
                If i = j Then
                    deltaWeight = synapseWeights(i, j) * nodeActivation(j) * (-1.0 + contextActivation(i)) + (1.0 - synapseWeights(i, j)) * nodeActivation(j) * contextActivation(i)
                Else
                    deltaWeight = synapseWeights(i, j) * nodeActivation(j) * (-1.0 + nodeActivation(i)) + (1.0 - synapseWeights(i, j)) * nodeActivation(j) * nodeActivation(i)
                EndIf
            ElseIf rule = 3 Then
                If i = j Then
                    f = diffPrePost(nodeActivation(j), contextActivation(i))
                    If f > 0 Then
                        deltaWeight = (1.0 - synapseWeights(i, j)) * f
                    Else
                        deltaWeight = synapseWeights(i, j) * f
                    EndIf
                Else
                    f = diffPrePost(nodeActivation(j), nodeActivation(i))
                    If f > 0 Then
                        deltaWeight = (1.0 - synapseWeights(i, j)) * f
                    Else
                        deltaWeight = synapseWeights(i, j) * f
                    EndIf
                EndIf
            Else
                deltaWeight = 0
            EndIf

            tempWeight = synapseWeights(i, j) + nodespecs(i + 1).rate * deltaWeight
            synapseWeights(i, j) = sigmoid(tempWeight)
        Next j
    Next i
End Sub

' Main program
Dim As Integer cycles = 10
Randomize Timer

' Initialize node specs with example rules and rates
Dim As Integer i
For i = 1 To NUMNODES
    nodespecs(i).rule = i Mod 4
    nodespecs(i).rate = learning(i Mod 4)
Next i

For cycle As Integer = 1 To cycles
    Print "Cycle "; cycle

    ' Generate random sensor inputs
    For i = 0 To NUMINPUTS - 1
        inputLayer(i) = Rnd
    Next i
    ' Zero for motor neurons
    For i = NUMINPUTS To NUMNODES - 1
        inputLayer(i) = 0
    Next i

    ' Set activations from input
    For i = 0 To NUMNODES - 1
        nodeActivation(i) = inputLayer(i)
        contextActivation(i) = nodeActivation(i)
    Next i

    ' Update weights
    weight_delta()

    ' Print weights
    Print "Synapse weights:"
    For i as integer = 0 To NUMNODES - 1
        For j as integer = 0 To NUMNODES - 1
            Print Using "###.###"; synapseWeights(i, j);
        Next j
        Print
    Next i
    Print
Next cycle
sleep

Posted by: @robotbuilder

↑

For example, if `w = [1, 2, 3, 4]`, then `np.sum(w)` will give `10`.

Sorry, I still don’t get it.

Using your example: the array w= [1,2,3,4] is passed to the function.

  total = np.sum(w) = 10;

  10 does not equal 0 so return w / total which is what?  [1,2,3,4]/10 or 10/10 or [1/10,2/10,3/10,4/10] or … ?

This is what I’ve used to normalize sensor input data: Divide each sensor’s reading by the square root of the sum of the squares of all sensor readings.  In the context of this paper, should I take all the synaptic values coming into a node and apply this formula to bound the synapses to the range [0,1]?

Tom

@thrandell

Ok. Thanks for replying. I am still trying to figure it all out and ChatGPT is giving bad results. Back to doing it all myself one step at a time.

This is what I’ve used to normalize sensor input data: Divide each sensor’s reading by the square root of the sum of the squares of all sensor readings.

Which in the above example using your code gives,

0.182574178987209
0.365148357974418
0.547722536961627
0.7302967159488359

Will return if I can come up with a coherent properly tested example of what you are trying to duplicate.

Hi @oliver2003

I’m working out all the bits and pieces I’ll need to run this experiment on a Pico W.  I’ll need an image sensor, 4 distance sensors, 3 ambient light sensors and a reflectance sensor pointed at the floor.  Along with the sensors and motors I need some way to communicate with a host to turn on a light and to log the fitness function results for each run.  I think that the Pico W has the mojo to handle all this.  

Here are the sensors that I’ve selected:

Arducam’s HM01B0 image sensor board: https://docs.arducam.com/Arduino-SPI-camera/Legacy-SPI-camera/Pico/Camera-Module/Arducam-HM01B0-QVGA-Camera-Module/

Pololu’s VL53L1X breakout board for the distance/ambient light sensors: https://www.pololu.com/product/3415

Pololu’s QTRX-MD-01RC for the floor reflectance sensor: https://www.pololu.com/product/4341

The VL53L1X time of flight sensor is pretty impressive.  It has a 16 by16 receiving array of single photon avalanche diodes (SPAD) that you can programmatically divide into multiple regions of interest (ROI).  More jargon… I tried dividing it vertically into 4 equal sized ROIs and was able to identify the leg of a table!  Which is way better then my ancient iRobot floor vacuum can do.

The HM01BO image sensor also has a fair number of knobs that you can turn.  Frame size, exposure…

Floreano and Urzelai used a Khepera robot in their paper.  I’m hopeful that I’ll be able to tune these sensors and get the same performance.

Tom

@thrandell

It should be a very educational project learning to use those sensors.

I’m hopeful that I’ll be able to tune these sensors and get the same performance.

There is no reason I can see that you wouldn't be able to get the same performance.

One practical issue is the number of attempts required and the time to run so many trials using a physical robot.

In the article they wrote: "The experiments have been carried out first in simulations sampling sensor activation and adding 5% uniform noise to these values."

Something you might consider. A simulator can run thousands (millions?) of trials in a very short time. Modern AI did not need physical actions of a robot body and thus could evolve by testing their connections millions of times in seconds on super computers.

The publication, which I have been carefully reading, covers a lot of other considerations which are familiar to me but I don't want to distract from what you are trying to do here.