My personal training robot

@robotbuilder 

I have just started programming and have only learned with Python so far. I have used Open CV on the ESP32 as well as on my pc with my own webcams, I really like it because there is so much documentation and tutorials on it.

one thing I am wondering about, and this might be a dumb question, is I should be able to build an app for my pc that can take the image from the esp 32 and do the image recognition and distance measuring on my pc and just send commands to the robot via Bluetooth/Wi-Fi, so essentially my pc is the controller. Then essentially you just write a machine learning program to automate the robot....  add voice controls and make it a chat bot with a mobile arm 🙂

@dryden

to use opencv with your pc I would consider getting the image from a wireless webcam and then send suitable robot commands to the  esp32 robot controller.  You may be able to use an image from the esp32 on the pc, but on a gut feel I think it may be challenging.

@byron 

I was wondering about that.

What I use for video feed in my other Open CV projects goes something like 

cv2.videocapture(0)

Can I not put 

cv2.videocapture( http://192.xxx.xxx) what ever the web page is for the esp 32?

I haven't played around enough to know 

@dryden

I was thinking it could be challenging to get a camera linked to an esp32 to stream output over wifi.  I should have done a quick google as I see plenty of examples doing just that.  OpenCV can certainly work with an ip video stream so I think if the frame rates are good enough you should be ok. 😎 

@byron 

ya, its an esp32 AI thinker, it has a cam on it.

Just some new pix now that I put the new base on it.

Ok so I gave up on the shift registers and may revisit again in the future, in the mean time I bought an Arduino mega 2560, and using simple Arduino libraries I can control all the motors on the robot. My new problem I am having is building functions on the Mega that the Esp32 Cam can call... I've been experimenting with serial communication, and I apparently don't understand it... Here is the code that I used in my experiment.

On the Arduino mega I have this code:

#include <Stepper.h>
#include <L298N.h>
const int stepsPerRevolution = 2048;  // change this to fit the number of steps per revolution
// for your motor

Stepper myStepper1(stepsPerRevolution, 28, 30, 29, 31);
Stepper myStepper2(stepsPerRevolution, 32, 34, 33, 35);
Stepper myStepper3(stepsPerRevolution, 36, 38, 37, 39);
Stepper myStepper4(stepsPerRevolution, 40, 42, 41, 43);
Stepper myStepper5(stepsPerRevolution, 44, 46, 45, 47);
Stepper myStepper6(stepsPerRevolution, 48, 50, 49, 51);

const unsigned int IN1 = 23;
const unsigned int IN2 = 22;
const unsigned int EN_A = 5;
const unsigned int IN3 = 25;
const unsigned int IN4 = 24;
const unsigned int EN_B = 6;



void setup() {
  myStepper1.setSpeed(4);
  myStepper2.setSpeed(4);
  myStepper3.setSpeed(4);
  myStepper4.setSpeed(4);
  myStepper5.setSpeed(4);
  myStepper6.setSpeed(4);

  L298N motor1(EN_A, IN1, IN2);
  L298N motor2(EN_B, IN3, IN4);
  
  // initialize the serial port:
  Serial.begin(115200);
  Serial.println("Initializing Contact...");
  Serial2.begin(9600);

  while (!Serial2)
  {
    //do nothing
  }

}

void loop() {

  while  (Serial2.available()) {
  int msg = Serial2.read();
  
  L298N motor1(EN_A, IN1, IN2);
  L298N motor2(EN_B, IN3, IN4);
  
  if (msg = "stop move");
  motor1.stop();
  motor2.stop();

  if (msg = "move forward");
  motor1.setSpeed(250);
  motor2.setSpeed(250);

  motor1.forward();
  motor2.forward();

  if (msg = "move backward");
  motor1.setSpeed(250);
  motor2.setSpeed(250);

  motor1.backward();
  motor2.backward();

  if (msg = "pivot left");
  motor1.setSpeed(200);
  motor2.setSpeed(200);

  motor1.forward();
  motor2.backward();

  if (msg = "pivot right");
  motor1.setSpeed(200);
  motor2.setSpeed(200);

  motor2.forward();
  motor1.backward();
  }


}

on the Esp32 I up loaded this simple code to test if it worked:

void setup() {
Serial.begin(9600);

while (!Serial);
{
//do nothing
}
}

void loop() {

while (Serial.available())

delay(3000);

Serial.println("move forward");

delay(3000);

Serial.println("stop move");

delay(1000);

Serial.println("move backward");

delay(3000);

Serial.println("stop move");

delay(1000);

Serial.println("pivot right");

delay(3000);

Serial.println("stop move");

delay(1000);

Serial.println("pivot left");

delay(3000);

Serial.println("stop move");

delay(1000);

}

No real surprise here but It dsnt work!!!

I'm going to try Serial.write(stop move) and see if that has any affect... but I doubt it. any suggestions on what a normal way to communicate from the esp32 ai thinker to an Arduino would be? Is UART the same as serial communication? Should I be using the SCL/SDA pins on the Mega? Im spending the day pouring over this to figure it out. Any advice would be great.

@dryden 

A quick observation (but maybe off-topic) ...

I would strongly suggest that you use the AccelStepper and MultiStepper classes from AirSpayce instead of the standard Arduino stepper library.

Defining your steppers as AccelSteppers will give you speed control and acceleration control right out of the box. Linking them together with MultiStepper will allow you to control them as a group for synchronized movement.

Think of all the code you won't need to write.

One caveat is that the wiring is a bit picky, so test with one ULN driver and one BYJ and then set the rest of them up to the same pin arrangement that worked.

By the way:

Be prepared to replace your arm's BYJ steppers and ULN drivers with something like NEMA17 steppers with, maybe, A4988 drivers so that your arm can lift something besides itself 🙂

@dryden 

Doesn't Serial2.read() just read in a single byte ?

Also, in C, a single '=' is used to assign a value. When performing a logical test you should use '=='.

So

if (msg = "stop move");

should be

if (msg == "stop move");

@will 

Posted by: @will

Be prepared to replace your arm's BYJ steppers and ULN drivers with something like NEMA17 steppers with, maybe, A4988 drivers so that your arm can lift something besides itself

That is for sure!!! This thing can barely lift itself!!! BARELY!!! lol witch brings me to why I am not using multi Stepper, right now there dsn't seem to be enough power to move more than one stepper at a time, I can do it, but it dsn't seam to work as well as just moving one at a time. the stepper library will allow you to move more than one at a time too... Not sure I'm doing it right? but in the loop if I call 2 separate steppers without a delay between them they both move at once. when I have the money for some nema17's we will definatly be trying to move more than one at a time.

Posted by: @will

One caveat is that the wiring is a bit picky, so test with one ULN driver and one BYJ and then set the rest of them up to the same pin arrangement that worked.

For my stepper motors with all the Arduino libraries I'm finding they are setting them up as 

Stepper myStepper(stepsPerRevolution, 8, 9, 10, 11);

Where I have to change this to be 

Stepper myStepper(stepsPerRevolution, 8, 10, 9, 11);

I call this changing the firing order, as I understand in vehicle mechanics its like changing the firing of each piston. This is to me like a 4 cylinder engine. 

@will 

I will give that a go, but being that I believe that I am using Python, all my other stuff is just =

Posted by: @will

Doesn't Serial2.read() just read in a single byte ?

Does it just read single byte? I dont know? I just figured if I could Serial.print a string, I figured it would also read that string.... 

Posted by: @dryden

Posted by: @will

Doesn't Serial2.read() just read in a single byte ?

Does it just read single byte? I dont know? I just figured if I could Serial.print a string, I figured it would also read that string.... 

But when it's sending it already knows that there's data to move, how much and where it is. When it's reading, it doesn't know how much there is at the start.

@will

Posted by: @will

Also, in C, a single '=' is used to assign a value. When performing a logical test you should use '=='.

I tried changing that part and got error msg ISO C++ forbids comparison between pointer and integer [-fpermissive}

Posted by: @dryden

@will 

why I am not using multi Stepper, right now there dsn't seem to be enough power to move more than one stepper at a time, I can do it, but it dsn't seam to work as well as just moving one at a time. the stepper library will allow you to move more than one at a time too... Not sure I'm doing it right? but in the loop if I call 2 separate steppers without a delay between them they both move at once.

AccelStepper uses a different strategy to move. You set a target position for the stepper and then call the run() method each time through the loop. You can also tell it to move directly to the target (thereby blocking all other activity until it reaches that target).

It's a different mind set, but it has to be because it includes so many other features for free.