Notifications
Clear all

Parsing serial data  

  RSS

MadMisha
(@madmisha)
Eminent Member
Joined: 8 months ago
Posts: 44
2020-08-15 10:43 pm  

I am using an ESP32 and a Nextion screen. I am sending serial data from the screen to the ESP32. This is the format:

1*72*1*5*2*1*SEND#

I am trying to parse them out into an array for use. There is another set of data that it sends but it has more numbers between the delimiters(first one is an IP address). Send is what I am using as the command to execute.

 

I currently have it comming into the ESP and put it into a string so that I can easily print it for debug. That is working fine. I am trying to use strtok but it needs to be in a char array.

 

I first tried to make the char array size by using .length on my string but it said it needed to be a constant. So I just made it 80 to try but the strcpy says too many arguments on the new char array. And it says I have too many arguments in strtok.

 

Are there any suggestions? I am not opposed to taking the serial data directly to a char array. It would be preferred that I get it as a whole to do some error checking before it puts the numbers into my final array.

 

    while (Serial2.available()){
char Smake = (char(Serial2.read()));
if (String(Smake) != "#") {
comms += String(Smake);

}
else {
Serial.println(comms);
parseData();
Serial.print(char(Serial2.read()));
//comms = "";
}

void parseData(){
int n = comms.length();
const int m = n + 1;

char commsRet[80];
strcpy(commsRet, comms.c_str());

char* token = strtok(commsRet, "*");
// I have more here but I need this solved first

Quote
MadMisha
(@madmisha)
Eminent Member
Joined: 8 months ago
Posts: 44
2020-08-16 12:44 am  

Update:

I changed my code to parse it as it came in. So, here are my concerns.

 

The basic method already relied on the amount of time between receiving characters over serial to do it's business. (9600 baud) I am concerned that there is now so much to do that I will have issues later on. I will be implementing an Android app to connect through wifi to also make adjustments. As a possibility, because there are 2 cores, I could have one core listening to the wired serial and the other to wifi. I have never done that. I would also have to decide what operation should be interrupted to complete it's task.

 

So how much can happen between incoming characters before I would start seeing issues?

 

Here is my possible solution. It works and seems relatively simple. I will be adding functions to execute after this.

 

 

 while (Serial2.available()){
char Smake = (char(Serial2.read()));
if (String(Smake) != "#") {
comms += String(Smake);
if (String(Smake) != "*") {
comms1 += String(Smake);
}
else {
commParse[inputNumb] = comms1;
comms1 = "";
inputNumb = inputNumb + 1;
}

}
else {
commParse[inputNumb] = comms1;
inputNumb = 0;
comms = "";
comms1 = "";
commParse[5] = "NULL";
}
}

 


ReplyQuote
frogandtoad
(@frogandtoad)
Honorable Member
Joined: 1 year ago
Posts: 508
2020-08-16 2:44 pm  

@madmisha

Do you prefer to check and operate on each token on the fly, or store all of the tokens in an array first, and then perform the necessary operation on each token, according to what it is?

 


ReplyQuote
MadMisha
(@madmisha)
Eminent Member
Joined: 8 months ago
Posts: 44
2020-08-16 10:24 pm  

@frogandtoad I would prefer to sort all the tokens in the array first. My goal is to receive the data over serial and then manipulate the data. I might have the solution but I am still concerned about overloading the process and missing data. I am unsure if making the string(what I am using for debug readout of full message and to detect my stop character) and the string array. I guess I could just drop the string.


ReplyQuote
frogandtoad
(@frogandtoad)
Honorable Member
Joined: 1 year ago
Posts: 508
2020-08-17 2:42 am  

@madmisha

In the following example, I am simply printing each token directly from the serial stream:

// Test data: 1*72*1*5*2*1*SEND*

void setup() {
  Serial.begin(9600);
 }

void loop() {
  while(Serial.available() == 0); // Wait until data is sent...

  String token = Serial.readStringUntil('*');
  Serial.println(token);

  if(token == "SEND") {
    Serial.readStringUntil('\n'); // Remove any remaining junk...
    Serial.println("End of tokens... sending data!");
   }
 }

You can add each token to your array, then manipulate it as you wish.

Cheers!


ReplyQuote
MadMisha
(@madmisha)
Eminent Member
Joined: 8 months ago
Posts: 44
2020-08-17 4:47 am  

@frogandtoad I am receiving the data from a Nextion display and there is no function to send println that I know of. I could just include it in what I am sending but since it is working I think I will keep it the same. I am using a character instead. Seeing how simple your code is I might go back and see what I can do to make mine closer to your method. I like simple.

 

But I do have a question about your method. Is it collecting the characters in the serial buffer in your example? It also looks like readStringUntil will clear it like flush. My understanding of the way I did it was that it would take each character as it is received and I would need to put it somewhere.

 

I am currently doing a lot in the time between characters. Most is just there for debug and will be gone soon. Here is what I got so far. It mostly handles the data after it's all received but it also does a lot in between.

 

    while (Serial2.available()){

      char Smake = (char(Serial2.read()));

      if (String(Smake!= "#") { //Use if not stop character

        comms += String(Smake);   //**************Add to string for debug******************************

        if (String(Smake!= "*") { //If not delimiter

          comms1 += String(Smake);  //Add to parse field

        }

        else {      //Once delimiter is found

          commParse[inputNumb= comms1;  //Add complete section to parsed array

          comms1 = "";  //Clear parsing field

          inputNumb = inputNumb + 1;  //Increase index for parsed array

          }

      }

      else {

        commParse[inputNumb= comms1; // Enter last complete data set

        inputNumb = 0;  //Reset index for parsed array

        Serial.println(comms);  //**************************DEBUG***************************

        Serial.println("Parsed:");  //**************************DEBUG***************************

        Serial.println(commParse[0]);  //**************************DEBUG***************************

        Serial.println(commParse[1]);  //**************************DEBUG***************************

        Serial.println(commParse[2]);  //**************************DEBUG***************************

        Serial.println(commParse[3]);  //**************************DEBUG***************************

        Serial.println(commParse[4]);  //**************************DEBUG***************************

        Serial.println(commParse[5]);  //**************************DEBUG***************************

        comms = "";  //Clear whole message String DEBUG********************************************

        comms1 = "";  //Clear parsing field

        if (commParse[5== "send") {

          Serial.println("Sending Executed");  //**************************DEBUG***************************

          sendIR();  //Execute send command

          

        }

        else if (commParse[4== "prop") {

          Serial.println("Prop Executed");  //**************************DEBUG***************************

        }

        commParse[5= "NULL";  //Null out the field what will not be overwritten when shorter data is recieved

        

      }

ReplyQuote
frogandtoad
(@frogandtoad)
Honorable Member
Joined: 1 year ago
Posts: 508
2020-08-17 5:13 am  

@madmisha

Posted by: @madmisha

@frogandtoad I am receiving the data from a Nextion display and there is no function to send println that I know of. I could just include it in what I am sending but since it is working I think I will keep it the same. I am using a character instead. Seeing how simple your code is I might go back and see what I can do to make mine closer to your method. I like simple.

But you're reading from a com port, aren't you?
What is Serial2?

Posted by: @madmisha

But I do have a question about your method. Is it collecting the characters in the serial buffer in your example? It also looks like readStringUntil will clear it like flush. My understanding of the way I did it was that it would take each character as it is received and I would need to put it somewhere.

Yes, exactly... it is collecting the characters from the Serial buffer.  No, readStringUntil() will do exactly what it says 🙂  It will read all characters up to the delimiter you specify, and the next call eats everything up until the new line character.

Here is the reference: readstringuntil

If you do read from the serial buffer as per my example, then just add the token to your array - I just used println to display the output to the serial monitor so you can see it working.


ReplyQuote
MadMisha
(@madmisha)
Eminent Member
Joined: 8 months ago
Posts: 44
2020-08-18 12:13 am  
Posted by: @frogandtoad

But you're reading from a com port, aren't you?
What is Serial2?

It is the second serial port on the ESP32 and where I have the screen connected to. It allows me to debug and send commands from my computer without interruptions to the screen communications. Just another reason I love my ESP32s. The instruction set for the Nextion does not have a println. I would have to add /n manually(that's how I I'm sending my stop character anyways). I see that I could easily change the example you provided to use my # character instead of new line.

 

I will be trying out both methods to see what I prefer and feel the safest with. Thanks for showing me another way of doing this!

This post was modified 2 months ago by MadMisha

ReplyQuote
frogandtoad
(@frogandtoad)
Honorable Member
Joined: 1 year ago
Posts: 508
2020-08-18 5:00 am  

@madmisha

Posted by: @frogandtoad

But you're reading from a com port, aren't you?
What is Serial2?

It is the second serial port on the ESP32 and where I have the screen connected to. It allows me to debug and send commands from my computer without interruptions to the screen communications.

Ok, I see... I had thought it was the second Arduino serial port, but just remembered that was Serial1 anyway.

Posted by: @madmisha

The instruction set for the Nextion does not have a println. I would have to add /n manually(that's how I I'm sending my stop character anyways). I see that I could easily change the example you provided to use my # character instead of new line.

 

I will be trying out both methods to see what I prefer and feel the safest with. Thanks for showing me another way of doing this!

No worries, let us know how you got on.

Cheers!


ReplyQuote