Notifications
Clear all

using c++

30 Posts
5 Users
15 Likes
410 Views
Inq
 Inq
(@inq)
Noble Member
Joined: 11 months ago
Posts: 982
 
Posted by: @frogandtoad

Actually, I am aware of printf, but that wasn't the main objective of my example and comment - The main takeaway was the dynamic array, the range based for loop, and that doing that in C was much more difficult, and harder to learn - Do you disagree?

I was taking your bullets as individual, independent aspects.  As I have never used the cout versions of C++ because

  • In windows C++, I was always use the debugger and any I/O was to/from database or Windows GUI elements.  There was never a need to use cout do a console.  And I tended to use C#:C++ about 90:10.
  • In MPU's, I was used to bitching and moaning about printf and was comfortable in my ignorance. 😆 Especially since I wrap  printf only once in the code base.  
Posted by: @frogandtoad

No, I have never tried it for such duration, so I can't say that there are any issues in doing that. ISO C++ improves in every iterations, so it might be worth a new test 😉

You know... I've never really looked into the C++ code base.  Not even sure it's possible.  As a corporate developer, we simply weren't allowed to even look at Open-Source code for legal/ethical reasons.  I'm so used to that, it stuck.  

But I'll take this opportunity to write a response to this as a instructional piece for this thread.

VBR,

Inq

 

3 lines of code = InqPortal = Complete IoT, App, Web Server w/ GUI Admin Client, Access Point Manager, Drag & Drop File Manager, OTA, Performance Metrics, Web Socket Comms, Easy App API, All running on ESP8266...
Even usable on ESP-01S - Quickest Start Guide


   
ReplyQuote
Inq
 Inq
(@inq)
Noble Member
Joined: 11 months ago
Posts: 982
 

For those interested in how things work... in our INO/C/C++/Python code development.

  • Dynamic allocations:  new, malloc, delete, free and things like the String object, and these dynamic arrays that do malloc/free under the covers allocate small chunk of memory.  Also... Python does a butt-load of this ALL the time.
  • Depending on the time and size of the allocations and freeing of memory, the limited space of the heap get peppered with used pieces of memory. 
  • The holes in between are now not contiguous.  If even one memory allocation asks for a piece bigger than any available hole, it fails. 
  • Most developers don't check for this failure... me include (most of the time). 
  • This is probably the worst bug of all to find - mainly because the code doesn't have a bug.  It was just dumb, bad luck of the randomness of data being dynamically allocated and freed.
  • What appears as a random reboot might happen days down the road if you use ANY of these "features" to a large extent.
  • On big CPUs (Windows, etc) this is really never an issue.  
    1. They have a huge memory space to pepper.
    2. They have memory cleaning processes where it'll relocates these memory chunks to be contiguous.  IOW take out the holes and leave some big, clean space.
  • For ages... MPU recommendations were to NOT use these dynamic features AT ALL!  (<32K heap days)
  • Then later it was allocate at start-up and handle all the data usage down to the gnat's ass yourself within the allocated area.
  • Now that we have near 1MB spaces, people have started treating them like CPU's (me included)
  • To my knowledge, they do not have the CPU memory cleaning ability.
  • There are several ways of avoiding the peppering/holes issue
    1. Don't care if your processor reboots occasionally - This is a main reason for the ESP having a Watch Dog process.  If YOU write bad code, the ESP, will re-boot your ass.  99.9% of all developers rely on this for their stability fallback.
    2. Don't use dynamic code instructions - That would rule out Python.
    3. Do excruciating detailed bookkeeping on your used dynamic memory allocations and implement your own memory cleaning algorithm. 
    4. Allocate and free systematically.  For instance a FIFO buffer (stand alone) allocates a new chunk at the end and deletes the first chunk.  The used memory space just moves through memory.  Holes are not created.  BUT if you have some other portion of your code using a String object it'll allocate the space potentially in the middle of your FIFO inch-worm.  As things get freed, a hole forms.

IOW this is not only an advanced topic, it is usually one that only keeps compiler developers and operating system developers up at night.

VBR,

Inq

 

3 lines of code = InqPortal = Complete IoT, App, Web Server w/ GUI Admin Client, Access Point Manager, Drag & Drop File Manager, OTA, Performance Metrics, Web Socket Comms, Easy App API, All running on ESP8266...
Even usable on ESP-01S - Quickest Start Guide


   
ron bentley reacted
ReplyQuote
Inq
 Inq
(@inq)
Noble Member
Joined: 11 months ago
Posts: 982
 
Posted by: @frogandtoad

Yes, you can do everything you desire, and use the mouse to navigate through the 3D environment to your imagination - p5.js offers WebGL (OpenGL), directly in the browser, camera, lighting, textures, etc...

Would you happen to know... 

  1. Can it be off-line?  For instance, I used Google Charts for a while till I found you could not pull it down and put it on a server.  You are required to just add the link to their server in your HTML/Scripts.  IOW, an Internet connection is required.
  2. IF it and all its dependencies were legally pulled down to put on say and ESP8266 server, what would be the size of all that?  I guess I could always use the 15MB space on the ESP8266 Pro, but I'd prefer to avoid that restriction.

Don't go to any trouble... just if you have experimented with that aspect.  Sometime I'll get to the library and try pulling it down if I go that route.

As is my tendency, I may just fallback and use first principles and write my own 3D rendering library.  Sounds like a fun challenge.

VBR,

Inq

3 lines of code = InqPortal = Complete IoT, App, Web Server w/ GUI Admin Client, Access Point Manager, Drag & Drop File Manager, OTA, Performance Metrics, Web Socket Comms, Easy App API, All running on ESP8266...
Even usable on ESP-01S - Quickest Start Guide


   
ReplyQuote
robotBuilder
(@robotbuilder)
Noble Member
Joined: 4 years ago
Posts: 1661
Topic starter  
Posted by: @frogandtoad

@robotbuilder,

Here is a simple program example for you using a standard vector<string> type on an ESP32 - This is how arrays in real C++ have been used for over a decade:

# include <iostream>
# include <string>
# include <vector>

  using namespace std;

vector<string> myArray {"one", "two", "three", "four", "five"};

void setup() {
  myArray.push_back("six");
  myArray.erase(myArray.begin() + 2);
 }

void loop() {  

  for(string element : myArray) {
    cout << "Element: " << element << '\n';
   }
    
  cout << "\nmyArray contains: " << myArray.size() << " elements\n\n";

  delay(1000);
 }

Note:
1) No Serial object, no print statement (can mix and match different data types to stdout)
2) A dynamic array, with an element later added to the end, and then one element removed, and uses an internal member function to print its size
3) Range based for loop

Now, I know what it takes to reproduce this in C, and handle all the memory issues on your own, and the amount of code you need to write over and above what's shown, not to mention, this is very self documenting - I'm sure the C version is quite an ugly duckling in comparison.

The vector resizes and reallocates all the memory for you - This is the tip of the iceberg, but as you can already hopefully see, just how superior it is!

Please feel free (anyone), to criticise any pro's or con's you may see.

Cheers

I will have to create an ESP32 folder 🙂

Copied your code to CODE::BLOCKS to try it out.
I am able to write console programs I just didn't have much success installing graphic libraries.

 

# include <iostream>
# include <string>
# include <vector>

using namespace std;

vector<string> myArray {"one", "two", "three", "four", "five"};

void setup() {
  myArray.push_back("six");
  myArray.erase(myArray.begin() + 2);
 }

void loop() {

  for(string element : myArray) {
    cout << "Element: " << element << '\n';
   }

  cout << "\nmyArray contains: " << myArray.size() << " elements\n\n";

 }

 int main(){
   setup();
   loop();
 }

C++ programmers have made a lot of C++ stuff available to FreeBASIC programmers.

Implementation of dynamic data types for Freebasic. This implementation is built on the basis of templates (macros), which makes it possible to use it for almost any data type.

In stock:

1) MAP - dictionary (associative array), created on the basis of a balanced tree
2) VECTOR - dynamic array, with the ability to insert and delete any number of cells and much more useful opportunities
3) LIST - doubly linked list
4) QUEUE
5) STACK
6) HashTable - dictionary (unordered data).

 

 

#include "VECTOR.bi"

type Q
   i as Long
   s as Zstring*10
End Type

type pQ as Q ptr

MVectorTemplate(pQ ,  1 )

dim V as TVECTORPQ

for i as Long = 0 to 10
  dim t as Q
  t.i = i
  t.s = "value" & i
  V.push_back(@t)
Next

for i as Long = 0 to V.size()-1
  ? V[i]->i , V[i]->s
Next

V.clear()
Sleep

end

 

 


   
ReplyQuote
robotBuilder
(@robotbuilder)
Noble Member
Joined: 4 years ago
Posts: 1661
Topic starter  

@frogandtoad 

For me the language is secondary to the algorithm which in theory could be written in any language.

Being able to quickly convert from one language to another was always an issue.  It used to be translating Z80 to 6502 to 68000 to x86. I would like to be able to write code in a way that makes it easy to produce a Python version and a Processing version.

The problem with using a language's special features is in translating to another language.

Sticking to the features standard to all languages makes the translation easier.

 


   
ron bentley reacted
ReplyQuote
Will
 Will
(@will)
Famed Member
Joined: 2 years ago
Posts: 2235
 
Posted by: @robotbuilder

Sticking to the features standard to all languages makes the translation easier.

And makes the code available to a much broader audience.

Experience is what you get when you don't get what you want.


   
ron bentley reacted
ReplyQuote
frogandtoad
(@frogandtoad)
Noble Member
Joined: 4 years ago
Posts: 1506
 

@inq

Posted by: @inq
Posted by: @frogandtoad

Actually, I am aware of printf, but that wasn't the main objective of my example and comment - The main takeaway was the dynamic array, the range based for loop, and that doing that in C was much more difficult, and harder to learn - Do you disagree?

I was taking your bullets as individual, independent aspects.  As I have never used the cout versions of C++ because

  • In windows C++, I was always use the debugger and any I/O was to/from database or Windows GUI elements.  There was never a need to use cout do a console.  And I tended to use C#:C++ about 90:10.
  • In MPU's, I was used to bitching and moaning about printf and was comfortable in my ignorance. 😆 Especially since I wrap  printf only once in the code base.  
Posted by: @frogandtoad

No, I have never tried it for such duration, so I can't say that there are any issues in doing that. ISO C++ improves in every iterations, so it might be worth a new test 😉

You know... I've never really looked into the C++ code base.  Not even sure it's possible.  As a corporate developer, we simply weren't allowed to even look at Open-Source code for legal/ethical reasons.  I'm so used to that, it stuck.  

But I'll take this opportunity to write a response to this as a instructional piece for this thread.

VBR,

Inq

You can easily download a draft copy of the ISO C++ Standard for various versions in a google search.  No worries, I'll check it out, and you haven't answered my question either 😉

Cheers


   
ReplyQuote
frogandtoad
(@frogandtoad)
Noble Member
Joined: 4 years ago
Posts: 1506
 

@inq

Posted by: @inq
Posted by: @frogandtoad

Yes, you can do everything you desire, and use the mouse to navigate through the 3D environment to your imagination - p5.js offers WebGL (OpenGL), directly in the browser, camera, lighting, textures, etc...

Would you happen to know... 

  1. Can it be off-line?  For instance, I used Google Charts for a while till I found you could not pull it down and put it on a server.  You are required to just add the link to their server in your HTML/Scripts.  IOW, an Internet connection is required.
  2. IF it and all its dependencies were legally pulled down to put on say and ESP8266 server, what would be the size of all that?  I guess I could always use the 15MB space on the ESP8266 Pro, but I'd prefer to avoid that restriction.

Don't go to any trouble... just if you have experimented with that aspect.  Sometime I'll get to the library and try pulling it down if I go that route.

As is my tendency, I may just fallback and use first principles and write my own 3D rendering library.  Sounds like a fun challenge.

VBR,

Inq

Yes, it can be off-line - You can download the whole p5.js library script, or use it online via a CDN link.  The one I downloaded previously was just over 4MB - All you need to do is create a server on your ESP8266 for it to run.

Check out the orbit control under the 3D link for some interactive examples - The documentation is great, and the code is very simple.

Cheers


   
ReplyQuote
frogandtoad
(@frogandtoad)
Noble Member
Joined: 4 years ago
Posts: 1506
 

@inq

Posted by: @inq

For those interested in how things work... in our INO/C/C++/Python code development.

  • Dynamic allocations:  new, malloc, delete, free and things like the String object, and these dynamic arrays that do malloc/free under the covers allocate small chunk of memory.  Also... Python does a butt-load of this ALL the time.
  • Depending on the time and size of the allocations and freeing of memory, the limited space of the heap get peppered with used pieces of memory. 
  • The holes in between are now not contiguous.  If even one memory allocation asks for a piece bigger than any available hole, it fails. 
  • Most developers don't check for this failure... me include (most of the time). 
  • This is probably the worst bug of all to find - mainly because the code doesn't have a bug.  It was just dumb, bad luck of the randomness of data being dynamically allocated and freed.
  • What appears as a random reboot might happen days down the road if you use ANY of these "features" to a large extent.
  • On big CPUs (Windows, etc) this is really never an issue.  
    1. They have a huge memory space to pepper.
    2. They have memory cleaning processes where it'll relocates these memory chunks to be contiguous.  IOW take out the holes and leave some big, clean space.
  • For ages... MPU recommendations were to NOT use these dynamic features AT ALL!  (<32K heap days)
  • Then later it was allocate at start-up and handle all the data usage down to the gnat's ass yourself within the allocated area.
  • Now that we have near 1MB spaces, people have started treating them like CPU's (me included)
  • To my knowledge, they do not have the CPU memory cleaning ability.
  • There are several ways of avoiding the peppering/holes issue
    1. Don't care if your processor reboots occasionally - This is a main reason for the ESP having a Watch Dog process.  If YOU write bad code, the ESP, will re-boot your ass.  99.9% of all developers rely on this for their stability fallback.
    2. Don't use dynamic code instructions - That would rule out Python.
    3. Do excruciating detailed bookkeeping on your used dynamic memory allocations and implement your own memory cleaning algorithm. 
    4. Allocate and free systematically.  For instance a FIFO buffer (stand alone) allocates a new chunk at the end and deletes the first chunk.  The used memory space just moves through memory.  Holes are not created.  BUT if you have some other portion of your code using a String object it'll allocate the space potentially in the middle of your FIFO inch-worm.  As things get freed, a hole forms.

IOW this is not only an advanced topic, it is usually one that only keeps compiler developers and operating system developers up at night.

VBR,

Inq

 

Indeed, this is a very advanced topic, and you've highlighted a lot of critical points for those who want to delve deep into this stuff - I'll add one more thing you didn't directly mention, is that memory fragmentation of the heap is most likely a problem with interpreted languages, which use some form of garbage collection to address this issue.

Note: The ISO C++ is very careful with this kind of thing, and it uses special memory allocators in clever ways... as such, the ISO standard guarantees that a C++ vector's elements are contiguous.

Cheers


   
ReplyQuote
frogandtoad
(@frogandtoad)
Noble Member
Joined: 4 years ago
Posts: 1506
 

@robotbuilder 

Posted by: @robotbuilder

I will have to create an ESP32 folder 🙂

Copied your code to CODE::BLOCKS to try it out.
I am able to write console programs I just didn't have much success installing graphic libraries.

No problem,

All you have to do is update your library and compiler paths in the options/preferences, and ensure you also have a windows environment variable set to the compiler path.

Posted by: @robotbuilder

C++ programmers have made a lot of C++ stuff available to FreeBASIC programmers.

Implementation of dynamic data types for Freebasic. This implementation is built on the basis of templates (macros), which makes it possible to use it for almost any data type.

In stock:

1) MAP - dictionary (associative array), created on the basis of a balanced tree
2) VECTOR - dynamic array, with the ability to insert and delete any number of cells and much more useful opportunities
3) LIST - doubly linked list
4) QUEUE
5) STACK
6) HashTable - dictionary (unordered data).

Indeed, C++ also offers these containers and many more.
Using templates in C++ is on the advanced side of things, but they are not that hard if you really put your mind to it - They can cut your C code by a 3rd or more in many cases, and add a whole lot of flexibility, lending to creating some really advanced stuff, but too advanced for microcontroller programming.

Cheers


   
ReplyQuote
frogandtoad
(@frogandtoad)
Noble Member
Joined: 4 years ago
Posts: 1506
 

@robotbuilder

Posted by: @robotbuilder

@frogandtoad 

For me the language is secondary to the algorithm which in theory could be written in any language.

Being able to quickly convert from one language to another was always an issue.  It used to be translating Z80 to 6502 to 68000 to x86. I would like to be able to write code in a way that makes it easy to produce a Python version and a Processing version.

The problem with using a language's special features is in translating to another language.

Sticking to the features standard to all languages makes the translation easier.

 

That's true, and I believe that what you call algorithm, I call programming concepts and fundamental constructs - That's what makes a programmer, not the language, which is just the tool.

Anyway, this is a different topic in and of itself - All I'm trying to do is promote the use of more modern C++ features, for use in the more powerful microcontrollers, for those people who want to learn more and make their programming life easier.

Cheers


   
ReplyQuote
frogandtoad
(@frogandtoad)
Noble Member
Joined: 4 years ago
Posts: 1506
 

@robotbuilder 

Posted by: @robotbuilder

Sticking to the features standard to all languages makes the translation easier.

Quite true!

That's why everything in my code examples use standard built in features since C++11.

Cheers


   
ReplyQuote
frogandtoad
(@frogandtoad)
Noble Member
Joined: 4 years ago
Posts: 1506
 

@robotbuilder, et, al...

For completeness, I've just put together a 'C' version alongside a 'C++' in this same post, so that everyone can compare which one they think is easier, especially for beginners:

C++ version: - Added iterator for the element search, so no magic numbers:

# include <iostream>
# include <string>
# include <vector>
# include <algorithm>
  using namespace std;

vector<string> myArray {"one", "two", "three", "four", "five"};

void setup() {
  myArray.push_back("six");

  // Find and remove the 3rd element from our array
  string element = "three";
  vector<string>::iterator position = find(myArray.begin(), myArray.end(), element);
  myArray.erase(position);
 }

void loop() {  

  for(auto element : myArray) {
    cout << "Element: " << element << '\n';
   }
    
  cout << "\nmyArray contains: " << myArray.size() << " elements\n\n";

  delay(1000);
 }

C version: - Using 'calloc' & 'realloc' to create and manipulate a dynamic array to mimic the C++ version above:

# include <stdio.h>
# include <stdlib.h>
# include <string.h>

# define ARRAY_SIZE 5
# define STRING_LENGTH 30

void setup() {
  //...
 }

void loop()
 {
  int idx = 0; // For C programming...

  char** myArray = (char**)calloc(ARRAY_SIZE, sizeof(char*));

  for(idx = 0; idx < ARRAY_SIZE; idx++) {
    myArray[idx] = (char*)calloc(STRING_LENGTH, sizeof(char));
   }

  strcpy(myArray[0], "one");
  strcpy(myArray[1], "two");
  strcpy(myArray[2], "three");
  strcpy(myArray[3], "four");
  strcpy(myArray[4], "five");

  // Resize our array to add a new element
  myArray = (char**)realloc(myArray, (ARRAY_SIZE + 1) * sizeof(char*));
  myArray[5] = (char*)calloc(STRING_LENGTH, sizeof(char));
  strcpy(myArray[5], "six");

  // Remove the memory we allocated for the 3rd element
  free(myArray[2]);

  // Remove the 3rd element from our array
  for (idx = 3; idx < ARRAY_SIZE + 1; idx++) {
    myArray[idx - 1] = myArray[idx];
   }

  // Resize the array again after removing our element
  myArray = (char**)realloc(myArray, (ARRAY_SIZE) * sizeof(char*));

  for(idx = 0; idx < ARRAY_SIZE ; idx++) {
    printf("%s\n", myArray[idx]);
   }

  // Note: ARRAY_SIZE must be *manually tracked* at each array size modification
  printf("myArray contains: %d", ARRAY_SIZE);
  printf(" elements\n");

  for(idx = 0; idx < ARRAY_SIZE; idx++) {
    free(myArray[idx]); // Free the memory for each pointer in our array
   }

  free(myArray); // Free the memory for the pointer for our array

  delay(2000);
 }

I hope that by seeing this difference alongside each other, people might start to wake up and stop criticising cleaner, safer and easier to learn and read code.

Cheers


   
ReplyQuote
Inq
 Inq
(@inq)
Noble Member
Joined: 11 months ago
Posts: 982
 
Posted by: @frogandtoad

The main takeaway was the dynamic array, the range based for loop, and that doing that in C was much more difficult, and harder to learn - Do you disagree?

Well... if you expect everyone to answer every question you pose, you're just setting yourself to be disappointed.  I have no valid opinion is why I did not answer.  I used dynamic List, Dictionary, etc on Windows all the time but don't use it on MPU's... for the stated reasons I wrote in the above long winded post above.  Yes, it is easier.  Yes, I could get away with it any many of my projects, but I don't care to. 

Posted by: @frogandtoad

Yes, it can be off-line - You can download the whole p5.js library script, or use it online via a CDN link.  The one I downloaded previously was just over 4MB - All you need to do is create a server on your ESP8266 for it to run.

Check out the orbit control under the 3D link for some interactive examples - The documentation is great, and the code is very simple.

I will definitely take a look at it when I start getting the point cloud data pumping.  It sounds impressive.

VBR,

Inq

3 lines of code = InqPortal = Complete IoT, App, Web Server w/ GUI Admin Client, Access Point Manager, Drag & Drop File Manager, OTA, Performance Metrics, Web Socket Comms, Easy App API, All running on ESP8266...
Even usable on ESP-01S - Quickest Start Guide


   
ReplyQuote
frogandtoad
(@frogandtoad)
Noble Member
Joined: 4 years ago
Posts: 1506
 

@inq

Posted by: @inq

Yes, it is easier.  Yes, I could get away with it any many of my projects, but I don't care to. 

Thanks for at least acknowledging that it was easier!

Cheers


   
ReplyQuote
Page 2 / 2