Managing Memory At Compile Time

It's not quite that simple.  If it happens at compile time, you'll get an error and it won't upload.  The problem that you might have been warned about always happens at run time.  It depends how your program allocates memory (heap) or uses temporary memory (stack) at run time.  Since all pieces of a program use both - your specific coding and the libraries you use, you can run out at any time if it does not clean up after itself (memory leaks) or flat out tries to use more than microcontroller can supply.  You usually get a fatal error and it locks the device.  Some micros like ESP8266 and ESP32 have what is called a watchdog process that if it detects such a lockup, it'll simply reboot the device.  Obviously, this is not always a good idea, but we don't have a "dog" in the fight.  Pun intended.  😆 

Here is a topic that might explain it better/differently than I just did.

https://www.esp8266.com/viewtopic.php?p=69937#p69937https://www.esp8266.com/viewtopic.php?p=69937#p69937

@silverstar, The memory numbers you see at compile time are simply the space that the program takes up AND the amount of static, global variables that your program uses.  The stack and heap I described above are dynamic at runtime only and the compiler can't give you any expectation of whether you'll run out or not.

Hi @silverstar,

  @inq has provided you some good hints.

  I don't know about any specific contemporary processors, since it is a long time since I last had to look into these aspects, but the oversimplified story used to be something like:

The compiler + linker starts by allocating memory for the code, plus global variables, and constants etc. that there will only be a single instance. This is the basis of the numbers you show in your screenshot.

-----------

This leaves a chunk of memory that is available for the "rest" of the memory requirements.

--------

The top address of this chunk is where the stack starts .. and it goes downwards.

e.g. when entering a function/subroutine the memory required to save registers, return address, variables local to the function and so on are pushed onto the stack, so the first stack 'still available' address goes down.

If the code calls another function, then the same process repeats, so the stack address continues downwards.

However, when the processor returns from a function,the memory is released, and the address rises.

By contrast, the heap address starts at the bottom of the chunk of available memory.

Here, programming like the C memory allocation instruction malloc(), requests a chunk of memory which is allocated at the heap address, pushing the start of address of unallocated heap memory upwards.

It is possible for the program flow to free up allocated heap memory, but the available memory can get fragmented, etc. causing the top of heap address to progress upwards. 

--------

Note that both the heap and stack processes depends on the program flow, so the compiler/linker cannot predict how much heap and stack memory a program will use, leaving the possibility that the top of the heap will overlap with the bottom of the stack.

--------

Sometimes this causes corruption of the memory contents, which may include program addresses, etc. and the processor will start trying to execute junk codes.

As @Inq says, this condition might be detected by a watchdog, which can force the processor to abort and reset.

-----

It is theoretically possible for the processor to check the stack and heap pointers before they overlap, but the processor's options are very limited, so at best, the processor will probably be forced to run some kind of fatal error exception.

-------

Basically, the answers are a combination of good programming practices and having sufficient memory for the task.

--------

In my explanation I have suggested that the processor has a single block of memory ... in many cases the memory map is more complex than that, but to the best of my knowledge, the principles are unchanged.

Hope this is helpful. Best wishes, Dave

@davee Thank you all for your prompt responses. I think I am reading that just because the numbers do not indicate that too much memory was used, its the runtime process that that might force a problem.

Is there guidance pertaining to the amount of memory used following a compile that could be used as an alarm bell?

@silverstar This is part of what being a programmer is about. First just knowing how stacks and heaps are managed and used is sort of in the 101 category. The answer to your question is to practice good memory management. Managing the stack is generally the easier of the two, just avoid too much recursion. The heap is a bit more problematic. If you will be using it a lot I would build my own memory manager (think linked lists) that allows optimal allocations and freeing, also I would obviously build in 'garbage collection'. If these terms are new to you, I am sure there are a few thousand google hits for you to study. I have been retired for almost 20 years now and foolishly discarded all my old code at some point. Now I am trying to rebuild at least a few of the basics and the list manager is the first.

@thats what I was looking for, thanks

@zander 

i wrote my first program 63 years ago. No language then, just 1's and 0's and no stacks, heaps of whatever.

Hi @silverstar,

  I would generally agree with what Ron @zander says. Assuming my simplistic understanding is still reasonably accurate, then the stack generally keeps 'well-behaved' in that memory is allocated and unallocated on a symmetric basis in normal programming ... assuming as Ron points out, you carefully watch recursion.

By contrast, allocation on the heap can easily become chaotic if the programmer is not very careful.

But don't think that the stack is 'bullet-proof' ... calling a function like

main ()
{
   greedy ();
}

void greedy (void)
{
  int numbers [1000000];

     < code that does things with array 'numbers' and reports the result to the user> 

}

will obviously give a problem to microcontroller with 50 kBytes of memory, as it tries to set up the stack on entry to function greedy.

By the way, if you build some simple test programs to check out things like this, be aware that compilers are sometimes smart enough to realise that variables, etc. are not actually being used, and delete them from the program, so make sure your test program actually uses them all.

-----

Another point, is that micocontrollers usually have non-volatile memory, which is used to hold the user program, because their contents are retained through power  cycles, and volatile memory (RAM), which is used for holding variables, stack, heap etc.

Obviously the different properties of these memories means that their allocations are not interchangeable.

Best wishes, Dave

@silverstar I hear you, nice to know there is another old timer here, my first computer was a 1959 Ferranti Packard Canada Analog. It was a gift to my high school. It couldn't do much, the inputs were a plugboard and the output was a meter. Most of the folks playing with the Arduino and such have no idea that 80% to 90% of the code is already written for them in terms of the visible and invisible libraries. I have no wish to go back to 14 KLOC programs in assembler, I am only interested in the result, and how I get it is no longer important to me.

In terms of your concern, try asking in the Arduino forum, somebody there may have a more detailed and useful answer than my generic answer.

@davee To add to @davee advice, look for the macros that control variable allocations. I don't have the exact names in my head at the moment, but something like PROGMEM and the F for Flash in Serial.println(F("Hello, I am stored in FLASH now not SRAM"));

There is one more at least and it is used for Interrupt code to store the ISR in the fastest memory, it is the macro IRAM_ATTR.

A quick google will give you syntax details etc. 

Good luck.

Posted by: @silverstar

↑

@davee Thank you all for your prompt responses. I think I am reading that just because the numbers do not indicate that too much memory was used, its the runtime process that that might force a problem.

 

Is there guidance pertaining to the amount of memory used following a compile that could be used as an alarm bell?

There is no guidance that would be accurate.  It all depends on how you are using the memory.  It would make it easier if you specify the type of microcontroller you are using and how you're using them.  Do you use heap functions (malloc, new, free, delete) in you Sketches?  Are you simply taking a reading of a sensor and displaying it on every loop()?  Are you doing some convoluted branching logic that changes on every loop()?  What kind of libraries are you using?  

For instance...

1. Using an Arduino board there are no hidden things going on.  So everything in your Sketch is at your control.  If you stay away from the heap using functions (malloc, calloc, new, free, delete) you're not using the heap.  The stack takes care of itself.  If your program runs through the loop and accesses every piece of code on every loop and it doesn't die in the first second, it'll probably run forever.  This is also true for some libraries you might be using.  Other libraries might misbehave cause memory leaks and die after hours, days or months of usage.
2. At the other end of the spectrum, are boards like the ESP8266 and ESP32 that if you are using WiFi or Bluetooth and possibly hosting a website that might have one or hundreds of browsers connecting, your processor might die in the first second or after hundreds of days.  In that case, you have no way to see how code beneath your sketch may be allocating and deleting willy-nilly and building up fragmented heap usage.  See the picture in the link I posted above.

Some boards have the ability to query how much memory they have available at any time.  You can use this to see how much you are using based on your program current state.  Here is a graph illustrating this over time as the microcontroller is being asked to do various tasks.

@silverstar Have a look at LINK  I think there are some ideas there.

@silverstar Here is another page to look at LINK

Also, check out the library AvrHeap. It is only for UNO but the concepts are probably portable.