Cloning ESP32 for Arduino Cloud

Hi Thomas EmbSysDev,

  I have not tried doing it, but I note there are YouTube videos on cloning the flash memory contents, presuming that is where the details you want to copy are held, showing use of the 'esptool.py'. In principle, if you could discover the address of the specific details, then perhaps the method could be adapted to that specific cause, but the video I saw was copying the entire memory, presumably including the part normally used for downloading code to the Arduino.

Thus, please be aware, you may end up 'bricking' your ESP32, if the procedure fails, and you cannot fix the problem. Also, if you make a mistake, you could overwrite the 'master' ESP32, and lose the details you wish to clone.

The video I watched was

www.youtube.com/watch?v=2GwzbBn7uRw

but there are also several others. It is not a specific recommendation, just an example. You might like to watch a few.

If you decide to follow such advice, I would recommend you 'practice' on a couple of ESP32s that you do not mind losing their contents. I am assuming the details you want to clone are held in the main flash memory. I have a suspicion some ESP32s have more than 1 memory, so you may need to check and modify the procedure. Also ensure the 'donor' and 'recipient' ESP32 boards match, as there are a lot of different models available.

Of course, anything you do is completely your responsibility and risk. This has the look of a mission that could work easily if everything is done correctly first time, or could go horribly wrong, and be difficult to diagnose and fix, due to a tiny mistake or a mismatch of boards, etc.

Best wishes and good luck, Dave

@embsysdev The ESP-NOW utility will let you clone ESP32 boards. As to whether it would "fool" the IoT Cloud, I'm not sure, but it would be easy enough to try.  I suspect the cloud is looking at the MAC address of the ESP32 to match it with the secret keys, but that's only a guess.

😎

Bill

@embsysdev @davee @dronebot-workshop I haven't played with the IOT Cloud in a while but my gut feel is it is not difficult to do what you want. I can give it a try later today, I think it just requires adding the new board then associating it to the old Thing. I am a big fan of DeMarco's Beneficial Wishful Thinking concept BUT maybe not.

Posted by: @embsysdev

↑

Hello all,

I just set up my first iot project with this tutorial :

Arduino IoT Cloud Fundamentals | 2024 Step-by-Step Guide

It worked flawlessly !

My query is :

Suppose the ESP32 has to be replaced, is it possible to clone a new ESP32 with the same Device ID and Secret Key ?

Cheers

Thomas

My first major IOT project was controlling my RV water system. Because it was a critical system, I had 3 ESP32 boards that were identical. I no longer have the system as the RV is up for sale but I still have the boards that are labelled 1, 2, 3.

I have to admit it does seem strange that it would work as I am sure Bill is correct re MAC and that is hard to change, but I do know it can be spoofed so maybe not that hard.

A quick test will confirm, but I work slow so it will still be a while.

Thanks for the replies !

I guess I misspoke a bit about my requirement.

Actually, I had made a demo for monitoring voltages in a voltage stabilizer.All the code is in a separate uC board. The uC board sends the data to the ESP32 on its second UART.And then I misplaced the ESP32. So I was wondering if it would be possible to use a new ESP board with the old device id and key(which I have downloaded in a pdf).

@embsysdev @davee @dronebot-workshop Unless I am totaly misunderstanding your question, I just simulated replacing one ESP32 for another. All that is needed is to select the sketch them when the editor opens, select a different board. You will of course need to add the board the usual way before hand. I now have two boards running the same sketch.

NOTE: When I plug in the original board, the LED blinks as it should, but looking at the devices page still shows the new board as online. I doubt that will be a problem since the scenario we are testing is a board gone bad, so simply delete it from the cloud.

I have yet to see a convenient fool proof way to 'clone' BUT I do have 2 boards running identical code.

I have done a bit of cleanup, deleting the old board, editing whatever needed to be done and I think even they keys etc are updated now. I am not good at documenting my speedy trips down every rabbit hole but I thought it was obvious.

Bottom line is if your board ceases to work, you can very quickly get a new board working with the old sketch. It will require some amount of hand work to 'balance the books'. Whether the board will work correctly in a more complex multi board, multi thing environment is not tested. I also did not test Triggers or Dashboards. My best guess is some of that will be easy, some may be hard, hopefully none is impossible.

This is as far as I want to go, I think I have answered the question asked and the answer is yes you can clone and it is somewhat easy.

@embsysdev According to what I just attempted, there is no need to worry about the keys, just upload the old sketch to the new board. You may have to fiddle with some associations etc in the cloud but from what I have seen it should still work.

@ron

Sir, you are the man :).

I don't know what I was doing wrong yesterday. But I connected the new ESP32 and again set up the "network" parameters in the "Setup" tab of the "Things" menu.

Now the new ESP shows up Online again. I will hook it up to the stabilizer and check it tomorrow with the dashboard. I think it should work (famous last words ?).

The nice thing is , thanks to the error, I landed up on this forum. The bad thing is I am still trying to locate the other ESP32.

Thanks again for your kind support !!

Thomas

@embsysdev Thomas, my handle is @zander. If you just click the Reply link bottom Right it will stick it in automatically.

Well now I am a bit worried. When things go that well after some hunches on my part I get a little worried.

Since I posted on this Topic before, I had an idea that there was/is a relationship between all the elements of the Cloud (Devices, Things/Sketches, Vars, (Triggers), Widgets and Dashboard) 

So it appears that if a device is changed, the only thing affected is the relationship between Device and Thing/Sketch.

Please keep me informed especially if things go south.

@zander  Well I tested with the dashboard and the stabilizer connected, and it worked perfectly.

Thanks a lot !!

@embsysdev Ok, but I am still nervous. I was all gung ho about this but now with the telescope adapter and GS camera on their way, I am EXTREMELY distracted!

I will do my best to document the relationships between the various pieces of the entire Arduino IOT but also would like to look into the approaches Bill (@dronebot-workshop) and Dave (@davee) discovered.

The part that confuses me and likely Bill and Dave as well, is the role the MAC plays. AFAIK, the MAC is 'hard coded' into the chip, but of course there are ways to 'spoof' it but I doubt this is the case here. I suspect my understanding of 'hard coded' is flawed so maybe someone who knows for sure can explain it. 

HOWEVER, IIRC, when I changed the relationship, the secret key became the secret key of the new board if my memory is correct, then it worked as a result of changing the relationship NOT changing any MAC or secret key.

Comments?

Hi Ron @zander,

  Your query about MAC address is valid, but might not be relevant.

I have no idea what the particular cloud service uses to 'recognise' a client. I think at least one licensing/anti-piracy software system has used the MAC address. However, it is commonly possible (for PCs, routers, etc.), to override the internal default MAC address, so for a 'cloud' service, although it maybe a convenient handle, I have my doubts regarding its 'security' for identification.

Whilst I accept that any particular cloud service might use MAC address as a kind of 'machine-username' (of the usual username+password set), I wonder whether they would bother.

---------

The original answer I gave was to the question "is it possible to clone a new ESP32 with the same Device ID and Secret Key ?", in which I assumed that the ID and Key would be held, in some form, within the non-volatle memory  of the ESP32.  Hence by copying the contents of the entire (non-volatile) memory of one ESP32 to the another ESP32 with the same memory map,, etc, it is likely to be an exact clone, from the server's viewpoint.

So whilst you are right to point out that it might not work, as I hadn't even thought to considering MAC addresses, I think there is a fair chance, it will. Had I been faced with ta similar question, I would have tried cloning in the manner I suggested, and only looked further, if it didn't work.

Best wishes, Dave

NB I have been presuming that the question had a legitimate purpose. I do not condon any illegal behaviour, but as with many things in life, the same knowledge and behaviour can be both 'good' and 'bad', depending upon the context, which is impossible to ascertain with any certainity from a forum mesaage.

NB 2 : I am using the term non volatile memory to include any memory which retains its content through power cycles, regardless of whether its technology, such as flash, etc. I am not specifically referring to memory devices labelled NVM, which can be continually rewritten for diagnostic purposes, etc.

@davee FYI @embsysdev We agree, the MAC can be 'spoofed' it can't be over ridden as the mfg part is hard wired but the net effect is the same AFAIK.

You are right I think the Arduino IOT at least is not using MAC. I had 3 esp32 boards all with the same code running my RV water system. At the time it didn't even occur to me that it would be an issue, I just naively loaded the sketch onto 3 boards all with different secret keys and passwords and it just worked.

While I am sure your method or Bill's (@dronebot-workshop) will work, it isn't that sophisticated, but it may make more sense. 

In order to wrap up this topic, would you kindly document the commands required and the steps etc?

Hi Ron @zander,

   If simply loading in the same sketch, with the same 'data', such as passwords, etc., works, then that is much easier and safer.

The OP asked for how to 'clone' .. i.e. take the contents of 1 ESP32 and transfer them to another ESP32, without needing access to the original Arduino sketches, etc.. and that was the question I tried to answer.

I haven't done the procedure myself, as I have't yet had a need. It is one of those cases that the video is probably better than a description, as it involves copying and modifying lines in the Arduino compile and download sequence. The reference I quoted seemed "adequate" for the job, albeit the presenter drifts away from the central plot a few times. There are other videos with similar titles. so hopefully, anyone who needs the process can easily find one to their 'taste'.

I suspect, reading the Espressif documents on how to  reload the bootstrap loader, will tell a fairly similar story, albeit the need to be more aware of the details of the microcontroller's memory map, which varies to some extent. Again, I have only glanced at it, as I haven't needed it yet.

Best wishes, Dave