Communicating between Raspberry Pi and ESP32

I'm not clear how your current system communicates to the Arduino cloud from the rabbit shed as I though you would need an internet connection in the shed to do that as I've only communicated to the internet via a home network router.  But I expect you have some other device in the shed to achieve that.  

I don't think you will be able to send mqtt messages over esp-now.  As far as I understand it esp-now is an alternative to an ethernet wifi network to communicate wirelessly between esp devices.   It has a maximum message payload of 256 bytes whereas mqtt messages can send up to  268435455 bytes (approx 260MB) per message.  Not that you would need to send that much per message but I don't think you will find a mqtt client library that would work over esp-now.

To use mqtt, which in my experience is a very desirable and easy to set up way to go, you would need an ethernet home network that can be extended to reach the rabbit shed, and you would need to set up a mqtt broker.  The rpi is ideal to be set up as a mqtt broker.  The rpi does not have to be dedicated to this task and it can be both a broker and also run mqtt client programs.    

If your rabbit shed is not currently reachable from your home network then its really quite easy to extend the network to reach, either via an external grade ethernet cable,  or maybe some external wifi repeaters, or perhaps a point to point wifi extender designed to link buildings such as seen in the link below.

https://www.broadbandbuyer.com/products/19209-tp-link-cpe510kit/?gclid=CjwKCAjwve2TBhByEiwAaktM1OhqLrPCmcY1EEPLAd84Sw0UcDypPezh-3IQZBVGZzRDXs3U3DTDYBoCCvMQAvD_BwE

Alternatively you could use esp-now from an esp board in the shed to communicate to another esp-now enabled esp board in the house that in turn communicates to a rpi perhaps via the likes of a UART link.  But imho using a fully home networked solution and then using http or mqtt  messages for the communications would be much easier to achieve (and probably more reliable)

I have a rpi set up where I use the rpi touch-screen to display temperature information etc. and have on screen buttons and sliders to control various functions.   In my set up I currently use pyqt5 for the gui (previously I had a tkinter version) but these are programmed in python.  The comms to various boards (other rpi's, rpi pico's, esp-32's, etc.) is via mqtt on the home network.  The communication to various outbuildings is achieved by extending the home network to those buildings.

@Byron, that's for the response.

Posted by: @byron

I'm not clear how your current system communicates to the Arduino cloud from the rabbit shed as I though you would need an internet connection

Yes, the wifi signal does reach from the house to the shed.

Posted by: @byron

I don't think you will be able to send mqtt messages over esp-now

The ESP-NOW would be used from the shed to an ESP32 in the house.  I am using this approach for two reasons.  First, I want to try ESP-NOW.  Second, using ESP-NOW removes the wifi as a point of failure.  I was looking at MQTT to communicate between the ESP32 in the house and at raspberry Pi that will be hosting the GUI.  After I posted my question last night I realized that I am adding wifi back into the system.  I started looking at communicating via the UART channel, but haven't yet found any examples.

Posted by: @byron

Alternatively you could use esp-now from an esp board in the shed to communicate to another esp-now enabled esp board in the house that in turn communicates to a rpi perhaps via the likes of a UART link. 

As I mentioned above, I will start researching this approach next.

Posted by: @byron

But imho using a fully home networked solution and then using http or mqtt  messages for the communications would be much easier to achieve (and probably more reliable)

This will be my fallback approach!

Posted by: @huckohio

First, I want to try ESP-NOW.  Second, using ESP-NOW removes the wifi as a point of failure. 

I too will be giving ESP-Now a whirl sometime soon to check it out for a small project where my wifi does not currently reach.  I have not experienced problems with the wifi network itself, with has proved rock solid, but I have had issues with some of the boards that attach to the network proving less than reliable.  

In particular I had some issues with ESP 8266 boards proving less than rock solid, but that is due to the wireless radio abilities in the circuits in the ESP board, whereas I have never had any issues with rpi zero boards in a similar locations.  I think it likely the same issues with the ESP wifi  abilities will also be found if using ESP-Now as its the wireless radio ability of the ESP that is the issue.

I have solved the occasional wifi issues with my ESP 8266 boards by using a resilient wifi/mqtt library that was designed for situations where the ESP board may be on the edge of good wifi reception and may be liable to disconnections.  This program will happily wait until wifi is automatically reestablished and then will continue to send and receive mqtt messages.  This resilient library is written in micropython so I don't suppose you want to go there, but maybe there are similar C++ libraries to be found.  

Setting up a mqtt broker and coding mqtt clients on the ESP-32 and the rpi seems a bit of an overkill just to link two boards that are probably in close proximity just to send a small amount of data for each message, and I would think a UART link would be the way to go.

Have fun with your project and let us know if your find the ESP-Now link proves to be robust. 

@huckohio

Posted by: @huckohio

I have a system in my wife’s rabbit shed that is using an ESP32-PICO to control two fans, heat lamps, and water bowl heaters.  The current system communicates through the Arduino Cloud and displays the temp and device status on an iPhone.  I have been using this system for almost a year and have found the Arduino Cloud approach unreliable. 

I am working on updating the system by using an ESP32-S3 in the rabbit shed to control the devices and receive input from the temp sensor.  I would like to change from using the iPhone to using a 7” touch screen display to display the temperature inside the shed and show the current status (on/off) for all devices.  I want to have the ability to override any of the devices (if they are off then turn them on and if on then turn off) and override state will persist until the override is removed.  I would also like to change the temperature threshold for triggering the devices. 

I plan to use ESP-NOW to communicate from the ESP32-S3 in the shed to another ESP32 inside the house (~200 feet away).  The second ESP32 (in the house) is only used to send/receive data between the shed and the house (remove Arduino Cloud).  Now I need to get a display.  My plan is to use a Raspberry PI to receive/send data between the PI (database/GUI) and the ESP32.  The Pi GUI will display the temp and device status, and provide override buttons and data fields to change the device trigger temperature.   

So here’s my question:  My thought was to use MQTT between the house ESP32 and the Pi, but I noticed in Bill’s ESP-NOW sketch that he sets the wifi mode on the Arduinos to WIFI_STA.  Can I use MQTT to communicate between the ESP32 and the Pi if the ESP32’s are both set to Station mode (not familiar with WIFI modes)?  Sorry for the long lead-in to the question.  Just wanted to give a quick idea of the system structure.

According to the following article, you should be able to do exactly what you describe, by simply setting up the WiFi mode to act as; both an access point and Wi-Fi station (WIFI_AP_STA):

ESP-NOW Web Server Sensor Dashboard (ESP-NOW + Wi-Fi)

@frogandtoad Thank you for the link!

FYI, I was able to successfully test data passing via ESP-NOW from the house to the rabbit shed (~200 feet) as long as the house ESP32 is not in the basement (where my workshop is located).  It was a simple test of passing an incrementing float between the ESP32-S3s.

@frogandtoad 

Posted by: @frogandtoad

According to the following article, you should be able to do exactly what you describe, by simply setting up the WiFi mode to act as; both an access point and Wi-Fi station (WIFI_AP_STA):

I've been working on this for a couple days and I think the only capability you get with the 'wifi' setup in this article is to run a web server on the ESP32.  Originally I was establishing a wifi connection to get date/time data concurrently with sending sensor data via ESP-NOW; the ESP-NOW data would fail to send.  I followed the article example, which doesn't seem to be establishing a wifi channel, and was able to send the sensor data via ESP-NOW.  But, I don't think you can really do anything with wifi from the article code, so next i added code to establish a wifi connection to ntpServer.  I was back to ESP-NOW send data failures.  I think this approach is a bust.  Question: Is it possible to open a wifi channel, get date/time info, and then close the channel so the sensor data can be sent over ESP-NOW (or is that a dumb idea)?

Posted by: @huckohio

Is it possible to open a wifi channel, get date/time info, and then close the channel so the sensor data can be sent over ESP-NOW (or is that a dumb idea)?

I dont think its a dumb idea, and I dont see why not, but I've not tried to chop and change the esp wifi on the fly so it will be interesting to see what you find.  

However my comment is to suggest an alternative for getting the correct time into your esp32's could be to get the appropriate time string needed to set the esp32's rtc from the rpi you will be linking to.  So the esp32 in the shed communicates via esp now to the esp32 in the house that holds the current time as it has requested it from the rpi it links to.   A convoluted route but that should also work.

And a final thought, although it may not be relevant for you as you want to use the esp now, but for a recent project I've been using an esp32 S3 with wifi and I've had occasional wifi dropouts plus I've had to put in some extra code to get set its time when it comes back if I get a power cut.  I've now replaced it with a rpi Zero board in preference and the wifi has been solid, and it can be set to wait for the network to be present before continuing to boot process.  And a real bonus is that, as its at a remote location and connected to wifi, I can sit at my office desk and program or otherwise communicate with the board via ssh or vnc viewer on my desktop pc.  No more having to get up close and personal to the board for a usb link.  

I'm still planning on setting up esp32 Now comms twix a couple of esp32 boards for another project so I'm watching your progress on this. 👍 

@byron Thank you for the reply and the recommendation.  I am going to stick with the ESP32S3s (for now) since I have three of them and no PI Zeros.  After I posted my last comment I ordered an RTC module for the project that I will load with the correct time/date and then add to the ESP32S3 in the shed.  The RTC module just seamed easier than turning the wifi on-off. 

@huckohio That is exactly how I would do it PROVIDED you are using an ESP32 with built in RTC. Do all that in setup and end by destroying the original wifi. Now in main loop start over as that give you the ability t0 reconnect if it fails, something along the line of 

loop()

  if (wifi NOT connected)

    conect

  else do everything{

  }

}  // end loop

YMMV

@byron Simple answer is yes, in fact another fellow on here looking for help in a similar situation did exactly that. Sorry I don't remember the thread now.

@huckohio The ESP32S3 has a built in RTC. If you can put the board into one of it's sleep modes the batteries will last much longer. It even has a separate pin to power the RTC. Spend a few hours with the spec sheet, you may be pleasantly surprised.

I am thinking these new S3 chips will be ideal for game cameras, even going so far as separate power supplies. One for the rtc (AA) one for main power (4 to 10 18650) and a 50W solar panel up the tree.

@huckohio I am sure you have seen this, but if not the links to datasheets etc will fill you in re RTC, RTC separate power, sleep modes and power requirements.

I am going to compare this to both the Pico and Pi 0 2 W to see which give me the best power consumption and if I take camera ability into the equation that will be an interesting second step.

I personally think this is a great time to be doing these types of projects, many new and better choices.

https://esp32s3.com

@huckohio
@zander

Posted by: @zander

@huckohio The ESP32S3 has a built in RTC. If you can put the board into one of it's sleep modes the batteries will last much longer. It even has a separate pin to power the RTC. Spend a few hours with the spec sheet, you may be pleasantly surprised.

I am thinking these new S3 chips will be ideal for game cameras, even going so far as separate power supplies. One for the rtc (AA) one for main power (4 to 10 18650) and a 50W solar panel up the tree.

Ron is correct, the ESP32 can handle real time clock and sync to an NTP time server without the addition of an external RTC module.  It should be pretty easy to incorporate that functionality into your ESP32 at the shed end, and then send that time string directly to your RPI (even without MQTT).