Notifications
Clear all

LoRa or WiFi Client-Server Library (Help and/or Interest)

10 Posts
4 Users
3 Likes
305 Views
Inq
 Inq
(@inq)
Member
Joined: 2 years ago
Posts: 1900
Topic starter  

I've not seen what I'm requesting on the forum before.  I'd like to see if there is any interest in a team type project.

Preamble

I was a Windows C/C++ and later C#.NET developer for about 22 years before I retired.  I primarily did server side code - either massively multi-threaded data crunching or secure database design, storage and retrieval.  Towards the end, I worked for SAP, AG.  The product I worked on was a massively, monolithic database program.  Holding most of the Fortune 500's financial and other critical information security was paramount.  Having been well established before I joined SAP, I had little say in the architecture of the system.  I always felt there was a better way to generalize the problem.  I've wanted to tackle such a project.  I think I've stumbled upon one...

10,000 Foot View

I've been on the forum a little over a year and a half.  I subscribed to New Posts and I try to help others when I can.  I like the robot aspects, but from what I've seen in that time, it seems like there is more interest here for IoT type projects. 

I'd like to propose building a generalized client-server system.  

Nominally it would contain two pieces of code.

  1. An Arduino library that we can incorporate into the Arduino Library System if we choose.  This library would take care of all the communications to a server.  My belief is this library can have a very small API... maybe even just two or three lines of code to add to our sketches.  It would:
    1. allow sending of any number of sensors' data in any format - ints, floats, doubles, string buffers.  
    2. Handle multiple communications paths by a simple #define type selection.
    3. Support multiple MPU - Arduino, RasPi Pico, ESP8266, ESP32, etc.
    4. Support multiple Protocols - LoRa, TCP, UDP
    5. Optionally handle sleep states for battery operated sensor nodes.
  2. A server program running on a Raspberry Pi.  This server side program would:
    1. accept data and store it generically - Flat files or Database, user preference.
    2. insure data integrity
    3. insure reception of all packets for unreliable communications protocols - LoRa, UDP
    4. dynamically organize scheduling so as to reduce contention - LoRa, UDP
    5. expose a web interface so that any device on your own private LAN could view and/or control the server
    6. support custom browser based dashboards.
    7. upload data to cloud services of choice

It is my belief this is all achievable and that once complete, there would be very little need to update it for different user scenarios.  IOW, it could be a black box we just use.  I can imagine that the time to implement a sensor node of your specifications might take minutes.  That you'd spend far more time designing and wiring the sensors to your MPU.

Documentation would be thorough for implementing the system as well as examples of hardware wiring diagrams for implementing the sensor node with an assortment of sensors.  

Question to You

Would you have any interest in helping out with a team project... in any capacity?

  1. Primary design/implementation of one piece or the other?
  2. Help in design aspects?
  3. Help out coding aspects?
  4. Help Beta testing aspects?
  5. Make suggestions?
  6. None of the above, but you would want to be an end-user?
  7. None of the above, but a casual viewer of the build thread?

 

For my part, I don't have all the skills to fully implement this as described.  Although, I've created RasPi web and database servers, I've never programmed apps or interfaced with electronic components on the RasPi.  I'm looking forward to learning from someone who has... even if it involves Python. 😜  I  haven't used all these MPU types and would like your experience there.  If there is no interest, I'd probably take a stab at this project on my own, but I'd only build the parts I need and wouldn't even mess with documentation.  I'd just use it in my little world. 😎 

I just realized the forum doesn't have a survey or poll ability... If even you're only a #6 or #7 on the list, would you kindly give a thumbs up.  If any of the others, would you give a reply stating your level of interest.

VBR,

Inq

3 lines of code = InqPortal = Complete IoT, App, Web Server w/ GUI Admin Client, WiFi Manager, Drag & Drop File Manager, OTA, Performance Metrics, Web Socket Comms, Easy App API, All running on ESP8266...
Even usable on ESP-01S - Quickest Start Guide


   
Quote
Ron
 Ron
(@zander)
Father of a miniature Wookie
Joined: 3 years ago
Posts: 6530
 

@inq 7

First computer 1959. Retired from my own computer company 2004.
Hardware - Expert in 1401, and 360, fairly knowledge in PC plus numerous MPU's and MCU's
Major Languages - Machine language, 360 Macro Assembler, Intel Assembler, PL/I and PL1, Pascal, Basic, C plus numerous job control and scripting languages.
Sure you can learn to be a programmer, it will take the same amount of time for me to learn to be a Doctor.


   
ReplyQuote
(@davee)
Member
Joined: 3 years ago
Posts: 1547
 

Hi @inq,

   Sorry, I don't presently have any great ambitions as an end-user about a general purpose IoT station, but curiosity etc. will almost certainly mean I will be at least at casual observer level, probably more interested in technology (soft, hard or whatever) bits that are different or unusual, especially if they are relevant in other spheres.

I'm reluctant to take on open-ended commitments but obviously, if something piques my interest, I'll see what I can do, and if requested, may be willing to help out on specific bits.

Sorry if that sounds rather negative, but hope it makes sense.

Best wishes and good luck with whatever you decide, Dave


   
ReplyQuote
Inq
 Inq
(@inq)
Member
Joined: 2 years ago
Posts: 1900
Topic starter  

Posted by: @davee

Hi @inq,

   Sorry, I don't presently have any great ambitions as an end-user about a general purpose IoT station, but curiosity etc. will almost certainly mean I will be at least at casual observer level, probably more interested in technology (soft, hard or whatever) bits that are different or unusual, especially if they are relevant in other spheres.

I'm reluctant to take on open-ended commitments but obviously, if something piques my interest, I'll see what I can do, and if requested, may be willing to help out on specific bits.

Sorry if that sounds rather negative, but hope it makes sense.

Best wishes and good luck with whatever you decide, Dave

I understand and that's great.  I always appreciate when you keep me between the fence posts. 😆 

Clarification - in case others got the same impression.  My intention is not to necessarily solicit for any kind of commitments.  It is primarily to see if there might be any interest. 

It has always been my contention when writing code (even if it is just for me) to consider... "Will I ever use this again even once?"  If the answer is no, the code goes into the Sketch file.  If the answer is even potentially... maybe, I'll put it in a library.  I'll comment it up more, so I can at least say... look at that $#!+... what was I thinking?  If I think it might be useful and consider publishing it, I'll write cleaner code so as not to show my underwear in public and double the comments in the code and formally document the capabilities, limitations and the API.  I'll also entertain people's suggestions... in the hope it'll be even more generalized for things I didn't think of, but might say someday... hey I'm glad I added that.

I know some people would want to code their own even if this system did exist.  But I think there might be a contingent here that just want to solve something simple like, get/set the temperature in the work shop or green-house a hundred yards away.  They don't want to learn all the intricacies and pitfalls of LoRa, Deep Sleep, RTC, GUI, Database, or Web servers so they never realize the feature.  If a library turns those challenges into a 5 line program, someone might benefit.

In this specific posting.  If there is not enough interest,   I have no need for most of the MPU's.  I won't add those.  I'll pick one LoRa library to use.  I won't support a WiFi version.   I'll hard code features instead of exposing them for optional configuration.  It'll just remain a private library.

 

 

3 lines of code = InqPortal = Complete IoT, App, Web Server w/ GUI Admin Client, WiFi Manager, Drag & Drop File Manager, OTA, Performance Metrics, Web Socket Comms, Easy App API, All running on ESP8266...
Even usable on ESP-01S - Quickest Start Guide


   
DaveE, Inst-Tech and Ron reacted
ReplyQuote
(@80stech)
Member
Joined: 2 years ago
Posts: 3
 

 I have interest in the project but am not savvy enough to help with anything. Would the portal be able to handle video or security camera stills ?  Sounds like a very good project either way 🙂

Cheers,

Rainer 


   
ReplyQuote
Inq
 Inq
(@inq)
Member
Joined: 2 years ago
Posts: 1900
Topic starter  

Posted by: @80stech

 I have interest in the project but am not savvy enough to help with anything. Would the portal be able to handle video or security camera stills ?  Sounds like a very good project either way 🙂

Cheers,

Rainer 

I too would be interested in that.  Unfortunately the library could not be written to handle those.  The LoRa communication is geared to "Long Range" (miles) and extreme minimal power (to run on batteries for months and even years).  To do that it sacrifices bandwidth (how fast it can send it) and size of the data sent (< 256 bytes).  It was really designed for remote sensors running off batteries.  As you can see, it would not be capable of either video or images.

Sorry,

Inq

 

3 lines of code = InqPortal = Complete IoT, App, Web Server w/ GUI Admin Client, WiFi Manager, Drag & Drop File Manager, OTA, Performance Metrics, Web Socket Comms, Easy App API, All running on ESP8266...
Even usable on ESP-01S - Quickest Start Guide


   
ReplyQuote
Ron
 Ron
(@zander)
Father of a miniature Wookie
Joined: 3 years ago
Posts: 6530
 

@80stech If your interest is video and stills, get an ESP32CAM, but get the one with a MotherBoard. Here is a link on Amazon https://amz.run/7MGF    but those are relatively expensive. I have probably a dozen or more but most were bought on Aliexpress, likely 10 for $20.

First computer 1959. Retired from my own computer company 2004.
Hardware - Expert in 1401, and 360, fairly knowledge in PC plus numerous MPU's and MCU's
Major Languages - Machine language, 360 Macro Assembler, Intel Assembler, PL/I and PL1, Pascal, Basic, C plus numerous job control and scripting languages.
Sure you can learn to be a programmer, it will take the same amount of time for me to learn to be a Doctor.


   
ReplyQuote
(@80stech)
Member
Joined: 2 years ago
Posts: 3
 

 Yep I have quite a few as well but am currently using them remotely using port forwarding and would really like to have a portal to use for this and other projects.

Cheers 🙂 

I have some interest in LORA as well but was hoping to use that to avoid using the internet. 


   
ReplyQuote
Ron
 Ron
(@zander)
Father of a miniature Wookie
Joined: 3 years ago
Posts: 6530
 

@80stech I don't understand what you mean by portal. The sample code that comes with the board is a web server that allows you to control the camera. It sounds like you are doing that already for some purpose. Remember these are 2MP cameras, very low quality.

LORA can not be used for data intense applications, it is for passing small messages such as instrumentation (temperature, humidity, pressure) It has NO retry ability, no encryption, no ack/nak. You must build all that capability your self if you want it. I have designed several protocols in my days as an employee so have a 5th draft done so far. For me #5 is usually final, but this is a tricky beast. I have it down to 11 bytes of overhead. I could go smaller, but I am placing data integrity as my #1 priority.

Try to rephrase your question and ask again.

First computer 1959. Retired from my own computer company 2004.
Hardware - Expert in 1401, and 360, fairly knowledge in PC plus numerous MPU's and MCU's
Major Languages - Machine language, 360 Macro Assembler, Intel Assembler, PL/I and PL1, Pascal, Basic, C plus numerous job control and scripting languages.
Sure you can learn to be a programmer, it will take the same amount of time for me to learn to be a Doctor.


   
ReplyQuote
(@80stech)
Member
Joined: 2 years ago
Posts: 3
 

@zander

No question really,  I was just replying/commenting 🙂 


   
ReplyQuote