InqPortal - An ESP8266 IoT, Application, Web Server

Posted by: @frogandtoad

Question... Do you expect to profit off your library advertisements here?

If not, then why is your code not open source?

If it isn't clear in many other places - It is free!  It will always remain free.  It is also royalty free if someone would use it in a product.  

The post https://forum.dronebotworkshop.com/postid/27948/ highlights my only motives.  If you were to look at my website, it would also show you that I have no financial motives.  There are no advertisements on it.  I don't even do affiliate links which is all the rage.  I pay for its hosting out of my own pocket.  I don't tweet or Facebook.  Where I live now, is technically, software developmentally challenged.  My only peer is a gifted 9 year old in my library User's Group.  These forums are my main connection to peers... my social connection.

I don't know if Bill has even noticed my posts.  I hope I have not violated his trust of keeping out spammers which seems to be your mind-set that I am.  Maybe, he is still making the decision on whether I am a benefit or a liability.  If he contacts me and wants to let me know and put some constraints on how/what I write, I will follow his directives.  I do not want to get kicked off the forum...

Besides the social aspects, I am here because I am keenly interested in creating projects - I would like to make a robot.  I am hardware challenged and by-in-large, people here look to be kind and helpful.  I am trying to give back by answering other people's questions in any way I can.  I offer my library here because I think it can be very helpful, especially to beginners.  I also think it can offer (to a lesser degree) benefits to advanced developers.  It frees the tedium of interacting with a project.  They can focus on the project... not the logistics.  I have thousands of hours invested in it trying to make it bullet-proof and faster than any other library.

I think its main "value-add" is:

1. Simplicity to the developer
2. Eliminate or greatly reduce WiFi, Web type code.  On another forum, I see around one new post every day of how hard it is for new people to get any wifi related aspects working - UDP, TCP, Web Sockets, dropped connection woes.  
3. An application API that frees them to focus on your project.  

Why not open source?

I have hundreds of reasons.  I will not itemize all of them.  

Over the years InqPortal was based on many open and closed-source code bases.  Originally, I took a stab at writing to ONLY Espressif's API.  I later switched to the Core ESP8266 WiFi.  Later still, the ESPAsyncWebServer.  I've used SPIFF's and LittleFS, I've used the EEPROM library.  The first time I switched away from Espressif's API, was in an update of theirs in 2017 that broke my library.  I could not get any help on the several forums I tried.  It much later turned out that the fault was not with Espressif, but some other library that did not update accordingly.  Every one of those libraries above let me down at one point or another.  It might be because of the original developer or some well meaning contributor.  Getting someone to fix or...  me spending time fixing other peoples work became a never ending battle. 

InqPortal does depend on the Arduino Core code base ONLY for letting it compile in the Arduino IDE and to expose the setup() and loop() methods.  There is a surprising amount of code in just that part that I have not totally rooted out why's it exists.  A PlatformIO program of the same capability is nearly 100KB smaller.  But my target ... again... is the beginner... thus the Arduino IDE.

InqPortal is now back to being based on the core Espressif API for everything else.  It incorporates my own WiFi connection manager, web server, web sockets, file system and persisting libraries.  All of those sub components are either faster than the older equivalents or offer some other benefit that those open-source libraries do not.  However, their benefit is primarily due to they being optimized to ONLY do what is necessary for the broader context of the InqPortal server.  They are not full-featured, generalized, stand-alone libraries. The extra bloat of making them so is not something I care to do nor is the massive amount of writing I would have to document and support other developers breaking and bloating the code-base.

But for now... the buck stops here.  If someone finds a bug... they come to me to fix it.  If someone wants a capability, I have the right to choose if the extra binary size is worth the benefit to enough people. 

If someone doesn't want to use it because they can't easily look into the black-box, they are welcome to move to some other solution or write their own.  I'm not trying to hide it or say my I.P. is better than anything someone else can write.  In fact, I have always been glad to share how things are done in the library, and do offer code pieces with constructive people.

@inq Very much appreciate your well thought out response to @frogandtoad, and further more see the cost/benefit of not making the code open-source as you explained.. I do however will be attempting to build this project in the near future, and because I am woefully software challenged in other languages other than VBA, will probably require your expertise in the coding aspects so that I can learn to code more efficiently, while learning how things actually work in the coding environment. I am, however, much more hardware capable as I was in industrial Electrical/Instrumentation for over 40 years..Also worked as a electronics tech in the Navy on Electronic Warfare Systems (EWS) like D.A.S.H and ASROC weapons system back in the mid 1960's.  Bottom line is I would hope that Bill, and others, would see that this could possibly be very useful to help us code challenged folks out here that do wish to learn more about the software aspects.

Thank you, and all the other code buffs out there ,for making out projects more interesting and functional..

And particularly Bill for providing this forum so that we all can benefit together..

kind regards,

LouisR

@inq I once did a project for a hardware vendor porting an open source product to their hardware. My experience with the open source community was very disappointing. First of all this one was not really open, only a few individuals could modify the source code. Secondly, the 'community' could be best described as chaos which shouldn't be a surprise but I was perhaps naive.Through a fluke of luck I actually got to meet in his place of work the author of the main book about the product and major player in the open source world. I was there to look at some of his corporate source code for porting to a different platform. Let's just say I was less than impressed meeting him in person. Bottom line, open source is a mixed bag, if you went that route your entire life would be consumed by it if it became popular. The experience I had was with a large player in the networking world but maybe a product with only a few thousand users wouldn't be so bad.

@inst-tech 

Thank you for your kind words.  As I'm new to the forum, I am trying to not step on toes, but I am also weighing the friction versus benefit.  I don't need friction in my life and posts like yours and several other individuals mean a lot to me to counterbalance others.  Thank you.

Please ask questions.  I always take questions and constructive criticism as an opportunity to improve my software, documentation or how I teach (I'm quite new at it) my local students.  I'm in the process of creating a step-by-step in the InqWeather thread and I'm always interested in seeing other uses of the library.

VBR,

Inq

@inq I was just looking at your sample weather app. I see the usual SSID and PW items. Bill recently showed us how to get rid of those by using WiFiManager. You might want to look at incorporating that into your samples to make the code more secure and portable.

It is difficult to spit out everything that everyone wants/needs to know.  I'm just a little too verbose as it is. 😆 

The aspect of having code that does something malicious and is connecting to the Internet is a concern.  However there have been documented cases where such behavior existed for years in open-source code and no one caught it, because it was open-source.  And... because it was open-source, someone got away with it.  There was one case where Microsoft did something questionable and it was found almost immediately.  Because someone did check.  Also, there was no ambiguity of who to point the finger.   

Using Wire-Shark or equivalent is way too easy to get and check if a code is miss-behaving.  

• The InqPortal makes no connections out on Internet whatsoever.  
• It connects to your router by your choice only.
• When you browse to any page exposed by your InqPortal server based project, you are only connecting your browser to your server.  
• There is only one exception to that - When you browse to the Admin built into every project and are working in the Admin tabs (Settings, Files, Logs, History, App) they have context sensitive Help available that will go to the website to give you detailed, cross-referenced help (just like the Arduino IDE does)  The space to provide that on-line help on the ESP8266 would be impossible.  And even still... you will note... the server is still not making that Internet connection.  That is your Browser making the connection to the website.  

@inq I hope I misunderstand, I want to be able to look at the weather on my phone when away from my router, is that not possible?

@inq Your are quite welcome..and as for teaching..the trick I learned trying to teach other experienced technicians, whom many had a few more years themselves, both in experience and years on me..was to KISS it ("keep it simple Stupid")  if you think it's a challenge to teach young minds ( basically a blank slate) then try to explain things to those that think they already know everything, you will get a lot of push back..lol

But the positive side of that is, you will probably learn more about that subjest than they will, because of all the questions they will surely ask, that you may, or may not know the answer to. So your further research will provide you with more knowledge as your already ahead of the curve, hence, the reason why you are teaching them..  lol

In my work world, technician were also know to be the proverbial "know it all"..so dealing with them took a certain amount of both patients, and ingenuity.. or wearing more than one hat, so to speak..I think in the end, I probably befitted from the exercise more than they did sometimes.. but never the less,  it was ,(is) challenging. The focus however is being determined to get them to engage and participate in your ideas, and then it becomes a win-win!  I think you'll do just fine..

regards,

LouisR

@inst-tech Whenever I did any teaching or a presentation at a user group that I was asked to do, I am 100% positive I learned as much or more most of the time.

Posted by: @zander

@inq I was just looking at your sample weather app. I see the usual SSID and PW items. Bill recently showed us how to get rid of those by using WiFiManager. You might want to look at incorporating that into your samples to make the code more secure and portable.

InqPortal lets you do both - easily.

The reason I show hard-coding it in some examples is that is easier while developing.  You can leave them set to NULL or "" and it will allow you to configure them in the Admin.  Once configured, they are persisted, just like WiFiManager.  There is no extra libraries or coding required for that capability.  

Here is where it is discussed in the on-line manual:

https://inqonthat.com/inqportal-bare-essentials/#LAN

Here is the context-sensitive Help if you are already in the Admin and wanting to configure the LAN connections.  Up to 5.

https://inqonthat.com/inqportal-help-ts/#lLAP   

Posted by: @zander

@inq I hope I misunderstand, I want to be able to look at the weather on my phone when away from my router, is that not possible?

Sometimes it is possible, but you must be aware of the risks it presents and some Internet Service Providers do not allow it.

The risks are significantly less than kids that setup their gaming computers to allow other gamers to connect and play games together as they are exposing a real computer to the Internet.  Its also far less risky than having a security cam or smart door lock. 

Exposing a little ESP8266 isn't as risky.  At the very worst someone could set up a Denial of Service Attack and you couldn't get to your weather station from the road.  But why would they?

Then there is the problem of whether your ISP changes your IP address.  Some provider give your home router its IP and it stays there for months.  Other change it daily.  It's a crap shoot.

Smart Devices

Most all mainstream smart devices require you to connect to their servers.  And people are starting to get concerned about their, Fitbit, smart watch, TV, locks, web cams, even your smart wall outlets and lightbulbs, etc... pushing data to some servers somewhere in the world.  I've always focused InqPortal project smart devices as always staying inside your local area network to eliminate those concerns.  The other area students want is directly connect their phone to the ESP8266 in their robot cars and use their smart phone as a controller.  That happens real time ONLY between the phone and the car.  It doesn't even go to their home router.

Cloud

The other alternative is to have your ESP8266 server push data to a cloud service.  Then you'd log in on the road to that service and you look at your data, but its typically in a table or graph.  The InqPortal UI's you might create would not be accessible by that method.

There are many popular/free cloud services out there, but they limit the amount and frequency of data you can upload.  Fine for a weather station.  But maybe not for some other things.  That would be a great topic for Bill to do.  

InqPortal doesn't have this capability yet.  It is on the TODO list, but it opens the trust issues you brought up in your previous post.  I'll probably do it as tack-on library.  I haven't researched how easily it is to make it generic enough to work with multiples of these services.  Again... a TODO.

Hope this explains... ask if you need some clarification on some part.

VBR,

Inq

@inq I need to think on all that for a while, I misunderstood. Have you looked at the Arduino cloud yet as a possible web cloud? I am playing with it but find it very primitive at least so far.

I am not an experienced web or network guy, I started on tubes and discrete components so all this is new to me.

@inq Ok, that looks great, I think the best thing for me to do is try to do some reading and when the sensors get here fire up the sample weather station so I have better questions.

Posted by: @zander

@inq I need to think on all that for a while, I misunderstood. Have you looked at the Arduino cloud yet as a possible web cloud? I am playing with it but find it very primitive at least so far.

I am not an experienced web or network guy, I started on tubes and discrete components so all this is new to me.

Yes, I'll definitely add it to the list, but that is exactly kind of service Cloud was referencing.

@zander

Posted by: @zander

@frogandtoad Yes, used the sample verbatim. The sensor is dead.

Oh well, was worth a try.