Inqling Junior - Robot Mapping, Vision, Autonomy

Posted by: @robotbuilder

@inq 

My "science project" is to handle the environment and navigation more like a human ...
In my Inqling series, I'm want to tackle the "getting the object to my FOV". IOW, My beer isn't in the floor in the middle of the room where Inqling starts. I want Inqling to know that my beer is in the fridge and to get to the fridge in the kitchen no matter which room it starts.

First it has to recognize which room it is in. It would handle the environment more like a human if it could look and see what room it is in and then it could look and see where the door was (and if it is open or closed) and then it could navigate to that door. It might look and see it is already in the kitchen because it can see the fridge! Then it has to look and see if the beer is in the fridge and not already taken or moved to another part of the fridge.

Although the VL53L5CX may be a very useful sensor for many things I think using it to recognize a can of beer or any other object will be difficult.  I don't know if it can be used like sonar to map a room and even use the returned distances to recognize the room?

 

I think you are being systematic in what I'll need to do for the full, long-term project (multiple years maybe 😋 ).  I did mention, Inqling Jr. is only to do the part about recognizing the room and planning the path and getting to get to the fridge.

Beer selection will require true vision with a camera.  It will need to understand when I ask it to get an IPA or a Stout.  😉 

I completely agree the VL53L5CX's 8x8 pixels is a non starter, but I think it will serve the purposes, I have laid-out.  Besides, I just received this notice of a 0.5 Mpixel version...  This will certainly overload an ESP8266 trying to process all the data.  Hope it is reasonably priced.

https://www.globenewswire.com/news-release/2022/02/22/2389403/0/en/STMicroelectronics-transforms-digital-vision-with-the-market-s-first-0-5Mpixel-depth-image-sensor.html

VBR,

Inq

Posted by: @byron

Posted by: @inq

My beer isn't in the floor in the middle of the room where Inqling starts.  I want Inqling to know that my beer is in the fridge and to get to the fridge in the kitchen no matter which room it starts.

Posted by: @inq

... and the tune is Rock and Roll! 😉 🤩 

And this maybe what happens when the bot gets to the beer fridge 😀 

 

An olde, but a goodie!

Posted by: @davee

I suspect 'simple' AI systems will struggle to realise which things are 'solid' navigation points.

So maybe, some kind of 'help' is needed. Here you may need to be inventive, or tolerant of 'invading' objects, or both.

Lot's of good input!  Thanks.  Some, I have started the thought processes required, but at least one was one I hadn't broached yet.

Only indoors.  No thoughts of handling terrain and sunlight screwing up the sensors.  I'll only do indoors!

My other thoughts so far were to map the rooms, passing the data to a computer (for now) and allow me to label things.  That massaged database would then be backloaded to Inqling.    Letting the room geometry be the identifying characteristic to "discover" where it is and then handle the path determination and navigation.

I would like Inqling not to require the computer during discovery and/or navigation.  I do not feel that computer/me massaging the data is cheating (at this point)... it is equivalent of telling a baby, don't go up those steps, don't stick your tongue in the power socket.

I had not thought about the aspect of changes causing a misidentification of the room during the discovery phase.  I had thought about changes found during navigation as in a new obstruction... but not the discovery. 

Must ponder!

VBR,

Inq

@inq For me it reminds me of my poor ability to predict the future. In the late 70's I was at a conference on robotics. This was an IBM event but there were some partners from outside. I don't remember if it was Honda but it may have been. I recall pointing out how incredibly complex human walking was. WOW these robots have exceeded my expectations by quite a lot. In my feeble defense the computer of the day was room sized and had a clock speed of 5MHz. My watch outdoes that by several orders of magnitude today. The dog like creature in the video is already undergoing trials and has been for a couple years with several army groups, and that last dude I think is undergoing trials at Amazon. I don't often wish I was younger, but I envy the young engineers of today, they will make huge advances affecting humans directly, imagine that dog scaled down a bit as a tool for the elderly, carrying not just the groceries, but even the human for at least short distances. I am simply blown away by the progress Boston Dynamics has made. Clearly they mastered walking!

UPDATE

I've finished the chassis/body design and the printing.  I'm still bouncing back and forth between turning the sensor by stepper or servo.  At the moment, I have returned to using a servo.  Reasons for my wobbling...

• I was starting to design the body for the sensor placement and realized that I had to raise it so that the sensor's FOV didn't "see" the body itself.  I don't really want to handle that blind spot / or logic of negating that region.
• That thought process also made me realize, if the robot starts up with the sensor (using Steppers) would have to have some kind of "centering" process.  I could not come up with something easy without adding hall sensors or something to the head.  Thus I'm returning to using the servo which will know where center is.
• The bad part of using servos... the MPU must use processing power to even hold a position, while it would not if I used a stepper.  I'm sure, I wobble some more in the future.

Introducing Inqling Jr. to the Fridge.

These are "Training Wheels" while I play with the balancing logic.  I don't want to do too many face plants!  The "skis" are easily swappable.  The ones shown allow up to 5 degrees of tilt, while others allow progressively more tilt.  Removing the skis altogether, still has the round lattice pieces that don't hit until about 45 degrees and thus it will simply roll over versus doing a very hard face plant.

@inq Without some sort of indoor sensor system building a map of the environment will be a challenge unless the state of artificial vision and recognition is sufficiently advanced (may be but IDK) As a V1 you might enter all the data points manually, but isn't the tech for navigating using this pre-built DB exactly the same as the tech needed to build it?

Posted by: @zander

@inq Without some sort of indoor sensor system building a map of the environment will be a challenge unless the state of artificial vision and recognition is sufficiently advanced (may be but IDK) As a V1 you might enter all the data points manually, but isn't the tech for navigating using this pre-built DB exactly the same as the tech needed to build it?

Maybe I'm not understanding your point, or maybe you missed that I will have such a sensor.  The VL53L5CX is a "Time of Flight" sensor that measures the distance out to about 4 meters at 64 points (8 x 8) with a FOV or 45 degrees in the vertical and horizontal.  It can do a snapshots of the 64 points at 15 Hz.

I feel this will do a better job even than a rotating Lidar.  I will be able to handle the vertical aspects of the room.  It even proports that it can detect hand signals/waving where as a single-beam Lidar could not.  

My hope is that the robot can trundle around the room/house and map the walls/furniture.  With the vertical component, I hope to see the ceiling which can hopefully distinguish between furniture (not going all the way up) and the walls that do go all the way up.

Here is the post where I show my first testings of it: https://forum.dronebotworkshop.com/user-robot-projects/inqling-junior-robot-mapping-vision-autonomy/paged/3/#post-31051

VBR,

Inq

Hi @inq,

  I am pleased you found something interesting.

 I note you say:

My other thoughts so far were to map the rooms, passing the data to a computer (for now) and allow me to label things.  That massaged database would then be backloaded to Inqling.   ... etc.

I totally agree this is not cheating ... and maybe you could push all of the 'hard thinking' to the 'base station' computing, during the development phase, since, clearly, this is much more than a single Arduino or ESP can handle, and I suspect will go through a fair degree of metamorphosis during development.

Thus, Inqling would include and control the mechanical, sensor and so on aspects, including the ability to move around, etc., but the Where am I? What shall I do next? thinking is sub-contracted to the base station. And baby Inqling might also go through metamorphic stages to provide the necessary mechanical, etc. attributes.

Re-integrating them later, as Inqling grows up, whilst far from trivial, is probably one of the easier things to do ... and might use computing boards, etc. that don't exist at present.

This way, you are less constrained in finding enough computing resources and making them mobile, whilst still discovering what is actually needed.

The downside, apart from a sense of it not being a truly independent being in phase 1, is that you need reliable comms between baby Inqling and the basestation. Again, not always trivial, but hopefully do-able.

Just a random thought. Kids grow up in many different ways. Maybe Inqlings can as well. Dave

@inq My bad, I now do recall you discussing that, but I forgot that in the recent back and forth. 

Posted by: @davee

The downside, apart from a sense of it not being a truly independent being in phase 1, is that you need reliable comms between baby Inqling and the basestation. Again, not always trivial, but hopefully do-able.

Oh!  That is the easy part.  The InqPortal library (see signature) handles all this with very high throughput and near trivial coding.  That will be working all in the R/C type phase, long before map coding starts.  I'll be layering in Inqling Senior code in strait away.  It's only about 100 lines of code (not counting the library).  I can push almost 200 KB/sec to computer telemetry.  I've not benchmarked computer to ESP8266, but I suspect it is not quite as good.

VBR,

Inq

Hi Inq,

I just noticed:

The bad part of using servos... the MPU must use processing power to even hold a position,

As a 'relative newcomer' to the Arduino, etc. world, I am not familiar with the specific resources for all the different board products, or servos, for that matter. I rely on reading data sheets when I need to know something ...

So this is a semi-theoretical comment, in that I haven't identified specific products. Economics dictated by range and availability of very low cost boards, etc., may be the controlling factor that constrains choices to processors with limited peripherals.

----------------

The servo I have seen, simply takes in a voltage and moves to an angular position, proportional to that voltage. Thus, a constant 'specified' voltage is all that is required to hold a given angle. I am using this as my 'model' of any servo you might choose.

----------------

In theory, I do not see why providing a 'specifiable' constant voltage MUST use processing power, a single write of a number to the "right" place should be enough.

I am guessing you are referring to 'creating' this voltage using a PWM method, and the mark-space ratio which defines the voltage, is created by the processor running a tight timing based loop to change the level. i.e. The computing equivalent of a treadmill... which is fine unless you want the micro to actually compute something as well.

To overcome this (mis)use of processing power, some microcontrollers include digital-analogue converters (DACs), others provide "PWM engines" ... essentially controllable counters, which the microcontroller initialises a register or two with required counting values, and leaves the PWM counters to actually produce the waveform.

In addition, DACs are available as standalone parts, and even 'homebrew' DACs consisting of n-bit latch (or a few spare GPIO pins) and a small pile of resistors is sometimes 'good enough'.

So are any of these options suitable and available for your project?

(Of course,  I am not discussing the merits of servos and steppers .. this is only appropriate if you prefer to use a servo and you are looking for a convenient solution.)

---------

Just a few simple thoughts ... Best wishes, Dave

Hi Inq,

re: Oh!  That is the easy part.  The InqPortal library (see signature) handles all this .... etc.

Yes, you have a lot of useful experience that I am sure will be very handy .. so maybe this is an opportunity to use it.

However, I was, in part at least, thinking of the possibility of 'dead spots' where the WiFi (or equivalent) signals contrive to cancel each other out, or can't get to because of walls, etc. 

Of course, you may have perfect signal coverage on the path from your couch to your fridge, so Inqling will never be able to hide or get up to any mischief, without base station knowing and taking appropriate action. 😀 (I know kids can find hiding places ... Inqlings I can only guess about .. but I always take the skeptical view of everything!)

Also, if base station is receiving video/image data, then the necessary bandwidth can easily get higher.

Dave

Posted by: @davee

So this is a semi-theoretical comment, in that I haven't identified specific products. Economics dictated by range and availability of very low cost boards, etc., may be the controlling factor that constrains choices to processors with limited peripherals.

Your whole post is correct/valid and contains many options that can be considered.  I typically am in the habit of trying to minimize MPU time usage... and layer it in until it pukes.

And things do vary greatly...

I believe Arduino boards do have PWM implemented in hardware.  Because the ESP processors are so much faster many things that Arduinos do in hardware is actually done in software on the ESP.  PWM is one of those.  So... if the MPU gets overloaded... say doing robot balancing, sending WiFi data out, receiving WiFi data in, it may miss-time a PWM pulse.  When this happens the servo sums a false reading and all of a sudden the servo will "glitch" and come back.  This is under very high MPU usage.

If I use a stepper motor for the head and an A4988 stepper driver... it only uses MPU when I want to move the head... sending pulses.  Once its at a new position, no MPU background processing is required (unlike PWM).

Now, I could use other things like the D2A things you mentioned, but at less than a $1, I have a mess of ESP8266, ESP-01... I'll just throw another processor at it.  Hardware being my weak point, I tend to work optimize code until I work a processor to death and then add another or stronger one.  I shy away adding IC's, chips, or even resistors and capacitors unless someone/some cookbook says I have to.  🤗 

Posted by: @davee

Of course, you may have perfect signal coverage on the path from your couch to your fridge, so Inqling will never be able to hide or get up to any mischief, without base station knowing and taking appropriate action.

I'm pretty good with WiFi within our small little house, but the Internet sucks... as I've whined about before.  In fact, I have not had any Internet since last Thursday at the house.  Tourist up for 4th of July are killing it.  I had to come into the library today to even see if I had email!

As you perfectly outlined above... during the prototyping phase, I'll simply do the WiFi to my computers.  But Inqling 3rd will be a complete new design.  It'll be cooler, bigger, taller, faster and more modular to handle upgrades more readily.  It will also get the Raspberry Pi 4 to handle all the mapping/higher-order processing on the robot.  It'll still have telemetry output to computers, but won't need them to do anything.  

Hi Inq,

re: I shy away adding IC's, chips, or even resistors and capacitors unless someone/some cookbook says I have to

In the present drought of components that has international car manufacturers halting production for the sake of parts costing a cent or two each, I feel embarassed to suggest almost anything that isn't piled high at Amazon, etc., but I really would recommend you discard a little of this shyness ... treat it like an extension to your 3D printing expertise!

(Confession, I have never tried 3D printing ... yet ... so I know how the 'shyness' can arise!)

Best wishes, Dave