Notifications
Clear all
Topic starter
2025-08-09 2:28 am
I became interested in comparing the accuracy of sensors when results were gathered at different heights above the ground after reading about how height can influence the results of a sonar range finder. So I used that idea when I designed this little experiment.
Sensor. I used a digital distance sensor that is based on STMicroelectronics - VL53L1X Time of Flight laser ranging sensor. The designers of this sensor board have added an 8 bit MCU host that uses the VL53L1X driver to communicate and configure the Time-of-Flight sensor via I2C. The board takes continuous distance measurements and outputs them as digital pulse width signals. The VL53L1X was programmed to use a 15 degree field of view.
Method. I wrote a program that gathers readings continuously, displaying results for a sample size of 31, calculating the accuracy and precision for each distance I wanted to measure. A sheet of white paper was used as the target in a room with low ambient lighting. The true distances I measured were 152mm (6”), 305mm (12”), 610mm (24”), 914mm (36”), 1219mm (48”) & 1524mm (60”). Three runs where made, one with the sensor at a height of 79mm (3.125”) above the ground, a second where the sensor was at a height of 197mm (7.75”) and the third with the sensor at a height of 470mm (18.5”).
Accuracy. A sensor’s accuracy is the difference between its (average) output m and the true value v to be measured. accuracy = 1 - ((| m - v |) / v). So approaching 1 means more accuracy.
Precision. The sensor’s precision is given as 1 divided by the sensor’s variance (v). Variance tells you the degree of spread in the data set. The more spread out the data the larger the variance is in relation to the mean. Variance is calculated by summing the squared differences between the measured distance (d) and the mean (m) and then dividing that by the number of measurements (s) - 1.
precision = 1 / (sum of ((d - m)^2)/s -1). So approaching 1 means greater precision.
I paraphrased this from a book I’m reading.
Precision is a measure of repeatability of a signal, whereas accuracy describes a systematic error that is introduced by the sensor’s physics.
Results/Performance.
Conclusion. Overall the precision graph seems to indicate that repeatability for all heights got worse as the distance measured increased, except for a little bump at 914mm (36”).
The accuracy of the low mount sensor started to trend downward after the distance measured exceeded 610mm (24”), when compared to the others. This result along with the precision for the low height sensor seems to indicate that the best practice for that height is to use the sensor for short range obstacle detection, i.e. < 914mm. This matches what I’ve read about sonar ranging sensors, i.e. that as the distance to be measured increases, the sensor will be more accurate when mounted higher off the ground. Although this result is not surprising I did think it was interesting how this sensor’s accuracy hit 1.0 at 60” for the mid and high mounted measurements. Seems like in my environment this accuracy might be good at scanning a room for corners and openings…
Tom
To err is human.
To really foul up, use a computer.
Lee G reacted
2025-08-12 9:01 pm
Caveat:
These are just casual observations. I'm not an avionics guy and have no experience with the device.
Something seems amiss here. The test data implies that the sensor is aware of direction (i.e., ground level.) Since the sensor can be oriented in any direction, the concept of up or down is arbitrary. This can be seen using the picture from the user manual
The X and Y axis may be oriented in any manner desired by orienting the sensor. Unless I'm mistaken, this is the expected condition with avionics. A plane 1000m above ground shouldn't affect a sensor due to a change in altitude of 10m.
Somewhere in the User manual, datasheet, or web page it is stated that the test condition was for a target directly in front of the sensor. Assuming these results use the same test condition, I can't see a relationship to height above ground.
The sensor timing budget and lighting have an effect but I haven't been able work through them so far.
The one who has the most fun, wins!
2025-08-13 12:09 pm
Hi Tom @thrandell & Tim @tfmccarthy,
Unfortunately, the opening sentence "I became interested in comparing the accuracy of sensors when results were gathered at different heights above the ground after reading about how height can influence the results of a sonar range finder." does not include a reference, so I couldn't check the context. So, apologies if, yet again, I'm getting the wrong end of the stick, in which case I apologise. Meanwhile, I'll let my imagination wander a little...
-----------
In the case of an aircraft needing an accurate height above ground measurement, typically during takeoff and landing, then a downward facing transmitter and sensor pair, based on sound, light or radio waves, is the obvious configuration. In this case, an important challenge for the instrumentation designer is likely to be the natural variation between adjacent measurements, resulting in 'noisy data'. Assuming such variations cannot readily be reduced to a negligible variation, by improving the transmitter and/or sensor, then the designer is likely to resort to filtering mechanisms to provide the 'best estimated value', for the pilot's readout. This appears to be directly analogous to Tom's experimental study.
In principle, this system could also be used for measuring distances to vertical objects in a 'flat earth' scenario, by simply rotating the transmitter and sensor combination through 90 degrees. So in a simple scenario, the transmitter & sensor combination is unaware of the direction in which it is pointed, which I believe is related to Tim's point.
Of course, there are countless examples of this collision avoidance approach, including the ubiquitous ultrasound modules for robot collision avoidance and car parking sensors, plus lidar modules for range finding, etc. In these cases, the principal direction of the wave is 'approximately' horizontal, but height is usually a factor to consider. The 'problem' being that the 'wave' usually 'follows' a cone-like volume, which expands as the distance from the transmitter increases. This implies that in many cases, when the distance from the transmitter substantially exceeds the height above the ground, at least some of the 'cone' will intersect with the ground. When this happens, some of the wave is likely to be reflected back to the sensor, where it may be mistaken for a nearby vertical object. In principle, a well-collimated, narrow laser beam, analogous to a 'laser pointer' could travel a long distance before it intersected with the ground. However, this is very difficult/impossible to achieve with ultrasound or radio waves, and the low-cost lidar modules I have seen have cone angles of at least a few degrees, so excepting tall robots, etc., or very short range detection only, height is likely to be a consideration in many real cases.
----
So sorry, if this diatribe is only adding confusion, but I hope it is a useful hint for further thought.
Best wishes to all, Dave
Lee G reacted
2025-08-16 10:25 pm
I've been reading my copy of the datasheet and user manual for the VL53L1X with the idea of how to replicate your and their accuracy and precision results. But the more I read the less informed I become. So before I go further I wanted to verify I'm working from the same sources you have.
The datasheet I have is from STMicroelectronics: Datasheet - VL53L1X
The User manual is here: VL53L1X API user manual(UM2356)
Lastly, I'm assuming you used the github AdaFruit library: Adafruit_VL53L1X
Are these the same reference materials you have, or do you have something else?
You also mentioned a book reference. Could you provide that as well?
I'd really appreciate it.
Thanks,
- Tim
The one who has the most fun, wins!
Topic starter
2025-08-18 3:17 pm
I wanted to try mounting this sensor: https://www.pololu.com/product/5493 to a servo to implement a simple range scanner and graph the data points I collect to see if I could identify simple features like corners and openings.
Back in September 2019 here on dronebot: https://forum.dronebotworkshop.com/sensors-modules/object-detection-and-avoidance-using-ultrasonics/ @pugwash, @robo-pi & @robotbuilder worked on this same idea with sonar range finders. They each had a different goal: pugwash wanted to let his robot follow the contours of its environment via umbra and penumbra zones detected by the sensor (wall following?); James wanted to get his robot to identify its location via a pre-determined map of his home (localization?) & Casey wanted to gather data to analyze. It looks like I missed the boat by a few years.
I attached the sensor to a Hi-Tec HS-311 servo that has a range of +/- 60 degrees. I powered the servo with an external 5V power source and used my laptop USB connection to power the sensor and Raspberry Pi Pico.
Prusa Orange!
Because this sensor is accurate, but not very precise I decided to come up with a sample size that I could average and use for a data point. I settled on 10 readings per sample and a servo rotation rate of 5 degrees per sample. The only bummer about this is that now the data acquisition is slowing down the servo speed. The sensor is spec’d to identify objects up to 300cm away so in my program I report any reading over 300cm as 400cm and treat that sample as ‘object not detected’. The output of the program is the Cartesian and Polar coordinates for the 25 data points.
In this graph the sensor sits at the origin. This is a scan of the corner of a room with double doors, one door is open and the other is closed. The line of dots on the right side between zero and 25 degrees is the open door resting against a wall. Between 25 and 45 degrees is the opening that cannot be detected. The line of dots between 45 and 65 is the closed door. To the left of that door the wall makes a slight angle that you can see between 75 and 80 degrees. Beyond 100 degrees is a sliding closet door that is set back from the wall. Clear as mud?
Extracting features like lines from this data is still a big jump for me, but line recognition can help with localizing a robot and creating a map of its environment which I guess is where James was going back in 2019.
Tom
To err is human.
To really foul up, use a computer.
Lee G reacted
2025-08-18 6:56 pm
Extracting features like lines from this data is still a big jump for me, but line recognition can help with localizing a robot and creating a map of its environment which I guess is where James was going back in 2019.
Keep in mind the dots (lines) are relative to the robot and do not provide absolute coordinates. As you probably know Simultaneous Localization and Mapping (SLAM) techniques are used to create maps from such data.
Connecting the dot points might be as simple as connecting each point to its nearest neighbour which will be greater the further they are away from the sensor. If the distance is beyond the expected then it is a break.
This reminds me of @inq and his experiments with the VL53L5CX
https://forum.dronebotworkshop.com/user-robot-projects/inqegg-one-step-forward-two-steps-back/
Lee G reacted
2025-08-18 10:36 pm
I wanted to try mounting this sensor:...
OK, this is quite a bit to chew. Gimme a bit to pull things together. I'm gonna use the HC-SR04 sonar sensor ('cuz it's what I have) but it should be able to be substituted to a degree.
The one who has the most fun, wins!
2025-08-20 9:46 pm
Extracting features like lines from this data is still a big jump for me, but line recognition can help with localizing...
I think there are going to be memory constraints there.
In any case, I'm bogged down getting a "pan and tilt" mount built so in the meantime, have you considered using a linear regression algorithm, specifically a line fitting algorithm? I DL'd a C+= version but haven't looked into it. Still, I think you need coordinate pair containers and the pico only has 256 kb.
The one who has the most fun, wins!
Lee G reacted
2025-08-21 12:03 am
I'm following this with interest, may try something similar (just for grins...) I'm sure you're aware, if you go up to a Pico2W, you’ll double (almost) that SRAM. Flash goes from 2MB to 4MB. It’s only a couple of bucks more (~$7.00) at MicroCenter. Regards, Lee
2025-08-21 5:53 pm
I'm gonna use the HC-SR04 sonar sensor ('cuz it's what I have) but it should be able to be substituted to a degree.
Software solutions for converting the input data to a fixed map can be worked on using a simulated robot. ROS provides such simulators and probably has algorithms to stich together this kind of data into a map. Your algorithms can be tested on the simulation although you need to keep in mind that real hardware may not behave exactly in the same way as the simulated hardware such as @thrandell mentions needing to use averaging over 10 samples for the real hardware.
In the display of a simulator example below the problem is to write an algorithm to convert the TOF data on the right into the absolute coordinates of the layout on the left as the robot moves about (using what motor control algorithm?). Each input has to be stitched together to make a constant representation of the map of the left in the robots memory. This image show the data collected by spinning the robot around on the spot.
As we rotate our body the image on our eyes rotates as will the input on the right in the display below (relative data). However we experience ourselves as rotating while the world remains still despite it moving on the retina. If you press an eyeball (through the eyelid skin not directly on the eye!) you will actually experience the world moving not your eye. This shows how the brain uses signals controlling the eye to stabilize the image as a robot vision system might do by using the signals to the motors.
Robot vacuum cleaners create such maps using lidar or vision and display them on your iPhone so maybe these algorithms can be adapted for use with the tof sensor @thrandell is experimenting with.
2025-08-22 6:56 am
You're wearing your software developer's hat while I'm wearing my test engineering hat. I'm focused on getting a high degree of confidence in data collection rather than how that data will be used. I'm interested in trying to measure the various parameters used to collect sensor data before considering using a simulator.
I'm not qualified to discuss how eyesight works, other than to say I believe it's an analog as opposed to a digital model and deeply integrated with other biological systems. That said, I made a solem promise to my college chemistry teacher that I would stay as far as possible from chemistry or anything remotely related to it if I passed the course. I keep my promises. So I must excuse myself from further comment. I have nothing of worth to offer.
- Tim
The one who has the most fun, wins!
2025-08-22 9:47 am
A lot of engineering solutions can come from observing the biological world.
@thrandell said the data is accurate but not precise meaning the average is close to the true value. So now we can get on with figuring out how to use those average values to create a map while the robot moves about its environment. Each time the robot moves its input changes but if there are things in common between each input frame we can use them to stich all the frames into one big map.
Take the two frames below of the wood floor in my office. Automate the identification of some visual features in common and write code to use the change in position and rotation of those features to rotate and slide one over the other to match and keep repeating this until you make a larger map of the floor. The robot can then use this larger map with any current input to determine where it is and what direction it is pointing.
2025-08-23 3:41 pm
OK, this is quite a bit to chew. Gimme a bit to pull things together. I'm gonna use the HC-SR04 sonar sensor ('cuz it's what I have) but it should be able to be substituted to a degree.
So are you going ahead with this?
I don't have the sensor being used by @thrandell but have used the sonar sensor on a servo motor for obstacle avoidance experiments and as a sonar radar setup as there are examples on how to do this on the internet. To use this data for mapping there might be, as you wrote, memory constraints to deal with.
Don't want to hyjack this VL53L1X thread so will have to start another one.
2025-08-23 4:02 pm
So are you going ahead with this?
Yeah, I'm working on it.
I've pulled together various sketches and I'm testing them. I have to add serial port support to connect to the laptop. I've ordered the Adafruit "pan and tilt" that should arrive next week but I'm cobbling together a simple pan that should replicate the radar sketch. This to show how to get a "floorplan" map.
When I have the pan and tilt working, I plan to demonstrate producing a mesh to give a 2D result and then connect that to a solid modeling library to get a 3D result. Then a use SDL to add imagery to align with the model.
The one who has the most fun, wins!
2025-08-24 1:31 am
Just to be clear, I don't mean to "abandon" your thread here. My effort should apply to your work as well, simply by substituting one sensor for another: they're both distance sensors.
So, no offense meant, OK?
Tim
The one who has the most fun, wins!
Page 1 / 2
Next