Notifications
Clear all
Robot Projects
126
Posts
7
Users
18
Reactions
4,678
Views
Topic starter
2025-03-02 9:52 pm
As you are interested in computer vision I wonder if you would be better off to make a small investment in one of these AI cameras and taking your computer vision programs to the next level. This probably does mean stepping into a bit of the bleeding edge of computer vision and I don't think you will find so many nice examples of using them.
I have watched all Bill's videos on these AI cameras with great interest.
At this stage I haven't considered "taking it to the next level". Training such networks hundreds of times for some specialized tasks makes sense but they still have limits.
https://bdtechtalks.com/2020/03/02/geoffrey-hinton-convnets-cnn-limits/
For many professional and hobby vision projects they might be the best or only practical solution. However they do fail at times and thus you wouldn't want to use them where a failure might have a catastrophic outcome.
I have been interested in AI for a long time and that included neural networks. They became practical with back propagation, new architectures and very fast computers with lots of memory. That interest continues and I read a lot about them and how they work.
2025-03-03 4:12 am
D.J.Peters from the FreeBASIC forum wrote a webcam capture for Windows.
ok, there's a lot on the plate right now. I have FBImage, 2 escapi samples, a demo jig to build, and dozens of tests.
It is really great that you have come on board, thank you.
Thank you. I'm enjoying it so far, but I have to say, I have some doubts about the end result. I can't see a pathway to getting it off the laptop unless you replace freeBASIC. We'll cross that bridge when we get to it.
The one who has the most fun, wins!
Topic starter
2025-03-03 7:13 am
FreeBASIC runs on the RPi. A link to using FB with Linux and OpenCV.
https://www.freebasic.net/forum/viewtopic.php?t=28361
I do have the RPi camera connected but not sure if I can grab and process images from it with FreeBASIC or any language for that matter.
Anyway the next step I think is to use recognized visual features of known positions to work out where the robot is and in what direction it is pointing.
2025-03-03 3:19 pm
I do have the RPi camera connected but not sure if I can grab and process images from it with FreeBASIC or any language for that matter.
Say what-what?
You have a RPi with a RPi camera connected to it?
There has to be software that communicates with the camera. I suspect there's a C or C++ library to do so. The task would be to interface that library with freeBASIC.
I'm focused on getting FBImage to capture and save images to a file. There are toolchain issues to overcome.
The one who has the most fun, wins!
Topic starter
2025-03-03 4:54 pm
You have a RPi with a RPi camera connected to it?
I posted an image of it here, I use a wireless keyboard and mouse.
https://forum.dronebotworkshop.com/user-robot-projects/robot-base-new-build/paged/6/
I never got around to using the RPi beyond reading some buttons and turning LEDs on/off using FreeBASIC. I bought the LCD touch screen and RPi some years ago now.
Just did a search on the internet on using the camera. I see examples favour using Python.
https://pyimagesearch.com/2015/03/30/accessing-the-raspberry-pi-camera-with-opencv-and-python/
I'm focused on getting FBImage to capture and save images to a file. There are toolchain issues to overcome.
Not sure what a "toolchain" is. I just use bsave to save images.
I remember now you write rather complicated code compared with mine 🙂
My focus is on testing algorithms to extract useful information from images.
2025-03-04 12:47 am
I posted an image of it here, I use a wireless keyboard and mouse.
OK, that's good to know. We'll circle back to that later. I haven't started to work with the RPi but I expect that I would at some stage.
Not sure what a "toolchain" is. I just use bsave to save images.
"Toolchain" here is the software used to build the application, i.e., compiler, library manager, linker, debugger, editor, etc. They're the usual suspects that an IDE is made of.
My problem is I use the Microsoft toochain and freeBASIC uses the *nix toolchain (gcc, et al.) The problem is the object files aren't compatible. So, I need to adjust to use the gcc tools and I'm not as familiar with them. I should be able to get past this. nmake is my friend.
My focus is on testing algorithms to extract useful information from images.
We're calling that image processing.
Once that is under control the info would be translated into a direction to pass to the navigation process.
And I'm working on image capture to produce images that image processing code accepts.
So, Pinky, are you pondering what I'm pondering?
The one who has the most fun, wins!
Topic starter
2025-03-04 2:10 am
What I call image processing is things that change the whole image such as contrast enhancement, passing the image through filters that smooth the image or remove noise, reduce the resolution or range of pixel values and so on...
Extracting information I call making lists of things like edges, areas and so on to build up a description of what is there. A square for example has four interconnected straight edges. A triangle has three interconnected straight edges. How might you extract a description of 3d data like say a banana or a pear or a mushroom or a child's play block to be picked up by that robot arm?
At a the highest level there is a goal description and the creation of a plan to move toward that goal.
In the case of a robot vacuum cleaner that goal is to cover all the floor area to suck up the dirt. A sub goal triggered by an interrupt signal might be to find the charger when the battery voltage is getting low.
For navigation the goal is to go to location X. It has to plan what move to make next to achieve that outcome.
Have you run the code I posted? In this example I added a test call to invert the image.
#include "escapi.bi"
const IMGW = 640
const IMGH = 480
screenres IMGW,IMGH,32
dim shared as ubyte gray1(IMGW,IMGH),gray2(IMGW,IMGH)
dim shared as any ptr image
image = imagecreate(IMGW,IMGH)
'=============== ESCAPI CODE SETUP ============================
Dim As Integer nDevices=setupESCAPI()
If nDevices<1 Then
Print "No active capture device found!"
Beep:Sleep:End
Else
if nDevices > 4 then
nDevices = 4
end if
print "number of devices=";nDevices
End If
'list of pointers
Dim shared As Any Ptr lpImage(4) 'array of pointers
for k as integer = 0 to nDevices-1
lpImage(k)=ImageCreate(IMGW,IMGH) 'create bitmaps for cams
next k
Dim Shared As SimpleCapParams Params(4)
for k as integer = 0 to nDevices-1
Params(k).mWidth = IMGW
Params(k).mHeight = IMGH
Params(k).mTargetBuf=cptr(Integer Ptr,lpImage(k))
Params(k).mTargetBuf+=8
doCapture(k)
next k
for k as integer = 0 to nDevices-1
initCapture(k, @Params(k))
next k
' =================================================
sub getCamImage(cam as integer)
doCapture(cam)
While isCaptureDone(cam)<>1:Sleep(10):Wend
end sub
Sub DisplayImage(gray() as ubyte)
dim as ubyte ptr target = image
'copy to gray array to bitmap
for j as integer = 0 to IMGH-1
for i as integer = 0 to IMGW-1
target[(i*4+j*4*IMGW)+32] = gray(i,j) ' Blue level.
target[(1+i*4+j*4*IMGW)+32] = gray(i,j) ' Green level.
target[(2+i*4+j*4*IMGW)+32] = gray(i,j) ' Red level.
next i
next j
put (0,0),image,trans
End Sub
sub invertImage(gray() as ubyte)
for j as integer = 0 to IMGH-1
for i as integer = 0 to IMGW-1
gray(i,j)=255-gray(i,j)
next i
next j
end sub
sub convertToGrayArray(gray() as ubyte, cam as integer)
dim as ubyte ptr target = lpImage(cam)
dim as ulong v,r,g,b
for j as integer = 0 to IMGH-1
for i as integer = 0 to IMGW-1
r = target[(2+i*4+j*4*IMGW)+32]
g = target[(1+i*4+j*4*IMGW)+32]
b = target[(i*4+j*4*IMGW)+32]
v = (r+g+b)/3
gray(i,j)=v
next i
next j
end sub
dim as integer cam = 0
do
getCamImage(cam)
convertToGrayArray(gray1(), cam)
'PROCESS IMAGE HERE
'eg invert image
invertImage gray1()
if inkey=" " then 'if space key is pressed
bsave "imageSave.bmp",lpImage(cam)
bsave "grayImage.bmp",image
end if
screenlock
cls
DisplayImage gray1()
screenunlock
sleep 2
loop until multikey(&H01)
for k as integer = 0 to nDevices-1
imageDestroy(lpImage(k))
next k
2025-03-04 6:14 am
At a the highest level there is a goal description and the creation of a plan to move toward that goal.
In the case of a robot vacuum cleaner that goal is to cover all the floor area to suck up the dirt. A sub goal triggered by an interrupt signal might be to find the charger when the battery voltage is getting low.
@robotbuilder I don’t think that’s right.
The goal description and creation of a plan falls into what Rodney Brooks might call “Good Old Artificial Intelligence”. His group at MIT in the 1980’s moved away from that traditional AI approach and took on the challenge of creating autonomous robots that could negotiate a real world environment. This lead to the development of Behavior-Based Robotics.
Joe Jones, one of the inventors of Roomba (the robotic vacuum cleaner from iRobot) states in his book ‘Robot Programming’, that Roomba uses a behavior-based programming scheme, i.e. not the GOAI approach that you think was used.
Joseph Jones (2004) Robot Programming “A Practical Guide to Behavior-Based Robotics”.
Check out Rodney Brooks, he’s Australian! https://en.wikipedia.org/wiki/Rodney_Brooks
Tom
To err is human.
To really foul up, use a computer.
2025-03-04 8:38 am
Have you run the code I posted? In this example I added a test call to invert the image.
No, I haven't because I'm working on getting the build process under control. However, I did build and run the image invert app. It didn't work on my system. I don't know why. I think it has to do with the interaction between the console and escapi. It doesn't save the image when press the spacebar.
Now we don't need to use escapi to test the code. Just directly load any 640x480 bitmap image file, process it, and save the result. I modified the code to load a image of a color gradient to highlight the transformation.
I extracted the image transfer operation out of DisplayImage into xfr_image and moved the screenlock / unlock inot DisplayImage. This allowed me to show the image tranformation in stages. I also reorganized the code so that the main section is at the bottom.
The result is not what I was expecting but it does invert the greyscale image. I'm not sure that is what is wanted to detect an edge. Depends on the defintion of an "edge" but the inversion seems to indicate a shadow where there isn't one and visa-versa.
I didn't work on the screen indexing but feel that could be simplified.
A request: attach the source files to the post rather than print them out. It makes it easier tot read the post and, by naming the file, reduces confusion. You can quote just relevant sections in the post.
The one who has the most fun, wins!
Topic starter
2025-03-04 9:05 am
Thank you for the interest. I am always happy to use a simpler method to achieve the same outcome. I am familiar with the subject matter you alluded to.
Casey
Topic starter
2025-03-04 9:42 am
Pity my code didn't run. Your code ran ok. Will look at it later.
An edge is a change in adjacent pixel values. I have two edge routines one for monochrome and one for color. Different colors can come out as the same or similar gray levels. Thus if you edge a color image the edges can be stronger (brighter) particularly if their gray counter part is similar in intensity.
As you can see in the attached image. The edge between the colors are easy to see but when converted to monochrome they look much the same!!
A request: attach the source files to the post rather than print them out. It makes it easier to read the post and, by naming the file, reduces confusion. You can quote just relevant sections in the post.
Ok.
My thought was that some might like to read it without downloading to find out what part of the source code was being referred to in the post. I can name the code before the printing the code and name the code in the source code itself.
2025-03-04 2:00 pm
Pity my code didn't run.
In reality, it was a benefit. See, I don't have your set-up. I just have the one webcam on my laptop. Of course, that just captures what's in front of the laptop, which would be my mug. Now I know I'm a handsome devil but there are reports of sudden illness and crying children upon seeing my handsomeness, unawares.
An edge is a change in adjacent pixel values.
That's not a very strong definition. The color depth of the image would almost insure a change for every pixel. Changing the palette to a monochrome palette will reduce the effective color depth and may obscure an edge that would otherwise be apparent, e.g., two contrasting light colors.
Thus if you edge a color image the edges can be stronger (brighter) particularly if their gray counter part is similar in intensity.
I not sure about this. For example, in the color gradient image, for the blue and brown squares, middle row, have similar monochrome values but are high contrasting colors that would indicate an edge.
But further, your algorithm inverts the monochrome colors, so bright becomes dark, etc., so the brightness is inverted and it may indicate an edge in a brightly lit area.
Try to mark the detected edges on the monochrome and inverted image with color. Use a simple image like the gradient image.
This post was modified 1 month ago by TFMcCarthy
The one who has the most fun, wins!
Topic starter
2025-03-04 9:44 pm
Just found your reference to Pinky which is the cartoon series Pinky and the Brain?
With my setup I also have a laptop webcam. The cam = 0 defaults to the laptop webcam if I disconnect the other one. You can capture and display images from more than one webcam which is handy if I want to play with stereo images. Viewing yourself at my age is best avoided although my grand daughter says she prefers the way I look now after seeing pictures of me in my youth.
I will let the Brain decide on the definition of an edge in image processing?
You could define it as the output of a particular edge processing routine.
https://blog.roboflow.com/edge-detection/
Your are right about the monochrome outputs. I just assumed I was getting the correct set of gray values. I didn't realise the results would be so different.
Vision is complex and color is complex. When you learn what they now know about the human visual system you realise how far these AI things are from how we process and experience vision.
These are the pixel colors used.
rgb( 18 ,170 , 6)
rgb(121 , 86 ,105)
rgb(247 , 24 ,156)
rgb( 90 , 16 ,106)
rgb(128 ,111 , 3)
rgb(237 , 39 ,111)
There are two ways given of converting them to a gray version.
gray_value = int((r+g+b)/3)
gray_value = int((0.3 * r) + (0.59 * g) + (0.11 * b))
2025-03-05 5:23 am
Just found your reference to Pinky which is the cartoon series Pinky and the Brain?
The very same. I always enjoyed that line.
You could define it as the output of a particular edge processing routine.
I'm ignorant about vison algorithms, so I enjoyed the article. However, it contains a fair bit of math (derivatives, matrix algebra, etc.) that I haven't worked with since college. And I have to admit, your interest in it is a surprise. Understanding that is certainly not less complex than programming. Also, I don't think those algorithms are suitable for run-time calculation.
The one who has the most fun, wins!
Topic starter
2025-03-05 8:29 am
After decades of programming I have learned a bit of math 🙂
Also, I don't think those algorithms are suitable for run-time calculation.
You would have to run them and see. If you can get the webcam capture code I posted to work then you can test edge detection on the captured images in real time and see how fast the code runs. It looks pretty fast to me including the target following code. You can move the target around to see how quickly it locates and identifies it.
Some image processing does use advanced math but there is plenty of simpler to understand methods to extract useful information from an image for practical use in a robot vision system.