RADXA X2L, 8Gb ram, nVME, Single Board Computer with Pico 4020 chip & GPIO.

Results are in and NOT good. 🙁

Turns out the Radxa X2L 8Gb is actually slower at processing local Large Language Model inference than the Raspberry Pi 5.  I did side by side comparison and the Pi5 wins every time... by a lot...  So back to the drawing board.

Having said that, I do like this little SBC. I installed both Linux Mint and Windows 10. Both straightforward, and worked. Wifi/BLE module not recognized on OS install, but I will look into that later. Probably just needs a driver.  I installed Thonny, put MicroPython on the RP4020 chip, and had access to the GPIO (presumably works, I didn't wire anything, but all looked normal, interpreting MicroPython on the 4020)

Ran fine on both a Raspberry Pi 5 OEM power brick, and my also Anker power bank.  Seemed to be drawing about 3-4 watts at idle and 9-10w at full 99% CPU while processing LLM requests. Just SLOW, too slow to make natural conversation with the robot. So back to the drawing board.

So for local (non-cloud) LLM in robotic application, either I need to:

1) Tackle this crazy power issue and continue the project with the Pi5, so it can run full tilt, un-inhibited. or...

2) Build a small SBC or NUC and use a DC-ATX power supply, which probably means Intel CPU and having the option to add pci Nvidia GPU. or...

3) Use a Jetson or other SBC/ dev board intended for AI inference as a one-stop-shop

Going forward, I will probably continue to work on a power solution for #1, start exploring hardware and build ideas using my scrap piles and parts drawer for #2, and start saving my pennies and writing letters to Santa for #3.

@whitneydesignlabs I don't understand why you think there is some mystery re the Pi5 power. If you are going with battery power then just match the output of the Pi5 wall wart. I think it is 5V @ 5A. I would use something bigger than the 18650 as it is almost impossible to find non-counterfeit cells. There are similar cells that are thicker and or taller. Also you could use LiPo, the issue there other thn safety is you bneed a special balancing charger. I recommend the SkyRC line of chargers for all your charger needs. May be more expensive, but much safer and they work correctly.

Posted by: @zander

↑

@whitneydesignlabs I don't understand why you think there is some mystery re the Pi5 power. If you are going with battery power then just match the output of the Pi5 wall wart. I think it is 5V @ 5A. 

Hi Ron, because there IS a big mystery with the Pi5 power.  I'll post a link to the thread where we explored it in depth:
https://forum.dronebotworkshop.com/raspberry-pi/raspberry-pi-5-dc-dc-buck-down-power-supply-from-36v-best-method/paged/4/#post-45769

I tried many options with the Pi5. Nothing I found so far will let it run 100% max CPU except for the factory wall wart.  If a new solution has popped up in the last 3 months, I will definitely explore it. Things that don't work:

1) Top of the line Anker power bank claiming USB C PD. (tried several)

2) Supplying 5vdc at 5A directly to the 5v pins on the header, (using high output DC-DC buck)

3) Top of the line, high wattage automotive cigarette lighter adapter, claiming USB C PD.

Options 1-3 do not work.

That is why I am currently using an autotmotive inverter and the factory wall wart to run the Pi 5 at full tilt on battery. It is kludge, but at least it works on battery.

I have plenty of big battery packs of all voltages and amperages. (18650s in 3s2p, 4s2p, ...10s... etc.) No shortage of power. I just can't get the Pi5 to accept the power without throttling itself.

@whitneydesignlabs The replacement battery packs will have to connect via the USB connector, NOT the 5V pin. I thought you knew that.

I originally thought just get a portable inverter, but that does waste some power so instead recommended a direct DC connection.

Maybe build your own inverter?

TBH, I don't understand how they shove 5A down those tiny USB-C wires. I obviously have a poor mental model of wire cross section and allowable current.

The only thing I have not done is use a USB C break out board, and shove 5vdc at 5A directly down the USB C power wires.  Which, as far as I can tell, maybe is what the mystery of the OEM Pi5 wall wart does - totally OUT of Spec for USB C. Naturally, this has some risks and I am hesitant to try it. To your point, small wires, high amperage can be a problem...

In answer to your question, normally they (other USB C devices/chargers) don't shove 5a at 5vd down those small wires. Normally USB -C charger/supply negotiates with device and increases voltage to get more total power. But the Pi5 does not do this. It seems to want 5vdc at 5A which not what a normal USB spec device will deliver. Hence the quagmire with Pi5 dc power.

The portable inverter does work, and is my current work around. I'd rather not custom build an inverter. Time could be spent elsewhere.

But I was trying to locate the LLM compute in the robot head (about human size). However, if an inverter becomes integral to the LLM compute, it might need to be located elsewhere in the bot. If I do that, I might as well build a dedicated LLM compute device, as small as possible, but whatever it takes in size, and locate it elsewhere.  I can use Pis in the head to handle display, animatronics, TTS & STT etc.  Main AI compute may need to go elsewhere, i.e. torso, wheeled base, frame etc.  AI vision compute maybe could still fit in the head, maybe goes elsewhere, too.

@whitneydesignlabs I just remembered, there is a secondary DC lane, maybe they send 3A down the normal lane, and 2 more down the secondary lane. What I would do is hook up a Pi5 and a Pi5 wall wart with a breakout board between them and measure the voltage and current (with clamp amp meter)

Hi @whitneydesignlabs, 

  Sorry you have hit problems with LLM performance ... but hopefully you are learning from it.

  As for the R-Pi 5 power.

  I haven't seen one, nor have I seen circuits for R-Pi 5 yet, but your experience doesn't make a lot of sense.

  From my memory of what I read a while back, USB-C in its higher voltage forms now goes up to 5A, albeit officially using higher rated cables, which have'smart electronics' to indicate their elevated current capability status to power supplies.

The awkward bit is that this does not apply to the 5V option, because the standards people thought 25W (5V @ 5A) was already covered at the intermediate voltages, at 3A. However, from the point of view of copper wires and contacts, if a cable can supply 5a at a voltage above 5V, it should cope at 5V as well. 

Of course, there may be issues in finding connecting a 5A cable at the power supply end, because 5V 5A USB based power supplies, excepting the R-Pi one, are going to be hard to find, so some form of cable surgery maybe required, but this is at the supply end, not the R-Pi end.

Alternately, a quick look at the R-PI 4 schematic https://datasheets.raspberrypi.com/rpi4/raspberry-pi-4-reduced-schematics.pdf

Suggests the 5V and GND pins on the 40-way connector go directly to the same net. (I don't know if R-Pi 5 does anything different.)  In principle, the tracking from the pins might be a bit too thin, or something, but I would have thought/hoped they would have coped. Of course, those PCBs are small and restrained to the same size a sth eoriginal R-Pi, so compromises are going to happen.

I am suspicious that the problems are mainly down to the supplies you have tried. Any supply with an USB plug or socket, excluding the R-Pi wart, is almost bound to fail by design, because 5V 5A is beyond the USB specs.

As for buck regulator you tried, I just don't know. Maybe it isn't as good as it claims? And some are very noisy, being aimed at driving LED lights and so on.

Whilst I realise you are looking for battery supply, if you want still want to power the R-Pi, I think I would start with a 'reliable' 5V mains power unit, rated at a bit more than 5A ..say 7-8A, and play with ways of getting the R-Pi to accept it. Hopefully you can find a supply, without spending too much, and use it as a learning curve.

I think I would start by trying to find a way of pushing it in via the USB, unless you can find any reliable info about putting it in through the 40-way, as clearly the board has been designed to be powered that way, and the PCB tracking might not be up to currents that high from the 40-way pins. As I mention above, that might mean surgery to a cable or handwiring a USB socket. There are sockets with breakout PCBs available, but make sure they include all of the pins you want to drive. Make sure it is a cable rated at 5A.

Of course, these are all speculative suggestions, and as with any research project, some problems only show up the expensive way, so the risks are onto you.

Best wishes and good luck, Dave

@davee I have the top quality 100W cables, and the wires are still impossibly thin. Did we not determine that the 5V line whether it be USB-A or USB-C is max 3A, the 20V is 5A yielding the 100 Watts (the latest is 48V at 5A for 240W) some of my cables are rated at. In any case, my hunch is that the Pi folks are taking advantage of the old USB-2 standard 5V 3A pins, and then using the VBUS and G pins to supply the extra power. What I suggested to the OP is to connect a Pi5 to a Pi5 wall wart with a breakout board in between then use a clamp on amp meter and VOM to determine what power is on what pins. My bet is the old USB-2 power will be 5V at 3A and the new VBUS will be the balance of up to 2A at 5V.

@davee I think I found the technique the Pi folks used. See attached pic and note the 3A OR 5A statement for 5V or 20V. Full article is at link

@davee I also found this interesting discussion on the Adafruit forum. As per the spec 5V is max 3A, BUT by using a buck converter you can take the 20V at 5A as input and get a bunch of amps at 5V. I doubt that is what the guys at Pi did though.

https://forums.adafruit.com/viewtopic.php?t=205201

@davee Just to confuse us even more, here is a pdf from Texas Instruments. It talks of 5 TO 20 V at 3 to 5 Amps. Clearly this includes 5V at 5A so the Pi5 mystery is no more!

It is beyond my skills to prove that from the TI pdf, but again, a VOM and clamp on ammeter will tell no lies.

Here is the pdf for those wanting a light read.

Fellas, friends, @Davee and @zander. I think I am just about over trying to figure out what the Pi5 has or wants or expects or negotiates or doesn't negotiate for power. I super-duper appreciate your speculations, but I have been doing real world testing on multiple Pi5s and multiple power sources. The Pi5 is not like the Pi4. VERY different power handling.  As of now, I will back burner trying to find a solution for Pi5 to run the primary LLM on DC at 100% CPU usage.  Something low power-(ish) with GPU(s) may be inevitable for my robotic ideas.

I had hoped the Radxa X2L would be an alternative. It is not. It is slower at AI, LLM, deep learning, tasks than the Pi5.

No worries, I am moving on to scrap yard SFF PCs and cheap entry level video cards, powered from DC-ATX power supplies or my own variant. The Pi5s will still hold a task in the robot. Modular design means everything can fall into its best use-case.