Good on you for coming back and finishing the project the way you intended to and posting about it.
The IMA ADPCM compression you discovered sounds very useful. If i continue with audio projects that would be very useful.
I finished my baby monitor project. It has a lot of simularities to your project. I used wifi, my home network as the conduit and that made it much more complicated. Home networks can be pretty busy these days and it caused delays and it was hard to track down as I was looking for an issue in the code but it was just the network being tied up. I had to compensate that with storing of packets both on the esp32 and on the client. It would of not been a problem if i was not trying to do something that depended on being close to real time as possible.
I used AI to help me on the project and it was awesome. AI exposed me to new techniques and help clear up some misunderstanding I've had. You have to be careful though, sometimes it can be wrong and go down rabbit holes.
I did not use radio because i thought the signal would be iffy though walls and floors but my daughter has a baby monitor, does not use the network and works great . They must have very good antenna's lol,