I would not recommend LocalAI. There documentation is somewhat lacking and it’s an all in one utility with many moving parts. The parts also tend to break, quite often.
I have a similar setup and it’s been a huge pain when I when I have to do the OS updates.
The Coral needs a dkms module, but the sources and Google’s own documentation for it are out of date. I would highly recommend using the iGPU for inference.
You raise some good points and I agree with most of them.
I live in Canada and the only option of I have to sideload is paying for a developer account which is $100 CAD/year, otherwise I am stuck with a version where I have to re-sign the apps every 7 days. I know that the EU version is cumbersome and still needs Apple to notarise apps. My point here is that it’s hard to sideload, within and outside of the EU.
I truly believe their notion of a “walled garden” is just nonsense. They use this to justify preventing other people from accessing their digital marketplace or creating their own marketplace. It’s just a money grab and a monopoly.
At least on Android, I have the option to use the Play Store, Samsung Store, or any other 3rd party App Store, or simply sideload for free. I don’t have this option with an iPhone.
That makes too much sense and you can’t shoehorn AI for that sweet sweet VC money.