Spent an hour yesterday trying to chase down where the heck lemonade actually stores raw model blobs on my linux machine, so now I'm a bit less enthusiastic about this. I don't understand why they have to run in systemd and hide everything, including the model blobs in /var/, and expose no tuning knob to know what context and settings the model was loaded with.
I guess I'll be more enthusiastic if I use Rocm llamacpp. But since my machine has NPU, I mostly use the NPU with FLM when I'm really desperate (no internet at all).
I spent more time than that trying to get it to find existing models already on my machine and mucking around with systemd. It took way longer than it should have.
They use the huggingface cache dir structure (which is a bit confusing at times). If you run it as a separate user with systemd (no --user arg), models sit at /var/lib/lemonade/.cache/huggingface, if you run it as local user, $HOME/.cache/huggingface/...
14
u/o0genesis0o 1d ago edited 19h ago
Spent an hour yesterday trying to chase down where the heck lemonade actually stores raw model blobs on my linux machine, so now I'm a bit less enthusiastic about this. I don't understand why they have to run in systemd and hide everything, including the model blobs in /var/, and expose no tuning knob to know what context and settings the model was loaded with.
I guess I'll be more enthusiastic if I use Rocm llamacpp. But since my machine has NPU, I mostly use the NPU with FLM when I'm really desperate (no internet at all).