god i can’t wait for the ai bubble to pop
There is already GPT4All.
Convenient graphical interface, any model you like (for Llama fans - of course it’s there), fully local, easy to opt in or out of data collection, and no fuss to install - it’s just a Linux/Windows/MacOS app.
For Linux folks, it is also available as flatpak for your convenience.
So it doesnt require internet access at all? I would only use these on a disconnected part of my network.
Yes, it works perfectly well without Internet. Tried it both on physically disconnected PC and laptop in airplane mode.
Google hosting their shit on Microsofts servers and telling you to sideload and not using their own software distribution method for their own OS is kind of crazy if you think about it
Wonder what this has over its competitors, I hesitate to think they released this for fun though
Excellent, I will be sure not to use this, like all Google shit.
deleted by creator
In a few years you won’t be able to anyway
I’m just reaching the end game faster then.
You never heard of ollama or docker model runner?
Android and iOS.
Why would I use this over Ollama?
Ollama can’t run on Android
That’s fair, but I think I’d rather self host an Ollama server and connect to it with an Android client in that case. Much better performance.
How is Ollama compared to GPT models? I used the paid tier for work and I’m curious how this stacks up.
It’s decent, with the deepseek model anyway. It’s not as fast and has a lower parameter count though. You might just need to try it and see if it fits your needs or not.
Yes, that’s my setup. But this will be useful for cases where internet connection is not reliable
Is there any useful model you can run on a phone?
Llama.cpp (on which ollama runs on) can. And many chat programs for phones can use it.
Try PocketPal instead
You can use it in termux
Has this actually been done? If so, I assume it would only be able to use the CPU
Yeah I have it in termux. Ollama is in the package repos for termux. The speed it generates does feel like cpu speed but idk
Alibaba also provides an OpenSource App, it even has support for their multimodal voice chat Model qwen2.5 omni: https://github.com/alibaba/MNN
Enclave on iOS does the trick for the rare times i need a local LLM
Didn’t know about this. Checking it out now, thanks!
Is the chat uncensored?
And unmonitored? Don’t trust anything from Google anymore.
What makes this better than Ollama?
Quote: “all running locally, without needing an internet connection once the model is loaded. Experiment with different models, chat, ask questions with images, explore prompts, and more!”
So you can download it and set the device to airplane mode, never go online again - they won’t be able to monitor anything, even if there’s code for that included.
never go online again - they won’t be able to monitor anything, even if there’s code for that included.
Sounds counter-intuitive on a smart phone where you most likely want to be online again at some point in time.
So trust them. If you don’t and want to use this, buy a separate device for it, or VM.
Can’t? This is not for you.
I won’t gonna use my smartphone as a local llm machine.
That is exactly what Ollama does too.
everything is unmonitored if you don’t connect to the network.
But not everything works in those conditions.
it does if you make it work in those conditions.
software that “phones home” is easy to fool.
Tell that to my Red Dead Redemption 2 install.
Just firewall the software or is there anything more fancy i would need to do?
typically the phone home is looking for a response to unlock.
use a packet sniffer to see what the request/response is and replicate it with a proxy or response server.
this is also know as a man-in-the-middle (mitm).
takes skill and knowledge to do, but once you do a few dozen it’s pretty easy since most software “phone homes” are looking for static non-encrypted responses.
Thanks for the info!
Censoring is model dependent so you can select one of the models without the guardrails.
