[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-devel
Subject:    Interest in building an LLM frontend for KDE
From:       Loren Burkholder <computersemiexpert () outlook ! com>
Date:       2023-12-01 2:53:22
Message-ID: SA3PR20MB5936B8C9D0ACA1C3B05DAC15C781A () SA3PR20MB5936 ! namprd20 ! prod ! outlook ! com
[Download RAW message or body]


Howdy, everyone!

You are all undoubtedly aware of the buzz around LLMs for the past year. Of course, \
there are many opinions on LLMs, ranging from "AI is the future/endgame for web \
search or programming or even running your OS" to "AI should be avoided like the \
plague because it hallucinates and isn't fundamentally intelligent" to "AI is evil \
because it was trained on massive datasets that were scraped without permission and \
regurgitates that data without a license". I personally am of the opinion that while \
output from LLMs should be taken with a grain of salt and cross-examined against \
trustworthy sources, they can be quite useful for tasks like programming.

KDE obviously is not out to sell cloud services; that's why going to https://kde.org \
doesn't show you a banner "Special offer! Get 1 TB of cloud storage for $25 per \
month!" Therefore, I'm *not* here to talk about hosting a (paywalled) cloud LLM. \
However, I do think that it is worthwhile opening discussion about a KDE-built LLM \
frontend app for local, self-hosted, or third-party-hosted models.

From a technical standpoint, such an app would be fairly easy to implement. It could \
rely on Ollama[0] (or llama.cpp[1], although llama.cpp isn't focused on a server \
mode) to host the actual LLM; either of those backends support a wide variety of \
hardware (including running on CPU; no fancy GPU required), as well as many \
open-source LLM models like Llama 2. Additionally, using Ollama could allow users to \
easily interact with remote Ollama instances, making this an appealing path for users \
who wished to offload LLM work to a home server or even offload from a laptop to a \
more powerful desktop.

From an ideological standpoint, things get a little more nuanced. Does KDE condone or \
condemn the abstract concept of an LLM? What about actual models we have available \
(i.e. are there no models today that were trained in a way we view as morally OK)? \
Should we limit support to open models like Llama 2 or would we be OK with adding API \
support for proprietary models like GPT-4? Should we be joining the mainstream push \
to put AI into everything or should we stand apart and let Microsoft have its fun \
focusing on AI instead of potentially more useful features? I don't recall seeing any \
discussion about this before (at least not here), so I think those are all questions \
that should be fairly considered before development on a KDE LLM frontend begins.

I think it's also worth pointing out that while we can sit behind our screens and \
spout out our ideals about AI, there are many users who aren't really concerned about \
that and just like having a chatbot that responds in what at least appears to be an \
intelligent manner about whatever they ask it. I have personally made use of AI while \
programming to help me understand APIs, and I'm sure that other people here have also \
had positive experiences with AI and plan to continue using it.

I fully understand that by sending this email I will likely be setting off a \
firestorm of arguments about the morality of AI, but I'd like to remind everyone to \
(obviously) keep it civil. And for the record, if public opinion comes down in favor \
of building a client, I will happily assume the responsibility of kicking off and \
potentially maintaining development of said client.

Cheers,
Loren Burkholder

P.S. If development of such an app goes through, you can get internet points by \
adding support for Stable Diffusion and/or DALL-E :)

[0]: https://github.com/jmorganca/ollama
[1]: https://github.com/ggerganov/llama.cpp


["signature.asc" (application/pgp-signature)]

[prev in list] [next in list] [prev in thread] [next in thread]