[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-devel
Subject:    Re: Interest in building an LLM frontend for KDE
From:       Ethan Barry <ethanbarry () howdytx ! net>
Date:       2023-12-01 3:07:04
Message-ID: bEZx41TeTWpvyhVYf4n2yrBJln1dAs-48TZDQSCCu99m5_OOhXCAWdp_mjPS-ZsTRDZ3YvojeJxkb5VsYJUpw-DDvMv2711x_gNqJNbNTa0= () howdytx ! net
[Download RAW message or body]

On Thursday, November 30th, 2023 at 8:53 PM, Loren Burkholder <computersemiexpert@outlook.com> wrote:
> 
> 
> Howdy, everyone!
> 
> You are all undoubtedly aware of the buzz around LLMs for the past year. Of course, there are many \
> opinions on LLMs, ranging from "AI is the future/endgame for web search or programming or even running \
> your OS" to "AI should be avoided like the plague because it hallucinates and isn't fundamentally \
> intelligent" to "AI is evil because it was trained on massive datasets that were scraped without \
> permission and regurgitates that data without a license". I personally am of the opinion that while \
> output from LLMs should be taken with a grain of salt and cross-examined against trustworthy sources, \
> they can be quite useful for tasks like programming. 
> KDE obviously is not out to sell cloud services; that's why going to https://kde.org doesn't show you a \
> banner "Special offer! Get 1 TB of cloud storage for $25 per month!" Therefore, I'm not here to talk \
> about hosting a (paywalled) cloud LLM. However, I do think that it is worthwhile opening discussion \
> about a KDE-built LLM frontend app for local, self-hosted, or third-party-hosted models. 
> From a technical standpoint, such an app would be fairly easy to implement. It could rely on Ollama[0] \
> (or llama.cpp[1], although llama.cpp isn't focused on a server mode) to host the actual LLM; either of \
> those backends support a wide variety of hardware (including running on CPU; no fancy GPU required), as \
> well as many open-source LLM models like Llama 2. Additionally, using Ollama could allow users to \
> easily interact with remote Ollama instances, making this an appealing path for users who wished to \
> offload LLM work to a home server or even offload from a laptop to a more powerful desktop. 
> From an ideological standpoint, things get a little more nuanced. Does KDE condone or condemn the \
> abstract concept of an LLM? What about actual models we have available (i.e. are there no models today \
> that were trained in a way we view as morally OK)? Should we limit support to open models like Llama 2 \
> or would we be OK with adding API support for proprietary models like GPT-4? Should we be joining the \
> mainstream push to put AI into everything or should we stand apart and let Microsoft have its fun \
> focusing on AI instead of potentially more useful features? I don't recall seeing any discussion about \
> this before (at least not here), so I think those are all questions that should be fairly considered \
> before development on a KDE LLM frontend begins. 
> I think it's also worth pointing out that while we can sit behind our screens and spout out our ideals \
> about AI, there are many users who aren't really concerned about that and just like having a chatbot \
> that responds in what at least appears to be an intelligent manner about whatever they ask it. I have \
> personally made use of AI while programming to help me understand APIs, and I'm sure that other people \
> here have also had positive experiences with AI and plan to continue using it. 
> I fully understand that by sending this email I will likely be setting off a firestorm of arguments \
> about the morality of AI, but I'd like to remind everyone to (obviously) keep it civil. And for the \
> record, if public opinion comes down in favor of building a client, I will happily assume the \
> responsibility of kicking off and potentially maintaining development of said client. 
> Cheers,
> Loren Burkholder
> 
> P.S. If development of such an app goes through, you can get internet points by adding support for \
> Stable Diffusion and/or DALL-E :) 
> [0]: https://github.com/jmorganca/ollama
> [1]: https://github.com/ggerganov/llama.cpp


I am anti-LLM on the grounds that the training sets were created without the original authors' consent. I \
see no issue with a libre/ethical LLM, if there is one, though. If a developer or team of developers \
wants to implement a Qt and KDE-integrated LLM app, I have no problem with that, but I believe KDE as an \
organization should probably steer clear of such a thorny subject. It's sure to upset a lot of users no \
matter what position is taken. On the other hand, for those people who do make use of AI tools, a native \
interface would be nice, especially one as feature-ful as you're describing...

Regards,

Ethan B.


[prev in list] [next in list] [prev in thread] [next in thread]