[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-devel
Subject:    Re: Interest in building an LLM frontend for KDE
From:       Ethan Barry <ethanbarry () howdytx ! net>
Date:       2023-12-01 3:07:04
Message-ID: bEZx41TeTWpvyhVYf4n2yrBJln1dAs-48TZDQSCCu99m5_OOhXCAWdp_mjPS-ZsTRDZ3YvojeJxkb5VsYJUpw-DDvMv2711x_gNqJNbNTa0= () howdytx ! net
[Download RAW message or body]

On Thursday, November 30th, 2023 at 8:53 PM, Loren Burkholder <computersemiexpert@outlook.com> \
wrote:
> 
> 
> Howdy, everyone!
> 
> You are all undoubtedly aware of the buzz around LLMs for the past year. Of course, there are \
> many opinions on LLMs, ranging from "AI is the future/endgame for web search or programming \
> or even running your OS" to "AI should be avoided like the plague because it hallucinates and \
> isn't fundamentally intelligent" to "AI is evil because it was trained on massive datasets \
> that were scraped without permission and regurgitates that data without a license". I \
> personally am of the opinion that while output from LLMs should be taken with a grain of salt \
> and cross-examined against trustworthy sources, they can be quite useful for tasks like \
> programming. 
> KDE obviously is not out to sell cloud services; that's why going to https://kde.org doesn't \
> show you a banner "Special offer! Get 1 TB of cloud storage for $25 per month!" Therefore, \
> I'm not here to talk about hosting a (paywalled) cloud LLM. However, I do think that it is \
> worthwhile opening discussion about a KDE-built LLM frontend app for local, self-hosted, or \
> third-party-hosted models. 
> From a technical standpoint, such an app would be fairly easy to implement. It could rely on \
> Ollama[0] (or llama.cpp[1], although llama.cpp isn't focused on a server mode) to host the \
> actual LLM; either of those backends support a wide variety of hardware (including running on \
> CPU; no fancy GPU required), as well as many open-source LLM models like Llama 2. \
> Additionally, using Ollama could allow users to easily interact with remote Ollama instances, \
> making this an appealing path for users who wished to offload LLM work to a home server or \
> even offload from a laptop to a more powerful desktop. 
> From an ideological standpoint, things get a little more nuanced. Does KDE condone or condemn \
> the abstract concept of an LLM? What about actual models we have available (i.e. are there no \
> models today that were trained in a way we view as morally OK)? Should we limit support to \
> open models like Llama 2 or would we be OK with adding API support for proprietary models \
> like GPT-4? Should we be joining the mainstream push to put AI into everything or should we \
> stand apart and let Microsoft have its fun focusing on AI instead of potentially more useful \
> features? I don't recall seeing any discussion about this before (at least not here), so I \
> think those are all questions that should be fairly considered before development on a KDE \
> LLM frontend begins. 
> I think it's also worth pointing out that while we can sit behind our screens and spout out \
> our ideals about AI, there are many users who aren't really concerned about that and just \
> like having a chatbot that responds in what at least appears to be an intelligent manner \
> about whatever they ask it. I have personally made use of AI while programming to help me \
> understand APIs, and I'm sure that other people here have also had positive experiences with \
> AI and plan to continue using it. 
> I fully understand that by sending this email I will likely be setting off a firestorm of \
> arguments about the morality of AI, but I'd like to remind everyone to (obviously) keep it \
> civil. And for the record, if public opinion comes down in favor of building a client, I will \
> happily assume the responsibility of kicking off and potentially maintaining development of \
> said client. 
> Cheers,
> Loren Burkholder
> 
> P.S. If development of such an app goes through, you can get internet points by adding \
> support for Stable Diffusion and/or DALL-E :) 
> [0]: https://github.com/jmorganca/ollama
> [1]: https://github.com/ggerganov/llama.cpp


I am anti-LLM on the grounds that the training sets were created without the original authors' \
consent. I see no issue with a libre/ethical LLM, if there is one, though. If a developer or \
team of developers wants to implement a Qt and KDE-integrated LLM app, I have no problem with \
that, but I believe KDE as an organization should probably steer clear of such a thorny \
subject. It's sure to upset a lot of users no matter what position is taken. On the other hand, \
for those people who do make use of AI tools, a native interface would be nice, especially one \
as feature-ful as you're describing...

Regards,

Ethan B.


[prev in list] [next in list] [prev in thread] [next in thread]