Kobold-Assistant

NOTE: This was a fun project in the early ChatGPT days. Now, it's outdated. I'd recommend looking at open-webui + llama.cpp + openedai-speech, until the true end-to-end multimodal models are available for this.

A fully offline voice assistant interface to KoboldAI's large language model API. Can probably also work online with the KoboldAI horde and online speech-to-text and text-to-speech models, if you really want it to.

It's reasonably good; at least as good as Amazon Alexa, if not better. It uses the latest coqui "jenny" text to speech model, and openAI's whisper speech recognition, and additionally the model is prompted to know that it's getting text through speech recognition, so is cautious and clarifies if it's not sure what was heard. Unfortunately it has been known to go meta and suggest that you to adjust your microphone! ;)

The assistant is called Jenny by default, per the speech model.

You can tweak the assistant name, speech-to-text model, text-to-speech model, prompts, etc. through configuration, though the config system needs more work to be user-friendly and future-proof.

Discord Server

We have a channel on the Kobold-AI server. Please try to file real bugs and requests on github instead, but this is a good place to initially discuss possible bugs, chat about ideas, etc. https://discord.com/channels/849937185893384223/1110256272403599481

Running

• Install as instructed below
• Make sure KoboldAI (preferably) (a.k.a, KoboldAI-Client), KoboldCPP or text-generation-webui are running, with a suitable LLM model loaded, and serving a KoboldAI compatible API at http://localhost:5000/api/v1/generate (see Configuration, below, if you need to change this URL). See KoboldAI below, for a quickstart guide.
• Run one or more of the commands below. If you get any errors about missing libraries, follow the instructions about that under Installation, below.