Inline chat

Using inline chat ⋅ The document is the context ⋅ Model configuration ⋅ Why?

Chat with AI directly inside a document using the current file as context.

Start a line with >, write a prompt, and run it as an inline command. The response is streamed directly into the document below.

Using inline chat

Inline chat prompts start with >:

> Explain the difference between a process and a thread.

Place the cursor on the line and run Inline Command:

Koi sends the prompt to the configured chat model and streams the response below it:

> Explain the difference between a process and a thread.
A process is an independent running program with its own memory space, while a thread is an execution path within a process...

The response is inserted as normal text without a prefix.

A working indicator is shown while the model is processing or generating a response. Press Escape to cancel the current request.

To continue the conversation, add another > prompt and run it the same way:

> How does that affect memory usage?
Processes generally have separate address spaces, while threads within the same process share...

The document is the context

Inline chat uses the current document as context.

The document can contain source code, notes, logs, previous responses, or anything else relevant to the next prompt.

For example:

We need to reduce allocations in this function.

def parse_items(data):
    return [item.strip() for item in data.split(",")]

> How could this be improved?

Because the context is ordinary text, you can edit it before sending the next prompt. Delete an irrelevant response, change an earlier instruction, paste in more code, or rearrange the document.

The > prefix distinguishes prompts from the rest of the document. Model responses remain ordinary text.

Saving conversations

An inline conversation is just part of the document.

Save the file normally and reopen it later to continue. Add another > prompt and run the inline command again.

There is no separate chat history or conversation format to maintain.

Model configuration

Code completion runs through Ollama.

Download and install Ollama, then choose a model from the ⁠Ollama model library. Models can run locally on your Mac, or you can use larger cloud models that would otherwise require much more memory and compute.

For example, to use a local model:

ollama pull gpt-oss:20b

Or a larger cloud model:

ollama pull gpt-oss:120b-cloud

Cloud models use the same Ollama interface as local models, but inference runs on Ollama's hosted hardware.

See the models available on your Mac with:

ollama list

Then use the model name in the Koi configuration:

[models]

inline_chat                     gpt-oss:120b-cloud

For inline chat, larger models can be useful when working with longer documents, code, or questions that require more reasoning. A smaller local model keeps everything on your Mac, while a cloud model gives you access to much larger models without requiring the hardware to run them locally.

Why?

Inline chat keeps the model, its context, and its output in the same editable document.

There is no separate chat panel and no special response format. Prompts are marked with >, responses are ordinary text, and the surrounding document provides the context.

You can see and edit that context directly using the same tools you use for everything else in the editor.

Inline chat is still under active development. Its context handling, configuration, and behavior may change as the feature develops.