Sep 3, 2026 by Michael Sjoberg

Using smaller models for fast code completion

There was a thread on Twitter/X questioning the usefulness of smaller, sub-27B parameter models for local coding.

I've had the same 1.5B parameter model running locally for about a year.

qwen2.5-coder:1.5b

It's not ChatGPT level thinking, but it does provide fast completions.

I find it mostly useful for newline suggestions. For example, after defining a function, it can suggest the first line based on how I've written nearby functions. Or, when defining a few constants, it can suggest the next one, and so on.

It does this fairly successfully.

This isn't a major hands-off AI coding feature, but that's also why I like it. It's almost invisible. It doesn't require any real attention from me, and sometimes it gets it right and I accept the completion instead of writing it myself.

For this kind of use, I don't really need a model capable of solving complex coding problems. I mostly want something small enough to always have running and fast enough that waiting for the suggestion isn't noticeable.

I still use a larger model for the more powerful inline chat feature in Koi.

glm-5.3-flash:cloud

This is served via Ollama Cloud.