In last blog post, I talked about running AI LLM models locally. Read that first if you want to follow along.

With local LLMs in place and running, I searched for ways to integrate them with VS Code. The answer is Continue extension.

Set up:

• Go to http://127.0.0.1:3000 and login. Then setting (gear icon) -> Account tab -> API keys. Generate one if non exists.
• Get some models. I have tried both deepseek-coder-v2 and qwen2.5-coder for chatting assistance and love them. For Autocomplete, I am using starcoder2.
• Install Continue extension. I’ve made the following changes to ~/.contine/config.json to get things going. Reading Continue doc, I think there are more useful features that may need additional customization. I’ll edit this post (or write additional blogs) as I discover additional useful feature.
  

  "models": [
    {
      "model": "qwen2.5-coder",
      "provider": "openai",
      "apiKey": "sk-mykey",
      "apiBase": "http://localhost:11434/v1",
      "title": "qwen2.5-coder",
      "contextLength": 16384
    },
    {
      "model": "deepseek-coder-v2",
      "provider": "openai",
      "apiKey": "sk-mykey",
      "apiBase": "http://localhost:11434/v1",
      "title": "deepseek-coder-v2",
      "contextLength": 16384
    }
  ],
  "tabAutocompleteModel": {
    "model": "starcoder2",
    "provider": "openai",
    "apiKey": "sk-mykey",
    "apiBase": "http://localhost:11434/v1",
    "title": "starcoder2"
  },

In my limited experience so far, I was very impressed. I started writing a C program that interfaces with CockroachDB using PostgreSQL library. Both qwen2.5-coder and deepseek-coder-v2 generated code that worked with slight tweaks. It also helps me reacquaint with VS Code by giving suggestions on build tasks.

I think AI/ML tools will be very useful for experienced developers. Less so for inexperienced technology professionals.

I’d love to hear about your experience using Continue.