Cloudflare

Configure Cloudflare Workers AI with Continue to access various models for chat and autocomplete, including Llama 3 8B and DeepSeek Coder through Cloudflare’s serverless AI platform

Cloudflare Workers AI can be used for both chat and tab autocompletion in Continue. Here is an example of Cloudflare Workers AI configuration:

YAML

config.yaml

name: My Config
version: 0.0.1
schema: v1

models:
  - name: Llama 3 8B
    provider: cloudflare
    apiKey: <YOUR_CLOUDFARE_API_KEY>
    model: "@cf/meta/llama-3-8b-instruct"
    defaultCompletionOptions:
      contextLength: 2400
      maxTokens: 500
    roles:
      - chat
    env:
      accountId: YOUR CLOUDFLARE ACCOUNT ID
  - name: DeepSeek Coder 6.7b Instruct
    provider: cloudflare
    apiKey: <YOUR_CLOUDFARE_API_KEY>
    model: "@hf/thebloke/deepseek-coder-6.7b-instruct-awq"
    defaultCompletionOptions:
      contextLength: 2400
      maxTokens: 500
    roles:
      - chat
    env:
      accountId: YOUR CLOUDFLARE ACCOUNT ID
  - name: DeepSeek 7b
    provider: cloudflare
    apiKey: <YOUR_CLOUDFARE_API_KEY>
    model: "@hf/thebloke/deepseek-coder-6.7b-base-awq"
    roles:
      - autocomplete
    env:
      accountId: YOUR CLOUDFLARE ACCOUNT ID

JSON

config.json

{
  "models": [
    {
      "title": "Llama 3 8B",
      "provider": "cloudflare",
      "accountId": "YOUR CLOUDFLARE ACCOUNT ID",
      "apiKey": "YOUR CLOUDFLARE API KEY",
      "contextLength": 2400,
      "completionOptions": {
        "maxTokens": 500
      },
      "model": "@cf/meta/llama-3-8b-instruct"
    },
    {
      "title": "DeepSeek Coder 6.7b Instruct",
      "provider": "cloudflare",
      "accountId": "YOUR CLOUDFLARE ACCOUNT ID",
      "apiKey": "YOUR CLOUDFLARE API KEY",
      "contextLength": 2400,
      "completionOptions": {
        "maxTokens": 500
      },
      "model": "@hf/thebloke/deepseek-coder-6.7b-instruct-awq"
    }
  ],
  "tabAutocompleteModel": {
    "title": "DeepSeek 7b",
    "provider": "cloudflare",
    "accountId": "YOUR CLOUDFLARE ACCOUNT ID",
    "apiKey": "YOUR CLOUDFLARE API KEY",
    "model": "@hf/thebloke/deepseek-coder-6.7b-base-awq"
  }
}

Visit the Cloudflare dashboard to create an API key.

Review available models on Workers AI

View the source