Use Ollama

What You’ll Learn

How to configure Foyle to use models served by Ollama

Prerequisites

1. Follow [Ollama’s docs] to download Ollama and serve a model like llama2

Setup Foyle to use Ollama

Foyle relies on Ollama’s OpenAI Chat Compatability API to interact with models served by Ollama.

1. Configure Foyle to use the appropriate Ollama baseURL

   foyle config set openai.baseURL=http://localhost:11434/v1
   

   • Change the server and port to match how you are serving Ollama
   • You may also need to change the scheme to https; e.g. if you are using a VPN like Tailscale

2. Configure Foyle to use the appropriate Ollama model

   foyle config agent.model=llama2 
   

   • Change the model to match the model you are serving with Ollama

3. You can leave the apiKeyFile unset since you aren’t using an API key with Ollama

4. That’s it! You should now be able to use Foyle with Ollama