Ollama Local Model Integration

​

Overview

​

What is Ollama?

​

Prerequisites

• Docker Desktop installed and running
• Weam repository set up locally

​

Step 1: Start Ollama Service

docker compose -f nextjs/docker-compose.ollama.yml --profile cpu up -d

Note: The initial download may take several minutes depending on your internet connection.

docker compose -f nextjs/docker-compose.ollama.yml --profile cpu down

​

Step 2: Configure Ollama in Weam

1. Navigate to Settings
   • Go to Settings → Configuration

3. Add New Model
   • Click Add Model
4. Select Ollama Provider
   • Select Ollama as the provider
5. Choose Your Model
   • Click Select Local Model
   • Choose your desired Llama model (e.g., llama3, qwen2, mistral)
6. Configure and Download
   • Click Configure
   • The model will be automatically downloaded and linked to Weam

​

Step 3: Verify Your Setup

1. Check Ollama Service
   docker ps
   
   You should see an Ollama container running.
2. Test Model Availability
   curl http://localhost:11434/api/tags
   
   This should return a JSON response with your installed models.
3. Test in Weam
   • Open a new chat in Weam
   • Select your configured Ollama model
   • Send a simple message like “Hello, how are you?”
   • Verify the response comes from your local model

​

Available Models

​

Large Language Models

• llama3 - Meta’s latest Llama model (8B, 70B variants)
• llama3.1 - Updated version with improved performance
• qwen2 - Alibaba’s Qwen series (7B, 14B, 72B variants)
• mistral - Mistral AI’s efficient models (7B, 8x7B variants)
• codellama - Code-specialized Llama model
• phi3 - Microsoft’s compact but capable model

​

Model Selection Tips

• For general use: Start with llama3 (8B) - good balance of performance and resource usage
• For coding tasks: Use codellama or qwen2-coder
• For resource-constrained systems: Try phi3 or mistral (7B)
• For maximum capability: Use llama3 (70B) if you have sufficient RAM

​

Resource Requirements

​

Troubleshooting

​

Common Issues and Solutions

• Ensure Docker Desktop is running
• Check if port 11434 is already in use
• Try: docker compose -f nextjs/docker-compose.ollama.yml --profile cpu down then restart

• Verify you have added an OpenAI API key
• Check that Ollama service is running
• Restart Weam after adding the API key

• Check available RAM - models need sufficient memory
• Consider using a smaller model if resources are limited
• Close other memory-intensive applications

• Check internet connection
• Ensure sufficient disk space
• Try downloading manually: docker exec -it <ollama_container> ollama pull llama3

​

Checking Logs

# Check Ollama container logs
docker logs <ollama_container_name>

# Check Weam logs
docker logs <weam_service_name>

​

Performance Optimization

​

For Better Performance

1. Allocate More Resources
   • Increase Docker memory allocation in Docker Desktop settings
   • Ensure your system has sufficient RAM for the model size
2. Model Selection
   • Use quantized models when available (e.g., llama3:8b-instruct-q4_0)
   • Start with smaller models and scale up as needed
3. System Optimization
   • Close unnecessary applications
   • Use SSD storage for better I/O performance
   • Ensure good cooling for sustained performance

​

Security and Privacy

​

Data Privacy Benefits

• Local Processing: All AI inference happens on your machine
• No Data Transmission: Your prompts and responses never leave your system
• Full Control: You control the model, data, and processing environment
• Compliance: Easier to meet data privacy regulations

​

Security Considerations

• Keep your system updated with security patches
• Use strong authentication for Weam access
• Consider network isolation if running in production
• Regularly update Ollama and model versions

​

That’s It!