Month: October 2024

Run LLMs locally without API costs using Ollama: # Install Ollama curl -fsSL https://ollama.ai/install.sh | sh # Pull a model ollama pull llama2 ollama pull codellama Node.js Integration npm install ollama import { Ollama } from ‘ollama’; const ollama = new Ollama(); // Simple completion const response = await ollama.chat({ model: ‘llama2’, messages: [{ role:…

Rate limiting is critical for AI APIs. Here’s a robust implementation: import Bottleneck from ‘bottleneck’; const limiter = new Bottleneck({ reservoir: 60, // 60 requests reservoirRefreshAmount: 60, reservoirRefreshInterval: 60 * 1000, // per minute maxConcurrent: 5, minTime: 100 // 100ms between requests }); // Wrap OpenAI calls const rateLimitedChat = limiter.wrap(async (prompt) => { return…