Getting started with AI in Node.js is simpler than you might think. The key libraries to know are: 1. OpenAI SDK npm install openai import OpenAI from ‘openai’; const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY }); const completion = await openai.chat.completions.create({ model: ‘gpt-4’, messages: [{ role: ‘user’, content: ‘Hello!’ }] }); console.log(completion.choices[0].message.content); 2. LangChain.js For…
Streaming responses from LLMs provides a much better UX. Here’s how to implement it properly: import OpenAI from ‘openai’; import { Readable } from ‘stream’; const openai = new OpenAI(); async function streamChat(prompt) { const stream = await openai.chat.completions.create({ model: ‘gpt-4’, messages: [{ role: ‘user’, content: prompt }], stream: true }); for await (const chunk…
Vector databases are essential for RAG (Retrieval Augmented Generation). Here’s a complete setup with Pinecone: npm install @pinecone-database/pinecone openai import { Pinecone } from ‘@pinecone-database/pinecone’; import OpenAI from ‘openai’; const pinecone = new Pinecone({ apiKey: process.env.PINECONE_API_KEY }); const openai = new OpenAI(); const index = pinecone.Index(‘documents’); // Generate embeddings async function embed(text) { const response…
Function calling lets LLMs interact with your APIs. Here’s a production pattern: const tools = [{ type: ‘function’, function: { name: ‘get_weather’, description: ‘Get current weather for a location’, parameters: { type: ‘object’, properties: { location: { type: ‘string’, description: ‘City name’ }, unit: { type: ‘string’, enum: [‘celsius’, ‘fahrenheit’] } }, required: [‘location’] }…
Run LLMs locally without API costs using Ollama: # Install Ollama curl -fsSL https://ollama.ai/install.sh | sh # Pull a model ollama pull llama2 ollama pull codellama Node.js Integration npm install ollama import { Ollama } from ‘ollama’; const ollama = new Ollama(); // Simple completion const response = await ollama.chat({ model: ‘llama2’, messages: [{ role:…
Rate limiting is critical for AI APIs. Here’s a robust implementation: import Bottleneck from ‘bottleneck’; const limiter = new Bottleneck({ reservoir: 60, // 60 requests reservoirRefreshAmount: 60, reservoirRefreshInterval: 60 * 1000, // per minute maxConcurrent: 5, minTime: 100 // 100ms between requests }); // Wrap OpenAI calls const rateLimitedChat = limiter.wrap(async (prompt) => { return…
LangChain agents can use tools autonomously. Here’s a complete agent setup: import { ChatOpenAI } from ‘@langchain/openai’; import { AgentExecutor, createOpenAIToolsAgent } from ‘langchain/agents’; import { DynamicTool } from ‘@langchain/core/tools’; import { ChatPromptTemplate } from ‘@langchain/core/prompts’; const tools = [ new DynamicTool({ name: ‘calculator’, description: ‘Performs math calculations’, func: async (input) => { return String(eval(input));…
Caching LLM responses saves money and improves latency: import { createHash } from ‘crypto’; import Redis from ‘ioredis’; const redis = new Redis(); const CACHE_TTL = 3600; // 1 hour function hashPrompt(messages, model) { const content = JSON.stringify({ messages, model }); return createHash(‘sha256’).update(content).digest(‘hex’); } async function cachedChat(messages, options = {}) { const { model =…
Chunking strategies greatly affect RAG quality: import { RecursiveCharacterTextSplitter } from ‘langchain/text_splitter’; // Basic chunking const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000, chunkOverlap: 200, separators: [‘ ‘, ‘ ‘, ‘ ‘, ”] }); const chunks = await splitter.splitText(document); Semantic Chunking async function semanticChunk(text, maxTokens = 500) { const sentences = text.match(/[^.!?]+[.!?]+/g) || [text]; const chunks…
Building a conversational AI with memory requires careful context management: class ConversationManager { constructor(options = {}) { this.maxTokens = options.maxTokens || 4000; this.systemPrompt = options.systemPrompt || ‘You are a helpful assistant.’; this.conversations = new Map(); } getHistory(sessionId) { if (!this.conversations.has(sessionId)) { this.conversations.set(sessionId, []); } return this.conversations.get(sessionId); } async chat(sessionId, userMessage) { const history = this.getHistory(sessionId);…
