Best Practices

Essential patterns and recommendations for building robust, secure, and maintainable AI applications with Orka.

1. Structured Error Handling

Orka provides a custom error hierarchy (OrkaError) with error codes and module context. Always catch errors by type to handle different failure modes appropriately — network timeouts, rate limits, and validation errors each require different recovery strategies.

import { OrkaError, OrkaErrorCode } from 'orkajs/errors';
 
try {
  const result = await orka.ask({
    knowledge: 'docs',
    question: 'What is Orka?',
  });
} catch (error) {
  if (OrkaError.isOrkaError(error)) {
    switch (error.code) {
      case OrkaErrorCode.LLM_TIMEOUT:
        // Retry with a longer timeout or fallback provider
        console.warn(`Timeout on ${error.module}: ${error.message}`);
        break;
      case OrkaErrorCode.LLM_RATE_LIMIT:
        // Back off and retry after delay
        await delay(5000);
        break;
      case OrkaErrorCode.KNOWLEDGE_SEARCH_ERROR:
        // Fall back to direct LLM call without RAG
        break;
      default:
        // Log and propagate
        console.error(error.toJSON());
        throw error;
    }
  }
}

Tip: Use OrkaError.isRetryable(error) to quickly determine if an error is transient and can be retried automatically.

2. Resilience: Retry + Fallback + Timeout

Production LLM applications must handle transient failures gracefully. Orka provides three complementary resilience mechanisms that can be composed together: ResilientLLM (automatic retry with exponential backoff), FallbackLLM (multi-provider failover), and per-adapter timeouts.

import { OpenAIAdapter } from 'orkajs/adapters/openai';
import { AnthropicAdapter } from 'orkajs/adapters/anthropic';
import { ResilientLLM } from 'orkajs/resilience';
import { FallbackLLM } from 'orkajs/resilience';
 
// Step 1: Configure adapters with timeouts
const openai = new OpenAIAdapter({
  apiKey: process.env.OPENAI_API_KEY!,
  timeoutMs: 30_000, // 30s timeout
});
 
const anthropic = new AnthropicAdapter({
  apiKey: process.env.ANTHROPIC_API_KEY!,
  timeoutMs: 45_000,
});
 
// Step 2: Wrap with automatic retry
const resilientOpenAI = new ResilientLLM({
  llm: openai,
  retry: {
    maxRetries: 3,
    initialDelayMs: 1000,
    backoffMultiplier: 2,
  },
  onRetry: (error, attempt) => {
    console.warn(`Retry #${attempt}: ${error.message}`);
  },
});
 
// Step 3: Add multi-provider fallback
const llm = new FallbackLLM({
  adapters: [resilientOpenAI, anthropic],
  onFallback: (error, adapterName) => {
    console.warn(`Falling back from ${adapterName}: ${error.message}`);
  },
});

This layered approach ensures: (1) transient errors are retried automatically, (2) persistent failures trigger a switch to an alternative provider, and (3) no request hangs indefinitely thanks to timeouts.

3. Security Best Practices

- SQL Injection Prevention

When using SQLToolkit, always enable readOnly mode in production. Orka's SQLToolkit automatically validates table names (alphanumeric + underscores only), blocks dangerous SQL keywords (DROP, DELETE, ALTER, etc.), and prevents multi-statement injection attacks.

import { SQLToolkit } from 'orkajs/agent/toolkits/sql';
 
const sqlToolkit = new SQLToolkit({
  execute: (query) => db.query(query),
  readOnly: true,  // ALWAYS enable in production
  maxRows: 50,     // Limit result size
  schema: schemaString, // Provide schema to avoid dynamic queries
});

- URL Validation (SSRF Protection)

When ingesting knowledge from URLs, Orka automatically validates that only http/https protocols are used, enforces a 30-second timeout, and limits response size to 50MB. Never pass user-controlled URLs directly without additional validation in your application layer.

// Orka handles this internally, but add your own allowlist for user-facing apps:
const ALLOWED_DOMAINS = ['docs.example.com', 'wiki.internal.com'];
 
function validateUserUrl(url: string): boolean {
  try {
    const parsed = new URL(url);
    return ALLOWED_DOMAINS.includes(parsed.hostname);
  } catch {
    return false;
  }
}
 
// Then use it before passing to Orka
if (validateUserUrl(userProvidedUrl)) {
  await orka.knowledge.add('docs', { url: userProvidedUrl });
}

- API Key Management

Never hardcode API keys. Use environment variables and validate their presence at startup.

// Validate at startup
const requiredEnvVars = ['OPENAI_API_KEY', 'PINECONE_API_KEY'];
for (const envVar of requiredEnvVars) {
  if (!process.env[envVar]) {
    throw new Error(`Missing required environment variable: ${envVar}`);
  }
}

4. Memory Management

Choose the right memory strategy based on your use case. Orka provides three strategies: sliding_window (keeps the N most recent messages), buffer (keeps messages up to a token limit), and summary (compresses old messages into a summary to preserve context while reducing size).

import { Memory } from 'orkajs/memory';
 
// For chatbots: use summary to preserve long conversation context
const chatMemory = new Memory({
  strategy: 'summary',
  maxMessages: 20,
  summaryThreshold: 8, // Summarize when 8+ messages overflow
});
 
// For short tasks: use sliding_window for simplicity
const taskMemory = new Memory({
  strategy: 'sliding_window',
  maxMessages: 10,
});
 
// For token-sensitive apps: use buffer with token estimation
const tokenMemory = new Memory({
  strategy: 'buffer',
  maxTokensEstimate: 2000,
});

Multi-session: Use SessionMemory for multi-user applications. It manages separate Memory instances per session with automatic TTL-based cleanup.

5. Observability & Tracing

Always configure the Tracer with limits in production to prevent memory leaks. Set maxTraces to cap stored traces and traceTtlMs to auto-expire old ones. Use hooks to export traces to your monitoring system.

import { Tracer } from 'orkajs/observability';
 
const tracer = new Tracer({
  logLevel: 'info',
  maxTraces: 500,          // Keep at most 500 traces in memory
  traceTtlMs: 3600_000,    // Auto-expire traces after 1 hour
  hooks: [{
    onTraceEnd: (trace) => {
      // Export to your monitoring system
      metrics.recordLatency(trace.name, trace.totalLatencyMs);
      metrics.recordTokens(trace.name, trace.totalTokens);
    },
    onError: (error, context) => {
      errorTracker.capture(error, context);
    },
  }],
});

6. Agent Event Listeners

All agents (ReActAgent, OpenAIFunctionsAgent, StructuredChatAgent) extend BaseAgent and support event listeners. Use them for logging, metrics, or real-time UI updates.

import { ReActAgent } from 'orkajs/agent/react';
 
const agent = new ReActAgent(config, llm, memory);
 
// Listen to agent lifecycle events
agent.on('step:start', (event) => {
  console.log(`[Agent] Starting step for: ${event.agentType}`);
});
 
agent.on('tool:start', (event) => {
  console.log(`[Agent] Calling tool: ${event.toolName}`);
});
 
agent.on('tool:error', (event) => {
  console.error(`[Agent] Tool ${event.toolName} failed:`, event.error);
});
 
agent.on('complete', (event) => {
  console.log(`[Agent] Completed:`, event.output);
});
 
const result = await agent.run('Analyze the sales data');

7. Caching Strategy

Caching LLM responses dramatically reduces costs and latency. Use MemoryCache for development and RedisCache for production. Orka uses SHA-256 hashing for cache keys, ensuring collision-free lookups.

import { CachedLLM } from 'orkajs/cache';
import { RedisCache } from 'orkajs/cache';
 
// Production: use Redis for persistent, shared caching
const cache = new RedisCache({
  url: process.env.REDIS_URL!,
  keyPrefix: 'orka:llm:',
  ttlMs: 3600_000, // 1 hour TTL
});
 
const cachedLLM = new CachedLLM({
  llm: openai,
  cache,
  ttlMs: 3600_000,
});
 
// Same prompt + options = cache hit (SHA-256 key)
const result1 = await cachedLLM.generate('What is TypeScript?'); // API call
const result2 = await cachedLLM.generate('What is TypeScript?'); // Cache hit!

8. Knowledge Ingestion at Scale

When ingesting large document collections, configure chunk size and overlap carefully. Smaller chunks improve retrieval precision but increase storage. Larger chunks preserve more context but may reduce relevance scoring.

// Recommended settings by use case:
 
// FAQ / Short answers: small chunks, low overlap
await orka.knowledge.create({
  name: 'faq',
  source: { path: './faq-docs' },
  chunkSize: 256,
  chunkOverlap: 32,
});
 
// Technical docs: medium chunks, moderate overlap
await orka.knowledge.create({
  name: 'technical',
  source: { path: './technical-docs' },
  chunkSize: 512,
  chunkOverlap: 64,
});
 
// Legal / Long-form: large chunks, high overlap
await orka.knowledge.create({
  name: 'legal',
  source: { path: './contracts' },
  chunkSize: 1024,
  chunkOverlap: 128,
});

9. Testing & Evaluation

Use Orka's built-in evaluation framework to measure quality before deploying. Define test datasets with expected outputs and run automated evaluations with multiple metrics.

const results = await orka.evaluate({
  dataset: [
    { input: 'What is Orka?', expected: 'Orka is a TypeScript AI framework' },
    { input: 'How to install?', expected: 'npm install orkajs' },
  ],
  metrics: ['relevance', 'faithfulness', 'completeness'],
  concurrency: 3,
});
 
// Check results
console.log('Average relevance:', results.summary.relevance.mean);
console.log('Average faithfulness:', results.summary.faithfulness.mean);
 
// Fail CI if quality drops below threshold
if (results.summary.relevance.mean < 0.8) {
  process.exit(1);
}

10. Architecture Recommendations

- Tree-Shaking: Import Only What You Need

Orka supports sub-path exports for optimal bundle size. Always import from specific modules rather than the root package.

// Good: specific imports (tree-shakeable)
import { ReActAgent } from 'orkajs/agent/react';
import { OpenAIAdapter } from 'orkajs/adapters/openai';
import { JSONParser } from 'orkajs/parsers/json';
 
// Avoid: importing everything
// import { ReActAgent, OpenAIAdapter, JSONParser } from 'orkajs';

- Adapter Composition Pattern

Compose adapters like building blocks. Each wrapper adds a capability without modifying the underlying adapter.

// Composition: Timeout → Retry → Cache → Fallback
import { OpenAIAdapter } from 'orkajs/adapters/openai';
import { ResilientLLM, FallbackLLM } from 'orkajs/resilience';
import { CachedLLM, RedisCache } from 'orkajs/cache';
 
//    OpenAI (with 30s timeout)
// → ResilientLLM (3 retries)
// → CachedLLM (Redis, 1h TTL)
// → FallbackLLM (+ Anthropic backup)
 
const openai = new OpenAIAdapter({ apiKey: '...', timeoutMs: 30_000 });
const resilient = new ResilientLLM({ llm: openai });
const cached = new CachedLLM({
  llm: resilient,
  cache: new RedisCache({ url: 'redis://...' }),
  ttlMs: 3600_000,
});
const llm = new FallbackLLM({
  adapters: [cached, new AnthropicAdapter({ apiKey: '...' })],
});

Quick Reference

Practice	Do	Don't
Error Handling	Catch OrkaError by code	Generic try/catch with console.log
Timeouts	Set timeoutMs on every adapter	Rely on default infinite timeout
SQL Agents	readOnly: true + provide schema	readOnly: false in production
Caching	RedisCache with TTL in production	No caching for repeated queries
Tracing	Set maxTraces + traceTtlMs	Unlimited traces in long-running apps
Imports	Sub-path imports for tree-shaking	Import * from root
API Keys	Environment variables	Hardcoded in source code
Memory	Choose strategy per use case	Unlimited message history