Storage Abstraction
Overview
AgentDock includes a comprehensive storage abstraction layer that provides a unified interface for multiple storage backends, enabling developers to switch between different storage solutions without changing application code.
Implementation
The storage abstraction layer has been fully implemented with:
Core Features
- Unified
StorageProvider
interface for all backends - Factory pattern with environment-based configuration
- Essential adapters: Memory, Redis, Vercel KV
- Full integration with SessionManager and OrchestrationManager
- Built-in TTL support across all providers
Extended Storage Support
- SQLite for local persistence
- PostgreSQL for production deployments
- PostgreSQL Vector for AI/embeddings
- MongoDB as optional document store
- Automatic Node.js adapter registration
Advanced Capabilities
- S3, DynamoDB, Cloudflare KV/D1 adapters
- Pinecone, Qdrant, ChromaDB for vector operations
- SQLite-vec for local vector search
- Consistent patterns across all adapters
Available Adapters (15 Total)
Always Available (Auto-registered)
- Memory - Default, in-memory storage
- Redis/Upstash - High-performance distributed cache
- Vercel KV - Vercel's Redis-compatible service
Core Storage (Server-side)
- SQLite - File-based local storage
- SQLite-vec - SQLite with vector search capabilities
- PostgreSQL - Production relational database
- PostgreSQL Vector - pgvector for embeddings
- MongoDB - Document storage (enable with
ENABLE_MONGODB=true
)
Additional Adapters (Not Auto-registered)
To keep build size small:
- S3, DynamoDB, Cloudflare KV/D1, Pinecone, Qdrant, ChromaDB
Implementation Details
Architecture
// Unified interface
interface StorageProvider {
get(key: string): Promise<any>
set(key: string, value: any, ttl?: number): Promise<void>
delete(key: string): Promise<void>
exists(key: string): Promise<boolean>
list(pattern?: string): Promise<string[]>
}
Configuration
# Simple env-based setup
KV_STORE_PROVIDER=redis
REDIS_URL=https://your-instance.upstash.io
REDIS_TOKEN=your-token
Usage
// Automatic adapter selection
const storage = getStorageFactory().getProvider();
await storage.set('key', 'value');
Key Achievements
- Zero Breaking Changes: Default memory storage maintains compatibility
- Simple Configuration: Environment variables control everything
- Production Ready: Battle-tested adapters for all use cases (15 total)
- Developer Friendly: Single API to learn, works everywhere
- Flexible Architecture: Easy to add new storage backends
- Build Optimization: Non-essential adapters not auto-registered to keep builds small
Technical Decisions
- Factory Pattern: Centralized adapter management
- Dynamic Imports: Node.js adapters load only when needed
- TTL First-class: Built into core interface
- Environment Config: Simple setup via env vars
Use Cases
Current
- Session Management: Redis for distributed sessions
- Message History: PostgreSQL for durability
- Vector Search: pgvector for semantic search
- Development: SQLite for local persistence
Future Considerations
- Storage migration utilities
- Multi-provider sync capabilities
- Compression middleware
- Encryption at rest
Documentation
Success Metrics
- 15 production-ready adapters (including SQLite-vec)
- Zero-config memory storage
- < 5 min setup for any adapter
- No client-side bundling issues
- Backward compatible