Response Streaming in AgentDock
AgentDock's response streaming system extends the Vercel AI SDK to provide enhanced functionality for orchestration, error handling, and state management.
AgentDockStreamResult
The core of AgentDock's streaming capabilities is the AgentDockStreamResult
interface, which extends Vercel AI SDK's StreamTextResult
:
export interface AgentDockStreamResult<T extends ToolSet = ToolSet, R = unknown>
extends VercelStreamTextResult<T, R> {
_orchestrationState?: {
recentlyUsedTools?: string[];
cumulativeTokenUsage?: {
promptTokens: number;
completionTokens: number;
totalTokens: number;
};
[key: string]: unknown;
} | null;
_hasStreamingError?: boolean;
_streamingErrorMessage?: string;
}
Key Enhancements
-
Orchestration State Tracking
- Maintains a record of recently used tools
- Tracks cumulative token usage across multiple requests
- Supports arbitrary state properties for extensibility
-
Enhanced Error Handling
- Includes flags to indicate streaming errors
- Preserves error messages for client-side handling
- Overrides
toDataStreamResponse
to properly include error information
-
Backward Compatibility
- Provides a type alias (
StreamTextResult
) for backward compatibility - Maintains the same streaming interface as Vercel AI SDK
- Provides a type alias (
Integration with LLMOrchestrationService
The AgentDockStreamResult
is primarily returned by the LLMOrchestrationService.streamWithOrchestration
method, which:
- Wraps the CoreLLM's
streamText
method - Injects orchestration-specific callbacks
- Updates token usage in session state
- Tracks tool usage for subsequent requests
Usage in AgentNode
The AgentNode
returns the AgentDockStreamResult
from its handleMessage
method, allowing:
- Clients to consume the stream directly
- Error handling at the adapter/route level
- Access to orchestration state for complex flows
Benefits
- Enhanced Reliability: Better error propagation and handling
- Improved State Management: Automatic tracking of tokens and tools
- Seamless Integration: Works with the existing Vercel AI SDK patterns
For more information about the AgentNode implementation, see Agent Node Documentation.