Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Streaming (v1.0.0)

This document covers the currently implemented streaming behavior.

Streaming Architecture

Token streaming is built around a process-global event sink in antikythera-cli::infrastructure::llm::streaming.

flowchart LR
    Provider[LLM provider parser] --> Emit[emit_stream_event]
    Emit --> Sink[STREAM_SINK global]
    Sink --> TUI[TUI mpsc channel → Conversation panel]
    Sink --> Terminal[Terminal sink → stderr]

StreamEvent

Provider parsers emit one of three variants:

VariantPayloadDescription
Startedprovider_id, session_idSignals that the stream has begun
Chunkprovider_id, session_id, contentOne content fragment from the model
Completedprovider_id, session_idSignals that the stream is done

Sink modes

SinkInstalled byBehaviour
TUI sinkset_stream_event_sink in submit_inputForwards Chunk.content over an mpsc::unbounded_channel; TUI drains it each frame for live preview
Terminal sinkinstall_terminal_stream_sink via --stream flagWrites chunks directly to stderr; Started prints a header, Completed prints a newline

Runtime Guarantees

  • Only one sink is active at a time; set_stream_event_sink replaces any previous sink.
  • clear_stream_event_sink drops the sender, cleanly ending the mpsc channel after each response.
  • Started and Completed events are forwarded to tracing::info! for the log panel under the cli:streaming source label.
  • Chunk events are not traced — they are too frequent and their content is visible in the chat panel.