Stream real-time processing status updates via Server-Sent Events (SSE). Perfect for monitoring indexing progress in real-time.
Request
No parameters required. This endpoint returns a long-lived SSE connection.
Request Example
curl -N http://localhost:60534/api/updates
import requests
import json
response = requests.get(
'http://localhost:60534/api/updates',
stream=True
)
for line in response.iter_lines():
if line:
decoded = line.decode('utf-8')
if decoded.startswith('data: '):
event = json.loads(decoded[6:])
print(f"[{event['type']}] {event['path']}")
const eventSource = new EventSource('http://localhost:60534/api/updates');
eventSource.onmessage = (event) => {
const data = JSON.parse(event.data);
console.log(`[${data.type}] ${data.path}`);
};
eventSource.onerror = (error) => {
console.error('SSE error:', error);
eventSource.close();
};
Each SSE event is sent in this format:
data: {"type": "file_parsing", "path": "/path/to/file.txt", "metadata": {...}}
Event Types
File is being parsed by MarkItDown
Detected file type/extension
AI is generating summary and keywords
Length of parsed content in characters
LLM backend being used: ollama or openai
Creating vector embedding for semantic search
Embedding model: e5-base-v2
Dimension of embedding vector: 768
File successfully indexed and ready for search
Total processing time in milliseconds
Error occurred during processing
Error message describing what went wrong
Pipeline stage where failure occurred
Event Stream Example
data: {"type": "file_parsing", "path": "/Users/username/Downloads/report.pdf", "metadata": {"file_size": 1048576, "file_type": "pdf"}}
data: {"type": "file_summarizing", "path": "/Users/username/Downloads/report.pdf", "metadata": {"content_length": 5234, "llm_backend": "ollama"}}
data: {"type": "file_embedding", "path": "/Users/username/Downloads/report.pdf", "metadata": {"model": "e5-base-v2", "embedding_dim": 768}}
data: {"type": "file_complete", "path": "/Users/username/Downloads/report.pdf", "metadata": {"total_time": 3245, "title": "Q4 Financial Report"}}
data: {"type": "file_failed", "path": "/Users/username/Downloads/corrupted.xlsx", "metadata": {"error": "Unable to parse file: File is corrupted", "stage": "parsing"}}
Use Cases
1. Progress Monitoring (TUI)
The TUI uses this endpoint to show real-time indexing status:
async for event in sse_stream('/api/updates'):
status_bar.update(f"Processing: {event['path']}")
2. Custom Progress UI
Build your own progress interface:
const stats = {
parsing: 0,
summarizing: 0,
embedding: 0,
complete: 0,
failed: 0
};
eventSource.onmessage = (event) => {
const data = JSON.parse(event.data);
stats[data.type.replace('file_', '')]++;
updateProgressBar(stats);
};
3. Error Monitoring
Track and log failures:
for event in sse_stream():
if event['type'] == 'file_failed':
logger.error(f"Failed to index {event['path']}: {event['metadata']['error']}")
This is a long-lived HTTP connection. Make sure your HTTP client supports streaming responses and doesn’t timeout on long connections.
The TUI automatically subscribes to this endpoint and displays updates in the status bar. You can see real-time progress without polling.