Request Lifecycle

When your application calls await orchestrator.build_context(session_id, query), ContextOS executes a carefully sequenced pipeline. This page breaks down the exact steps that occur in a single request lifecycle.

User / Agent
    ↓
ContextOS Orchestrator
    ↓
1. Persistence & Token Counting
    ↓
2. Embedding Generation
    ↓
3. Hybrid Retrieval (Dense + BM25)
    ↓
4. Reciprocal Rank Fusion (RRF)
    ↓
5. Late Hydration
    ↓
6. Reranking (Cross-Encoder)
    ↓
7. Memory Planning (Budgeting)
    ↓
8. Context Construction
    ↓
ContextResult
    ↓
Application LLM

1. Persistence & Token Counting

Before any search begins, the user's incoming query is converted into a ContextItem with role USER. Its exact token count is calculated using the configured tokenizer, and it is immediately persisted to the PostgreSQL context_items table.

2. Embedding Generation

The query string is passed to the configured EmbeddingService (e.g., local BGE or OpenAI). The resulting dense vector is persisted to the appropriate vector table (e.g., context_embeddings_bge) via pgvector.

3. Candidate Retrieval

The Retriever executes up to three parallel queries against the database:

• Recent Retrieval: The most recent N conversational turns from the session.
• Dense Retrieval: An HNSW cosine-similarity search against the vector table using the query embedding.
• Lexical Retrieval: A BM25 keyword search against the context_items table (if enabled).

4. Fusion & Hydration

To minimize latency and I/O, the dense and BM25 queries only return UUIDs and scores, not full text. ContextOS merges the dense and lexical rankings using Reciprocal Rank Fusion (RRF) to calculate a unified hybrid score.

The pool is truncated to the top pre_rerank_k candidates. Only these top candidates are "hydrated" by fetching their full text content from PostgreSQL.

5. Reranking

The hydrated candidates are passed to a Cross-Encoder reranker. The reranker computes a highly accurate semantic relevance score for each candidate relative to the query.

6. Memory Planning

The scored candidate pool and the configured Token Budget are passed into the MemoryPlanner. The planner deducts guaranteed items (like the System prompt) from the budget, sorts the remaining candidates by their final hybrid score (combining semantic relevance and chronological recency), and greedily packs them until the budget is exhausted.

7. Construction & Return

The selected items are sorted back into chronological order and passed to the PromptBuilder, which formats them into a single continuous string.

ContextOS returns a ContextResult containing the finalized prompt string, the list of selected items, and a detailed ExecutionTrace of the entire lifecycle.