Algorithms Overview

Combines BM25 keyword relevance, semantic similarity, temporal freshness, domain authority, evidence strength, diversity penalty, content depth, and source consensus into a single 0–1 score.

CSS selectors → Readability → Headless JS → PDF extraction → Screenshot OCR. Each layer is tried in order, falling back gracefully to handle any web content.

BM25-scored segment ranking combined with a greedy knapsack algorithm to pack maximum relevance within a token budget. Powers all detail tiers.

Classifies content into 8 typed segments: heading, paragraph, code, list, table, quote, metadata, other. Each type has a different token efficiency weight.

Analyzes query intent, complexity, and freshness requirements to automatically select the optimal combination of search backends for each request.

Spawns parallel Tokio agents (planner, searcher, extractor, synthesizer) to process complex research queries with inter-agent coordination via async channels.

Cross-session learning via SQLite. Tracks source trust scores, failure patterns, and query prediction to improve results over time.

SimHash-based fingerprinting identifies semantically equivalent queries and routes to cached results, reducing redundant backend calls.